There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k Price
Under the paper’s assumptions, eliminating the many-try disadvantage wipes out the entire first-try gain.
Yinuo Liu models RLVR as reweighting a base model’s strategies shared across prompts. Under exclusive specialization, the theorem shows that every update improving first-try accuracy must reduce success on some prompts. If those losses reach the hardest prompts, the base model eventually wins when allowed enough attempts. The paper also describes a diagnostic for measuring the trade-off from evaluation logs.
ArXiv · AI/CL/LG's note
Yinuo Liu models RLVR as reweighting a base model’s strategies shared across prompts. Under exclusive specialization, the theorem shows that every update improving first-try accuracy must reduce success on some prompts. If those losses reach the hardest prompts, the base model eventually wins when allowed enough attempts. The paper also describes a diagnostic for measuring the trade-off from evaluation logs.
ArXiv · AI/CL/LG's note
score 5