ThinkPrior: Zero-Rollout Difficulty Priors for Cold-Start Prompt Selection in RLVR
ThinkPrior uses an offline anchor model to pick RLVR prompts before spending target-policy rollouts.
The paper says uniform sampling wastes 39% of rollouts on groups where every answer is right or every answer is wrong, producing no reward-advantage gradient. ThinkPrior initializes prompt difficulty with verifier-scored anchor pass rates, then updates from training outcomes without changing the loss or optimizer. On Qwen2.5-Math-7B across sixteen seeds, it more than halves early silent groups and cuts wasted rollouts through step 30 by nearly a fifth, with no detected final-accuracy difference. In the reported 250-prompt setting, the gain is described as reallocation rather than a net saving; the ThinkPrior+DAPO setup reduced generated rollouts by 10.6% while keeping the same 3840-rollout update budget. ArXiv · AI/CL/LG's note
The paper says uniform sampling wastes 39% of rollouts on groups where every answer is right or every answer is wrong, producing no reward-advantage gradient. ThinkPrior initializes prompt difficulty with verifier-scored anchor pass rates, then updates from training outcomes without changing the loss or optimizer. On Qwen2.5-Math-7B across sixteen seeds, it more than halves early silent groups and cuts wasted rollouts through step 30 by nearly a fifth, with no detected final-accuracy difference. In the reported 250-prompt setting, the gain is described as reallocation rather than a net saving; the ThinkPrior+DAPO setup reduced generated rollouts by 10.6% while keeping the same 3840-rollout update budget. ArXiv · AI/CL/LG's note
score 5