Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning
A small trained concept generator can act as a reusable search policy for a larger frozen reasoning model.
The paper argues that repeated sampling often produces near-duplicate solutions because it relies on decoding noise. Its alternative samples problem-specific concepts, hints, or strategies before generating answers. The authors then train a small concept generator with reinforcement learning to improve a larger answer generator’s success rate. On hard math reasoning tasks, that trained generator beats naive repeated sampling at the same answer budget and transfers to other answer models.
ArXiv · AI/CL/LG's note
The paper argues that repeated sampling often produces near-duplicate solutions because it relies on decoding noise. Its alternative samples problem-specific concepts, hints, or strategies before generating answers. The authors then train a small concept generator with reinforcement learning to improve a larger answer generator’s success rate. On hard math reasoning tasks, that trained generator beats naive repeated sampling at the same answer budget and transfers to other answer models.
ArXiv · AI/CL/LG's note
score 5