Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
Training a model to study textbooks raised its proof accuracy from 51.5% to 79.5%.
The researchers paired a BookNavigator interface with reinforcement learning rewards, requiring Intern-S2-Preview-35B to find relevant textbook material and construct proofs without access to reference solutions. The method improved results across both in-domain and out-of-domain textbooks and subjects, outperforming RL-trained baselines using no textbook or fixed retrieval. Gains also carried over to BrowseComp-Plus and CL-bench, with improvements over the base model of 1.7 and 2.3 percentage points, respectively.
HF Daily Papers' note
The researchers paired a BookNavigator interface with reinforcement learning rewards, requiring Intern-S2-Preview-35B to find relevant textbook material and construct proofs without access to reference solutions. The method improved results across both in-domain and out-of-domain textbooks and subjects, outperforming RL-trained baselines using no textbook or fixed retrieval. Gains also carried over to BrowseComp-Plus and CL-bench, with improvements over the base model of 1.7 and 2.3 percentage points, respectively.
HF Daily Papers' note
score 5