Learning to solve hard problems in RL for LLMs by never giving up
The paper says RL gains are often concentrated on problems the model could already partly solve, while the hardest prompts barely move.
Noukhovitch calls this the “Matthew Effect” in RL for LLMs: easy tasks get easier, hard tasks often stay hard. The proposed fix, Never Give Up, samples a small group first, quickly filters solved prompts, and retries unsolved ones so compute shifts toward harder examples. In the reported math and code experiments, NGU improves the hardest subsets without meaningfully hurting easier ones, though the author says it may help less when nearly everything is already very difficult. HN · ArXiv's note
Noukhovitch calls this the “Matthew Effect” in RL for LLMs: easy tasks get easier, hard tasks often stay hard. The proposed fix, Never Give Up, samples a small group first, quickly filters solved prompts, and retries unsolved ones so compute shifts toward harder examples. In the reported math and code experiments, NGU improves the hardest subsets without meaningfully hurting easier ones, though the author says it may help less when nearly everything is already very difficult. HN · ArXiv's note
score 5
Discussions
- hn · 75 points
- hn · 79 points
- hn · 83 points · 4 comments
- hn · 86 points · 4 comments
- hn · 87 points · 4 comments
- hn · 90 points · 4 comments
- hn · 92 points · 4 comments
- hn · 95 points · 4 comments
- hn · 98 points · 6 comments
- hn · 101 points · 6 comments
- hn · 101 points · 6 comments
- hn · 103 points · 6 comments
- hn · 103 points · 6 comments
- hn · 103 points · 6 comments
- hn · 103 points · 6 comments
- hn · 103 points · 6 comments
- hn · 103 points · 6 comments
- hn · 103 points · 6 comments
- hn · 103 points · 6 comments
- hn · 106 points · 6 comments
- hn · 106 points · 6 comments
- hn · 106 points · 6 comments
- hn · 106 points · 6 comments
- hn · 109 points · 6 comments
- hn · 109 points · 6 comments
- hn · 111 points · 6 comments
- hn · 111 points · 6 comments
- hn · 111 points · 6 comments
- hn · 111 points · 6 comments
- hn · 111 points · 6 comments
- hn · 111 points · 6 comments
- hn · 111 points · 6 comments
- hn · 111 points · 6 comments
- hn · 111 points · 8 comments
- hn · 111 points · 8 comments
- hn · 111 points · 8 comments
- hn · 111 points · 8 comments
- hn · 111 points · 8 comments
- hn · 111 points · 8 comments
- hn · 111 points · 8 comments
- hn · 111 points · 8 comments
- hn · 111 points · 9 comments
- hn · 111 points · 9 comments
- hn · 111 points · 9 comments
- hn · 111 points · 9 comments
- hn · 111 points · 9 comments
- hn · 111 points · 9 comments
- hn · 111 points · 9 comments
- hn · 111 points · 9 comments
- hn · 112 points · 9 comments
- hn · 112 points · 9 comments
- hn · 112 points · 9 comments
- hn · 112 points · 9 comments
- hn · 112 points · 9 comments
- hn · 112 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 113 points · 9 comments
- hn · 114 points · 9 comments
- hn · 114 points · 9 comments
- hn · 114 points · 9 comments
- hn · 114 points · 9 comments
- hn · 114 points · 9 comments
- hn · 114 points · 9 comments
- hn · 115 points · 9 comments
- hn · 115 points · 9 comments
- hn · 115 points · 9 comments
- hn · 115 points · 9 comments