Language Models Learn to Improve Language Models
The authors report that LM2LM beat GPT-6-sol on held-out model-development tasks with 33% lower data-generation cost.
LM2LM uses reinforcement learning to train a language model to propose improvements to other language models. A frozen model predicts each proposal’s gains from experimental records, reducing the need for costly training runs. The researchers periodically execute highly ranked proposals and feed measured outcomes back into that model’s context to keep its predictions reliable. Across eight autoregressive and diffusion model-development environments, the trained Qwen3.8-27B outperformed GPT-6-sol even when the latter received 300 experimental records; it also generalized to a model family absent from training.
HF Daily Papers' note
LM2LM uses reinforcement learning to train a language model to propose improvements to other language models. A frozen model predicts each proposal’s gains from experimental records, reducing the need for costly training runs. The researchers periodically execute highly ranked proposals and feed measured outcomes back into that model’s context to keep its predictions reliable. Across eight autoregressive and diffusion model-development environments, the trained Qwen3.8-27B outperformed GPT-6-sol even when the latter received 300 experimental records; it also generalized to a model family absent from training.
HF Daily Papers' note
score 5