Scaling Automatic Research Agents via World Models
The paper says simulated execution can cut the training bottleneck for research agents.
The authors propose World Model RL, replacing real sandbox execution during training with a learned world model. They add online debiasing and inverse-variance denoising to handle biased or noisy rewards from that model. In their tests, WMRL speeds training by 3-4x and beats standard RL baselines. The post-trained 4B and 9B agents also outperform larger 48B and 120B open-weight agents on held-out benchmarks. HF Daily Papers' note
The authors propose World Model RL, replacing real sandbox execution during training with a learned world model. They add online debiasing and inverse-variance denoising to handle biased or noisy rewards from that model. In their tests, WMRL speeds training by 3-4x and beats standard RL baselines. The post-trained 4B and 9B agents also outperform larger 48B and 120B open-weight agents on held-out benchmarks. HF Daily Papers' note
score 5