Long-Horizon RL Needs Muon
The full training recipe achieved higher training success than the baselines on repository-level issue resolution across two model families.
The recipe combines Muon with token-averaged loss, truncated importance sampling, and penalties for length and tool errors. It uses a single rollout per issue, allowing asynchronous policy updates without waiting for multiple trajectories for that issue. The authors examine optimizer choices and recipe components for agents learning from sparse outcome rewards. The paper is marked as work in progress; its abstract reports no numerical gains.
HF Daily Papers' note
The recipe combines Muon with token-averaged loss, truncated importance sampling, and penalties for length and tool errors. It uses a single rollout per issue, allowing asynchronous policy updates without waiting for multiple trajectories for that issue. The authors examine optimizer choices and recipe components for agents learning from sparse outcome rewards. The paper is marked as work in progress; its abstract reports no numerical gains.
HF Daily Papers' note
score 5