Environment Evolution for Terminal Agents
The paper proposes off-policy “environment evolution” to keep terminal-agent training tasks difficult as models improve.
Instead of generating new environments from scratch or relying on on-policy rollouts, the method incrementally raises difficulty across training generations. The authors derive three difficulty directions from the multi-turn learning objective and implement them with a loop-engineered multi-agent harness. Rollout tests with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol found the evolved environments were consistently harder. In long-horizon RL training, Qwen3.6-27B and Qwen3.6-35B-A3B improved by 14.4 and 18.0 points on Terminal-Bench 2.1, respectively. HF Daily Papers' note
Instead of generating new environments from scratch or relying on on-policy rollouts, the method incrementally raises difficulty across training generations. The authors derive three difficulty directions from the multi-turn learning objective and implement them with a loop-engineered multi-agent harness. Rollout tests with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol found the evolved environments were consistently harder. In long-horizon RL training, Qwen3.6-27B and Qwen3.6-35B-A3B improved by 14.4 and 18.0 points on Terminal-Bench 2.1, respectively. HF Daily Papers' note
score 5