The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation
A controlled testbed finds that long-horizon planning improves most when models learn explicit state transitions, not just isolated skills.
The paper studies planning across pre-training, post-training, and multi-teacher distillation in a unified multi-turn environment. It reports that chain-of-thought state transition modeling gives stronger long-horizon generalization, while suboptimal trajectories hurt because errors compound. For post-training, OPD is described as more reliable than GRPO in low-quality and long-horizon settings. Multi-teacher OPD can integrate capabilities when teachers share compatible planning patterns, but conflicting patterns produce severe interference. HF Daily Papers' note
The paper studies planning across pre-training, post-training, and multi-teacher distillation in a unified multi-turn environment. It reports that chain-of-thought state transition modeling gives stronger long-horizon generalization, while suboptimal trajectories hurt because errors compound. For post-training, OPD is described as more reliable than GRPO in low-quality and long-horizon settings. Multi-teacher OPD can integrate capabilities when teachers share compatible planning patterns, but conflicting patterns produce severe interference. HF Daily Papers' note
score 5