DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents
On CodeGym with compressed histories, DeLoop cut Qwen3-8B’s share of tasks containing loops from 57.3% to 6.1%.
The model used about one third as many steps per task while maintaining similar task success. The researchers define loops as repeated tool calls that return exactly the same observation, adding no information. DeLoop pairs a looping action with the first later non-looping action in the same rollout, then trains with direct preference optimization without task rewards, gold answers or LLM judges. On Qwen3-32B, it reduced loops and steps while increasing task success from 37.7% to 44.8%.
HF Daily Papers' note
The model used about one third as many steps per task while maintaining similar task success. The researchers define loops as repeated tool calls that return exactly the same observation, adding no information. DeLoop pairs a looping action with the first later non-looping action in the same rollout, then trains with direct preference optimization without task rewards, gold answers or LLM judges. On Qwen3-32B, it reduced loops and steps while increasing task success from 37.7% to 44.8%.
HF Daily Papers' note
score 5