Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
The paper argues that changing the environment’s feedback can make long-horizon RL training less sparse and more stable.
The authors propose Feedback-Enriched Environments, shifting support from direct action guidance toward richer observations as agents explore and improve. They report gains on SciWorld and BFCL across Qwen3 model sizes and RL methods including GRPO, GSPO, and DAPO. Their analysis says the setup reduces entropy volatility, encourages harder-task exploration, and gets the guidance absorbed into policy weights. HF Daily Papers' note
The authors propose Feedback-Enriched Environments, shifting support from direct action guidance toward richer observations as agents explore and improve. They report gains on SciWorld and BFCL across Qwen3 model sizes and RL methods including GRPO, GSPO, and DAPO. Their analysis says the setup reduces entropy volatility, encourages harder-task exploration, and gets the guidance absorbed into policy weights. HF Daily Papers' note
score 4