PCSD: Persistent Consistency for Self-Distillation in Agentic Reinforcement Learning
PCSD weights self-distillation by whether the teacher’s advantage persists locally, not by one-off token gaps.
The paper targets sparse-reward RL for multi-turn language-agent tasks. Its method uses adaptive windows, decayed aggregation, trend modulation, and sigmoid gating to decide where teacher guidance should count. The objective is trained with GRPO, combining dense teacher signals with sparse environment feedback. Reported gains include ALFWorld Overall improvements over GRPO of 15.6 and 13.3 points across two backbones, plus a 15.8-point gain on the unseen ALFWorld split. HF Daily Papers' note
The paper targets sparse-reward RL for multi-turn language-agent tasks. Its method uses adaptive windows, decayed aggregation, trend modulation, and sigmoid gating to decide where teacher guidance should count. The objective is trained with GRPO, combining dense teacher signals with sparse environment feedback. Reported gains include ALFWorld Overall improvements over GRPO of 15.6 and 13.3 points across two backbones, plus a 15.8-point gain on the unseen ALFWorld split. HF Daily Papers' note
score 5