One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation
The review says OPSD’s central risk is collapse: models lose diversity in their reasoning paths.
Robert and Qader frame on-policy self-distillation as a cheaper alternative to teacher-based distillation, using the same model with privileged information as its own teacher. The paper argues that this setup can bias training even when it saves generated tokens. It organizes the collapse problem around three controls: token weighting, what privileged information the teacher sees, and how guidance changes over time. The authors limit the review to mathematical reasoning and report no new experiments. ArXiv · AI/CL/LG's note
Robert and Qader frame on-policy self-distillation as a cheaper alternative to teacher-based distillation, using the same model with privileged information as its own teacher. The paper argues that this setup can bias training even when it saves generated tokens. It organizes the collapse problem around three controls: token weighting, what privileged information the teacher sees, and how guidance changes over time. The authors limit the review to mathematical reasoning and report no new experiments. ArXiv · AI/CL/LG's note
score 5