On-Policy Self-Distillation without Any Supervision
The paper claims a model can improve its reasoning by distilling from its own consistent answers, without ground truth or a larger teacher.
U-OPSD samples multiple rollouts, uses majority agreement above a self-consistency threshold as a pseudo-solution, then trains on completions that disagreed. The authors say this lets the model correct places where it is confidently wrong using only its own generations. Across math benchmarks including AIME24, AIME25, HMMT25, MATH500, and AMC23, they report gains over Qwen3 base models and results matching or beating supervised OPSD and GRPO in the cited settings. HF Daily Papers' note
U-OPSD samples multiple rollouts, uses majority agreement above a self-consistency threshold as a pseudo-solution, then trains on completions that disagreed. The authors say this lets the model correct places where it is confidently wrong using only its own generations. Across math benchmarks including AIME24, AIME25, HMMT25, MATH500, and AMC23, they report gains over Qwen3 base models and results matching or beating supervised OPSD and GRPO in the cited settings. HF Daily Papers' note
score 6