On-Policy Self-Distillation without Any Supervision
The paper claims a model can improve by distilling from its own internally consistent answers, without ground truth or a stronger teacher.
U-OPSD samples multiple rollouts, uses majority vote above a self-consistency threshold to form a pseudo-solution, then trains on where the model’s longest wrong answer diverges. The authors report gains across math benchmarks and say the method matches or beats supervised OPSD and GRPO in several settings. On Qwen3 non-thinking mode, they report average improvements over base models of 8.5% at 4B and 10.7% at 8B. Source: ArXiv · AI/CL/LG's note.
U-OPSD samples multiple rollouts, uses majority vote above a self-consistency threshold to form a pseudo-solution, then trains on where the model’s longest wrong answer diverges. The authors report gains across math benchmarks and say the method matches or beats supervised OPSD and GRPO in several settings. On Qwen3 non-thinking mode, they report average improvements over base models of 8.5% at 4B and 10.7% at 8B. Source: ArXiv · AI/CL/LG's note.
score 5