Bridged Self-Distillation for On-Policy Learning
A copy of the student turns a stronger model’s reasoning traces into training guidance in the student’s own token space.
Bridged Self-Distillation gives that copy the teacher’s reasoning as context, while the student generates attempts without it. The copy scores those attempts, and a verifier checks outcomes; the scores guide updates on unsuccessful attempts. Because the external teacher supplies only text, the method supports shared tokenizers, different tokenizers and closed APIs. The authors report improvements over the base model and scenario-specific baselines across all three settings.
HF Daily Papers' note
Bridged Self-Distillation gives that copy the teacher’s reasoning as context, while the student generates attempts without it. The copy scores those attempts, and a verifier checks outcomes; the scores guide updates on unsuccessful attempts. Because the external teacher supplies only text, the method supports shared tokenizers, different tokenizers and closed APIs. The authors report improvements over the base model and scenario-specific baselines across all three settings.
HF Daily Papers' note
score 5