Negative Self-Distillation: Learning to Reason by Avoiding Flaws
The paper argues that self-improvement can work better by training a model away from its own flawed reasoning, not toward privileged answer traces.
The authors say on-policy self-distillation can hurt complex reasoning by making models imitate overconfident traces and suppress uncertainty. Their proposed Negative Self-Distillation has the model generate a question-specific “negative” reasoning mode, then pushes the student away from it. A dynamic gating mechanism is meant to target reasoning-critical tokens without damaging basic language ability. In their experiments, NSD outperforms OPSD and other label-free self-bootstrapping RL baselines. HF Daily Papers' note
The authors say on-policy self-distillation can hurt complex reasoning by making models imitate overconfident traces and suppress uncertainty. Their proposed Negative Self-Distillation has the model generate a question-specific “negative” reasoning mode, then pushes the student away from it. A dynamic gating mechanism is meant to target reasoning-critical tokens without damaging basic language ability. In their experiments, NSD outperforms OPSD and other label-free self-bootstrapping RL baselines. HF Daily Papers' note
score 5