Continually Post-training Small Models via Synthetic Self-distillation
The paper proposes SynthSelf to make continual post-training work on smaller models without losing prior capabilities.
The authors say standard Self-Distillation Fine-Tuning breaks down on small models because it depends on strong in-context learning and instruction following. They link both poor acquisition and forgetting to distribution shift during post-training. SynthSelf uses a minimally larger same-family aligner to generate in-distribution rewrites, then adds off-policy supervision and rejection sampling. Across 4 acquisition datasets, 14 retention benchmarks, and 3 small-model scales, it outperformed 6 comparison methods.
HF Daily Papers' note
The authors say standard Self-Distillation Fine-Tuning breaks down on small models because it depends on strong in-context learning and instruction following. They link both poor acquisition and forgetting to distribution shift during post-training. SynthSelf uses a minimally larger same-family aligner to generate in-distribution rewrites, then adds off-policy supervision and rejection sampling. Across 4 acquisition datasets, 14 retention benchmarks, and 3 small-model scales, it outperformed 6 comparison methods.
HF Daily Papers' note
score 4