Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation
A pool of nearby frozen experts gave math-reasoning students better supervision than a single OPSD teacher.
The paper introduces Neighborhood OPSD, which selects local parameter perturbations that add reference-aligned corrections at positions the base privileged teacher misses. At training time, routing picks an anchor token with MaxPeak, then chooses among experts that support it before distilling the expert’s full next-token distribution. On AIME 2024, AIME 2025, and HMMT February 2025, it improves Average@12 over OPSD by 2.75, 1.67, and 1.94 points on Qwen3-1.7B, 4B, and 8B. Inference uses only the distilled student. HF Daily Papers' note
The paper introduces Neighborhood OPSD, which selects local parameter perturbations that add reference-aligned corrections at positions the base privileged teacher misses. At training time, routing picks an anchor token with MaxPeak, then chooses among experts that support it before distilling the expert’s full next-token distribution. On AIME 2024, AIME 2025, and HMMT February 2025, it improves Average@12 over OPSD by 2.75, 1.67, and 1.94 points on Qwen3-1.7B, 4B, and 8B. Inference uses only the distilled student. HF Daily Papers' note
score 4