Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition
Lightning Weave trains one reasoning model to inherit both accuracy and token-efficiency gains from separate specialist models.
The paper frames each specialist’s advantage as a policy shift from a pre-training baseline, then composes those shifts during on-policy distillation. It caches anchor-model scoring once, so the student can train without running multiple live anchor models at the same time. On Qwen3.5-4B, the authors report higher accuracy with fewer response tokens on HMMT 2025 and LiveCodeBench v5. Code is not yet available; the paper says it will be released soon. HF Daily Papers' note
The paper frames each specialist’s advantage as a policy shift from a pre-training baseline, then composes those shifts during on-policy distillation. It caches anchor-model scoring once, so the student can train without running multiple live anchor models at the same time. On Qwen3.5-4B, the authors report higher accuracy with fewer response tokens on HMMT 2025 and LiveCodeBench v5. Code is not yet available; the paper says it will be released soon. HF Daily Papers' note
score 5