Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training
The paper claims reasoning post-training can be cut to 0.22% of full-data SFT exposure without losing performance.
The method builds Monte Carlo reasoning trees around intermediate chain-of-thought states to find where the teacher’s reasoning is uncertain. It first contracts the training corpus into a smaller teacher-informed pool, then routes optimization during SFT based on the student model’s learning state. Across reasoning benchmarks and model families, the authors report maintained or improved performance with more than 450x less training exposure. They also say the approach transfers across black-box teachers, heterogeneous students, and weak-to-strong settings. HF Daily Papers' note
The method builds Monte Carlo reasoning trees around intermediate chain-of-thought states to find where the teacher’s reasoning is uncertain. It first contracts the training corpus into a smaller teacher-informed pool, then routes optimization during SFT based on the student model’s learning state. Across reasoning benchmarks and model families, the authors report maintained or improved performance with more than 450x less training exposure. They also say the approach transfers across black-box teachers, heterogeneous students, and weak-to-strong settings. HF Daily Papers' note
score 5