RISE: Recursive Improvement via Self-Extrapolating Policy Distillation
RISE builds its teacher from the model’s own RLVR training trajectory, then refreshes that teacher as the student improves.
The paper says this turns sparse outcome-based updates into dense token-level supervision without an external teacher or privileged conditioning. It extrapolates the shift between a current checkpoint and a trailing anchor, either in parameter space or logit space. The authors frame the method as a loop: rewards steer reasoning toward correct outcomes, while the extrapolated teacher sharpens token decisions. They report gains over RLVR-only training and on-policy self-distillation across math, STEM, code generation, and multi-turn agent tasks. ArXiv · AI/CL/LG's note
The paper says this turns sparse outcome-based updates into dense token-level supervision without an external teacher or privileged conditioning. It extrapolates the shift between a current checkpoint and a trailing anchor, either in parameter space or logit space. The authors frame the method as a loop: rewards steer reasoning toward correct outcomes, while the extrapolated teacher sharpens token decisions. They report gains over RLVR-only training and on-policy self-distillation across math, STEM, code generation, and multi-turn agent tasks. ArXiv · AI/CL/LG's note
score 6