RISE: Recursive Improvement via Self-Extrapolating Policy Distillation
RISE turns a model’s own RL training trajectory into a refreshed synthetic teacher for token-level distillation.
The paper says the method extrapolates between a current checkpoint and a trailing anchor, either in parameter space or logit space. That converts sparse reward-driven updates into dense per-token targets without using an external teacher or privileged conditioning. The loop pairs RLVR with on-policy distillation, refreshing the teacher as the student improves. The authors report gains over RLVR-only training and on-policy self-distillation across math, STEM, code, and multi-turn agent tasks. HF Daily Papers' note
The paper says the method extrapolates between a current checkpoint and a trailing anchor, either in parameter space or logit space. That converts sparse reward-driven updates into dense per-token targets without using an external teacher or privileged conditioning. The loop pairs RLVR with on-policy distillation, refreshing the teacher as the student improves. The authors report gains over RLVR-only training and on-policy self-distillation across math, STEM, code, and multi-turn agent tasks. HF Daily Papers' note
score 5