Distilled Reinforcement Learning for LLM Post-training
The paper proposes adding selective teacher guidance directly into RL post-training, instead of relying on outcome rewards or full logit imitation.
Distilled RL uses reverse importance sampling with clipping, negative sample reset, and sequence-level geometric normalization. The authors say this lets a student model absorb knowledge from a teacher without blindly matching it. In their experiments, the method beats standard RL and on-policy distillation on pass@1 and pass@k, including cross-family distillation settings. HF Daily Papers' note
Distilled RL uses reverse importance sampling with clipping, negative sample reset, and sequence-level geometric normalization. The authors say this lets a student model absorb knowledge from a teacher without blindly matching it. In their experiments, the method beats standard RL and on-policy distillation on pass@1 and pass@k, including cross-family distillation settings. HF Daily Papers' note
score 5