When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation
A mismatch over which EOS token means “stop” can make distilled models run long.
The paper traces length inflation in on-policy distillation to cases where a base student and post-trained teacher put stopping probability on different EOS tokens. The authors find this across Qwen3, Llama, and Gemma, even when the models list the same stopping set. Simply aligning decoding stops does not fix it; treating equivalent EOS tokens as one semantic stop action reduces the inflation. Later OPD stages still show some length growth, so the mismatch is important but not the whole story. HF Daily Papers' note
The paper traces length inflation in on-policy distillation to cases where a base student and post-trained teacher put stopping probability on different EOS tokens. The authors find this across Qwen3, Llama, and Gemma, even when the models list the same stopping set. Simply aligning decoding stops does not fix it; treating equivalent EOS tokens as one semantic stop action reduces the inflation. Later OPD stages still show some length growth, so the mismatch is important but not the whole story. HF Daily Papers' note
score 5