Mitigating the Length-Scaling Tax with Online Distillation
The paper targets wasted verbosity from RL post-training on problems the model could already solve.
The authors define “length-scaling tax” as extra response length on already-solved queries without matching accuracy gains. Their proposed Length Self-Distillation routes solved prompts into on-policy distillation while keeping the RL objective for unsolved prompts. It uses an exponential moving average of the online policy as the teacher, avoiding an external model. Reported LST drops from 19.0% to -3.7% on single-turn reasoning and from 31.4% to 13.7% on multi-turn agentic tasks. HF Daily Papers' note
The authors define “length-scaling tax” as extra response length on already-solved queries without matching accuracy gains. Their proposed Length Self-Distillation routes solved prompts into on-policy distillation while keeping the RL objective for unsolved prompts. It uses an exponential moving average of the online policy as the teacher, avoiding an external model. Reported LST drops from 19.0% to -3.7% on single-turn reasoning and from 31.4% to 13.7% on multi-turn agentic tasks. HF Daily Papers' note
score 5