The Surprising Effectiveness of Deleting Weights in LLM Post-Training
Zeroing fewer than 0.05% of a model’s weights matched full fine-tuning across three post-training regimes, the authors report.
Deletion Tuning learns a binary mask over transformer blocks’ gate projections, leaving all other weights unchanged. The reported results cover reinforcement learning with verifiable rewards, supervised fine-tuning and on-policy distillation. Its trained masks are hundreds to thousands of times smaller than full fine-tuning checkpoints. In reinforcement learning experiments, prior-task accuracy stayed near the base model’s level, while full fine-tuning, LoRA and random sparse fine-tuning lost up to 5.4 points.
HF Daily Papers' note
Deletion Tuning learns a binary mask over transformer blocks’ gate projections, leaving all other weights unchanged. The reported results cover reinforcement learning with verifiable rewards, supervised fine-tuning and on-policy distillation. Its trained masks are hundreds to thousands of times smaller than full fine-tuning checkpoints. In reinforcement learning experiments, prior-task accuracy stayed near the base model’s level, while full fine-tuning, LoRA and random sparse fine-tuning lost up to 5.4 points.
HF Daily Papers' note
score 5