Self-Preservation in Language Models
The paper finds that replay chosen and generated by the model can reduce forgetting beyond the tasks explicitly protected.
The authors test continual instruction tuning on Qwen models using “Task Banks” as preservation targets. Generated replay helps retain capabilities outside the bank, and combining partial banks improves retention over either bank alone. At 4B scale, new-task-informed target selection beats random selection on held-out retention, while smaller models do not show the same advantage. Under a fixed example budget, more replay preserves old behavior but weakens gains on the new task. HF Daily Papers' note
The authors test continual instruction tuning on Qwen models using “Task Banks” as preservation targets. Generated replay helps retain capabilities outside the bank, and combining partial banks improves retention over either bank alone. At 4B scale, new-task-informed target selection beats random selection on held-out retention, while smaller models do not show the same advantage. Under a fixed example budget, more replay preserves old behavior but weakens gains on the new task. HF Daily Papers' note
score 4