Preserving Utility in Continual LLM Unlearning via Orthogonal Projections
COP kept models useful through successive unlearning requests without weakening forgetting in the authors’ benchmark tests.
Sequential deletions can interfere with one another, undoing earlier forgetting or compounding damage to model utility. COP restricts new gradient updates using activation subspaces saved from earlier deletion requests, with a proof of first-order non-interference. It stores a low-rank basis per layer rather than past gradients or deletion datasets, and works with gradient-based unlearning methods without changing their objectives. On TOFU and RWKU, the authors report that COP prevented utility collapse and outperformed continual-unlearning baselines on forgetting.
ArXiv · AI/CL/LG's note
Sequential deletions can interfere with one another, undoing earlier forgetting or compounding damage to model utility. COP restricts new gradient updates using activation subspaces saved from earlier deletion requests, with a proof of first-order non-interference. It stores a low-rank basis per layer rather than past gradients or deletion datasets, and works with gradient-based unlearning methods without changing their objectives. On TOFU and RWKU, the authors report that COP prevented utility collapse and outperformed continual-unlearning baselines on forgetting.
ArXiv · AI/CL/LG's note
score 4