Disentangling Forgetting from Utility: Lightweight LLM Unlearning via Orthogonal Null-Space Projection
UUE uses pluggable adapters to disrupt unwanted knowledge while keeping the underlying model frozen.
The method constrains parameter updates to a mathematical space designed to minimize interference with knowledge the model should retain. A closed-form solution pushes representations of unwanted knowledge toward their opposite points. The authors report that UUE and its low-rank variant, UUE-L, outperform existing unlearning methods on the TOFU and WMDP benchmarks. The supplied abstract gives no numerical results.
ArXiv · AI/CL/LG's note
The method constrains parameter updates to a mathematical space designed to minimize interference with knowledge the model should retain. A closed-form solution pushes representations of unwanted knowledge toward their opposite points. The authors report that UUE and its low-rank variant, UUE-L, outperform existing unlearning methods on the TOFU and WMDP benchmarks. The supplied abstract gives no numerical results.
ArXiv · AI/CL/LG's note
score 4