Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
FOM-UL tries to make LLM unlearning survive quantization by updating only the transformer layers most tied to the material being forgotten.
The paper defines a forget-to-retain significance score to choose layers with high influence on the forget set and low sensitivity to retained knowledge. Its evaluations report less residual memorization than several baseline unlearning methods while keeping retain-set utility close to the original model. The authors also say the method holds up better after 8-bit and 4-bit post-training quantization, with lower recovery under adversarial prompts. They do not claim a formal guarantee that the unwanted content is erased.
ArXiv · AI/CL/LG's note
The paper defines a forget-to-retain significance score to choose layers with high influence on the forget set and low sensitivity to retained knowledge. Its evaluations report less residual memorization than several baseline unlearning methods while keeping retain-set utility close to the original model. The authors also say the method holds up better after 8-bit and 4-bit post-training quantization, with lower recovery under adversarial prompts. They do not claim a formal guarantee that the unwanted content is erased.
ArXiv · AI/CL/LG's note
score 4