GRACE:Gradient-guided Coreset Selection for LLM Unlearning
GRACE builds the forget and retain sets itself from a small number of examples, instead of assuming those sets are already labeled.
The method uses gradients from seed examples to define the behavior to remove, then selects a compact forget coreset that approximates that direction. It separately chooses retain examples after removing the forget direction from the gradient space, aiming to preserve general model utility. In tests across two domains, two model families, and four unlearning algorithms, the authors report comparable forgetting with better retained utility, especially against earlier gradient-based selection methods. ArXiv · AI/CL/LG's note
The method uses gradients from seed examples to define the behavior to remove, then selects a compact forget coreset that approximates that direction. It separately chooses retain examples after removing the forget direction from the gradient space, aiming to preserve general model utility. In tests across two domains, two model families, and four unlearning algorithms, the authors report comparable forgetting with better retained utility, especially against earlier gradient-based selection methods. ArXiv · AI/CL/LG's note
score 5