From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution
Influence-selected samples mattered far more when rewritten than when merely reweighted.
The paper tests training data attribution by changing the responses attached to influential examples while leaving their instructions fixed. Across four open-weight LLMs, response rewriting produced stronger and more durable shifts in epistemic abstention than reweighting the same examples. The shifts worked in both directions and stayed concentrated on target-relevant behavior. The authors report the same qualitative contrast for safety refusal. HF Daily Papers' note
The paper tests training data attribution by changing the responses attached to influential examples while leaving their instructions fixed. Across four open-weight LLMs, response rewriting produced stronger and more durable shifts in epistemic abstention than reweighting the same examples. The shifts worked in both directions and stayed concentrated on target-relevant behavior. The authors report the same qualitative contrast for safety refusal. HF Daily Papers' note
score 5