To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing
LLM code patches often avoid deleting the code they are supposed to remove, even when the fix passes tests.
The paper measures this “deletion avoidance” across leading SWE-bench Verified models, finding they usually reach the right file but cut the exact required line less than 52% of the time. In 29.0% of passing patches, models instead wrap targeted code in guards or fallbacks. When the authors added tests that fail if the old code remains, four frontier models dropped from 63.2% to 41.9%. A deletion-only benchmark still left the best model failing one in five tasks, though post-training on deletion reduced the behavior. HF Daily Papers' note
The paper measures this “deletion avoidance” across leading SWE-bench Verified models, finding they usually reach the right file but cut the exact required line less than 52% of the time. In 29.0% of passing patches, models instead wrap targeted code in guards or fallbacks. When the authors added tests that fail if the old code remains, four frontier models dropped from 63.2% to 41.9%. A deletion-only benchmark still left the best model failing one in five tasks, though post-training on deletion reduced the behavior. HF Daily Papers' note
score 6