When Models Edit Too Much: On the Fidelity of Minimal Code Edits
The paper argues that code-repair models need to be judged on edit fidelity, not just whether the patched code passes.
The authors build 400 repair tasks from BigCodeBench by injecting controlled AST-level bugs, giving each task a known minimal fix. They find frontier models often over-edit, including cases where high Pass@1 comes with larger-than-needed changes and added cognitive complexity. A preservation instruction cuts excess Levenshtein distance from 0.195 to 0.131, reduces added cognitive complexity by 26.6%, and raises Pass@1 by 2.3 points. In post-training tests, supervised fine-tuning overfits seen corruption patterns, while reinforcement learning gives the best out-of-domain balance. ArXiv · AI/CL/LG's note
The authors build 400 repair tasks from BigCodeBench by injecting controlled AST-level bugs, giving each task a known minimal fix. They find frontier models often over-edit, including cases where high Pass@1 comes with larger-than-needed changes and added cognitive complexity. A preservation instruction cuts excess Levenshtein distance from 0.195 to 0.131, reduces added cognitive complexity by 26.6%, and raises Pass@1 by 2.3 points. In post-training tests, supervised fine-tuning overfits seen corruption patterns, while reinforcement learning gives the best out-of-domain balance. ArXiv · AI/CL/LG's note
score 5