From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale
ARCTIC tries to move AI-diff review toward intent, mismatch, and where humans should look first.
The paper says AI-generated code is arriving faster than traditional peer review can handle. Its system predicts the purpose of a change, checks whether the agent’s output drifted from that intent, and ranks the diff regions most worth human scrutiny. In offline tests, the authors report 0.86 F1 for intent prediction, QWK 0.907 for drift detection, and better spotlighting than a baseline AI reviewer using fewer tokens. In rollout, drift scores reduced misalignment by 5.76 points, intent prediction drew 90.2% approval, and the authors say no defects have been attributed to self-reviewed diffs since launch. ArXiv · AI/CL/LG's note
The paper says AI-generated code is arriving faster than traditional peer review can handle. Its system predicts the purpose of a change, checks whether the agent’s output drifted from that intent, and ranks the diff regions most worth human scrutiny. In offline tests, the authors report 0.86 F1 for intent prediction, QWK 0.907 for drift detection, and better spotlighting than a baseline AI reviewer using fewer tokens. In rollout, drift scores reduced misalignment by 5.76 points, intent prediction drew 90.2% approval, and the authors say no defects have been attributed to self-reviewed diffs since launch. ArXiv · AI/CL/LG's note
score 5