Specification-first convergence with an AI coding agent: a case study of dismantling a core architectural invariant across 189 files in a 717k-line codebase with no test oracle and no human code review
An AI agent reportedly completed a three-day refactor that the author judged rewrite-level in difficulty.
The case study describes a 717,725-line TypeScript app where a core UI invariant was removed: generation streams no longer had to die when their panel closed. The agent worked from a frozen specification, ran 14 refinement audits before implementation, then 17 verification audits after it. The paper says 201 defects were corrected before any human ran the program, with success defined as two clean verification passes. The change touched 189 files, and the author reports no observed bug across the first and roughly 30 later sessions. HF Daily Papers' note
The case study describes a 717,725-line TypeScript app where a core UI invariant was removed: generation streams no longer had to die when their panel closed. The agent worked from a frozen specification, ran 14 refinement audits before implementation, then 17 verification audits after it. The paper says 201 defects were corrected before any human ran the program, with success defined as two clean verification passes. The change touched 189 files, and the author reports no observed bug across the first and roughly 30 later sessions. HF Daily Papers' note
score 4