An Empirical Study of Harness Design for Coding Agents
The paper isolates which coding-agent harness parts actually change performance: context handling, planning, and action space.
Across four models on SWE-Bench Verified and Terminal-Bench 2.1, the authors test 176 matched harness settings. Context management helps most when the context window is tight, mainly by avoiding overflow failures. Planning helps weaker models with accuracy, but for stronger models it mostly cuts cost. Tool design depends on the model: predefined tools help agents weak at bash, while bash-capable models can do well with a cheaper bash-only setup. ArXiv · AI/CL/LG's note
Across four models on SWE-Bench Verified and Terminal-Bench 2.1, the authors test 176 matched harness settings. Context management helps most when the context window is tight, mainly by avoiding overflow failures. Planning helps weaker models with accuracy, but for stronger models it mostly cuts cost. Tool design depends on the model: predefined tools help agents weak at bash, while bash-capable models can do well with a cheaper bash-only setup. ArXiv · AI/CL/LG's note
score 6