Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification
HarnessLens verifies only the task behaviors a proposed agent-harness change is likely to affect.
The paper argues that fixed-task verification wastes rollout budget and can hide regressions behind aggregate scores. HarnessLens uses execution trajectories to propose harness changes, then applies an attributable-evidence gate to test each candidate on behavior-relevant tasks. In experiments across three harnesses and four benchmarks, it reports 7.6-13.6% average held-out performance gains while using substantially less evaluation budget than baselines. ArXiv · AI/CL/LG's note
The paper argues that fixed-task verification wastes rollout budget and can hide regressions behind aggregate scores. HarnessLens uses execution trajectories to propose harness changes, then applies an attributable-evidence gate to test each candidate on behavior-relevant tasks. In experiments across three harnesses and four benchmarks, it reports 7.6-13.6% average held-out performance gains while using substantially less evaluation budget than baselines. ArXiv · AI/CL/LG's note
score 5