EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?
The benchmark tests whether agents lose old abilities or adapt unevenly when their tool, skill, or agent harness changes.
EVOHARNESSBENCH shifts the moving target from the task stream to the harness itself, using 17 staged streams built from verifier-based benchmarks. The paper reports 802 tasks spanning 520 tools, 42 skills, and 62 agents. Its evaluations separate retention under harness expansion from adaptation as new capabilities arrive. The authors find harness-induced forgetting, inconsistent adaptation gains, and tension between preserving prior competence and learning to use new additions. HF Daily Papers' note
EVOHARNESSBENCH shifts the moving target from the task stream to the harness itself, using 17 staged streams built from verifier-based benchmarks. The paper reports 802 tasks spanning 520 tools, 42 skills, and 62 agents. Its evaluations separate retention under harness expansion from adaptation as new capabilities arrive. The authors find harness-induced forgetting, inconsistent adaptation gains, and tension between preserving prior competence and learning to use new additions. HF Daily Papers' note
score 5