Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI
The paper says a fixed “Builder” model can learn reusable rules for constructing better agent environments without changing either model’s weights.
The authors introduce “Meta-Skill,” a bank of principles learned from execution feedback on development tasks, then frozen and reused on unseen tasks. On Harness-Bench and NewtonBench, the full meta-skill bank raises macro-average performance by 8.95 percentage points over no-skill harness construction. It also beats giving the same bank directly to the Target by 12.02 points, suggesting the benefit is in how support is built into the harness. The paper further reports gains when the same model plays both Builder and Target, pointing to environment design as a route for system-level self-improvement. ArXiv · AI/CL/LG's note
The authors introduce “Meta-Skill,” a bank of principles learned from execution feedback on development tasks, then frozen and reused on unseen tasks. On Harness-Bench and NewtonBench, the full meta-skill bank raises macro-average performance by 8.95 percentage points over no-skill harness construction. It also beats giving the same bank directly to the Target by 12.02 points, suggesting the benefit is in how support is built into the harness. The paper further reports gains when the same model plays both Builder and Target, pointing to environment design as a route for system-level self-improvement. ArXiv · AI/CL/LG's note
score 5