The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents
Skills improved agents less by adding wins than by avoiding new failures.
The paper compares skilled and unskilled agents across nearly 6,000 office-automation runs. It separates true gains from “regressions,” where a task worked before the skill was added and failed after. The authors trace those losses to skill text changing behavior even unused, procedures displacing input grounding, and procedures suppressing verification. Their conclusion is that skill evaluation should report gains and regressions separately, with more weight on grounding and checking outputs. ArXiv · AI/CL/LG's note
The paper compares skilled and unskilled agents across nearly 6,000 office-automation runs. It separates true gains from “regressions,” where a task worked before the skill was added and failed after. The authors trace those losses to skill text changing behavior even unused, procedures displacing input grounding, and procedures suppressing verification. Their conclusion is that skill evaluation should report gains and regressions separately, with more weight on grounding and checking outputs. ArXiv · AI/CL/LG's note
score 6