SkillSpec: Intent-Masked Specification Reasoning for Agent Skill Correctness
The paper reports 763 confirmed defects in reusable agent skills, with many failures sitting between stated intent and implementation.
SkillSpec treats skill checking as a specification problem, converting mixed instructions, descriptions, and code into a graph. It compares expected behavior against inferred behavior while masking levels of intent context to reduce both bias and guesswork. Tested on 515 real-world skills, it found defects in 239 of them at 61.2% precision. The authors say code-node reasoning held up across model families, while plain-text nodes remained harder to verify. HF Daily Papers' note
SkillSpec treats skill checking as a specification problem, converting mixed instructions, descriptions, and code into a graph. It compares expected behavior against inferred behavior while masking levels of intent context to reduce both bias and guesswork. Tested on 515 real-world skills, it found defects in 239 of them at 61.2% precision. The authors say code-node reasoning held up across model families, while plain-text nodes remained harder to verify. HF Daily Papers' note
score 4