Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction
The paper turns human desktop demos into reusable GUI-agent skills, then tests whether those skills improve execution.
Work2Skill records screen video, mouse and keyboard events, and accessibility-tree data through its WorkRecorder tool. The authors use it to build WorkDemo-350, a screened set of 350 successful human GUI demonstrations. Their skill-induction method extracts workflow rules, dependencies, and completion checks from that evidence. Across the same four execution models, Work2Skill reports the best scores and success rates against no-skill baselines on MyPCBench Bounded Action and OSWorld-Verified. HF Daily Papers' note
Work2Skill records screen video, mouse and keyboard events, and accessibility-tree data through its WorkRecorder tool. The authors use it to build WorkDemo-350, a screened set of 350 successful human GUI demonstrations. Their skill-induction method extracts workflow rules, dependencies, and completion checks from that evidence. Across the same four execution models, Work2Skill reports the best scores and success rates against no-skill baselines on MyPCBench Bounded Action and OSWorld-Verified. HF Daily Papers' note
score 5