Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents
EvoSkill-GUI updates an agent’s GUI skills during deployment, using failures as edit instructions without retraining the model.
The paper frames GUI tasks as brittle because pop-ups, loading delays, and moved widgets can break fixed plans mid-run. Its proposed skill package includes metadata, executable plans, backup localization, recovery rules, accessibility utilities, and recorded failure cases. A reflect-revise-reuse loop lets the executor patch skills during rollouts, while a separated critic diagnoses failed trajectories. The authors report gains across MobileWorld, AndroidWorld, and OSWorld, with maximum improvements of 16.2%, 6.0%, and 10.5% respectively. HF Daily Papers' note
The paper frames GUI tasks as brittle because pop-ups, loading delays, and moved widgets can break fixed plans mid-run. Its proposed skill package includes metadata, executable plans, backup localization, recovery rules, accessibility utilities, and recorded failure cases. A reflect-revise-reuse loop lets the executor patch skills during rollouts, while a separated critic diagnoses failed trajectories. The authors report gains across MobileWorld, AndroidWorld, and OSWorld, with maximum improvements of 16.2%, 6.0%, and 10.5% respectively. HF Daily Papers' note
score 5