DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents
DeskForge generates dense desktop annotations at scale, then uses them to improve GUI grounding and task completion.
The paper introduces a controllable desktop environment that runs real applications while varying window layout, content, appearance, resolution, and app state. Its DeskForge-1M dataset contains 1.2 million annotated observations and 159.7 million element instances. Fine-tuning four vision-language models on 200,000 examples improved results on held-out desktop settings and five external GUI grounding benchmarks. Under a fixed planner, the fine-tuned Qwen3.5-4B model also completed more long-horizon tasks on WebArena-Infinity and OpenApps. HF Daily Papers' note
The paper introduces a controllable desktop environment that runs real applications while varying window layout, content, appearance, resolution, and app state. Its DeskForge-1M dataset contains 1.2 million annotated observations and 159.7 million element instances. Fine-tuning four vision-language models on 200,000 examples improved results on held-out desktop settings and five external GUI grounding benchmarks. Under a fixed planner, the fine-tuned Qwen3.5-4B model also completed more long-horizon tasks on WebArena-Infinity and OpenApps. HF Daily Papers' note
score 5