Hybrid Computer Use: A Unified Programmatic, Semantic, and Visual Interface for General Agents
HCU posted the highest Partial and Binary scores among the systems compared on OSWorld 2.0.
The interface lets agents combine code and command-line tools, accessibility data, and graphical interaction. Its accessibility tools return compact observations, batch actions, and report changes to reduce round trips. HCU also guards against actions based on stale state when agents switch channels. The authors report that every evaluated model improved with HCU, though models divided work across the channels differently.
HF Daily Papers' note
The interface lets agents combine code and command-line tools, accessibility data, and graphical interaction. Its accessibility tools return compact observations, batch actions, and report changes to reduce round trips. HCU also guards against actions based on stale state when agents switch channels. The authors report that every evaluated model improved with HCU, though models divided work across the channels differently.
HF Daily Papers' note
score 5