JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
JarvisGUI tests GUI agents on workflows that move state across Android, Windows, and Ubuntu.
The paper frames tasks as input-output transformations under a lightweight type system, letting it compose multi-step cross-device workflows automatically. Its authors argue that single-device GUI benchmarks miss the coordination, state transfer, and long-horizon dependencies common in real use. In virtual environments across the three operating systems, current open-source GUI agents struggle with those demands. The paper is accepted to EMNLP 2026 Main Conference. ArXiv · AI/CL/LG's note
The paper frames tasks as input-output transformations under a lightweight type system, letting it compose multi-step cross-device workflows automatically. Its authors argue that single-device GUI benchmarks miss the coordination, state transfer, and long-horizon dependencies common in real use. In virtual environments across the three operating systems, current open-source GUI agents struggle with those demands. The paper is accepted to EMNLP 2026 Main Conference. ArXiv · AI/CL/LG's note
score 5