RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?
RoboSPA tests VLA robot models against harder spatial reasoning and longer procedural planning than standard benchmarks.
The benchmark covers 10 task categories, 56 base tasks, and 280 difficulty-scaled variants. Its dataset includes 527K trajectories across multiple robot embodiments and scenes. The authors say representative VLA models still fail on complex spatial relations, precise low-level execution, and memory-heavy planning. Data and code are available, and the paper is accepted at EMNLP 2026. ArXiv · AI/CL/LG's note
The benchmark covers 10 task categories, 56 base tasks, and 280 difficulty-scaled variants. Its dataset includes 527K trajectories across multiple robot embodiments and scenes. The authors say representative VLA models still fail on complex spatial relations, precise low-level execution, and memory-heavy planning. Data and code are available, and the paper is accepted at EMNLP 2026. ArXiv · AI/CL/LG's note
score 5