LiveGeo: Benchmarking Geographic Agents in Continuously Evolving Worlds
Strong performance on static geographic tasks did not guarantee reliable decisions when conditions changed.
The researchers introduce GeoWorld, a controllable geographic environment with evolving states and verifiable histories, and LiveGeo, a benchmark built on it. Agents must gather relevant information, check whether earlier evidence still holds, and revise plans when needed. Even frontier models struggled with static tasks, with no model consistently leading across tasks. The analysis identifies difficulty turning relevant evidence into valid decisions and keeping those decisions correct as the environment evolves.
ArXiv · AI/CL/LG's note
The researchers introduce GeoWorld, a controllable geographic environment with evolving states and verifiable histories, and LiveGeo, a benchmark built on it. Agents must gather relevant information, check whether earlier evidence still holds, and revise plans when needed. Even frontier models struggled with static tasks, with no model consistently leading across tasks. The analysis identifies difficulty turning relevant evidence into valid decisions and keeping those decisions correct as the environment evolves.
ArXiv · AI/CL/LG's note
score 5