ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds
The paper tests whether AI systems can learn unfamiliar rules by exploring executable “alien” environments, rather than leaning on memorized knowledge.
ExplorationBench uses two sandboxes, AlienCode and AlienLogic, with flawed manuals, feedback, and tool-call interfaces.
The tasks are built so answers can be checked exactly, while the rules conflict with familiar knowledge.
Across 10 AI systems, the authors report that top systems can acquire and use new rules, but results vary sharply by exploration path.
They also find that more exploration can stall or undo earlier gains.
ArXiv · AI/CL/LG's note
ExplorationBench uses two sandboxes, AlienCode and AlienLogic, with flawed manuals, feedback, and tool-call interfaces.
The tasks are built so answers can be checked exactly, while the rules conflict with familiar knowledge.
Across 10 AI systems, the authors report that top systems can acquire and use new rules, but results vary sharply by exploration path.
They also find that more exploration can stall or undo earlier gains.
ArXiv · AI/CL/LG's note
score 6