AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments
GPT-6 Sol reached just 61.9% calibrated accuracy when agents had to discover hidden rules and adapt to unannounced changes.
AdaExploreBench tests whether agents can gather evidence, infer an environment’s rules and notice when those rules change. Across eight leading LLMs, supplying sufficient evidence or removing rule drift improved average performance by close to 10%. The authors identify a bottleneck: agents often reason effectively from useful evidence but struggle to find it and recognize when it has become stale.
HF Daily Papers' note
AdaExploreBench tests whether agents can gather evidence, infer an environment’s rules and notice when those rules change. Across eight leading LLMs, supplying sufficient evidence or removing rule drift improved average performance by close to 10%. The authors identify a bottleneck: agents often reason effectively from useful evidence but struggle to find it and recognize when it has become stale.
HF Daily Papers' note
score 4