BTS-AgentBench: A Deterministic, Replayable Pipeline from Read-Only Telemetry Logs to Agent Benchmarks
The paper turns industrial telemetry logs into replayable multi-turn agent tasks, with a 532-row benchmark release.
BTS-AgentBench normalizes metadata and raw histories into a read-only tool store, then builds static tasks with gold answers, evidence, and bounded operator-facing episodes. The release reports exact reproduction across two independent raw-to-episode builds, including the 356/87/89 train/dev/test split. A related XAI4HEAT run yields 204 episodes, with GPT-5.5 completing all 41 held-out retained executions. ArXiv · AI/CL/LG's note
BTS-AgentBench normalizes metadata and raw histories into a read-only tool store, then builds static tasks with gold answers, evidence, and bounded operator-facing episodes. The release reports exact reproduction across two independent raw-to-episode builds, including the 356/87/89 train/dev/test split. A related XAI4HEAT run yields 204 episodes, with GPT-5.5 completing all 41 held-out retained executions. ArXiv · AI/CL/LG's note
score 4