FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents
FM-Bench tests whether agents can manage consequences over 20 simulated years, not just finish short tasks.
The benchmark puts an LLM in charge of a football club through 26 tools and about 340 to 400 decision points. It compares 15 frontier models in solo runs and a shared Arena, with scoring handled by a deterministic engine rather than judges. Claude-fable-5 leads the solo results and the Arena, but the title rotates among ten models. The paper says performance comes down to management behavior: investing cash, timing contract renewals, and pulling back on slow-payoff spending late in the run. HF Daily Papers' note
The benchmark puts an LLM in charge of a football club through 26 tools and about 340 to 400 decision points. It compares 15 frontier models in solo runs and a shared Arena, with scoring handled by a deterministic engine rather than judges. Claude-fable-5 leads the solo results and the Arena, but the title rotates among ten models. The paper says performance comes down to management behavior: investing cash, timing contract renewals, and pulling back on slow-payoff spending late in the run. HF Daily Papers' note
score 5