SO-Arena: Benchmarking the Capability Spectrum and Reliability of System One Models
The highest-performing System One model ranked only seventh in decision reliability.
SO-Arena evaluated 21 models across 11 domains and five capability families. Beyond accuracy, it tested whether decisions stayed stable when answer options changed and whether models recognized when no valid answer was available. The benchmark also includes interactive tasks in which models execute actions in sandboxes. Gains from model scaling and additional reasoning varied substantially across domains.
HF Daily Papers' note
SO-Arena evaluated 21 models across 11 domains and five capability families. Beyond accuracy, it tested whether decisions stayed stable when answer options changed and whether models recognized when no valid answer was available. The benchmark also includes interactive tasks in which models execute actions in sandboxes. Gains from model scaling and additional reasoning varied substantially across domains.
HF Daily Papers' note
score 4