WorldCup Arena: Prospective, Leakage-Free Evaluation of Frontier LLMs on a Live Tournament
Six frontier LLMs mostly matched bookmaker favorites, not each other into better forecasts.
The paper tested six web-searching, extended-thinking LLMs before each 2026 World Cup kickoff, producing 4,494 scored predictions with no future answers available at query time. On match outcomes, they averaged 63.9%, roughly level with simply backing the bookmaker’s favorite. Majority voting did not improve results because the systems agreed more often than they were right. They were weakest in close fixtures, under-picked draws and goals, and clustered scorelines around one typical result. ArXiv · AI/CL/LG's note
The paper tested six web-searching, extended-thinking LLMs before each 2026 World Cup kickoff, producing 4,494 scored predictions with no future answers available at query time. On match outcomes, they averaged 63.9%, roughly level with simply backing the bookmaker’s favorite. Majority voting did not improve results because the systems agreed more often than they were right. They were weakest in close fixtures, under-picked draws and goals, and clustered scorelines around one typical result. ArXiv · AI/CL/LG's note
score 4