What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents
The highest reasoning setting cost up to 42 times more on easy tasks, without guaranteeing better results on individual problems.
Across four benchmarks covering single-call reasoning and longer agent tasks, higher effort sometimes failed where lower effort succeeded, even when it improved overall accuracy. Agents at higher settings spent more time checking their work and searching for solutions, extending runs without necessarily changing the outcome. Starting at the lowest effort and escalating only after a verified failure matched or beat fixed maximum effort across every tested model and benchmark, at lower cost for most proprietary models.
HF Daily Papers' note
Across four benchmarks covering single-call reasoning and longer agent tasks, higher effort sometimes failed where lower effort succeeded, even when it improved overall accuracy. Agents at higher settings spent more time checking their work and searching for solutions, extending runs without necessarily changing the outcome. Starting at the lowest effort and escalating only after a verified failure matched or beat fixed maximum effort across every tested model and benchmark, at lower cost for most proprietary models.
HF Daily Papers' note
score 5