GRUET: Quantifying Uncertainty of Agentic Reasoning-and-Acting Processes
GRUET scores how much an agent’s ReAct trajectory can be trusted by measuring uncertainty across its reasoning turns.
The paper argues that agent failures often come from accumulated uncertainty in intermediate reasoning, where the same task can lead to different reasoning branches and actions. GRUET models those possible reasoning branches as a graph, uses graph complexity to estimate turn-level uncertainty, then aggregates those scores over the trajectory. The authors report tests across nine LLMs and five benchmarks, evaluating selective generation with AUROC, AUPRC, and AUARC. ArXiv · AI/CL/LG's note
The paper argues that agent failures often come from accumulated uncertainty in intermediate reasoning, where the same task can lead to different reasoning branches and actions. GRUET models those possible reasoning branches as a graph, uses graph complexity to estimate turn-level uncertainty, then aggregates those scores over the trajectory. The authors report tests across nine LLMs and five benchmarks, evaluating selective generation with AUROC, AUPRC, and AUARC. ArXiv · AI/CL/LG's note
score 5