Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
The paper finds that agents can answer correctly while still mishandling uncertainty when evidence conflicts.
The authors evaluate “epistemic humility” through whether agents identify conflict, solve it, and escalate unresolved uncertainty. In tests across four agents, some high-accuracy setups noticed contradictions during execution but failed to carry that uncertainty into wrong final answers. The paper also reports that interventions can improve humility, but often reduce task accuracy. ArXiv · AI/CL/LG's note
The authors evaluate “epistemic humility” through whether agents identify conflict, solve it, and escalate unresolved uncertainty. In tests across four agents, some high-accuracy setups noticed contradictions during execution but failed to carry that uncertainty into wrong final answers. The paper also reports that interventions can improve humility, but often reduce task accuracy. ArXiv · AI/CL/LG's note
score 4