When Linguistic and Internal Confidence Diverge in Large Language Models
LLMs’ stated confidence often does not match their internal uncertainty signals.
The paper compares verbal confidence with logits-based confidence across classification tasks and semantic-entropy uncertainty in generation. The authors find weak instance-level alignment on average, with instruction-tuned models often sounding more confident while showing larger gaps and worse calibration. Prompt wording can shift reported confidence, but attitude cues inflate it without improving alignment. ArXiv · AI/CL/LG's note
The paper compares verbal confidence with logits-based confidence across classification tasks and semantic-entropy uncertainty in generation. The authors find weak instance-level alignment on average, with instruction-tuned models often sounding more confident while showing larger gaps and worse calibration. Prompt wording can shift reported confidence, but attitude cues inflate it without improving alignment. ArXiv · AI/CL/LG's note
score 4