Wrong Prediction, Right Answer: Recovering Evidence from Collapsed LLM Sequence Scores
The paper argues that failed sequence scores can hide reasoning evidence still present inside the model.
The authors report a “readout gap”: hidden-state probes can recover correct answers even when native sequence scoring collapses. A two-parameter, label-free additive correction trained on as few as 25 unlabeled examples restores 9 to 34 accuracy points for Qwen3.5 models and transfers to OLMo-2-1B and Llama-3.1-8B. The recovered decisions hold on harder cases and beat count-preserving permutation baselines, which the paper uses to argue the effect is not just lexical overlap. The authors say some zero-shot reasoning failures should be read more narrowly as expression failures, not missing internal logic. ArXiv · AI/CL/LG's note
The authors report a “readout gap”: hidden-state probes can recover correct answers even when native sequence scoring collapses. A two-parameter, label-free additive correction trained on as few as 25 unlabeled examples restores 9 to 34 accuracy points for Qwen3.5 models and transfers to OLMo-2-1B and Llama-3.1-8B. The recovered decisions hold on harder cases and beat count-preserving permutation baselines, which the paper uses to argue the effect is not just lexical overlap. The authors say some zero-shot reasoning failures should be read more narrowly as expression failures, not missing internal logic. ArXiv · AI/CL/LG's note
score 5