Auditing the Judge: Internal Signals for LLM Reasoning Assurance
At a 5% review budget, an internal-activation probe caught more errors than the models’ own judgment scores in all six test settings.
The study tested three open 7–12B models on GSM8K and MATH, finding gains of 1.3–22.5 percentage points in error recall, with a median of 7.4 points. At a 10% routing budget, probe-guided allocation improved accuracy over equal-cost random allocation in both GSM8K experiments, but beat judge-guided routing only for the weaker judge. Transfer across datasets reduced probe AUROC for every model in both directions, though precision and generation budgets also changed. The results support supervised, in-domain allocation—not domain-general confidence estimation.
ArXiv · AI/CL/LG's note
The study tested three open 7–12B models on GSM8K and MATH, finding gains of 1.3–22.5 percentage points in error recall, with a median of 7.4 points. At a 10% routing budget, probe-guided allocation improved accuracy over equal-cost random allocation in both GSM8K experiments, but beat judge-guided routing only for the weaker judge. Transfer across datasets reduced probe AUROC for every model in both directions, though precision and generation budgets also changed. The results support supervised, in-domain allocation—not domain-general confidence estimation.
ArXiv · AI/CL/LG's note
score 5