Megadose The AI curve, compressed.

When Expert Disagreement Hurts: Prestige and Selectivity in LLM Decision Revision

· HF Daily Papers ·
Expert disagreement reduced accuracy by up to 16.1 percentage points across five tested models.

The audit covered 1,989 evidence-grounded decisions in medicine, science and law, holding questions, evidence and initial answers fixed while changing the revision context. No model corrected wrong answers selectively enough under expert disagreement to improve accuracy. The expert label prompted more reversals in four of five models, but its added effect on accuracy ranged from a 5.2-point loss to a 2.4-point gain. That makes reversal counts alone a poor measure of harm.

HF Daily Papers' note

score 5

Categories: Research