Megadose The AI curve, compressed.

Correct Labels Are Not Enough: Reliable Metadata Can Silently Control Model Behavior

· HF Daily Papers ·
Models chose the wrong answer on 97–100% of CommonsenseQA items when a trusted reward annotation was moved to it.

Researchers fine-tuned 7–9B open-weight models from three families with “(Reward: 5)” always marking the correct option. Removing the annotation left most answers correct, allowing evaluations without it to miss the dependence. Models learned to follow even neutral or nonsense annotation words, with dependence rising sharply as training-time reliability increased. The authors propose audits that move or remove annotations while keeping the question and correct answer fixed.

HF Daily Papers' note

score 5

Categories: Research