Megadose The AI curve, compressed.

Rubric Rewards from Item Response Theory

· HF Daily Papers ·
RRT turns rubric verdict patterns into a reward signal and can cut judge calls without losing much performance.

The paper argues that simply summing rubric points hides which criteria are actually informative for a model’s current rollouts. Its Rubric Response Theory model estimates quality from criterion verdicts using item-response parameters for difficulty and discrimination. In experiments with Qwen3.5-4B, RRT beats GRPO by 1.7 macro criterion points across four datasets, with larger gains on hard criteria. With half the criterion budget, its adaptive selection stays within 0.1 points of full-judging GRPO. HF Daily Papers' note

score 4

Categories: Research