Scoring Higher, Answering Worse: Mitigating Reward Hacking in Rubric-Based RL via Protocol-Level Rubrics
The paper says rubric rewards can be gamed by satisfying secondary criteria while missing the decision that matters.
In clinical consultation tests, additive rubric scoring let models raise coverage while physician-rated appropriateness fell below the untrained model. The authors argue the issue was aggregation, not the rubric text itself. Their ProRubric method groups criteria into protocol-level dimensions that count only when the whole group holds and no failure clause triggers. It improved appropriateness by 10.8 points without losing coverage, according to the abstract. HF Daily Papers' note
In clinical consultation tests, additive rubric scoring let models raise coverage while physician-rated appropriateness fell below the untrained model. The authors argue the issue was aggregation, not the rubric text itself. Their ProRubric method groups criteria into protocol-level dimensions that count only when the whole group holds and no failure clause triggers. It improved appropriateness by 10.8 points without losing coverage, according to the abstract. HF Daily Papers' note
score 5