Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence
The paper tests whether a rubric can be built from only a query by forcing each candidate criterion to prove it separates better and worse answers.
Rubrics on Trial starts from no rubric and uses synthetic rubric-conditioned response pairs as its evidence. It screens proposed rubrics before adding them, rejecting criteria that are non-discriminative, too specific, or merely stylistic. The authors report experiments across five preference benchmark suites, with the method posting the best average accuracy and leading on six of seven evaluation sets. ArXiv · AI/CL/LG's note
Rubrics on Trial starts from no rubric and uses synthetic rubric-conditioned response pairs as its evidence. It screens proposed rubrics before adding them, rejecting criteria that are non-discriminative, too specific, or merely stylistic. The authors report experiments across five preference benchmark suites, with the method posting the best average accuracy and leading on six of seven evaluation sets. ArXiv · AI/CL/LG's note
score 5