PaperCritic: Training AI Reviewers to Assess the Soundness of Papers
PaperCritic rewards substantive, valid criticism and coverage of reference review points, rather than imitation of human reviews.
The researchers built PaperSound-23K from 1,000 ICLR submissions and 23,000 human-written and AI-generated review points. They trained PaperCritic with reinforcement learning, using an F1 reward that balances precision and recall. The authors report that it outperformed frontier models and alternative reviewer-training methods on an in-domain test set. They also report transfer to Nature-family papers, code review, legal contract review and story-writing feedback.
HF Daily Papers' note
The researchers built PaperSound-23K from 1,000 ICLR submissions and 23,000 human-written and AI-generated review points. They trained PaperCritic with reinforcement learning, using an F1 reward that balances precision and recall. The authors report that it outperformed frontier models and alternative reviewer-training methods on an in-domain test set. They also report transfer to Nature-family papers, code review, legal contract review and story-writing feedback.
HF Daily Papers' note
score 5