PaperCritic: Training AI Reviewers to Assess the Soundness of Papers
PaperCritic beat frontier models on an in-domain test of paper soundness, the authors report.
The team built PaperSound-23K from 1,000 ICLR submissions and 23,000 human-written and AI-generated review points. They trained PaperCritic with reinforcement learning, rewarding both sound, substantive criticisms and coverage of high-quality reference points. The task excludes judgments of interestingness and clarity. The authors also report transfer to Nature-family papers, code review, legal contract review and feedback on fiction.
ArXiv · AI/CL/LG's note
The team built PaperSound-23K from 1,000 ICLR submissions and 23,000 human-written and AI-generated review points. They trained PaperCritic with reinforcement learning, rewarding both sound, substantive criticisms and coverage of high-quality reference points. The task excludes judgments of interestingness and clarity. The authors also report transfer to Nature-family papers, code review, legal contract review and feedback on fiction.
ArXiv · AI/CL/LG's note
score 5