Negative-Only Policy Optimization for One-Sided Verifiable Rewards
Partial checks can reward wrong answers; NOPO learns only from failures those checks reliably catch.
The method treats responses that pass verification as unlabeled, since passing an incomplete check does not establish correctness. It suppresses verified failures and scales each update by the share of responses that escaped falsification. The authors report higher pass@k than baselines on code, logic and math benchmarks using unit tests, constraint checks and LLM self-verifiers, without oracle rewards. They say the gains persist with increased inference-time sampling and transfer across benchmarks.
ArXiv · AI/CL/LG's note
The method treats responses that pass verification as unlabeled, since passing an incomplete check does not establish correctness. It suppresses verified failures and scales each update by the share of responses that escaped falsification. The authors report higher pass@k than baselines on code, logic and math benchmarks using unit tests, constraint checks and LLM self-verifiers, without oracle rewards. They say the gains persist with increased inference-time sampling and transfer across benchmarks.
ArXiv · AI/CL/LG's note
score 5