Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation
The paper makes test-time RL work for code by rewarding behavioral agreement on generated probe inputs.
The authors say self-voting rewards used in test-time reinforcement learning fail for code because programs cannot be judged by surface-form agreement. Their method creates probe inputs from the problem statement, runs candidate programs on them, and uses the resulting consensus as a reward signal. Because that signal can still be gamed, ERPO adds rank masking for conservative negative updates and an entropy ceiling to limit policy drift. The paper reports substantial pass@1 and pass@k gains on coding benchmarks in both in-domain adaptation and zero-shot transfer. ArXiv · AI/CL/LG's note
The authors say self-voting rewards used in test-time reinforcement learning fail for code because programs cannot be judged by surface-form agreement. Their method creates probe inputs from the problem statement, runs candidate programs on them, and uses the resulting consensus as a reward signal. Because that signal can still be gamed, ERPO adds rank masking for conservative negative updates and an entropy ceiling to limit policy drift. The paper reports substantial pass@1 and pass@k gains on coding benchmarks in both in-domain adaptation and zero-shot transfer. ArXiv · AI/CL/LG's note
score 5