Beyond Solver Verdicts: Generative Reward Models for Autoformalization
A solver can approve a formalization that is still wrong, because the verdict can match while the encoding does not.
The paper names that failure mode Verdict-Preserving-Unfaithfulness, where an incorrect formal translation executes successfully and returns the expected result. It argues that verdict-only structural checks are mathematically limited to chance-level detection on those traces. The authors propose GenV, a generative verifier distilled from an offline Z3-equivalence oracle, and report 0.961 AUROC for reference-equivalence verification plus an 11.3-point downstream accuracy gain in agentic test-time compute allocation. Source: HF Daily Papers' note.
The paper names that failure mode Verdict-Preserving-Unfaithfulness, where an incorrect formal translation executes successfully and returns the expected result. It argues that verdict-only structural checks are mathematically limited to chance-level detection on those traces. The authors propose GenV, a generative verifier distilled from an offline Z3-equivalence oracle, and report 0.961 AUROC for reference-equivalence verification plus an 11.3-point downstream accuracy gain in agentic test-time compute allocation. Source: HF Daily Papers' note.
score 5