Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation
Restoring execution provenance recovered 85.5% of verifier judgment failures caused by abstraction.
The researchers compared native, abstracted and restored views of identical agent executions, keeping outcomes, ground truth and verifiers fixed. Across four benchmarks and seven verifier backbones, transport failures reached about 31% under macro abstraction and delegation. Direct verification more often returned incorrect verdicts, while evidence-grounded and deliberative protocols tended to abstain when evidence was limited. The findings show why accuracy on fully visible executions alone cannot establish verifier reliability across abstraction boundaries.
ArXiv · AI/CL/LG's note
The researchers compared native, abstracted and restored views of identical agent executions, keeping outcomes, ground truth and verifiers fixed. Across four benchmarks and seven verifier backbones, transport failures reached about 31% under macro abstraction and delegation. Direct verification more often returned incorrect verdicts, while evidence-grounded and deliberative protocols tended to abstain when evidence was limited. The findings show why accuracy on fully visible executions alone cannot establish verifier reliability across abstraction boundaries.
ArXiv · AI/CL/LG's note
score 5