How to Speculate about Uncertainty in Agentic Coding? A Draft-Model Gate Method
A small draft model is used as a gate to predict when a coding agent’s next actions are likely to fail.
The paper’s method, Speculative Uncertainty, scores an agent’s already-generated token trajectory without needing logits, weights, activations, or repeated samples. It separates reasoning and action spans, then calibrates those signals against a verifiable objective. In tests on Qwen3-Coder-480B and Claude 3.5 Sonnet agents, a pre-execution veto gate cut execution error rates by 6-8 percentage points and token costs by 14-19%. ArXiv · AI/CL/LG's note
The paper’s method, Speculative Uncertainty, scores an agent’s already-generated token trajectory without needing logits, weights, activations, or repeated samples. It separates reasoning and action spans, then calibrates those signals against a verifiable objective. In tests on Qwen3-Coder-480B and Claude 3.5 Sonnet agents, a pre-execution veto gate cut execution error rates by 6-8 percentage points and token costs by 14-19%. ArXiv · AI/CL/LG's note
score 5