Real-Time Detection and Repair of LLM Agent Failures
A lightweight monitor caught agent failures in real time and, when paired with deterministic checks, improved recovery without using an LLM judge.
Sunny Dubey tests telemetry-only failure detection across 2,823 agent episodes and reports 0.71 failure detection at a 5% false-alarm budget. A deterministic verification layer catches stated-total and missing-call errors with no false positives in the reported healthy set. Flagged runs are rolled back and re-run live, raising task success from 52% to 73% for about one extra model call per run. ArXiv · AI/CL/LG's note
Sunny Dubey tests telemetry-only failure detection across 2,823 agent episodes and reports 0.71 failure detection at a 5% false-alarm budget. A deterministic verification layer catches stated-total and missing-call errors with no false positives in the reported healthy set. Flagged runs are rolled back and re-run live, raising task success from 52% to 73% for about one extra model call per run. ArXiv · AI/CL/LG's note
score 5