Invariant-Aware Harness Enforcement to Protect LLM Agents from Silent API Failures
Blocking faulty API results cut wrong answers from 74.0% to 1.7% across seven open models in the paper’s target-fault tests.
The researchers found silent failures in 11 of 15 public research APIs, with counts inflated by up to 691 times despite successful HTTP responses. Their SilentGuard harness checks results against rules a correct API must satisfy and withholds values that fail. Giving agents a warning from the same check still left 70.1% of answers wrong. The system avoids errors by abstaining rather than recovering; on new fault types, wrong answers fell from 61.1% to 24.5%.
HF Daily Papers' note
The researchers found silent failures in 11 of 15 public research APIs, with counts inflated by up to 691 times despite successful HTTP responses. Their SilentGuard harness checks results against rules a correct API must satisfy and withholds values that fail. Giving agents a warning from the same check still left 70.1% of answers wrong. The system avoids errors by abstaining rather than recovering; on new fault types, wrong answers fell from 61.1% to 24.5%.
HF Daily Papers' note
score 5