From Evidence to Action: How Tool-Using Agents Fail
Correct results can mask agents acting without the evidence they needed first.
The paper introduces SafeActBench, a 656-case benchmark for testing how tool-using agents move from investigation to action. Across ten model-harness setups, agents could judge actions well in static settings but performed worse when executing interactively. The authors found failures often started before tool use, with agents stopping too early or acting before required evidence was established. Single actions were usually reliable once the evidence was in place; multi-step workflows added failures around unresolved prerequisites and incomplete execution. Source: HF Daily Papers' note.
The paper introduces SafeActBench, a 656-case benchmark for testing how tool-using agents move from investigation to action. Across ten model-harness setups, agents could judge actions well in static settings but performed worse when executing interactively. The authors found failures often started before tool use, with agents stopping too early or acting before required evidence was established. Single actions were usually reliable once the evidence was in place; multi-step workflows added failures around unresolved prerequisites and incomplete execution. Source: HF Daily Papers' note.
score 5