Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety
DonkeyRail missed 8 of 19 outputs that carried an injected procedure forward, while flagging 70 of 192 legitimate replies.
The study tested the text-similarity classifier in a controlled three-agent email relay, measuring whether injected instructions induced fresh computations in two downstream agents. Every completed reply involving authorized quotation or template reuse triggered a false alert. Recalibrating the threshold on one template set failed to transfer: it detected only one functional output and still raised 15 false alerts on new templates. The paper also shows that identical visible transcripts can require different safety decisions under different policies, arguing for evaluation grounded in behavior and authorization.
ArXiv · AI/CL/LG's note
The study tested the text-similarity classifier in a controlled three-agent email relay, measuring whether injected instructions induced fresh computations in two downstream agents. Every completed reply involving authorized quotation or template reuse triggered a false alert. Recalibrating the threshold on one template set failed to transfer: it detected only one functional output and still raised 15 false alerts on new templates. The paper also shows that identical visible transcripts can require different safety decisions under different policies, arguing for evaluation grounded in behavior and authorization.
ArXiv · AI/CL/LG's note
score 4