ASLEval: Measuring Privacy Exposure Displacement in LLM Agent Sessions
The paper says privacy checks that watch only one output can miss nearly half of the exposure in an agent session.
ASLEval defines “privacy exposure displacement” as the gap between a local proxy and what actually leaves through all visible exits. In the reported enterprise-style tests, checking only the expected outlet missed 46.9% of exposure found by the union of visible exits. Attacker self-reports were unreliable, mixing omissions with high false discovery. The authors argue benchmarks should predefine targets and authorization, declare the full visible boundary, and report privacy alongside task utility. ArXiv · AI/CL/LG's note
ASLEval defines “privacy exposure displacement” as the gap between a local proxy and what actually leaves through all visible exits. In the reported enterprise-style tests, checking only the expected outlet missed 46.9% of exposure found by the union of visible exits. Attacker self-reports were unreliable, mixing omissions with high false discovery. The authors argue benchmarks should predefine targets and authorization, declare the full visible boundary, and report privacy alongside task utility. ArXiv · AI/CL/LG's note
score 5