Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems
Even benign questions prompted the best-performing agent to disclose forbidden combinations of information in 7.08% of simulated answers.
The study tested 30 synthetic cybersecurity, medical and military scenarios, collecting 10,800 simulated answers and 1,440 answers from live agents in two open-source harnesses. GPT-5.6-Sol recorded that 7.08% rate on queries designed to require only one document, with worse disclosure rates in live harnesses. Classification headers and explicit aggregation rules substantially reduced pooled disclosures, but 32.37% of simulated answers across all tested intents remained unsafe despite those safeguards. Some refusals also revealed the prohibited conclusion.
HF Daily Papers' note
The study tested 30 synthetic cybersecurity, medical and military scenarios, collecting 10,800 simulated answers and 1,440 answers from live agents in two open-source harnesses. GPT-5.6-Sol recorded that 7.08% rate on queries designed to require only one document, with worse disclosure rates in live harnesses. Classification headers and explicit aggregation rules substantially reduced pooled disclosures, but 32.37% of simulated answers across all tested intents remained unsafe despite those safeguards. Some refusals also revealed the prohibited conclusion.
HF Daily Papers' note
score 5