The LLM-Attacker: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems
Giving attackers access to training gradients raised exact-document recovery from 13.71% to 37.77% in a controlled GPT-2 test.
Split language models keep input text on the client, but the intermediate representations sent to a server can expose it to reconstruction attacks. LLM-Attack Suite records attacker access, datasets and scoring choices so privacy evaluations can be checked and compared. Its scoring rule requires a linked undefended reference before a defence can pass, and reports flag untested attacks. The GPT-2 experiment tests how the evaluation responds to expanded attacker access; it does not establish recovery rates across other systems.
HF Daily Papers' note
Split language models keep input text on the client, but the intermediate representations sent to a server can expose it to reconstruction attacks. LLM-Attack Suite records attacker access, datasets and scoring choices so privacy evaluations can be checked and compared. Its scoring rule requires a linked undefended reference before a defence can pass, and reports flag untested attacks. The GPT-2 experiment tests how the evaluation responds to expanded attacker access; it does not establish recovery rates across other systems.
HF Daily Papers' note
score 4