The Implications of Linguistic Illegibility for LLM Security
Model self-reports are treated here as an unsound security boundary.
James Mickens argues that an LLM’s language outputs, chain-of-thought, and linguistically labeled probes may not faithfully expose the model’s internal computation. The paper calls this “linguistic illegibility” and says it is unavoidable when the real computation happens in activation space rather than natural language. On that view, safeguards such as chain-of-thought monitoring, constitutional self-critique, or language-defined activation probes cannot be complete on their own. The proposed floor is sandboxing that does not depend on interpreting the model’s words, including taint tracking, robust virtualization, and outside auditing of sandbox configurations. HN · ArXiv's note
James Mickens argues that an LLM’s language outputs, chain-of-thought, and linguistically labeled probes may not faithfully expose the model’s internal computation. The paper calls this “linguistic illegibility” and says it is unavoidable when the real computation happens in activation space rather than natural language. On that view, safeguards such as chain-of-thought monitoring, constitutional self-critique, or language-defined activation probes cannot be complete on their own. The proposed floor is sandboxing that does not depend on interpreting the model’s words, including taint tracking, robust virtualization, and outside auditing of sandbox configurations. HN · ArXiv's note
score 5
Discussions
- hn · 75 points · 29 comments
- hn · 75 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 76 points · 29 comments
- hn · 77 points · 29 comments
- hn · 77 points · 29 comments
- hn · 78 points · 29 comments