The Emergent Symbolic Structure of Artificial Neural Networks

The paper argues that neural-network vectors can be approximated as symbolic structures without greatly changing model behavior.

The authors say this held in small list-manipulation networks and in LLMs tested on arithmetic, logic, code, and language. They report replacing the representation-generating process with a closed-form symbolic equation while behavior remained largely intact. They also say targeted interventions on those representations changed LLM behavior, suggesting the identified structures are doing work inside the model. HN · ArXiv's note