The Emergent Symbolic Structure of Artificial Neural Networks
The paper argues that neural nets can carry symbolic structure inside their vector representations.
McCoy, Soulos, Linzen, and Smolensky say they can approximate a network’s representation-generating process with a closed-form symbolic equation while leaving behavior largely intact. They report the result across small list-manipulation networks and LLMs working on arithmetic, logic, code, and language. The authors also say those approximations let them make targeted changes to an LLM’s behavior through interventions on internal representations. ArXiv · AI/CL/LG's note
McCoy, Soulos, Linzen, and Smolensky say they can approximate a network’s representation-generating process with a closed-form symbolic equation while leaving behavior largely intact. They report the result across small list-manipulation networks and LLMs working on arithmetic, logic, code, and language. The authors also say those approximations let them make targeted changes to an LLM’s behavior through interventions on internal representations. ArXiv · AI/CL/LG's note
score 5