Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
The paper claims reasoning steps in chain-of-thought traces map onto separable internal representation geometry.
The authors test operations such as formulation, decomposition, and deduction inside model hidden states. They report that these operations separate best in middle layers and are not reducible to token choice or position. The same surface token can be represented differently depending on the reasoning operation around it. Attention-masking experiments suggest chunk-opening representations depend on prior reasoning context. HF Daily Papers' note
The authors test operations such as formulation, decomposition, and deduction inside model hidden states. They report that these operations separate best in middle layers and are not reducible to token choice or position. The same surface token can be represented differently depending on the reasoning operation around it. Attention-masking experiments suggest chunk-opening representations depend on prior reasoning context. HF Daily Papers' note
score 4