What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations
The paper finds code-model “universality” only in what concepts get circuits, not where or how those circuits are built.
Across Python and Rust, Qwen2.5-Coder-7B and DeepSeek-Coder-V1-6.7B agreed on which grammatical concepts earned dedicated circuitry. Their internal placement differed: Qwen concentrated processing around layers 17–19, while DeepSeek did so around layers 6–7. Rust constructs drew 2–3x more concept-specific circuitry than Python equivalents in both models. The author says the claims are limited to this 2x2 setup, with a third-model test left as the next step. ArXiv · AI/CL/LG's note
Across Python and Rust, Qwen2.5-Coder-7B and DeepSeek-Coder-V1-6.7B agreed on which grammatical concepts earned dedicated circuitry. Their internal placement differed: Qwen concentrated processing around layers 17–19, while DeepSeek did so around layers 6–7. Rust constructs drew 2–3x more concept-specific circuitry than Python equivalents in both models. The author says the claims are limited to this 2x2 setup, with a third-model test left as the next step. ArXiv · AI/CL/LG's note
score 5