RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons
RACE estimates whether Transformer neurons behave consistently across a target domain using forward-pass statistics.
The paper says existing neuron-analysis methods either stay at instance-level estimates or are too costly for broad domain analysis. RACE, short for Residual Alignment for Consistency Estimation, is presented as a way to measure domain-wide functional consistency. The authors report stronger domain specificity than gradient-based point estimates in perturbation tests, and token-distribution results linking selected neurons to the target domain. They also claim roughly two orders of magnitude less computational overhead than gradient-based methods. ArXiv · AI/CL/LG's note
The paper says existing neuron-analysis methods either stay at instance-level estimates or are too costly for broad domain analysis. RACE, short for Residual Alignment for Consistency Estimation, is presented as a way to measure domain-wide functional consistency. The authors report stronger domain specificity than gradient-based point estimates in perturbation tests, and token-distribution results linking selected neurons to the target domain. They also claim roughly two orders of magnitude less computational overhead than gradient-based methods. ArXiv · AI/CL/LG's note
score 4