Are Causal Neuron Circuits Unique and Transferable?
A few neurons often suffice to flip an answer, and separate subsets can produce the same result.
The study tested all 4,096 subsets of selected 12-neuron groups. On two-hop relation questions, patching neurons with activations from the first-hop question shifted answers toward the intermediate entity across three models; random neurons never did. The effects carried over to rewordings of the same question chain but were much weaker on other tested chains. The work identifies alternative neuron subsets that affect answers, without recovering full circuit connectivity.
HF Daily Papers' note
The study tested all 4,096 subsets of selected 12-neuron groups. On two-hop relation questions, patching neurons with activations from the first-hop question shifted answers toward the intermediate entity across three models; random neurons never did. The effects carried over to rewordings of the same question chain but were much weaker on other tested chains. The work identifies alternative neuron subsets that affect answers, without recovering full circuit connectivity.
HF Daily Papers' note
score 4