Language models can explain neurons in language models
OpenAI used GPT-4 to auto-generate explanations for GPT-2 neuron behavior and scores explainability, releasing a dataset of 77K neuron explanations.
Excerpt
We use GPT-4 to automatically write explanations for the behavior of neurons in large language models and to score those explanations. We release a dataset of these (imperfect) explanations and scores for every neuron in GPT-2.
Read at source: https://openai.com/index/language-models-can-explain-neurons-in-language-models