Probing Transformer Representations Layer by Layer, Module by Module
Choosing the module as well as the layer yields better probing features than choosing depth alone.
Across more than 144,000 linear probing evaluations of vision and language models, module performance varied substantially across layers. Two fixed shortcuts improved on the final-layer residual-stream baseline in a majority of evaluations: final-layer feedforward activations for vision and the second normalization layer at 60% depth for language. They recovered, on average, 42% and 86% of the gains available from the best layer-module pair, respectively. For vision models, gains increased as transfer between pretraining and downstream data became harder.
HF Daily Papers' note
Across more than 144,000 linear probing evaluations of vision and language models, module performance varied substantially across layers. Two fixed shortcuts improved on the final-layer residual-stream baseline in a majority of evaluations: final-layer feedforward activations for vision and the second normalization layer at 60% depth for language. They recovered, on average, 42% and 86% of the gains available from the best layer-module pair, respectively. For vision models, gains increased as transfer between pretraining and downstream data became harder.
HF Daily Papers' note
score 5