Knowledge That Is Linear Before Fine-Tuning Generalizes After It
Across 11 language models and 70 relations, linear readability predicted generalization with correlations of 0.87–0.92.
The researchers assigned arbitrary labels to existing answers, then fine-tuned on examples such as Paris → crab and tested whether models extended the mapping to Munich. Generalization tracked how readily a linear probe could recover the original relation from pretrained representations. The association persisted after controlling for recall accuracy, training-set size, answer count and subject frequency in pretraining text. It also held across different label assignments, fine-tuning seeds and label categories.
ArXiv · AI/CL/LG's note
The researchers assigned arbitrary labels to existing answers, then fine-tuned on examples such as Paris → crab and tested whether models extended the mapping to Munich. Generalization tracked how readily a linear probe could recover the original relation from pretrained representations. The association persisted after controlling for recall accuracy, training-set size, answer count and subject frequency in pretraining text. It also held across different label assignments, fine-tuning seeds and label categories.
ArXiv · AI/CL/LG's note
score 5