Representation Redistribution in Sparse Autoencoders
Changes in sparse autoencoder features can recur across prompt variants without causing downstream failure.
The researchers introduce a redistribution sensitivity score (RSS) to identify large feature responses that repeatedly shift in the same direction. Tests across 27 pretrained sparse autoencoders spanning eight language models found reproducible shifts on prompt variants excluded from feature selection. In two frozen safety readouts, RSS-selected sets with median sizes of 20 and 19 features, combined with the clean decision margin, predicted recurrent failure with AUROCs of 0.935 and 0.958. The method separates recurring representation changes from their contribution to a readout’s decision.
HF Daily Papers' note
The researchers introduce a redistribution sensitivity score (RSS) to identify large feature responses that repeatedly shift in the same direction. Tests across 27 pretrained sparse autoencoders spanning eight language models found reproducible shifts on prompt variants excluded from feature selection. In two frozen safety readouts, RSS-selected sets with median sizes of 20 and 19 features, combined with the clean decision margin, predicted recurrent failure with AUROCs of 0.935 and 0.958. The method separates recurring representation changes from their contribution to a readout’s decision.
HF Daily Papers' note
score 4