Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States
The paper proposes checking bias shifts inside a model’s hidden states before they surface in generated text.
The method compares related model variants by mapping sentences against a fixed set of anchor sentences, creating a shared “relative” representation space. It defines Representational Bias Shift, or ΔB, to measure how target groups move toward positive or negative attributes after changes such as fine-tuning. Across three model families and three benchmarks, ΔB tracked output-level bias changes in 15 of 18 tested settings. The authors present it as a faster complement to output-based audits, not a replacement. HF Daily Papers' note
The method compares related model variants by mapping sentences against a fixed set of anchor sentences, creating a shared “relative” representation space. It defines Representational Bias Shift, or ΔB, to measure how target groups move toward positive or negative attributes after changes such as fine-tuning. Across three model families and three benchmarks, ΔB tracked output-level bias changes in 15 of 18 tested settings. The authors present it as a faster complement to output-based audits, not a replacement. HF Daily Papers' note
score 4