Minimally Invasive Steering of Language Models
MISVO steers frozen language models at test time while penalizing shifts that would distort their token distributions.
The paper proposes adding position-specific vectors to final hidden states, then regularizing those interventions with local KL geometry. Its Fisher quadratic penalty is designed to preserve generation quality while still optimizing for a reward. The authors report results on preference and code-generation tasks across roughly 1B to 14B parameter models, with MISVO leading mean reward in six of seven model-task settings. Diversity and coherence stayed close to Best-of-N in their evaluations. ArXiv · AI/CL/LG's note
The paper proposes adding position-specific vectors to final hidden states, then regularizing those interventions with local KL geometry. Its Fisher quadratic penalty is designed to preserve generation quality while still optimizing for a reward. The authors report results on preference and code-generation tasks across roughly 1B to 14B parameter models, with MISVO leading mean reward in six of seven model-task settings. Diversity and coherence stayed close to Best-of-N in their evaluations. ArXiv · AI/CL/LG's note
score 4