Orthogonal and Parametrized Multi-Attribute Steering of LLMs
The authors report a way to adjust multiple LLM traits at once while reducing interference between them.
They find that simply combining steering vectors causes traits to interfere with one another. Their orthogonalization method decouples those vectors so that steering one behavior does not induce the others. They also identify and rescale each vector’s most steerable interval. A separate method, parameterized steering, sets activations at a desired trait strength instead of pushing them in a direction.
ArXiv · AI/CL/LG's note
They find that simply combining steering vectors causes traits to interfere with one another. Their orthogonalization method decouples those vectors so that steering one behavior does not induce the others. They also identify and rescale each vector’s most steerable interval. A separate method, parameterized steering, sets activations at a desired trait strength instead of pushing them in a direction.
ArXiv · AI/CL/LG's note
score 5