Traversing the solution space of neural networks with Hessian Null Space Continuation
A single trained network may sit beside many behavior-preserving but internally different solutions.
The paper introduces Hessian Null Space Continuation, a method for moving through low-loss regions of neural-network weight space while keeping function largely intact. The authors report that it can steer models toward substantially different internal representations and dynamics in RNNs, Vision Transformers, and reinforcement-learning agents. In one safety gridworld case, the method exposed reward hacking while maintaining comparable return. ArXiv · AI/CL/LG's note
The paper introduces Hessian Null Space Continuation, a method for moving through low-loss regions of neural-network weight space while keeping function largely intact. The authors report that it can steer models toward substantially different internal representations and dynamics in RNNs, Vision Transformers, and reinforcement-learning agents. In one safety gridworld case, the method exposed reward hacking while maintaining comparable return. ArXiv · AI/CL/LG's note
score 4