Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback
The paper finds that the best-performing one-bit Muon compression is also the placement its theory shows can diverge.
SignMuon compresses Muon updates to one bit per parameter by taking the elementwise sign after the Linear Minimization Oracle. The authors show that sign placement around the oracle can still produce ascent, and that error feedback on Muon’s output can fail regardless of smoothness, step size, or momentum. Error feedback applied to the gradient gives provable convergence for EF21-MuonUSign and EF21-MuonSign, but experiments on CIFAR-10 and nanoGPT favor the sign-after-LMO heuristic. ArXiv · AI/CL/LG's note
SignMuon compresses Muon updates to one bit per parameter by taking the elementwise sign after the Linear Minimization Oracle. The authors show that sign placement around the oracle can still produce ascent, and that error feedback on Muon’s output can fail regardless of smoothness, step size, or momentum. Error feedback applied to the gradient gives provable convergence for EF21-MuonUSign and EF21-MuonSign, but experiments on CIFAR-10 and nanoGPT favor the sign-after-LMO heuristic. ArXiv · AI/CL/LG's note
score 4