Muon on the Stiefel Manifold Admits an Exact Closed-Form Update
The paper claims Muon’s Stiefel-manifold update can be solved exactly, without heuristic or iterative approximations.
The authors study Muon for matrices constrained to have orthonormal columns. They derive a closed-form Stiefel Muon update and use it to define Skewon, an algorithm for orthogonality-constrained optimization. The paper also states first-order convergence guarantees for Skewon in smooth non-convex problems. Source: ArXiv · AI/CL/LG's note
The authors study Muon for matrices constrained to have orthonormal columns. They derive a closed-form Stiefel Muon update and use it to define Skewon, an algorithm for orthogonality-constrained optimization. The paper also states first-order convergence guarantees for Skewon in smooth non-convex problems. Source: ArXiv · AI/CL/LG's note
score 5