Module-Aware Optimization for Graph Neural Networks
GraphMuon reached target validation loss 1.32–2.42× faster than AdamW on three large-scale OGB benchmarks, the authors report.
Graph-propagated features can amplify a weight update’s effect on a GNN layer, even when the update is well normalized as a matrix. GraphMuon adapts Muon’s updates to the covariance of those features and scales steps by their effect on layer representations. Across 15 dataset–architecture configurations, it reduced time to target in 13 while maintaining competitive or better predictive performance.
HF Daily Papers' note
Graph-propagated features can amplify a weight update’s effect on a GNN layer, even when the update is well normalized as a matrix. GraphMuon adapts Muon’s updates to the covariance of those features and scales steps by their effect on layer representations. Across 15 dataset–architecture configurations, it reduced time to target in 13 while maintaining competitive or better predictive performance.
HF Daily Papers' note
score 4