The Precision Wall: An Optimizer-Dependent Boundary for Optimization under Power-Law Measurement Spectra
Adam can stall even when the optimization landscape has no spurious local minima.
In low-rank matrix sensing with power-law measurement covariance, the paper attributes the failure to weak curvature falling below an optimizer’s effective denominator floor. Its formula predicts Adam’s stall threshold within 0.45 in the power-law exponent across 16 held-out configurations, after calibration on four others. The approximation fails at rank two, and reducing the denominator setting below 10⁻⁸ encounters a separate dynamical floor.
ArXiv · AI/CL/LG's note
In low-rank matrix sensing with power-law measurement covariance, the paper attributes the failure to weak curvature falling below an optimizer’s effective denominator floor. Its formula predicts Adam’s stall threshold within 0.45 in the power-law exponent across 16 held-out configurations, after calibration on four others. The approximation fails at rank two, and reducing the denominator setting below 10⁻⁸ encounters a separate dynamical floor.
ArXiv · AI/CL/LG's note
score 4