Beyond Hessian Sharpness: Two Local Stability Edges in Practical Muon
Abstract
Edge of Stability connects Hessian sharpness to the local stability limit of gradient descent. We study this limit for practical Muon with its original five Newton–Schulz steps, normalization, Nesterov momentum, and weight decay. The key quantity combines the loss Hessian with the derivative of the matrix transformation. We derive an exact quadratic equation for two state multipliers per mode. They describe the growth or decay of small weight and momentum perturbations under the local linear map. For real modes, the stable interval has multiplier at the lower edge and at the upper edge. Finite Newton–Schulz steps can introduce negative modes under positive semidefinite curvature. Proposal scaling can preserve the next weight while moving modes across either edge. Two strongly convex quadratic constructions with clamp normalization connect these crossings to stable fixed points and a stable orbit of period two. Language model experiments support each part of the local calculation: negative derivative eigenvalues occur in all 240 sampled matrix and checkpoint pairs from 58M to 1.25B parameters, a convex objective with linearized logits isolates negative modes under positive semidefinite curvature, and interventions cross both edges in 58M and 124M models. In a 16 step replay with non-Muon parameters fixed, successive Jacobians reproduce the measured perturbation growth. Together, these results identify how the implemented matrix transformation and optimizer state determine Muon's local response.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.