High-Confidence Stationarity of Momentum Muon: A Heavy-Tailed Separation from Adam

As an emerging optimization method, Muon has been empirically shown to outperform Adam/AdamW on modern machine learning tasks, yet the mechanism underlying this advantage remains unclear. We identify a key mechanism behind Muon's high-probability advantage: its instantaneous singular-value compression provides the scale regularization sought by Adam's second-moment normalization while directly controlling unpreconditioned stationarity. By avoiding the confidence loss incurred when converting Adam's preconditioned adaptive energy into unpreconditioned stationarity, Muon achieves sharper confidence dependence under heavy-tailed noise. Specifically, under conditional $p$-moment noise for any $p\in (1,2]$, we prove a finite-time high-probability theorem for exact canonical pre-polar momentum Muon, with $\beta \in [0,1)$ fixed within each run. Choosing momentum and stepsize for the given horizon at fixed batch size yields the average-gradient rate $\mathcal{O}(T^{-a}(1+\log (1/\delta )))$, where $a=(p-1)/(3p-2)$. Even when stochastic gradients have infinite variance, the dependence on the confidence parameter $\delta$ remains only logarithmic. For the five-step Newton–Schulz (NS5) variant, we obtain the same rate up to an additive error floor under an explicit approximation-error condition. In contrast, for Adam with fixed ${\beta }_2\in (0,1)$, $ϵ>0$ and initial moments, we construct a uniformly $p$-moment-bounded two-dimensional family and prove a minimax stationarity lower bound $T^{-1}{\sum }_{t=1}^T\Vert \nabla f(X_t){\Vert }_{\text{F}}\ge c\min \{1,T^{-a}{\delta }^{-q}\}$ with probability greater than $\delta$.