1. The descent guarantee in Eq. (9) is not justified by the stated assumptions
Section 4.2 assumes twice continuous differentiability, while Appendix A.1 derives Eq. (9) by dropping the Taylor remainder at the current iterate and then applying a Rayleigh bound. A current-point Hessian does not, by itself, provide a finite-step upper bound on the loss.
For example, consider the strongly convex scalar function
f(w)=w+21w2+w4.
At w=0, both the gradient and Hessian equal one. The proposed current-Hessian threshold therefore permits η=1<2. Both GD and the scalar exact-polar update move to −1, but
f(−1)=21>f(0)=0.
The result can be repaired by explicitly assuming a common curvature upper bound along the entire update segment, or appropriate regional smoothness. Under that correction, the subsequent nuclear-norm algebra can remain valid. However, the learning-rate guarantees are not established as currently stated. The distinction between a local quadratic prediction and a guaranteed descent threshold needs to be maintained throughout Section 4.2.
2. The uniform convergence specialization in Eqs. (28)–(30) appears vacuous under the stated infinite full-rank trajectory assumptions
The product bound in Eq. (27) follows from the stated fixed-metric assumptions. The subsequent uniform specialization requires separate bounds
\le \sup_t\widetilde\beta_t<\infty.$$
Let $M=\sum_i m_i$. For full-rank exact-polar updates,
$$\|o_t\|_2=\sqrt M.$$
Together with the prescribed learning rate $\eta_t=1/\widetilde\beta_t$, a finite uniform upper bound $\overline\beta$ on $\widetilde\beta_t$ implies
$$\|W_t-W_{t-1}\|_2
\ge \frac{\sqrt M}{\overline\beta}>0.$$
However, the claimed geometric decrease, combined with the fixed-metric strong convexity already assumed on the common region $\Omega$ at the first iteration, implies $W_t\to W^\star$. Consecutive step lengths must then vanish, yielding a contradiction.
This concern is specific to the infinite-horizon uniform specialization; it does not invalidate the one-step argument or Eq. (27). A uniform lower bound on $\widetilde\alpha_t/\widetilde\beta_t$, allowing $\widetilde\beta_t\to\infty$, would be a natural alternative.
The treatment of rank deficiency also needs clarification. Even for
$$L(W)=\tfrac12\|W\|_F^2,\qquad W_0=\operatorname{diag}(2,1),$$
the step prescribed by Theorem 4 gives $W_1=\operatorname{diag}(1,0)$. The gradient becomes rank deficient before reaching the optimum, so the inverse-square-root preconditioner is no longer defined. This illustrates that full rank is not automatically maintained even on a simple quadratic.
#### 3. The scalar learning-rate comparison has limited scale-independent optimization significance
The paper correctly acknowledges objective scaling, but its implications for the main contribution need stronger treatment. Under $L_c=cL$, with $c>0$,
$$G_c=cG,\qquad
\operatorname{polar}(G_c)=\operatorname{polar}(G),\qquad
R_c=cR.$$
Thus, the condition $R>1$ can be reversed by changing the units of the objective. Meanwhile, dividing the GD learning rate by $c$ preserves its trajectory, and the exact-polar direction remains unchanged.
Consequently, the observation that $R_t>1$ at 99.8% of steps under a summed objective does not establish a scale-independent optimization advantage.
This limitation can also be seen within the paper’s common Euclidean smoothness bound. Write
$$S=\sum_i\|G_i\|_*,
\qquad M=\sum_i m_i.$$
Optimizing the quadratic descent bound over the scalar step size gives guaranteed decreases
$$D_{\mathrm{Muon}}=\frac{S^2}{2\beta M},
\qquad
D_{\mathrm{GD}}=\frac{\|g\|_2^2}{2\beta}.$$
Since $S^2\le M\|g\|_2^2$, this particular bound does not yield a stronger progress guarantee for Muon.
This observation concerns the explanatory power of the analysis, rather than ruling out actual Muon advantages. Comparisons involving directional curvature, achieved loss decrease, update distance, or independently tuned optimization trajectories would make the learning-rate contribution substantially more informative.
#### 4. The empirical section does not directly test the proposed cross-block sufficient condition
The introduction promises controlled changes to the spectral quantities and regimes in which the conditions weaken or fail. However, the sufficient condition in Eq. (59) is not evaluated. Figure 2 reports its desired consequence—a larger regularized curvature ratio—without establishing whether Eq. (59) holds or whether changing cross-block scales changes the observed behavior as predicted.
A small positive-definite multi-block quadratic experiment would provide a direct and inexpensive test. It could independently vary within-block spectra and cross-block scales, report the condition margin, and compare predicted curvature ratios with actual one-step and trajectory-level progress. Since the condition is sufficient, its failure need not imply that Muon performs worse; nevertheless, examining both sides would clarify its information content and scope.
The curvature diagnostic also needs a more complete account of regularization. Each probe uses only 1,024 tokens. For a 3,072-dimensional MLP factor, as in the 12-block architecture specified in Appendix B, its empirical covariance has rank at most 1,024. Such singularity is structural. Please provide the complete matrix shapes used in Section 5.2 and distinguish structural nullspaces from numerical conditioning.
I appreciate the qualification on p. 9 that Fig. 2 is not a direct theorem verification. However, the paper does not sufficiently specify which factors are eigenvalue-floored, the reference scale, the order of regularization relative to inverse powers, or sensitivity to the $10^{-6}$ floor. These choices could materially affect the reported 50–2,000-fold improvement.
Moreover, the absolute curvature ratios are approximately $10^{-15}$–$10^{-12}$. Substitution into the proposed contraction bound would remain extremely loose over the reported training horizon. Additional evidence is needed to connect this diagnostic to observed optimization speed.
#### 5. The optimization comparisons do not establish an advantage after independent learning-rate tuning
In the NoNorm experiment, every tested SGD learning rate diverges at step 3. This demonstrates a difference over the selected grid, but no stable SGD baseline is identified. Extending the SGD search to find its stable and effective regime is necessary for a convergence-speed comparison.
The normalized experiment compares the methods at the same scalar learning rates and reports only the three smallest values in the sweep. SGD validation loss continues improving as the learning rate increases from 0.03 to 0.3, whereas Muon validation loss worsens. This suggests that the two methods require different tuning ranges. Please report the complete sweep and compare separately selected learning rates, preferably using steps to a fixed objective value.
There is also a gap between the analyzed and implemented updates. The appendix uses finite Newton–Schulz iterations, rectangular matrix rescaling, and a reference AdamW optimizer for other parameters. These choices are reasonable, but the theoretical thresholds are derived for the specified exact-polar joint update. The paper should state the precise correspondence, including the effect of block-dependent rescaling. A controlled experiment matching the theoretical update would help resolve this issue.
#### 6. The contribution relative to existing theory needs sharper positioning
Much of the development consists of standard smoothness/PL descent arguments and spectral identities for Kronecker and block-diagonal matrices. The most distinctive component appears to be the cross-block sufficient condition, whose predictive value is not yet established.
The relationship to Davis and Drusvyatskiy, *When do spectral gradient updates help in deep learning?* (arXiv:2512.04299), should be made more explicit, particularly regarding sufficient conditions for spectral versus Euclidean progress. Gonon et al., *Insights on Muon from Simple Quadratics* (arXiv:2602.11948), is also relevant to the limitations of local proxies and the distinction between exact and approximate polar dynamics. These results do not automatically subsume the present work, but they help define the additional explanatory contribution that needs to be demonstrated.