acceptodds
Under review as a conference paper at ICLR 2027

The Illusion of Speed: Uncovering the Trade-Offs of Muon’s Spectral Flattening

Abstract

Muon has become a promising optimizer for accelerating neural network training by flattening the momentum update spectrum (momentum orthogonalization). Because it often improves model capabilities, the literature has focused on its theoretical understanding and applications while largely overlooking the underlying costs and drawbacks of the acceleration mechanism behind Muon. Our analysis reveals a fundamental tradeoff behind Muon’s speed: this mechanism alters implicit biases that help neural networks generalize in non-synthetic, noisy data settings, degrading performance in real-world environments. In practice, by equalizing the singular values of the update step, momentum orthogonalization assigns equal weight to update directions driven by the "clean" data distribution and those driven by rare, weak, or purely noisy data examples. We develop a theoretical account of this tradeoff, showing that spectral equalization encourages high-rank learning (while gradient-based methods are usually known as low-rank learners), removes the natural separation between signal and noise during training, and can move optimization away from simple solutions. We further identify asymmetric label corruption as a realistic failure mode: corrupted examples create coherent but weak directions that Muon promotes relative to Stochastic Gradient Descent (SGD). Experiments with Vision Transformers on CIFAR-100 support this prediction. Muon matches AdamW on clean data but underperforms SGD under label corruption, with a larger gap under asymmetric than symmetric noise. These findings motivate a new direction for optimizer design that retains the speed of spectral equalization while preserving the robustness of amplitude-sensitive learning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Muon, momentum orthogonalization, Noisy data, Noisy labels

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_BufWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the robustness cost of Muon's spectral equalization, motivated by the fact that flattening singular values accelerates weak modes regardless of whether they correspond to useful signal or noise. It analyzes how spectral equalization affects update rank, signal and noise update energy, mode wise learning rates, and drift near clean optima, and connects asymmetric label corruption to coherent weak spectral modes that are promoted by Muon relative to SGD. Experiments under label corruption are used to support the analysis.

Critical strengths

Label noise analysis. The paper connects spectral equalization to asymmetric label corruption by showing that corrupted subpopulations can form coherent weak spectral modes, and derives how Muon increases their relative update energy compared with SGD.

Experimental support. The experiments compare asymmetric and symmetric label corruption and show a larger robustness gap between Muon and SGD under asymmetric corruption, which is consistent with the analysis.

Critical weaknesses

Motivation. The motivation is not new. Prior work already shows that spectral equalization speeds up weak modes, removes the simplicity bias of SGD, and can promote spurious or tail features [1,2,3]. This paper mainly treats weak modes as noise and studies this under label corruption.

Theory. Several results put the conclusion into the assumptions. Theorem 3.2 assumes G=Gs+GnG=G_s+G_n, orthogonal signal and noise singular subspaces, and a spectral gap, so the amplification result follows almost directly. The issue is not the decomposition itself, but the assumption that signal and noise are already separate singular modes of the full gradient. Under a clean versus corrupted or conditional expectation decomposition, they do not need to remain separate singular modes, so the α2\alpha^2 result does not generally follow. Lemma 3.3 assumes different mode wise rates for amplitude sensitive dynamics and equal rates for equalized dynamics, then derives the learning time gap. Section 4 derives corrupted label components, but still assumes that they remain singular modes of the full gradient, with additional orthogonality assumptions for deep linear and ReLU networks. Overall, several results mainly restate properties already included in the assumptions.

Method novelty. SUMO is incremental. It applies Muon in a top singular subspace and uses the original gradient outside it. This is close to Subspace Momentum [4], which applies momentum in a low dimensional subspace and SGD to the orthogonal residual, and GaLore [5], which uses leading singular directions to build optimization subspaces. The main difference is applying Muon orthogonalization inside the selected subspace.

Experiments. The experiments show that SGD is more robust than Muon under synthetic label corruption, but do not verify the proposed mechanism. The paper does not show that corrupted examples form the weak singular subspaces assumed by the theory. SGD is also more robust under white label noise, so the results do not isolate the coherent noise explanation. The SUMO comparison also changes step sizes, momentum treatment, and orthogonalization, so it does not isolate the effect of restricting spectral equalization.

References

[1] Sara Dragutinović, Yedi Zhang, and Rajesh Ranganath. To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters. arXiv preprint, 2026.
[2] Bhavya Vasudeva, Puneesh Deora, Yize Zhao, Vatsal Sharan, and Christos Thrampoulidis. How Muon’s Spectral Design Benefits Generalization: A Study on Imbalanced Data. ICLR, 2026.
[3] Shuche Wang, Fengzhuo Zhang, Jiaxiang Li, Cunxiao Du, Chao Du, Tianyu Pang, Zhuoran Yang, Mingyi Hong, and Vincent Y. F. Tan. Muon Outperforms Adam in Tail-End Associative Memory Learning. ICLR, 2026.
[4] Thien Hang Nguyen and Huy Nguyen. Lean and Mean Adaptive Optimization via Subset-Norm and Subspace-Momentum with Convergence Guarantees. ICML, 2025.
[5] Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, and Yuandong Tian. GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection. ICML, 2024.

Questions for authors
  1. Section 4 assumes that corrupted examples come from a different feature subpopulation, but the CIFAR and MNIST experiments randomly flip labels within each class. The corrupted and clean examples therefore have the same feature distribution in expectation. Why should this setting produce the separated weak mode assumed in Section 4?
  2. The theory assumes exact spectral equalization, while the experiments use finite Newton Schulz iterations on the momentum matrix. Does the α2\alpha^2 amplification result still hold for the actual Muon update? If not, what amplification should we expect?
  3. Theorem 3.1 concerns a single update at fixed Frobenius norm, but the experiment uses final weight stable rank to support a higher rank bias. Muon moves about 9 times farther from initialization than SGD. Does the stable rank gap remain when displacement or path length is matched?
  4. The author list for Muon Optimizes Under Spectral Norm Constraints seems to be incorrect; it should be cited as Chen et al. (Lizhang Chen, Jonathan Li, and Qiang Liu).
  5. The submission seems to be missing the required AI use statement.
4 comments
AuthorsOpenReview ↗

Response to Reviewer BufW

Thank you for the detailed review, and for recognizing that the paper connects spectral equalization to asymmetric label corruption, showing that corrupted subpopulations can form coherent weak spectral modes whose relative update energy Muon increases compared with SGD, and that our experiments show a larger Muon–SGD robustness gap under asymmetric than symmetric corruption, consistent with this analysis. We quantify how spectral equalization redistributes update energy under label corruption and test how it affects memorization speed. The paper does not propose an optimizer. SUMO (NeurIPS 2025) is an existing method, not ours,and not a contribution of this paper, so its novelty relative to subspace methods does not bear on our contributions. We will add the citation, together with Subspace Momentum and GaLore.

New evidence. We added three controls. On matrices with known orthogonal signal and noise modes, both finite NS variants increased the noise-to-signal energy ratio in all 80 configurations (Q2). In 18 ViT-B/16 runs on CIFAR-100, five-step NS fit corrupted labels sooner in every noisy seed pair at matched update norms (W3). On the actual momentum matrices of ViT-B/16 training runs, five-step NS raised the corrupted-label share of update energy early in training in all 8 run/noise/source combinations we measured (W1, Q2).

W1. Theory and assumptions. We agree that a coherent label-dependent component need not be a separate singular mode of the full gradient. Theorem 3.2 assumes orthogonal signal and noise singular subspaces and a spectral gap. Section 4 gives sufficient conditions for this setting but retains assumptions on representation geometry and mode preservation in the full gradient (Corollaries 4.7 and 4.9; Remark 4.10). It does not prove that arbitrary deep networks satisfy these conditions. We will narrow the statements that suggest otherwise.

To measure the decomposition directly, we computed G_obs, G_clean, and their difference for all 48 hidden matrices of ViT-B/16 (CIFAR-100, 30% asymmetric and symmetric noise). We also tracked the corrupted-label part of the momentum along the training trajectory. Early in training (5% of steps, near the validation-selected checkpoint), the results were as follows.

Five-step NS raised the corrupted-label share of update energy in all 8 run/noise/source combinations (App. C.2, Fig. 4):

Run (trajectory, noise)Momentum: raw → NS5 → exact SVDGradient: raw → NS5 → exact SVD
Muon, asymmetric0.40 → 0.49 → 0.480.32 → 0.53 → 0.53
Muon, symmetric0.50 → 0.59 → 0.59−0.20 → 0.61 → 0.61
SGD, asymmetric0.20 → 0.31 → 0.310.11 → 0.32 → 0.32
SGD, symmetric0.07 → 0.44 → 0.430.12 → 0.49 → 0.49

The corrupted-label energy was 2.5–30× less concentrated than the clean energy in the leading singular directions, which hold half of the observed gradient's energy (App. C.3, Fig. 5):

Fraction of energy in leading directionsMuon, asym.Muon, sym.SGD, asym.SGD, sym.
Clean component0.190.300.380.20
Corrupted-label component0.050.120.010.06

This supports the energy-redistribution mechanism without relying on exact subspace separation.

W2. Motivation. The manuscript already cites Dragutinović et al. (2026) (lines 71, 130, 215). We now discuss explicitly in Section 2 their analysis of Muon’s loss of simplicity bias and its behavior with spurious features, and how our energy-ratio analysis complements it. Our analysis quantifies how much spectral equalization changes the noise-to-signal energy ratio under an explicit decomposition. Theorem 3.2 gives an α² lower bound for exact equalization under its assumptions. The ViT control tests how practical NS changes memorization speed.

W3. Experiments and SUMO. SUMO is not our contribution (so its limited novelty shouldn’t affect the assessment of our paper), and neither the abstract nor the contributions (lines 95–107) claim it. It is an existing optimizer that orthogonalizes the momentum only within a top-k gradient subspace and applies the remaining directions at their gradient amplitudes. We include it because it varies the quantity our analysis studies, namely how much of the spectrum is equalized. SGD equalizes no directions, SUMO equalizes the top k, and Muon equalizes all of them. SUMO also illustrates the direction our conclusion motivates: keeping equalization while preserving amplitude information. The comparison is exploratory. Appendix B.5 details how tuning, step size, momentum, and orthogonalization also differ between SUMO and Muon. We will state in Section 5 that SUMO is an existing baseline.

To test the spectral transform itself, we ran a controlled ViT-B/16 comparison at matched update norms. Five-step NS reached 75% accuracy on the assigned labels of corrupted examples at epoch 6 vs. 9 under asymmetric noise and at epoch 9 vs. 11 under symmetric noise, in every seed pair.

AuthorsOpenReview ↗

Response to Reviewer BufW continue

Q1. Feature distribution under label noise. This is a valid distinction. Section 4 assumes subpopulation-dependent corruption, whereas the CIFAR-100 and MNIST experiments use class-conditional random flips. Before training, the clean and corrupted subsets of a source class have the same feature distribution in expectation. Comparing corrupted examples with clean examples of the destination class does not establish the separation assumed in Section 4. Training may separate their representations, but we have not measured this. We will distinguish the two corruption models and qualify their connection.

Q2. Newton–Schulz vs. exact equalization. The α² lower bound applies to exact equalization. For finite NS, the output noise-to-signal energy ratio is the sum of squared transformed noise singular values divided by the corresponding sum for signal. At a gap of 4, exact SVD amplified this ratio by 16×, five-step NS by 40.60×, and twenty-step NS by 6.29×. The quintic iterates oscillate: the weak modes have larger output singular values than the strong modes after five steps and smaller ones after twenty. On actual MNIST momentum inputs, twenty steps still left 22–23% relative Frobenius error to the SVD polar factor in the second hidden layer. The implemented polynomial maps 1 to 0.701, so it cannot converge to the exact polar factor for a nonzero input.

Controlled amplification of the noise-to-signal energy ratio, with 64 orthogonal signal and 64 noise modes (App. C.7, Table 8, Fig. 9):

Gap αExact SVDNS5NS20
24.00×5.40×1.76×
416.00×40.60×6.29×
864.00×110.38×33.12×
16256.00×312.45×300.49×
12816,384×7,325×19,231×

For a K-step iteration the amplification is A_K = α² [f_K(1/α) / f_K(1)]², where f_K is the iteration's scalar map. This matches the measured values to within 0.013%.

The ViT-B/16 momentum matrices show the same picture (App. C.6, Fig. 8):

TransformRel. error to polar factorCorrupted-label share of update energy
Raw—0.397
NS50.370.486
NS200.220.471
Exact SVD00.480

Despite their numerical error, the finite iterations raised the corrupted-label share almost exactly as much as exact SVD did. The amplification is therefore not an artifact of exact equalization.

We will correct the numerical-exactness wording in Appendix B.2.

Q3. Displacement vs. rank. We matched the norm of each matrix update, and five-step NS still fit corrupted labels sooner (W3). Table 2 measures final weight stable rank. Muon retained 72% of its initial stable rank, and SGD retained 17%, with a 9.3-fold difference in displacement. We do not yet know whether the rank difference persists at matched displacement. We will present Table 2 as an empirical observation. Theorem 3.1 concerns the spectrum of one update.

Q4. Citation fix. We will correct the citation to Chen, Li, and Liu, "Muon Optimizes Under Spectral Norm Constraints." Thank you for catching this.

Q5. AI use statement. Thank you for flagging this. The page-one footnote pointed reviewers to ICLR's Reviewer Guidelines and AI Policy for Reviewers, and we will remove it from the revised manuscript. We reported our AI use in the submission form and omitted the dedicated author disclosure from the manuscript. We will add it and explain how we checked the work.

Reviewer_BufWOpenReview ↗

Thank you for your rebuttal

Weaknesses

1. Theory: Overly Strong Orthogonality Assumption

Partially resolved. C.1–C.3 decompose clean and corruption gradients and show that corruption energy is more concentrated in weak spectral directions. However, they do not establish that the two singular subspaces are orthogonal during actual training, nor do they relax the assumptions of Theorem 3.2.

2. Experiments: Insufficient Mechanistic Validation

Partially resolved. The authors add C.1–C.8, directly measuring corruption energy, noisy-label memorization speed, and learning order. However, causal intervention experiments that link spectral amplification to the observed generalization gap are still missing. Moreover, why the robustness gap between SGD and Muon is larger under asymmetric than under symmetric noise remains insufficiently explained.

Questions

Q1: Why Do Random Label Flips Produce Weak Spectral Directions?

Partially addressed. C.3 shows that corruption energy is more concentrated in weak spectral directions, but does not establish the orthogonality and spectral gap required by the theory.

Q4: Incorrect Reference Author List

Unresolved. I could not find any source, including Google Scholar, that lists the authors in the incorrect order. The authors did not explain how this error occurred.

Q5: Missing AI Use Statement

Unresolved. The AI use statement is still missing from the revised manuscript.

Related ICLR Papers

For the authors' reference, I list some related papers accepted at ICLR:

  1. Why Is SAM Robust to Label Noise? ICLR 2024.
  2. Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks. ICLR 2024.
  3. Does SGD Really Happen in Tiny Subspaces? ICLR 2025.
  4. How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data. ICLR 2026.
  5. Muon Outperforms Adam in Tail-End Associative Memory Learning. ICLR 2026.

Current Assessment

Compared with papers accepted at previous ICLR conferences in closely related areas, the experimental scale and quality of this paper fall far short of the level demonstrated by comparable accepted papers.

Therefore, I maintain my rating. In my view, acceptance would require substantial revision. The paper is not ready for acceptance in its current form.

AuthorsOpenReview ↗

Revised Manuscript Uploaded (with Rebuttal Edits & New Appendix Experiments)

Thank you for following up. We’ve just uploaded the revised manuscript, which incorporates the main-paper edits detailed in our rebuttal along with additional experiments in the appendix. We hope these updates address your comments and that you’ll take them into account in your final assessment.

Reviewer_wqFHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how Muon's spectral equalization may affect generalization and noise robustness. The analysis shows that orthogonalizing momentum can weaken the singular-value hierarchy and increase the relative contribution of weak noise directions. Experiments with asymmetric label corruption on CIFAR-100 and MNIST show behavior consistent with this mechanism.

Critical strengths

A unified theoretical framework connects spectral equalization to weakened low-rank implicit bias, relative noise amplification, and non-vanishing updates near clean optima.

Critical weaknesses

The analysis relies on strong orthogonality and spectral-separation assumptions that remain unverified in realistic networks.

Questions for authors

With other optimization factors held fixed, can you get similar conclusion?

1 comment
AuthorsOpenReview ↗

Response to Reviewer wqFH

We thank the reviewer for recognizing the unified theoretical framework connecting spectral equalization to weakened low-rank implicit bias, relative noise amplification, and non-vanishing updates near clean optima, and for asking for a comparison that holds the other optimization choices fixed. We ran this control.

W1. Controlled comparison. We fine-tuned ViT-B/16 with TorchVision ImageNet-1K weights on CIFAR-100 using clean labels, 30% asymmetric noise, and 30% symmetric noise. We ran three paired seeds per condition (18 runs). The norm-matched momentum and Muon arms shared Nesterov momentum, initialization, minibatch order, label flips, the auxiliary optimizer, gradient clipping and decay. We chose one per-layer update norm schedule on held-out clean validation (peak update RMS 10⁻⁵). The arms differed only in whether we applied five-step NS before rescaling.

W2. Results. In every noisy seed pair, Muon reached 75% accuracy on the assigned labels sooner: epoch 6 vs. 9 for asymmetric noise and 9 vs. 11 for symmetric noise. Final clean-test accuracy improved by 1.65 ± 0.70 and 1.92 ± 0.16 percentage points, respectively (mean ± sample SD of the three paired differences). With checkpoints selected on clean validation, the gains were 2.80 and 0.89 points. The final gap on clean training labels was 0.12 points. We will report clean accuracy by protocol and limit the conclusion of this ablation to faster memorization at matched update norms.

W3. Evidence for the mechanism. The control shows that applying NS can speed up fitting of corrupted labels. To apply Theorem 3.2 to real gradients, we would still need to show that the signal and noise occupy the assumed singular subspaces. We will make that distinction explicit. Independently of that assumption, we measured the decomposition on real ViT-B/16 gradients and momenta. Early in training, five-step NS raised the corrupted-label share of update energy in all 8 run/noise/source combinations (App. C.2, Fig. 4; table in our reply to BufW, W1).

Reviewer_gqmtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies the cost of Muon's orthogonalized update UV⊤UV^\top. It claims 4 consequences, all in Section 3:

  1. The update has maximal nuclear norm for its Frobenius norm.
  2. Weak "noise" directions get more relative update energy than under SGD.
  3. Strong and weak modes are learned at the same rate, so early stopping no longer helps.
  4. Muon keeps moving near a clean optimum.

Section 4 argues that asymmetric label noise creates such a weak noise direction. Experiments on CIFAR-100 (ViT-B/16 and a small ViT), MNIST, and a synthetic two-mode teacher are presented as support.

Critical strengths
  1. Muon is widely used, and it is worth asking what is lost when the gradient's singular values are discarded.
  2. The paper has reported experimental results as an attempt to support its claims.
Critical weaknesses

The theory in this paper is trivial and carries no novelty. It also does not support the claims built on it.

Thm 3.1 is the standard inequality ∣X∣∗≤r ∣X∣F\\|X\\|_* \le \sqrt{r}\,\\|X\\|_F, which follows from Cauchy–Schwarz on the singular values, with equality when they are all equal. It says nothing new about Muon. It also does not imply a "high-rank bias", since GG and UV⊤UV^\top have the same rank.

Thm 3.2 reduces to

RMuonRSGD=mean of (σis)2mean of (σjn)2≥α2.\frac{R_{\rm Muon}}{R_{\rm SGD}} = \frac{\text{mean of } (\sigma^s_i)^2}{\text{mean of } (\sigma^n_j)^2} \ge \alpha^2.

This holds as soon as one assumes that the noise forms separate singular directions of the matrix being orthogonalized. That assumption carries all the content, and it is too restrictive.

Lemma 3.3 is on a simple linear ODE. It only compares the two modes within each flow. It never compares τamp\tau^{\rm amp} with τeq\tau^{\rm eq}, and it cannot, because cc and ρ\rho are free: changing their values can reverse the comparison.

Thm 3.4 uses only ∣UV⊤∣F=rank⁡\\|UV^\top\\|_F = \sqrt{\operatorname{rank}} and says little. What remains is the known fact that a normalized method with a fixed step size does not converge, which goes away with a decaying learning rate.

Therefore, all four results follow from elementary facts, and none of them fits the actual Muon setting well.

Questions for authors
  1. Thm 3.1, 3.2 and 3.4 follow from ∣X∣∗≤r∣X∣F\\|X\\|_*\le\sqrt r\\|X\\|_F and ∣UV⊤∣F=r\\|UV^\top\\|_F=\sqrt r. What do they show about Muon beyond these two facts?
  2. Lemma 3.3 models Muon with a step proportional to the residual, while Thm 3.4 says Muon's step does not depend on the residual. Which one describes Muon?
2 comments
AuthorsOpenReview ↗

Response to Reviewer gqmt

Thank you for the detailed review, and for noting that, given Muon’s wide adoption, it is worth asking what is lost when the gradient’s singular values are discarded; this is the question our paper addresses, and we have added experiments to support it. Theorems 3.1 and 3.4 use standard linear algebra to characterize the update spectrum and step magnitude under stated conditions. We quantify the change in noise-to-signal energy under a specified decomposition and test how the spectral transform affects learning from corrupted labels. At matched update norms, practical Muon fit corrupted labels sooner in every noisy ViT seed pair.

W1. Theorem 3.2 assumptions. The bound requires signal and noise to occupy separate singular subspaces of the matrix being equalized. Section 4 gives settings where this holds under additional assumptions. In particular, Corollaries 4.7 and 4.9 require the label-dependent components to remain singular components of the complete gradient. We will make this conditional scope explicit and separate it from the empirical claim that Muon fits corrupted labels faster. Separately, on real ViT-B/16 gradients and momenta, five-step NS raised the corrupted-label share of update energy early in training in all 8 run/noise/source combinations we measured, for example from 0.40 to 0.49 on Muon's own momentum under asymmetric noise. The corrupted-label energy was also 2.5–30× less concentrated than the clean energy in the leading singular directions (App. C.2–C.3, Figs. 4–5).

W2. Lemma 3.3. The lemma compares the ordering of hitting times within each assumed flow. It does not compare absolute convergence times across optimizers, since c and ρ are free. We will state that scope explicitly. More importantly, we assume an equal-rate residual flow rather than deriving it from general Muon dynamics. We will present it as a conditional two-mode calculation and remove claims that it proves the loss of an early-stopping window for Muon in general. In Appendix B.2, W = max(0, τweak − τstrong) can be zero when the weak mode is learned first. We will report the signed gap and distinguish reversed ordering from equal hitting times.

We have now computed the signed gap τweak − τstrong in the two-mode setting (10 seeds per cell; positive means the strong mode is learned first; App. C.5, Fig. 7):

ActivationαGDExact SVDNS5
ReLU2+1102−516−908
ReLU4+1724−1224−1861
ReLU8+2066−2217−3253
Linear2+488−363−712
Linear4+768−870−1361
Linear8+922−1582−2412

GD learned the strong mode first. The exact polar factor and five-step NS learned the weak mode first in every seed of all six configurations, and the reversal grew with α. The clipped statistic would have reported 0 in all of these cases.

Q1. Scope of the theoretical results. Theorem 3.1 identifies the maximally spread update spectrum at fixed Frobenius norm and active rank. The gradient and its polar factor have the same algebraic rank, so this does not prove a higher-rank final solution. Theorem 3.2 quantifies relative noise-energy amplification under the stated decomposition. Theorem 3.4 gives the local step norm η√l at a clean stationary point with a nonzero rank-l residual noise gradient. It does not prove an increase in clean loss or rule out convergence with a decaying learning rate. Remark A.2 already notes the latter limitation; we will bring it into the main text.

Q2. Lemma 3.3 vs. Theorem 3.4. The exact equalized update in Theorem 3.4 has a norm that is independent of the nonzero residual singular values. Lemma 3.3 instead assumes a residual-proportional flow. We have not derived that flow from the update or established a transition between the two descriptions. We will remove the implication that they describe two proven phases of Muon training and retain Lemma 3.3 only as a conditional illustration of rate separation.

Q3. Ethics and AI disclosure. The page-one footnote pointed reviewers to ICLR's Reviewer Guidelines and AI Policy for Reviewers, and we will remove it from the revised manuscript. We reported our AI use in the submission form and omitted the dedicated author disclosure from the manuscript. We will add it and explain how we checked the work.

AuthorsOpenReview ↗

Response to Reviewer gqmt continue

Q4. Practical NS. We tested five-step and twenty-step NS on matrices with known orthogonal signal and noise modes. At a gap of 4, exact SVD gave 16× amplification, five-step NS gave 40.60×, and twenty-step NS gave 6.29×. The amplification persists for practical NS in all 80 tested configurations, but its magnitude is not bounded below by α² (App. C.7, Table 8, Fig. 9). On real ViT-B/16 momentum matrices, the finite iterations raised the corrupted-label share of update energy almost exactly as much as exact SVD (App. C.6, Fig. 8):

TransformRel. error to polar factorCorrupted-label share of update energy
Raw—0.397
NS50.370.486
NS200.220.471
Exact SVD00.480

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.