- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The manuscript is so confusing that summarizing it thoroughly would require a significant effort. Therefore, I will restrict my summary to the part I tried to understand: motivated by the recent success of Muon, this work aims to explain the implicit bias of Muon through the analysis of the dynamics of a projected polar flow under a linear regression model.
Critical strengths
None
Critical weaknesses
The manuscript is almost unreadable to me, and its main claim seems either tautological or insufficiently supported, as elaborated below.
-
The first motivation for this study comes from the success of the Muon optimizer. In the second sentence of the manuscript, it is defined as the polar factor of the gradient. This does not match the definition given by Jordan et al. (2024), which use momentum and the Newton–Schulz approximation. This is particularly confusing since, in Section 3, one refers to a momentum buffer that is not defined. It is difficult to assess claims about algorithms that are not defined consistently.
-
The projected polar flow seems to be a new algorithm introduced in this work. It is unclear why this algorithm is relevant to understanding Muon's properties. One motivation to replace the gradient with the weights is the SF condition under which . But we have , so under this condition the projected polar flow is constant... Also, if the goal is to study what interpolating solution is reached by Muon on a regression problem, using at each step a projection on the interpolating step considerably changes the problem.
-
The motivations of this work are very confusing. In the introduction, to explain the problem they are addressing, the authors wrote
Muon has become a practical optimizer for transformer training (Liu et al., 2025), but empirical acceleration alone does not explain what the polar map selects. Two natural explanations are tempting. One is that Muon is just matrix-gradient normalization; another is that spectral-norm geometry should lead to nuclear-norm, hence low-rank, implicit bias. We show that both are incomplete. Muon is better understood as a flat-update-spectrum method: it keeps the gradient singular directions while allocating equal update amplitude across them.
Where do these "natural" explanations come from? Are they coming from the authors or previous work? Also, it is unclear what matrix normalization refers to, and why this alone would explain the interpolator solution reached by Muon (should one guess that it would be the same as GF since "matrix normalization" would only induce a reparametrization of GF? But Muon is clearly not a time-reparametrized version of GF...). The second explanation is also unclear to me. Why should Muon reach a minimal nuclear norm interpolant? This would be an implicit bias theorem and not trivial. It is unclear to me how the previous statement explains anything. More importantly, the new interpretation provided by the authors: "Muon is better understood as a flat-update-spectrum method" is essentially tautological and not related to the previous problem of characterizing the reached interpolant.
- The first stated contribution is "A one-step spectral-bias theorem. Under explicit alignment assumptions, we prove that the polar profile maximizes first-order spectral-entropy gain among bounded updates that follow the gradient singular directions and ignore the current weight spectrum". I didn't find any corresponding result in the main text.
Questions for authors
According to Theorem 1, i) cannot increase; isn't it in contradiction with the left part of Figure 2, where and seem to increase?