acceptodds
Under review as a conference paper at ICLR 2027

Spectral Momentum Steering Mitigates Optimizer Mismatch of Muon Finetuning

Abstract

Muon has emerged as a compelling alternative to Adam for training matrix-valued parameters and has been adopted in the pretraining of frontier language models. Its use in post-training, however, is hindered by optimizer mismatch: fine-tuning an Adam-pretrained checkpoint with Muon can yield worse performance than fine-tuning with Adam itself. Because most publicly available checkpoints are pretrained with Adam, resolving this mismatch would enable broader application of Muon to post-training. Analyzing how pretrained singular directions align with layer representations, we find that pretrained weights read from input directions associated with spectral outliers beyond both edges of the Marchenko–Pastur (MP) distribution, yet write predominantly into the subspace spanned by the principal left singular vectors. Building on this, we propose Spectral Momentum Steering (SMS), which performs steepest descent under a plasticity-weighted anisotropic norm constraint that limits movement along the principal output directions encoding foundational representation structure. Across Adam-pretrained model families, SMS consistently improves pretrained capability preservation while matching or exceeding the downstream adaptation performance of matched Adam or standard Muon fine-tuning, substantially alleviating optimizer mismatch.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Post training, Finetuning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_KvJEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies optimizer mismatch when Muon is used to fine-tune AdamW-pretrained language models. It proposes Spectral Momentum Steering (SMS), which uses the pretrained left singular basis to restrict updates and reduce plasticity along directions with large singular values. The update is derived from an anisotropic spectral-norm constraint and retains Muon's Newton-Schulz implementation. Experiments on Qwen3-4B-Base and Llama3.1-8B-Base show substantial improvements in capability preservation with comparable or better math adaptation. Additional experiments examine code adaptation, full-weight fine-tuning, and controls for update magnitude and spectral ordering.

Critical strengths
  1. The preservation gains are substantial and practically relevant. Llama also benefits, particularly in knowledge retrieval and reading comprehension. These results support SMS as a promising way to use Muon with existing AdamW-pretrained checkpoints.
  2. The method has a clear optimization formulation. Deriving the update from an anisotropic spectral-norm constraint makes the role of directional plasticity explicit. SMS retains Muon's Newton-Schulz implementation and avoids requiring calibration data for its construction. This provides an interpretable optimizer-level approach that can be applied to both LoRA and full-weight fine-tuning.
  3. The ablations address important alternative explanations. Appendix B.1 separates subspace restriction from spectral steering, B.2 compares approximately matched update magnitudes, and B.3 reverses the plasticity ordering. Together, these controls support a contribution from update geometry. The code experiment in B.5 also provides useful evidence that the benefits extend beyond math adaptation.
Critical weaknesses
  1. The mechanistic interpretation remains partly a hypothesis. In Section 3.1, output variance already contains a squared singular-value factor, so high variance alone does not identify knowledge-critical directions. Figure 6 measures changes in sorted singular values rather than rotation of the principal output subspace. The inverse-profile control supports the spectral ordering, but more direct representation measurements would strengthen the claim about preserving optimizer-specific foundational geometry.
  2. The comparisons leave some uncertainty about the best achievable preservation-adaptation trade-off. The Muon learning-rate sweep and matched-magnitude control in Appendices B.1-B.2 are reassuring. However, the forgetting-aware baselines are evaluated only with Muon. Their poor performance therefore does not establish superiority over their standard AdamW implementations. A comparison at similar adaptation accuracy with one representative baseline would strengthen the broader claim.
  3. The average preservation score can obscure substantial forgetting in individual capabilities. For Llama in Table 1, SMS achieves Avg1 of 101.30, although knowledge retrieval and reading comprehension retain only 61.42% and 49.42% of their base scores. Improvements in instruction following compensate for these losses in the average. The authors acknowledge ratios above 100%, but the overall preservation claims should more clearly distinguish these improvements from actual retention.
Questions for authors
  1. Can the authors report principal left-singular-subspace overlap or activation drift for a few representative layers, or qualify the mechanistic interpretation more explicitly? This would help distinguish spectral preservation from preservation of knowledge-critical representations.
  2. Can the authors summarize preservation at comparable adaptation accuracy and clarify whether the baseline ranking holds with standard AdamW implementations?
Reviewer_shdNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SMS for fine-tuning AdamW-pretrained models with Muon. It uses the pretrained weight spectrum to reduce updates along the principal output singular directions. This gives the update PPolar⁡(PM)P\operatorname{Polar}(PM), where PP assigns less plasticity to directions with larger pretrained singular values. In the LoRA experiments, SMS is applied to the output factor.

On Qwen3-4B and Llama3.1-8B, SMS substantially reduces forgetting after math fine-tuning, with a smaller improvement in math accuracy. The appendix adds code adaptation, full-weight fine-tuning, and controls for update magnitude, column-space restriction, and the ordering of the plasticity weights.

Critical strengths
  1. The retention gains are large. For Qwen3-4B, Avg1 improves from 78.22 with AdamW and 67.96 with Muon to 98.20 with SMS, while math accuracy also improves slightly. The main results average three seeds, and the additional code experiment is useful evidence beyond the main math setting.

  2. The appendix addresses several obvious alternative explanations. In particular, the benefit survives approximate matching of global update norms, and reversing the plasticity ordering removes most of it. Table 5 also separates the column-space restriction from the spectral weighting. These controls support the importance of the update geometry.

Critical weaknesses
  1. The evidence does not isolate an optimizer-mismatch mechanism. Protecting principal pretrained directions is already motivated by MiLoRA and OPLoRA. The identity oi=σi2iio_i=\sigma_i^2 i_i partly explains output-variance concentration through linear scaling; it does not locate knowledge. The inverse-plasticity control supports the ordering of protection, but remains consistent with general anti-forgetting regularization. Comparable steering with AdamW and SMS on the Muon-pretrained checkpoint are not tested. Changes in sorted singular values also do not establish preservation of the original directions.

  2. The practical comparison omits AdamW combined with existing preservation methods. MiLoRA, CorDA, LoRA-Null, and OPLoRA are evaluated with Muon, while SMS improves adaptation over AdamW alone by only 0.82 and 0.70 points. The relevant retention–adaptation comparison therefore remains incomplete. Avg1 also obscures capability losses: Llama scores 101.30 overall despite retaining only 61.42% on knowledge retrieval and 49.42% on reading. This average does not establish broad preservation of foundational capabilities.

Questions for authors
  1. Does the same spectral protection help AdamW, and does SMS help the Muon-pretrained checkpoint? These comparisons would distinguish an optimizer-mismatch remedy from general protection of pretrained weights.

  2. How does SMS compare with AdamW plus existing preservation methods under comparable tuning budgets? Please report per-capability retention–adaptation tradeoffs and clarify whether matched update norms are measured on effective weights or adapter factors.

Reviewer_8s1wOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies forgetting when Muon is used to fine-tune AdamW-pretrained language models. Spectral Momentum Steering (SMS) defines an anisotropic update constraint using the pretrained weight's left singular vectors and downweights dominant singular directions. The resulting update applies the plasticity operator before and after momentum orthogonalization. LoRA experiments on Qwen3-4B and Llama-3.1-8B show improved preservation with competitive math adaptation; the appendices add matched-update controls, spectral ablations, a Muon-pretrained checkpoint comparison and code adaptation.

Critical strengths
  • The method has a clear constrained-descent interpretation and a practical LoRA implementation. Preservation gains are large on the main models, with adaptation maintained or improved.
  • The appendices address important alternative explanations: subspace-only and reversed-profile ablations, matched update magnitudes, learning-rate sweeps and code adaptation. Together these provide meaningful evidence that the spectral direction of the update matters.
Critical weaknesses
  • Table 1's Avg1 can conceal substantial forgetting. For Llama, SMS reaches 101.30 overall while knowledge and reading retain only 61.42% and 49.42%; instruction-following gains offset these losses. The headline preservation claim therefore depends strongly on normalization and aggregation.
  • The central importance proxy in Section 3 remains partly heuristic. Output variance satisfies o_k = sigma_k^2 i_k, so its association with singular values does not itself establish which directions store indispensable capabilities. The fixed, monotone plasticity profile and hard output-subspace restriction need stronger functional justification, particularly when adaptation requires changing dominant or previously unused directions.
Questions for authors
  • Would Table 1's conclusion hold with capped retention ratios, worst-category retention and absolute per-benchmark score changes?
  • Beyond output variance, what evidence links a direction's singular value to the capability loss caused by perturbing it?
  • Does Figure 2's read/write pattern hold across attention and MLP projections, layers and both model families?
  • How much useful target-task gradient is excluded by the hard output-subspace constraint in Eq. 12? When could this restriction hinder learning new features?
  • What practical rule selects gamma for a new task, and how sensitive are the gains to this choice when dominant directions must change?
  • Can Table 1's Llama comparison use independently validation-tuned learning rates for each optimizer and compare preservation at matched adaptation accuracy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.