acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR

Abstract

Muon (MomentUm Orthogonalized by Newton–Schulz) is a matrix-aware optimizer that leverages Newton–Schulz (NS) iterations to enforce spectral gradient orthogonalization by driving all singular values of the momentum matrix toward 1. While this uniform spectral whitening enhances exploration and outperforms AdamW in LLM pretraining, we show it could lead to fundamental limitations beyond pretraining in two increasingly important regimes: (i) cross-modality vision-language-action (VLA) training, where inherently low-rank action-module gradients cause amplification of noisy tail directions, and (ii) reinforcement learning with verifiable rewards (RLVR), where low-SNR gradients and the need to preserve per-head specialization inherited from prior training make whitening unstable. To address these challenges, we propose Pion (sPectral hIgh-pass Optimization on momeNtum), a drop-in replacement for Muon that preserves its computational efficiency while replacing uniform spectral whitening with a two-stage Promotion+Suppression mechanism, which we call the high-pass NS iteration. This design induces a sharp spectral high-pass effect, anchoring dominant singular values at 1 while suppressing noisy tail components toward 0, with controllable filter strength. To preserve pretrained per-head heterogeneity, Pion also supports a per-head mode that applies updates independently across attention heads via a simple reshape, at no extra cost. Extensive experiments demonstrate consistent gains over Muon and AdamW across both VLA and RLVR regimes. In VLA training on LIBERO and LIBERO-Plus, Pion consistently outperforms both baselines across ℓ₁-regression (VLA-Adapter) and flow-matching (VLANExt) architectures, e.g., reaching 100% success rate on LIBERO Object after 1,500 training steps with VLA-Adapter, vs. 97.0% for Muon and only 32.2% for AdamW. The advantage of Pion further extends to a real Franka Research 3 robot with a π₀.₅ backbone under the DROID setup on three grasp-and-place tasks. In RLVR post-training on Qwen3-1.7B/4B with GRPO and GMPO, Pion also outperforms AdamW on MATH and GSM8K while Muon collapses to zero.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Muon, Vision Language Action Model, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MyNdOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Pion, a spectral high-pass optimizer designed to address Muon’s limitations in VLA and RLVR post-training. It replaces uniform spectral whitening with two-stage promotion–suppression iterations that preserve dominant singular directions while attenuating noisy tails, and adds per-head processing for RLVR. Experiments show improved VLA performance with a module-specific hybrid optimizer configuration and stable RLVR training in settings where Muon collapses.

Critical strengths
  1. Useful ablations. Module-wise optimizer comparisons, per-head experiments, learning-rate sweeps, and reverse-filtering ablations provide evidence supporting the proposed design.
  2. Physical validation. The VLA experiments cover regression and flow-matching action models, along with a real-robot evaluation.
Critical weaknesses
  1. No clear criterion for choosing Pion. The paper identifies low effective rank and low SNR as relevant characteristics, but does not establish a validated rule for selecting Pion over Muon or AdamW. Neither condition alone guarantees that small singular directions contain noise rather than useful information. The directional-SNR analysis supports this assumption in one RLVR setting, but its generality remains unclear.

  2. RLVR validation is limited. Experiments use only Qwen3-1.7B and Qwen3-4B,and two mathematical reasoning datasets. The author should provide more results on other benchmark and model familes.

  3. The method’s scope is insufficiently characterized. VLA imitation learning and language-model RLVR differ substantially in objectives, supervision, architectures, and optimization dynamics. A shared spectral explanation is plausible, but success in these two settings does not yet establish a predictive characterization of where Pion should work. The paper needs stronger evidence that the proposed spectral conditions predict gains across tasks and models.

Questions for authors

See weakness above.

Reviewer_B15xOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Muon was widely adopted in LLM training such as Kimi model training. This paper argues that the proposed Pion model can address the limitations of the Muon and can perform well in VLA as well as RLVR training.

The core hypothesis are that Muon has the limitations. Specifically in VLA training, Muon is not working well due to that the action module often has low-rank structure in gradients, Muon would amplify the weak gradient direction and introduce noise. In RLVR post training, because policy gradient has low signal noise ratio (SNR), Muon can amplify the noise and make the training unstable.

Pion is a better optimizer that replace the Muon's uniform whitening with two-stages Promotion and Suppression. This change creates a spectral high-pass filter that preserve the domainant singular directions while suppressing weaker directions.

In experiments on the popular VLA datasets LIBERO and LIBERO-plus, the Pion outpeforms AdamW and Muon on both VLA-Adapter and Flow-matching architectures.

In RLVR experiments, training with Pion with Qwen3-1.7B/4B and GRPO and GMPO are doing better on MATH and GSMK8K datasets than AdamW and Muon.

Critical strengths

This paper is well motivated and it has successfully identified the core weakness in Muon optimization. The proposed Pion optimizer seems to be a good alternatives to address those limitations.

The paper provides a clean two set of experiments on VLA and RLVR. There are real-robot experiments as well as the additional ablation studies.

Critical weaknesses

The first critical concern i have is that i am not sure if weak directions are always noise? The core idea in Pion is that we keep strong directions and reduces weak directions. The paper needs to show more convincing evidences. I think VLA and RL VR examples are good, but they are not conclusive.

The 2nd concern is that the paper are missing out some of the alternative approaches, for example, can we add 3 baselines. One is to have SGD baseline. We need to run RL with SGD (Nesterov and weight decay). If we still want to use Muon, we should have alternatives. One is to do variance-aware dampening, this is to integrate a secondary state to track gradient variance and selectively dampen orthogonal updates when signal-to-noise ratio drops. The other is to do Lazy Muon, e.g. reduce Newton-Schulz iteration intensity to act more like standard SGD.

The 3rd minor issue is that the paper has a claim that does not have strong support. That is the small singular-value directions mostly contain noise. But low rank does not mean that all small singular directions are uninformative. There are no experiments on those.

Questions for authors
  1. Can we have more fair experiment setting. For example, let's say we can fix the optimizer config for VLA, and change the action module with only Muon or Pion.

  2. Can we provide direction evidences to show that the suppressed small singular-value directions are mostly noise? What happen if those directions still have useful information, or gradient has high rank and SNR, would Pion still work?

Reviewer_xjahOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors investigate how the Muon optimizer behaves beyond language model pretraining, focusing on multimodal training of VLA models and RL post-training. They observe that conventional Muon is not adaptive to rank heterogeneity across modules of VLA models, resulting in suboptimal performance. They also observe that Muon's whitening operator amplifies noisy directions in low-SNR gradients during RL post-training, leading to performance degradation. To mitigate these issues, they propose Pion (and its per-head mode) by splitting the NS iterations into a two-stage Promotion + Suppression sequence and provide experimental results on VLA and RLVR setups.

Critical strengths
  1. The motivation is clear, and the paper is well-organized overall. I find that understanding Muon's advantages beyond pretraining is an important direction that has been overlooked in previous literature.
  2. They provide a unified understanding of two pitfalls of the Muon optimizer. The distinction between the informative head and the noisy tail is insightful and easy to follow.
  3. The proposed Pion optimizer is based on a simple modification of the polynomial used in the NS iterations. In contrast to previous works, it provides a principled and scalable alternative for improving performance in VLA and RLVR setups.
Critical weaknesses

None

Questions for authors
  1. Can you suggest some theoretical analyses or toy experiments to support the claim that the Muon optimizer performs suboptimally when the gradient is relatively low-rank and the remaining directions represent noise? I think the explanation in the paper is intuitive enough, but some toy results or experiments could further support the claim.
  2. Shumaylov et al. (2026) reported that how we transform the gradient's singular value spectrum does not necessarily affect the performance of spectral optimizers, although their work focuses only on small-scale experiments and pretraining. Why do you think their approach worked well in their setting?
  3. There is some controversy surrounding the use of Muon for RL post-training. One prominent view is that a mismatch between the optimizers used during pretraining and post-training can cause the post-training dynamics to collapse [Qu et al., 2026; Liu et al., 2026]. Since this paper uses the Qwen3 family of models, which were pretrained using the AdamW optimizer, the collapse observed with Muon might be affected by this mismatch. Do you have any experimental results on RLVR setups using other language models pretrained with Muon (e.g., Moonlight)?

References

  • Shumaylov et al., Muon is Not That Special: Random or Inverted Spectra Work Just as Well, 2026.
  • Qu et al., Can Muon Fine-tune Adam-Pretrained Models?, ICML 2026.
  • Liu et al., Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less, 2026.

Comments on the paper

1 comment

Thank you for addressing my questions. I will keep my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.