- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper asks why orthogonalized momentum optimizers like Muon often collapse in GRPO post-training whereas the polynomial spectral-filter optimizer Pion stays stable. The authors trace part of this to a scale mismatch in public Pion RLVR code, which runs Muon at (25x the Moonlight scale and ~13x AdamW's step RMS). Proposition 1 proves that Pion's quintic map has a repelling fixed point at , so matrices with decay doubly exponentially toward zero as iterations grow (and empirically 47% of BF16 probes underflow at 5 steps), while surviving updates shrink to RMS. Matching per-matrix step RMS to AdamW (Eq. 3) gives flat polar updates a 1.4-3.0x LR headroom over 60 steps on 0.6B-1.7B models (4B is inconclusive) due to ~3x smaller input-aligned change . Yet matching alone still collapses as often or earlier than AdamW; only 1-step KL matching equalizes collapse.
Critical strengths
- Proposition 1 cleanly analyzes the quintic iteration around , and combining this asymptotic result with 5-step BF16 measurements explains why 47% of probe matrices zero out and surviving steps shrink by .
- The three-tier matching experiments (Frobenius RMS in Eq. 3, input-aligned , and 1-step policy KL across 32 seeds in Table 15) and pre-registered falsification checks (Table 37) are carefully designed.
Critical weaknesses
- The input-aligned metric fails across scales and horizons. In Sec. 5.4 (Table 15), matching alone collapses as often or earlier than AdamW on 0.6B, while on Qwen3-4B (Table 25) differs 4x between RMS-matched polar updates and AdamW without separating collapse LRs (and polar matched in is 3/3 stable at vs. AdamW's 2/3 collapse). Over 200 unpenalized steps (Table 13), all runs collapse and fails to order survival times; across 9 families (Tables 26, 28), has worse log-loss (0.71) than LR (0.68) or op-norm (0.66) and underpredicts flat-step KL by 1.09-1.88x.
- In Eq. (3), RMS-matched polar updates must track Adam's second moment to rescale by each step, and on 4B matching estimates input norms from 256 rows of micro-batch 1 (2 sequences, App. E.6, Table 11).
- Sec. 2 omits key orthogonal and RL optimizers.
Questions for authors
- Does the 4B mismatch (Table 25) persist if input norms use the full batch instead of 2 sequences in micro-batch 1?
- Could Sec. 2 discuss Learning from Streaming Video with Orthogonal Gradients, AdamO: A Collapse-Suppressed Optimizer for Offline RL, and the spectrum-preserving Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation?
Above all, my major concern is the stability metric breakdown, I'm open to discuss.
2 comments
Thank you for your detailed rebuttal. It has addressed all my concerns, so I will raise my score to 4.
Response to Reviewer bFYF
[Edited on 13 Nov 2026: updated to the final PDF; W1 rewritten; the 4B one-step KL result added.]
Thank you. Your analysis of a_t drove the main change in this revision.
Q1 first: 4B input norms. With 2048 rows spread over 64 of the 512 micro-batches, the AdamW/polar ratio of a/eta is 3.89, vs 3.93 from the first micro-batch.
W1: we agree; the robustness claim does not rest on a_t. Its cross-scale constant fails at 4B, and a_t fails for the public Pion filters, random rotations and the sign of the momentum. The one-step KL is not offered as a stability metric either, but as a control for one paired question: at equal policy change per step, does the flat direction collapse less than Adam's? The match can separate directions: under it the sign of the momentum (5/16, against our prediction) and the edge-truncated direction (0/8, pre-specified) collapse less often than AdamW (24/32 at 3e-6), while the flat step collapses in 26/32 in that cell. For those two directions we cannot tell robustness from policy change the KL misses. The paired test:
Is either yardstick reliable? a_t is not; no claim uses it as one. The one-step KL is the trust-region quantity (TRPO, PPO), used only as a paired control, and it does separate other directions. Its precision is limited: the closed- and open-loop protocols differ by 22 points (+8 vs +30), yet neither shows an advantage. Without it, at matched RMS the flat step's collapse curve is AdamW's shifted 2.2x with equal slope (a weak test).
W2: Eq. 3 needs Adam's second moment. Correct, and intentional. Eq. 3 is a measurement control that holds the per-matrix step RMS equal to AdamW's; it is not a proposed optimizer. It reuses AdamW's moments (peak memory 14.5 vs 14.6 GB on 0.6B). For practice: use the moment-free Moonlight scale (our kappa=0.2 arm) and tune the rate; the 1.4-2x starting rate in our conclusion applies to Eq. 3 only.
Q1 details. In four pre-specified 4B runs the two estimates agree within 1% (ratio 0.991-1.002); at 0.6B the small micro-batch inflates the ratio by under 10% (new Table 36).
W3 / Q2: related work. The revised Sec. 2 adds: