- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Biax, a matrix optimizer that combines Muon2-Factorized-style temporal row/column second-moment profiling before polar direction extraction with Hyperball’s fixed-initial-Frobenius-radius parameter update. The paper provides an operator-level characterization of the method, including a profiled spectral linear-oracle interpretation and fixed-radius angular dynamics. Empirically, Biax is compared with Hyperball-Muon and Muon in 20B-token pretraining runs at 500M and 1B parameters using four paired seeds. At 500M, Biax obtains a small but statistically resolved improvement over Hyperball-Muon, although the two configurations use different inherited learning rates. At 1B, where the learning rate is matched, the comparison is effectively null. The paper appropriately frames these experiments as comparisons between complete optimizer configurations rather than as causal evidence for the profiling component.
Critical strengths
- Careful experimental accounting and appropriately calibrated claims. The paper clearly distinguishes matched and unmatched comparisons, reports paired-seed confidence intervals, discloses the learning-rate mismatch at 500M, and explicitly treats the 1B result as null. This substantially increases confidence in the claims that are actually made.
- Clean structural formulation of the optimizer. The decomposition of Biax into factored temporal profiling, spectral direction extraction, and fixed-radius projection is technically clear. The profiled-oracle and angular-dynamics viewpoints provide a useful conceptual description of how the combined optimizer behaves.
Critical weaknesses
- The experiments do not establish that the proposed profiling mechanism improves Hyperball-Muon. The only positive primary result is the 500M comparison, but Biax and Hyperball-Muon use different global learning rates, and this difference also propagates to the AdamW fallback parameters. At 1B, where the learning rates are matched, the difference is essentially zero. Consequently, there is currently no controlled empirical evidence that the defining profiling component of Biax provides an advantage. The authors themselves acknowledge this limitation.
- The methodological novelty appears incremental relative to the closest components. Biax is primarily an instantiation that combines Muon2-Factorized-style profiling with Hyperball-style radius control. Since the factored estimator, pre-polar adaptation, and fixed-radius projection are inherited ideas, the empirical burden for demonstrating that this particular composition is important is relatively high. The current scale-limited and mixed-result experiments do not yet meet that burden for me.
Questions for authors
- Clarification materially affecting my score: Can the authors report a learning-rate-matched 500M comparison between Biax and Hyperball-Muon, ideally with the same small learning-rate sweep for both methods? If the 500M advantage persists under this control, this would substantially strengthen the paper.
- Clarification materially affecting my score: Can the authors provide a direct mechanistic ablation that isolates the pre-polar profiling component, rather than comparing complete configurations? In particular, it would be useful to understand whether the null 1B result reflects scale, learning-rate sensitivity, or simply the absence of a profiling benefit.