- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
FIRMO augments Muon with offline, shape-specific Newton–Schulz coefficients and percentile-based clipping before orthogonalization. Evaluation includes 1B-model pretraining, nine-task evaluation after 100B tokens, component ablations, and a clipping-focused CIFAR-10 experiment.
Critical strengths
The 100B-token experiment provides meaningful end-to-end validation. Testing orthogonalization on gradients separate from the calibration sample also provides useful evidence beyond performance on the fitted spectra.
Critical weaknesses
Limited novelty without decisive comparisons. Improved NS polynomials and pre-orthogonalization magnitude control have direct precedents in CANS/Polar Express and AdaMuon, acknowledged in §2.2 but absent from the main comparisons. A global coefficient triplet refitted on the same calibration data is also missing. Consequently, the experiments do not establish whether shape conditioning and this particular combination offer advantages over existing alternatives. This is central to assessing the contribution, not merely a request for additional baselines.
Insufficient evidence for robustness and reliability. Section 4.5 selects the calibration mixture using larger-model stability results that are not presented, while the displayed experiment favors a different mixture. A central robustness argument therefore cannot be verified. The 0.53-percentage-point average accuracy improvement over Muon is also reported without uncertainty estimates, leaving its reliability unclear.
Questions for authors
Clarification—comparative evidence: Are existing results available against CANS/Polar Express with identical clipping, AdaMuon, or globally refitted coefficients that establish the value of FIRMO’s specific design?
Clarification—robustness evidence: What are the configurations and results of the larger-model stability experiments invoked in §4.5?