- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper asks whether the sign-vs-spectral choice can instead be turned into a measurable, per-layer, possibly stage-dependent decision. For this interesting question, the authors derive a finite-batch ratio of two conservative descent-lemma lower bounds and then propose a measurement-driven assignment method (G-Scion). For experiments, on 31M GPT-2/OWT, the output-head score routes from spectral to sign and G-Scion (embedding, output) lowers validation loss vs. fixed Scion; on ViT-B/16/CIFAR-100, a spectral-edge assignment beats fixed Scion by 7.9–11.9 accuracy points. In conclusion, notably, the paper is unusually candid that its evidence "supports a diagnostic framework … rather than a general claim of optimal routing or practical optimizer superiority."
Critical strengths
- Well-posed question. With Muon and mixed sign/spectral recipes now standard, "which geometry on which layer, decided by measurement rather than convention" is a genuinely important and underexplored optimization question.
- The distinction between a curvature-free proxy (which selects the wrong geometry) and a directional-curvature-aware score is the paper's best idea, and the controlled input-rank experiment (opposite selections in all seven settings) is a convincing isolation of that point.
Critical weaknesses
- The proposed method seems to lose to a trivial baseline. By the authors' own reporting, G-Scion's online gate does not beat the best static assignment and AdamW+Muon "remains a stronger practical reference." All reported gains are over fixed Scion only. Moreover, the comparison of experiments is also insufficient, and many key hyperparameter selections are not explained. For a paper whose headline contribution is a measurement-driven assignment method, being dominated by a plain always-spectral run is close to fatal.
- The theory does not bind to the method. The analytical ratio "does not certify the implemented gate at a threshold of one," its decisive regions ( or ) are never estimated in the LM experiments, and the routing score is explicitly a different quantity from . So the elegant signal-efficiency curvature factorization is essentially decorative with respect to the decisions G-Scion actually makes.
Questions for authors
Please see weaknesses.
- Could you report G-Scion against always-spectral (and always-sign) on the edge groups, at matched well-tuned setting? Since static spectral already beats the gate, what concrete setting (model, stage, layer) does the online gate win in?
- In the 31M LM runs, what are the measured values and how often do layers fall in the certification regions ( or ) vs. the indeterminate band ? Without this, how should a reader interpret the threshold-of-one gate?
- Could you provide at least one larger-scale run with complete directional records?