ChromaMuon: Spectral Optimizer Diagnostics and Design with Retained Newton–Schulz Coordinates
Abstract
Practical Muon optimizers approximate a polar-like update with a short Newton–Schulz recurrence. Intermediate iterates preserve singular directions while applying distinct spectral responses. Building on prior iterate mixtures, ChromaMuon uses these responses as coordinates for optimizer diagnostics and design. We characterize coefficient-level controls for fitted maps, signed edits, and time-varying paths, and relate each response to its signal-relative contribution profile . For gradient inputs, this profile gives first-order modal descent up to scale. Matched language-model experiments show that finite-coordinate fits to bounded weak-tail targets worsen mean loss in the tested family, while static negative corrections have mean effects that depend on the base map. Under native cooldown, checkpoint continuations show a shrinking local advantage of deeper geometry; transient-edit probes show early loss gains followed by endpoint penalties. These observations motivate two representative paths that retain strong matched deep-control performance. On Modded-NanoGPT Track 3, their complete configurations first reach evaluated mean loss below 3.28 before the tested Muon-family references. Transferred to NanoChat without coefficient or schedule retuning, each configuration finishes below Tuned NS12 at all 40 paired endpoints across three separately analyzed token-budget protocols, spanning 537M–2.8B parameters. ChromaMuon connects spectral characterization, matched diagnostics, and complete-path evaluation for practical optimizer design.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.