acceptodds
Under review as a conference paper at ICLR 2027

ChromaMuon: Spectral Optimizer Diagnostics and Design with Retained Newton–Schulz Coordinates

Abstract

Practical Muon optimizers approximate a polar-like update with a short Newton–Schulz recurrence. Intermediate iterates preserve singular directions while applying distinct spectral responses. Building on prior iterate mixtures, ChromaMuon uses these responses as coordinates for optimizer diagnostics and design. We characterize coefficient-level controls for fitted maps, signed edits, and time-varying paths, and relate each response to its signal-relative contribution profile . For gradient inputs, this profile gives first-order modal descent up to scale. Matched language-model experiments show that finite-coordinate fits to bounded weak-tail targets worsen mean loss in the tested family, while static negative corrections have mean effects that depend on the base map. Under native cooldown, checkpoint continuations show a shrinking local advantage of deeper geometry; transient-edit probes show early loss gains followed by endpoint penalties. These observations motivate two representative paths that retain strong matched deep-control performance. On Modded-NanoGPT Track 3, their complete configurations first reach evaluated mean loss below 3.28 before the tested Muon-family references. Transferred to NanoChat without coefficient or schedule retuning, each configuration finishes below Tuned NS12 at all 40 paired endpoints across three separately analyzed token-budget protocols, spanning 537M–2.8B parameters. ChromaMuon connects spectral characterization, matched diagnostics, and complete-path evaluation for practical optimizer design.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.