Same Function, Different Dynamics: Understanding Muon’s Response to Reparameterization

We study how function-preserving changes in parameter representation affect optimizer updates and subsequent predictions. Focusing on Muon in SwiGLU language models, we develop Libra, a state-resolved framework that connects reciprocal channel scaling to local predictive response. Under consistent optimizer-state transport, update homogeneity explains the response to uniform scaling. After scale compensation, exact-polar derivatives separate column-space motion from singular-frame rotation, while finite Newton–Schulz updates also change spectral amplitudes. We decompose the difference between these derivatives. To connect update changes to predictions, we propagate the original-rule update defect through the full parameter–optimizer state and use Jacobian/Fisher readouts to estimate predictive changes. Experiments on 106.57M-parameter language models find that off-diagonal response contributes 65.3–82.1% of the derivative-difference energy between five-step Newton–Schulz (NS5) and exact-polar updates. This range spans checkpoint groups in the tested random directions. In the larger-dose exploratory cohort, observable projection lowers median relative Kullback–Leibler (KL) error compared with calibrated scalar defect proxies for one-step responses with transported optimizer state. Measurements with retained optimizer state further reveal how memory and update responses reinforce or cancel each other. These measurements characterize local, short-horizon predictive responses under the specified optimizer-history conventions.