acceptodds
Under review as a conference paper at ICLR 2027

After the Fact: Coupled Updates Shape the Implicit Bias of Muon

Abstract

When Muon updates one matrix layer, its motion changes gradients seen by surrounding parameters, so learning depends on how those signals are transformed downstream. We study this interaction through covariance, starting from Qwen3-0.6B Muon trajectories where similar NS5 update magnitudes can hide very different local responses, including negative elasticities. Persistent motion then turns these responses into curvature gradients. For stable cycles, we characterize when coupled spectral updates admit scalar curvature objectives, identify power compatibility, and show why cube-root updates remain aligned under decay. Under noisy training, Gaussian averaging can restore stability even when instantaneous responses are negative, while finite covariance fluctuations can change whether a scalar objective exists. Together, these results connect spectral response, persistent matrix motion, and optimizer coupling within one theory of curvature dynamics.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.