acceptodds
Under review as a conference paper at ICLR 2027

Is Muon Geometrically Unique? Orthonormal Rank-One Systems for Normalized Optimizers

Abstract

Recent normalized optimizers such as Lion and Muon highlight the importance of geometry in modern optimizer design. Yet it remains unclear whether Muon's two-sided singular-vector orthogonality is essential to its effectiveness or instead represents one realization of a broader geometric structure. We extend the Lion- perspective by representing the momentum variable in a possibly time-varying orthonormal rank-one system and defining as the corresponding coefficient-wise -type potential. Different selection rules over orthonormal rank-one systems recover SGD, Lion, and Muon within a single geometric framework. Within this framework, one-sided orthogonality defines a broader family that contains Muon's two-sided orthogonality as a structured special case. To probe whether this additional two-sided structure is necessary, we introduce Random Orthogonal Basis Normalization (ROBN) and Split Polar Update (SPU) as realizations of one-sided orthogonality. ROBN shows that one-sided orthogonality alone can retain a meaningful portion of Muon's empirical performance, while SPU shows that a realization based on one-sided orthogonality can match Muon across both vision and language modeling. These results suggest that Muon is best understood as a distinguished realization of two-sided orthogonality within the broader geometric principle of one-sided orthogonality, rather than as a uniquely privileged geometric structure.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.