Rethinking Spectral Constraints in Muon: Beyond Norm Preservation

Training large neural networks requires optimizers that make effective progress at a practical computational cost. Muon controls the scale of its updates, but does not directly constrain the weights themselves. Spectral constraints address this gap by controlling the scale of weight matrices. However, how these constraints are enforced matters: keeping the norm unchanged does not necessarily preserve the model's ability to learn. We revisit spectral constraints in Muon through their effects on parameter updates and computational cost. We establish conditions for norm preservation and show how certain constraint designs can unnecessarily restrict the changes available to the model. Guided by this analysis, we propose two Muon-based optimizers. Spectral Block Correction (SBC) selectively corrects update blocks while preserving the spectral norm. Regularized Spectral Projection (RSP) shrinks trial weights and caps only singular values exceeding a prescribed bound. Under the reported single-seed configurations, SBC reduces final BERT-Large validation loss on Wikipedia by 14.3% versus AdamW. On Imagenette, SBC and RSP reduce final ViT-L/16 validation cross-entropy by 16.2% and 13.2%, respectively, versus Muon. SBC also reduces the mean cross-width learning-rate transfer penalty by 69.6% relative to Muon. These results highlight that spectral control must account for learning and per-step cost, not norm preservation alone.