DAWN: Scalable Matrix-Whitening Optimization via Spectral Selectivity

Matrix-whitening optimizers such as Muon improve the data efficiency of language-model pretraining, but their full-spectrum polar updates are expensive and treat all singular directions as equally reliable. We argue that this may be unnecessary: across training, gradient matrices have a stable low-rank head and a noisy residual tail. Polarizing the tail amplifies noise to unit magnitude and wastes the most expensive part of the per-step compute. We propose DAWN (Decoupled Adaptive Whitening of Noisy gradients), which polarizes only the empirical head and applies a row-wise RMS update on the residual—a block-Schatten partition with one coarse spectral block on the head and singleton row blocks on the tail. DAWN reduces the per-layer whitening cost from $O(d^3)$ to $O(d^{2}k)$, matches Muon on GPT-2XL-scale (1.5B) pretraining, and lowers optimizer wall-clock time.