Beyond Gaussianity: From Source Properties to Muon for Noise Optimization

Reward-guided initial-noise optimization seeks to maximize a learned reward while keeping the latent within a region that a pretrained generator maps to high-quality samples. Existing methods approximate this feasible region through regularization or projection toward Gaussian source properties. However, we find that pretrained generators tolerate substantial deviations from Gaussian entry-wise distributions while still producing realistic images across three recent models. We partly explain this tolerance through exact agreement of first- and second-order statistics after the first embedding. At the same time, controlled rearrangements can destroy or restore generation quality despite preserving the same entry values and token norms. These observations motivate a weaker spectral description of the admissible source space that is independent of the entry-wise distribution. Rather than explicitly enforcing Gaussianity, we control how reward-driven updates perturb this spectral structure. Remarkably, this control emerges simply by using Muon, whose momentum orthogonalization flattens the update spectrum. Experiments on aesthetic and text-aligned generation show that Muon reaches higher target rewards while retaining held-out scores, without an additional Gaussianity penalty or latent projection.