acceptodds
Under review as a conference paper at ICLR 2027

Beyond Gaussianity: From Source Properties to Muon for Noise Optimization

Abstract

Reward-guided initial-noise optimization seeks to maximize a learned reward while keeping the latent within a region that a pretrained generator maps to high-quality samples. Existing methods approximate this feasible region through regularization or projection toward Gaussian source properties. However, we find that pretrained generators tolerate substantial deviations from Gaussian entry-wise distributions while still producing realistic images across three recent models. We partly explain this tolerance through exact agreement of first- and second-order statistics after the first embedding. At the same time, controlled rearrangements can destroy or restore generation quality despite preserving the same entry values and token norms. These observations motivate a weaker spectral description of the admissible source space that is independent of the entry-wise distribution. Rather than explicitly enforcing Gaussianity, we control how reward-driven updates perturb this spectral structure. Remarkably, this control emerges simply by using Muon, whose momentum orthogonalization flattens the update spectrum. Experiments on aesthetic and text-aligned generation show that Muon reaches higher target rewards while retaining held-out scores, without an additional Gaussianity penalty or latent projection.

Primary area:
generative models
Keywords:
Muon, Noise Optimization, Gaussianity

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_qkujOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the properties of initial noise in generative models and challenges common assumption that Gaussian property must be preserved with non-Gaussian initial noise example. The paper further demonstrates that having coherent spatial structure in initial noise degrades output quality. In initial noise optimization algorithms, the paper connects these observations to the existing Muon optimizer, using momentum orthogonalization to improve reward–quality trade-offs in initial-noise optimization.

Critical strengths
  1. I find the investigation into non-Gaussian initial noise particularly interesting. The observation that generative models can tolerate substantial deviations from Gaussian distributions while maintaining generation quality is insightful.
  2. The experiments showing that different spatial arrangements of the same noise values can lead to dramatically different generation quality are also interesting.
Critical weaknesses
  1. While the analysis of spatial structures in initial noise is interesting, the connection between these observations and the choice of Muon remains somewhat indirect. It is unclear whether this theoretical motivation fully explains the observed improvements in generation quality
  2. Although the paper provides examples of structured momentum emerging during optimization, it remains unclear how consistently these harmful patterns arise across different prompts, reward models, and optimizers, and to what extent their suppression explains Muon's performance improvements.
Questions for authors
  1. How consistently do harmful spatial patterns emerge during reward-guided optimization across different prompts, reward models, and optimizers?
  2. Is mitigating reward hacking one of the intended goals of using Muon for initial noise optimization? If so, are there direct empirical results demonstrating that Muon reduces reward hacking compared with existing methods? If not, do the authors expect Muon to be effective against reward hacking, and why? It would be interesting to investigate whether suppressing harmful spatial structures also improves robustness against reward-model exploitation, like maybe comparing with DNO (PRNO)
3 comments

Thank you for your thoughtful review. Harmful momentum patterns consistently arise in our experiments, motivating our investigation of source properties and the resulting Muon update.

[W1] From source observations to Muon

We clarify below the connection between our source observations and the resulting Muon update.

Generators tolerate broad non-Gaussian entry-wise distributions (Section 3.1) but can fail under coherent arrangements of the same entry values (Section 3.2). Singular spectra capture forms of concentration without requiring Gaussian marginals (Section 4.1). To prevent a few singular components from dominating each update, we flatten momentum's singular values while preserving its singular vectors (Section 4.2; Appendix E explains why we process momentum instead of the latent). This yields Muon's orthogonalized update (Section 4.3).

If there is a particular step in this connection that the reviewer still finds indirect, we would greatly appreciate any clarification so that we can address the concern more precisely.

[W2, Q1] Consistent harmful momentum patterns

To answer this question, we added Appendix G with six pages of randomly sampled examples across two generators and three rewards, including a VLM-based reward. The examples were selected entirely at random, with no cherry-picking. Using the observed momentum as initial noise severely degrades generation under the same prompts. In our experiments, further random sampling has consistently revealed the same harmful momentum patterns. We would be happy to provide additional randomly sampled examples if the reviewer would find them helpful. These observations motivate suppressing such structures in the momentum, as illustrated by the singular-value flattening in Figure 7.

[Q2] Reward hacking and existing methods

Yes. Mitigating reward hacking, understood here as increasing the target reward at the expense of image quality, is an explicit goal of our work. Our submission compares against MPGR, What Is Noise, and GPER, which all address harmful latent structure and were introduced after DNO/PRNO (Tang et al., ICML 2025). In particular, MPGR and GPER also report improved reward–quality trade-offs over PRNO in their evaluations.

We additionally provide the suggested PRNO comparison on FLUX.1/Aesthetic (Figure 8, Table 11). PRNO's best tested Aesthetic mean is 6.786, whereas Muon reaches 7.986 with comparable or higher means on all three held-out rewards. Adding PRNO leaves the conclusion unchanged: Muon achieves a better reward–quality trade-off in this comparison.

Reviewer_qkujOpenReview ↗

Thank you

Thank you very much for the detailed response, additional experiments, and revisions.

I find the non-Gaussian source analysis interesting, and I think the newly added Appendix E provides further evidence that momentum is an important component to investigate in noise optimization. The additional experiments also help strengthen the motivation for the proposed approach. Overall, I appreciate the authors' efforts in addressing my concerns, and I will raise my score.

Additionally, I have one minor comment regarding Appendix G. I find the experiment of using momentum directly as initial noise somewhat confusing, since momentum is not used as the initial noise during the actual optimization process. That said, I understand the authors' intention to isolate and visualize the potentially harmful structure in momentum. Perhaps explicitly clarifying the diagnostic purpose would make its results easier to interpret?

Thank you very much for your encouraging follow-up and helpful suggestion. We would like to address your remaining concern with the new examples in Figures 26 and 27.

Our intention in using normalized momentum as initial noise was to show that these vectors carry structure that is harmful to generation when used as a source. As you rightly point out, this diagnostic alone does not show how such patterns affect generation when momentum enters through small updates during optimization. We have made this diagnostic purpose explicit in Appendix G.

We therefore added Figure 26 to show this process directly. We would particularly appreciate your attention to the yellow-highlighted latent and incoming rows. The two runs use the same model, prompt, and initial latent. In this comparison, Muon is obtained by adding momentum orthogonalization to SGDM while keeping all other optimization settings, including update RMS, fixed.

Under SGDM, strong patterns in the incoming vectors visibly accumulate in the latent as the generated sailboat loses realism. Muon's raw momentum also contains strong patterns, but orthogonalization substantially weakens them in the incoming vectors. Correspondingly, its latent shows very little visible coherent structure, and the generated sailboat remains realistic throughout the displayed trajectory. This comparison illustrates how suppressing incoming patterns before accumulation helps preserve image quality during actual optimization.

Figure 27 extends this view across the full FLUX.1/Aesthetic experiment set. The largest singular value and low-frequency energy fraction capture complementary aspects of concentration, reflecting different features of the underlying patterns. Both increase in SGDM's latent, alongside substantial deterioration in all three held-out rewards. With Muon, concentration is reduced in the incoming direction before each update, and latent concentration stays close to its initial level. The target reward rises steadily while the mean held-out scores remain near or above their starting levels. These trajectories show the same pattern of reduced accumulation and better reward–quality behavior beyond the individual example in Figure 26.

Taken together, Figures 19 and 21 in Appendix E suggest that stronger suppression of momentum patterns tends to yield better reward–quality trade-offs in the tested settings. In particular, Figure 21 shows that flattening improves upon clipping, while Figure 19 provides an intuitive visual comparison of how these operators weaken patterns in the same momentum snapshot.

We are thankful for the time and care you have devoted to reviewing our work, and sincerely appreciate your thoughtful assessment. We hope these additional examples address your remaining concern and make the practical role of momentum processing clearer.

Reviewer_DamDOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the problem of optimizing the latent noise input of a diffusion or flow generative model to lead to realistic image outputs which maximize a reward function. Since directly maximizing the reward function leads to high reward for nonsense outputs, achieving this goal requires restricting the optimization to a region of the latent noise input space where the generator can produce realistic images. The paper begins by establishing that transformer diffusion models still generate realistic images for a broad range of independent entry-wise distributions whose first and second moments match the Gaussian, but cannot successfully perform generation for latent inputs that have significant correlated structures between different entries. This motivates the decision to perform reward optimization using gradient updates that have singular values which are nearly equal, since this should prevent any directional information from accumulating in the optimized noise. In particular, the Muon optimizer is used, since this optimizer by definition uses a weight update that approximately equalizes singular values in the update matrix. Experiments on recent diffusion and flow models show that the proposed method can significantly boost the reward of generated images on text-alignment and aesthetic reward while maintaining image quality across held-out metrics.

Critical strengths
  • The analysis of the compatibility of independent entry-wise non-Gaussian distributions for good generation and the failure of generation for coherent arrangements is quite interesting in its own right. Proposition 1 provides a nice theoretical explanation and the empirical results strongly support the claims made.
  • The use of the Muon optimizer naturally follows from the exposition in Section 3, and provides a simple and well-motivated method for handling the optimization in (1). The empirical results for reward-optimized noise in the experimental section show a clear benefit over baseline methods. Overall, this paper makes a strong and principled case for using Muon for reward-driven noise initialization.
Critical weaknesses
  • While not necessarily critical, it would be helpful to provide more visualized examples than the two presented in Figure 9 that compare the proposed method with others on reward-driven noise initialization. The held-out metrics provide good evidence that reward optimization with Muon maintains image quality, but at the end of the day qualitative assessment still is useful in this domain. The appendix is extensive and provides a lot of additional results, but does not contain further examples that similar to Figure 9, which are the most important type of qualitative evidence in for comparing the different methods.
Questions for authors
  • In the appendix, can you provide more qualitative comparisons that the two examples presented in Figure 9? Uncurated examples would be best.
1 comment

Thank you very much for your thoughtful and encouraging review. We sincerely appreciate your recognition of the contributions of our source analysis and the principled connection to Muon, as well as the empirical evidence supporting our approach.

Following your helpful suggestion, we have added further qualitative examples across three generators and two target rewards in Figure 25. These examples supplement Figure 9 and allow a broader visual comparison of Muon and the baseline methods. We are grateful for the time and care you devoted to our paper and for your suggestion to strengthen its qualitative evidence.

Comments on the paper

4 comments

Thank you for your careful review and constructive suggestions, particularly your close reading of Appendix E. We have clarified the connection from source properties to momentum processing and added direct evidence linking incoming spatial structure, latent accumulation, and generation quality.

[W1] Contribution and the Fourier-block result

Our contribution is a new source-based derivation and interpretation of Muon for reward-guided noise optimization. Section 3.1 establishes tolerance to broad entry-wise distributions, while Section 3.2 isolates sensitivity to arrangement. These observations motivate a statistic that captures structural concentration without requiring strict Gaussian marginals, leading to singular spectra in Section 4.1. Then, Section 4.2 applies this statistic to weaken incoming patterns by flattening momentum's singular values, yielding Muon's orthogonalized update (Section 4.3). This connection from source behavior to update design, supported by Appendix E's controlled comparisons of where and how to intervene, is the paper's central contribution.

We explicitly credit Muon's update, Muse's reshape, Chen et al.'s attention-distillation application, and GPER's spectral preconditioning. The Fourier-block advantage shows that the representation used for processing matters. Establishing how to choose the structural measure, representation, and processing strength remains future work, as discussed in Appendix E. Muon provides a simple practical instance of this perspective.

[W2, Q1] Using an incomplete structural statistic in optimization

The token-permutation experiment was designed to isolate the effect of arrangement while keeping the token values fixed. As you correctly note, it preserves the original token–feature singular spectrum. Figure 6 illustrates a different intervention: rearranging individual entries in 64×64 matrices, which can change the singular spectrum. These observations establish both the importance of arrangement and the incompleteness of singular spectra as source diagnostics. The arrangements encountered during actual optimization are examined separately below.

Appendix G.2 presents six pages of randomly sampled examples across two generators and three rewards, including a VLM-based reward. These examples consistently exhibit pronounced momentum patterns. Even when normalized to zero mean and unit variance, these vectors produce severely degraded images when used as the entire initial noise. This diagnostic illustrates the problematic structure that optimization can introduce into the latent.

Figure 26 follows its accumulation during actual updates. From the same Gaussian start and at the same update RMS, SGDM progressively accumulates visible latent structure as generation deteriorates, while Muon preserves a recognizable image. Despite similar Aesthetic gains of +1.72 and +1.89, PickScore changes by −4.77 for SGDM versus +0.13 for Muon. Preventing this separation between reward improvement and image quality is our objective.

Appendix E explains why constraining the latent's spectral profile is insufficient. Figures 17 and 18 show that flattened profiles can retain harmful patterns and yield artifacts. Figure 20 demonstrates this during optimization: the upper-row latent satisfies the spectral reference intervals yet produces an artifact, whereas Muon preserves a realistic image at a nearly identical Aesthetic Score despite exceeding those intervals. Spectral compliance therefore cannot guarantee a suitable source.

The question is consequently how to use this incomplete statistic effectively. Actual momentum carries strong patterns accompanied by spectral concentration, as shown in Figure 7. SV flattening preserves the singular directions but equalizes their weights. At fixed update RMS, this attenuates dominant components and can weaken visible patterns, as illustrated in Figures 7 and 26. Figure 26's incoming-vector rows show this directly: Muon's raw momentum remains structured, but the processed direction carries substantially weaker patterns into the latent.

Proposition 2 establishes that a flat spectrum minimizes a worst-case spectral-displacement bound for a fixed update norm and rank budget. The 2×2 example in Appendix D.3 explicitly illustrates why this need not minimize every realized displacement. We therefore evaluate the effects on spatial structure and generation quality directly.

Figure 27 quantifies the connection across all cases in our FLUX.1 Aesthetic optimization experiments, tracking the largest singular value, low-frequency DCT energy, and rewards at every step. Processing reduces momentum's time-averaged low-frequency energy share from 22.70% to 2.02%, with reductions in all Muon trajectories. The final latent share remains near initialization (1.325% → 1.333%), versus 1.810% for SGDM. Muon increases the target while retaining all three final held-out means above Base, whereas SGDM's held-out scores decline.

Figure 21 then compares where and how to intervene. Across the tested operators, momentum processing reaches higher target rewards than latent projection at comparable held-out PickScore values, and flattening improves over clipping. For example, at RMS 0.005, replacing momentum SV clipping with flattening improves Aesthetic from 7.668 to 7.986 and PickScore from 21.806 to 21.963.

Together, these results support flattening momentum's singular values to suppress incoming structure before accumulation as an effective use of an incomplete statistic. This operation coincides with Muon's update. Which aspects of structure to control and how strongly to control them remain open questions, as discussed in Appendix E.

[W3, Q2] Mean, variance, and token energy

We measured the initial and final values in our 100-update aesthetic score optimization experiments with FLUX.1-schnell:

MethodMaximum absolute entry meanMean entry varianceMean token-energy 99th percentile
Before optimization (shared)0.00300.99561.4585
SGDM0.02761.03361.5948
Muon0.01131.02801.5809

Although SV flattening can tighten worst-case bounds on changes in these statistics, SGDM and Muon differ only modestly in the measured values, yet produce strikingly different optimization results and image quality, as shown in Figures 26 and 27. The key distinction in these trajectories is the accumulation of harmful spatial patterns, which mean, variance, and token energy alone do not fully capture. We therefore focus on suppressing these patterns before they enter and accumulate in the latent.

[Q3] Regularization and RMS selection

We used published regularization coefficients. Appendix F.5 now tests five coefficients spanning a 100-fold range for MPGR and What Is Noise at fixed update RMS with three seeds for each reward. In Figure 23 and Tables 17 and 18, these sweeps do not close the gap to Muon. For example, the highest mean Aesthetic scores are 6.829 and 6.599, versus 7.986 for Muon at comparable held-out scores.

We also tested the reported learning rates and adapted update rules without fixed-RMS scaling, under the same 100-update budget (Table 19 and open markers in Figure 23). MPGR reaches PickScore 24.703, but ImageReward falls to −0.149, versus 1.202 for Muon at PickScore 23.777. The higher target obtained by MPGR comes with substantial held-out degradation.

For RMS, we sweep separately for each model, reward, and method. Our reported reward-range comparisons select the highest target mean for which all three held-out means remain at or above Base. The full curves show the available operating points and their trade-offs.

[Q4] Compositional accuracy and diversity

Following your suggestion, we have added Appendix F.6, which reports all eight T2I-CompBench++ metrics for images generated after PickScore optimization on FLUX.1. These metrics are held out of optimization. Trends vary across categories, and decreases also occur with other optimizers, consistent with imperfect alignment between PickScore and individual compositional criteria.

Furthermore, we have added Appendix F.7, which reports DreamSim and LPIPS pairwise distances and Vendi scores for optimized samples from all three generators. All four measures exceed Base at both reported Muon RMS settings, supporting preserved within-prompt diversity.

Thank you for the detailed response.

  1. The motivation for using Muon orthogonalization updates was explained in detail.
  2. The new trajectory analysis provides further support for the authors’ explanation of why Muon preserves image quality better than SGDM.
  3. A 100-step aesthetic reward optimization experiment on the FLUX.1-schnell dataset was added. The results show that mean, variance, and token energy alone do not fully capture harmful spatial patterns during optimization.
  4. The authors test a broad range of regularization coefficients and specify how RMS is selected. They also report compositional-accuracy and diversity evaluations. The diversity scores exceed the baseline, suggesting no clear loss of within-prompt diversity. These results address my main concerns, therefore I recommend upgrading its rating to clear acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.