- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The authors propose MSIS, which improves the initial noise to enable diffusion models to generate better images with very few denoising steps. The core approach is to precompute the mean of 1% of the training images, select an intermediate noise starting point, add the scaled mean to newly sampled random noise, and use the original model to complete the remaining denoising steps, thereby accelerating training. The approach is validated on the ImageNet dataset using the SIT model.
Critical strengths
- The paper is clearly written and easy to follow.
- The experiments are reasonably thorough and cover several datasets and samplers.
Critical weaknesses
- The text-to-image evaluation is narrow: Stable Diffusion 2.1 on MS-COCO alone is not enough. Does the method still help FLUX.2, Z-Image, or Qwen-Image on complex prompts?
- The method may sacrifice diversity by discarding data covariance. For a VP path, the true intermediate covariance is (a_t^2I+b_t^2\Sigma), whereas MSIS uses (I). Since , MSIS underestimates variance by along any data-covariance eigenvector with eigenvalue . Under a locally linear denoising map , define output diversity as the expected squared distance between independent outputs. Then D_{\mathrm{true}}-D_{\mathrm{MSIS}}\ \=2b_t^2\operatorname{tr}!\left[J(\Sigma-I)J^\top\right].\ This is positive when the retained directions are dominated by data variance above one. Thus, diversity loss is a concrete risk, although not an unconditional result for nonlinear denoising. The full-covariance variant also improves recall from 0.503 to 0.536. Why do the authors not directly test diversity across seeds for the same prompt?
- The choice of z over a_{t_s}z is not tested. The authors argue that scaling the noise causes oversmoothing, but provide no direct comparison. This is a central design choice, not a detail that can be justified by intuition alone.
- Lower FID does not establish better handling of complex prompts, and average CLIP Score offers limited evidence. Does MSIS improve composition, text rendering, and instruction following on Qwen-Image-Bench (Text-to-Image)?
- The experiments leave out models already trained for few-step generation. Does MSIS offer any additional benefit on DMD/LCM-distilled models? Its applicability to MeanFlow and GAN-based generators is also unclear. Without these results, the practical value of the acceleration claim is difficult to judge.
Questions for authors
No
10 comments
Response to Q1 & Q4-2: Generalization to Z-Image and Fine-Grained Text-to-Image Evaluation
Detailed Results and Analysis
(a) Aesthetics: Visual Composition and Aesthetic Quality
The detailed results for Aesthetics are presented below.
Table 2: Detailed comparison of Aesthetics subcategories.
| Subcategory | Raw | Ours | Paired Δ |
|---|---|---|---|
| Aesthetics (L1) | 46.29 | 46.97 | +0.67 |
| Composition | 46.06 | 44.55 | -1.52 |
| Color Harmony | 60.61 | 60.71 | +0.10 |
| Lighting | 52.99 | 53.81 | +0.81 |
| Anatomical Portraiture | 6.59 | 5.29 | -0.99 |
| Emotional Expression | 48.31 | 49.39 | +0.88 |
| Style Control | 40.62 | 44.90 | +4.17 |
(b) Creative Generation: Text Rendering and Creative Generation Capabilities
The detailed results for Creative Generation are presented below.
Table 3: Detailed comparison of Creative Generation subcategories.
| Subcategory | Raw | Ours | Paired Δ |
|---|---|---|---|
| Creative Generation (L1) | 39.06 | 39.87 | +0.80 |
| Imagination | 57.69 | 60.00 | +2.61 |
| Feature Matching | 43.93 | 44.55 | +0.43 |
| Logical Resolution | 37.41 | 34.04 | -1.03 |
| Text Rendering | 29.70 | 30.27 | +0.57 |
| Design Applications | 42.16 | 41.98 | -0.35 |
| Visual Storytelling | 55.86 | 55.54 | -0.25 |
(c) Alignment: Instruction Following and Prompt-Image Consistency
The detailed results for Alignment are presented below.
Table 4: Detailed comparison of Alignment subcategories.
| Subcategory | Raw | Ours | Paired Δ |
|---|---|---|---|
| Alignment (L1) | 49.75 | 49.94 | +0.19 |
| Attributes | 56.59 | 56.20 | -0.39 |
| Actions | 36.10 | 36.11 | +0.98 |
| Layout | 50.42 | 52.04 | +1.47 |
| Relations | 40.44 | 41.90 | +1.46 |
| Scene | 60.78 | 59.78 | -1.01 |
Across the three dimensions, we observe positive changes in several fine-grained capabilities relevant to the reviewer's concerns. In particular, Text Rendering exhibits a positive change of +0.57, while Layout and Relations show improvements of +1.47 and +1.46, respectively. Meanwhile, Composition exhibits a decrease of -1.52, and some other subcategories also show negative changes. Nevertheless, all three L1 dimensions exhibit positive paired differences, indicating that MSIS does not lead to an overall degradation in these evaluated capabilities.
Discussion
We emphasize that MSIS is not designed to directly enhance composition, text rendering, or instruction following. Its primary objective is to reduce discretization errors introduced by large sampling steps in few-step generation. Since these fine-grained capabilities can also be affected by discretization errors, MSIS may indirectly alleviate their degradation by reducing such errors.
Our experimental results are consistent with this expectation. Under the same 10-step sampling budget, all three evaluated L1 dimensions exhibit positive paired differences. Furthermore, Text Rendering, Layout, and Relations show positive changes, suggesting that MSIS can help preserve these fine-grained capabilities and mitigate potential degradation in certain aspects. Although some individual subcategories still exhibit decreases, the overall results do not indicate systematic degradation caused by MSIS.
We acknowledge that, due to the limited computational budget during the rebuttal period, our additional evaluation covers only 500 of the 1,000 official prompts and focuses on three L1 dimensions most relevant to the reviewer's concerns. Further evaluations on models such as FLUX.2 and Qwen-Image, as well as a comprehensive evaluation across all benchmark dimensions, would be valuable future work. We will incorporate these additional results and clarify the corresponding experimental settings and limitations in the revised manuscript.
We thank the reviewer again for these valuable suggestions, which have helped us further validate the applicability of MSIS across different text-to-image models and better understand its effects on complex text-to-image generation.
Note: Paired Δ is computed by averaging per-prompt score differences over prompts with valid scores under both configurations. Since the Raw and Ours scores may be aggregated over different sets of valid prompts, Paired Δ may differ from the difference between the two reported average scores.
Response to Q2-1: Potential Impact of Covariance Approximation on Generation Diversity
Thank you for raising the concern that approximating the intermediate covariance with an isotropic Gaussian may sacrifice generation diversity. We address this concern from both theoretical and empirical perspectives.
Theoretical Analysis: Covariance Mismatch Does Not Necessarily Imply Reduced Output Diversity
We first clarify that an initialization-level covariance mismatch does not necessarily translate into reduced diversity in the final generated images.
Consider the VP diffusion process:
\qquad \epsilon\sim\mathcal N(0,I),$$ where $x_0$ and $x_1$ are independent, and the data distribution has covariance $\Sigma=\operatorname{Cov}(x_1)$. The covariance of the true intermediate distribution is $$\Sigma_t^{\mathrm{true}} =a_t^2I+b_t^2\Sigma.$$ In contrast, MSIS uses an isotropic initialization with covariance $$\Sigma_t^{\mathrm{MSIS}}=I.$$ Using the VP condition $a_t^2+b_t^2=1$, their covariance difference is $$\Delta\Sigma_t =b_t^2(\Sigma-I).$$ Therefore, MSIS indeed underestimates the initialization variance along data-covariance eigenvectors with eigenvalues greater than one, while overestimating it along directions with eigenvalues smaller than one. However, this covariance mismatch alone does not establish that the final generated images necessarily exhibit reduced diversity. To understand this distinction, consider a local linear approximation of the denoising mapping: $$F(x)\approx Jx+d,$$ where $J$ denotes the local Jacobian of the denoising mapping. Defining output diversity as the expected squared distance between two independent outputs, we have $$D=\mathbb E\left[ \|F(x^{(1)})-F(x^{(2)})\|_2^2 \right] =2\operatorname{tr}(J\Sigma_xJ^\top).$$ Consequently, the difference in output diversity between the true intermediate distribution and MSIS is approximately $$\begin{aligned} D_{\mathrm{true}}-D_{\mathrm{MSIS}} &=2\operatorname{tr} \left[J(\Sigma_t^{\mathrm{true}}-I)J^\top\right]\\ &=2b_t^2\operatorname{tr} \left[J(\Sigma-I)J^\top\right]\\ &=2b_t^2\sum_{i=1}^{d} (\lambda_i-1)\|Ju_i\|_2^2, \end{aligned}$$ where $\lambda_i$ and $u_i$ are the eigenvalues and corresponding eigenvectors of the data covariance matrix. This expression shows that the effect of covariance mismatch on output diversity depends not only on the covariance eigenvalues, but also on how the denoising network responds to the corresponding directions. In particular, even when some data-covariance eigenvalues exceed one, the resulting output-diversity difference is not necessarily positive. Therefore, although the reviewer correctly identifies a directional covariance mismatch at initialization, this does not imply that MSIS necessarily reduces the diversity of final generated images. ## Empirical Evidence: Covariance Ablations and Direct Prompt-wise Diversity Evaluation We further investigate this concern through two complementary sets of experiments. **(a) Recall Comparison across Gaussian, Mean, and Mean + FC Initialization.** First, we revisit our covariance ablation experiments on conditional ImageNet 256 × 256 using DDIM at NFE = 6, 8, and 10. We compare three configurations: the standard Gaussian baseline, mean-based initialization (Mean), and mean initialization with full covariance matching (Mean + FC). For both Mean and Mean + FC, we use a training-data fraction of $\rho=0.01$. The complete comparison is presented below. |NFE|Initialization|FID ↓|sFID ↓|IS ↑|Precision ↑|Recall ↑| | -----| ----------------| --------: | --------: | ---------: | -------------: | ----------: | |6|Baseline|20.3680|23.5155|127.7730|0.5895|0.4330| |6|Mean|9.10|6.76|180.33|0.718|0.503| |6|Mean + FC|7.98|8.09|179.60|0.712|**0.536**| |8|Baseline|10.0229|14.3662|179.8244|0.7087|0.4642| |8|Mean|5.56|5.68|211.15|0.770|0.517| |8|Mean + FC|5.09|6.79|209.56|0.760|**0.545**| |10|Baseline|6.5124|11.3127|209.7852|0.7631|0.4784| |10|Mean|4.22|5.57|227.75|0.796|0.520| |10|Mean + FC|4.11|6.78|224.92|0.785|**0.542**| The results reveal a consistent trend across all three sampling budgets. Introducing mean-based initialization substantially improves recall over the Gaussian baseline. Furthermore, incorporating full covariance information provides additional recall improvements over mean-only initialization. Specifically, recall increases from 0.4330 to 0.503 and further to 0.536 at NFE = 6. At NFE = 8, it increases from 0.4642 to 0.517 and further to 0.545. Similarly, at NFE = 10, it increases from 0.4784 to 0.520 and further to 0.542. These results provide empirical evidence that incorporating data-dependent statistics into the initialization does not necessarily sacrifice generation coverage. In particular, full-covariance matching can further improve recall over mean-only initialization in our evaluated settings.
Response to Q1 & Q4-1: Generalization to Z-Image and Fine-Grained Text-to-Image Evaluation
Thank you for raising these important concerns regarding the generalization of MSIS to other large-scale text-to-image models and its impact on generation quality under complex prompts.
First, we would like to clarify the primary objective of MSIS. Our method is not designed to directly enhance fine-grained generation capabilities such as composition, text rendering, or instruction following. Instead, its main objective is to reduce the discretization errors introduced by excessively large sampling steps in few-step generation. Since these fine-grained capabilities can also be affected by discretization errors, reducing such errors may indirectly alleviate the degradation of these capabilities in few-step generation, or at least avoid introducing additional degradation due to the initialization strategy.
To further investigate this effect and address the reviewer's concerns about cross-model generalization, we conduct additional experiments using Z-Image on Qwen-Image-Bench. This allows us to evaluate MSIS on another large-scale text-to-image model and examine its effects on fine-grained generation capabilities.
Experimental Setup
Qwen-Image-Bench contains 1,000 prompts and provides a hierarchical evaluation framework covering various aspects of text-to-image generation. However, both Z-Image generation and benchmark evaluation involve substantial computational costs. Given the limited computational budget and hardware resources available during the rebuttal period, we adopt a reduced evaluation protocol.
Specifically, we randomly select 500 prompts from the official 1,000-prompt benchmark and focus on three L1 dimensions most relevant to the reviewer's concerns:
Regarding MSIS initialization, since the original training dataset of Z-Image is not available to us for computing the global data mean, we randomly select an additional set of 100 prompts, which are completely disjoint from the 500 evaluation prompts. We reverse-sample these 100 prompts to the 0.98 point and use the resulting samples to estimate the mean for MSIS initialization.
We compare the original Z-Image initialization (Raw) with our MSIS initialization (Ours). Both configurations use exactly 10 sampling steps, ensuring that the sampling budget remains identical and that the only difference is the initialization strategy. All other generation settings follow the corresponding official Z-Image configurations and are kept consistent between the two configurations.
For evaluation, we follow the Qwen-Image-Bench evaluation protocol, aggregating scores hierarchically from L3 evaluation facets to L2 subcategories and then to L1 dimensions. N/A entries are excluded from score aggregation.
Overall Results
The overall results for the three selected L1 dimensions are summarized below.
Table 1: Overall comparison between Raw and Ours on Qwen-Image-Bench.
The results show positive changes across all three evaluated L1 dimensions. Specifically, Aesthetics, Creative Generation, and Alignment exhibit paired improvements of 0.67, 0.80, and 0.19 points, respectively.
These results provide further evidence of the cross-model generalization of MSIS to Z-Image. More importantly, under the same 10-step sampling budget, introducing MSIS does not degrade the overall scores of these three dimensions. Instead, all three exhibit positive changes, suggesting that MSIS can preserve overall generation quality in complex text-to-image tasks.
To further investigate the effects on the fine-grained capabilities raised by the reviewer, we examine the corresponding L2 subcategories.