- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies joint initial-noise design for diffusion models. It proposes a low-cost repulsive coupling that introduces negative correlations among samples while preserving each noise sample’s standard Gaussian marginal distribution. The authors evaluate its effect on gallery-level diversity across image and video generation, examine its combination with existing diversity methods, and conduct ablations on gallery size and coupling strength.
Critical strengths
Strengths
- The direct repulsive-coupling method requires neither model training nor test-time noise optimization and adds negligible computational overhead.
- By preserving each noise sample’s Gaussian marginal distribution, the method can be applied to pretrained diffusion models without modifying their sampling procedures.
- Experiments across image and video generation show improvements in gallery-level pixel and perceptual diversity while largely maintaining output quality and prompt alignment.
- Coupled initialization is complementary to existing diversity methods and yields further gains when combined with them.
Critical weaknesses
Weakness
-
Although the paper evaluates pixel- and feature-space diversity, it remains unclear whether these gains translate into meaningful diversity for users. The exploratory multimodal-judge study shows only a modest preference for coupled galleries; a larger human evaluation would strengthen this claim.
-
The gains diminish substantially as gallery size increases. At , the pixel-level improvement is small, while the perceptual and feature-space gains are inconclusive, limiting the method’s utility for larger candidate sets.
-
The theoretical results establish maximal repulsion in noise space and increased separation for linear features, but do not guarantee greater perceptual or semantic diversity after a nonlinear generator. A clearer characterization of when noise-space repulsion transfers to output diversity would strengthen the theoretical motivation.
Questions for authors
See the weakness
3 comments
Dear Reviewer zDcT,
Thank you again for your thoughtful review. We hope our response, including the new blinded human study, has addressed your questions about perceived diversity, why the gains diminish with larger galleries, and when repulsion in noise space increases output diversity. We would greatly appreciate any further thoughts you may have.
Best, Authors
Dear Reviewer zDcT,
We would like to kindly follow up, as the discussion period will close in less than a few hours. We hope our responses have addressed your concerns, and we would be happy to clarify any remaining questions in the time left. If you find our responses satisfactory, we would greatly appreciate it if you could take them into account in your final evaluation.
Thank you!
Authors
Thank you for the careful review and for recognizing the method's simplicity and empirical gains!
Summary of our changes:
We agree that a user study is important for assessing whether the gains are meaningful to people. We therefore completed a blinded study of perceived diversity and best-image preference.
We clarified how gallery size shapes the diversity gains. We strengthened the connection between input and output diversity with a sufficient condition and supporting experiments on neural features.
1. Human judgments of diversity and best-image choice
Thanks for the suggestion! We conducted a blinded SD3 study with 30 participants, each comparing coupled and IID galleries for ten prompts at K=2,3. They separately judged diversity and best-image choice, with ties allowed, yielding 300 judgments per question.
These results show a strong diversity preference, with the same direction at both gallery sizes, and a modest best-image preference. We will do our best to recruit more participants and extend this evaluation.
Please see Section 5.2 and Appendix D.12 (blue text), including Table 23, for details and complete counts.
2. Larger galleries
We agree that the gains diminish as K increases. This is consistent with maximal repulsion's pairwise correlation of −1/(K−1), whose magnitude decreases with K. Our experiments at K=16,32 confirm this trend. Nevertheless, we believe small K is important in practice. For example, image-generation software often presents users with a small set of candidate images to compare and choose from. Meanwhile, prior work likewise evaluates small galleries: CNO uses K=3 or 5, and Particle Guidance uses K=4 for image generation.
3. How input diversity transfers to output diversity
Good point! We have added a theoretical guarantee and supporting experiments to explain when diversity in the initial noise carries through to generated features.
The intuition is as follows. Assuming the generated features are linear in the initial noise, then repulsion increases their expected squared distance by a known factor. For nonlinear features, diversity gain can be similarly explained by “how linear the feature is”.
Our existing analysis in Appendix A.7 (Proposition A.4) gives an exact expression for how noise correlation changes squared feature distance. The result separates linear and higher-order effects. We now derive a sufficient condition that can be directly estimated from data.
New sufficient condition. We use a linear predictor from noise to generated features and compare its prediction error with the total variance. If the error is small enough, Corollary A.5 guarantees an improvement in diversity under repulsion. The generator and feature extractor can both be nonlinear.
Supporting experiment. We fit the predictor offline, then evaluate the condition and output distances using 256 independent noise pairs (K=2) on a frozen CelebA-HQ model. The table reports ALADIN at c=−0.1 and −1, and DINOv2 at c=−1, relative to IID (c=0).
Interpretation.
Please see Appendix A.7 (blue text), including Proposition A.4, Corollary A.5, and Figure 4, for the analysis and experiments.
We hope these additional results clarify when and how diversity in the initial noise translates into diversity in the generated outputs.