acceptodds
Under review as a conference paper at ICLR 2027

Posterior Overlap Reduction for Diverse Text-to-Image Generation

Abstract

Text-to-image generative models can produce high-quality images yet often yield visually repetitive groups for a single prompt. Existing group diversification methods commonly compare the model's clean predictions directly or through visual embeddings. However, at an intermediate timestep, the model's conditional clean prediction estimates only the mean of a posterior over possible clean images. We argue that different posterior means can still correspond to distributions that share substantial probability mass. We call this shared mass posterior overlap and introduce Posterior Overlap Reduction (POR), a training-free method that reduces pairwise posterior overlap within a fixed group of noisy states. Under a condition on posterior variance, we derive an upper bound on overlap that decreases with symmetrized KL divergence. We obtain a model-based estimate of this divergence without evaluating posterior densities. Using this estimate, POR jointly updates the group at a single timestep early in sampling, then resumes the original sampler. Across Stable Diffusion 3 and FLUX.1-dev, POR shifts the quality–diversity frontier outward on COCO and achieves the highest Vendi Score and lowest mean pairwise similarity among the compared methods on COCO and DPG-Bench, while retaining competitive image quality and prompt alignment.

Primary area:
generative models
Keywords:
Diffusion Models, Flow Matching, Text-to-Image Generation, Sample Diversity, Posterior Overlap, Training-Free Sampling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_K16HOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Posterior Overlap Reduction (POR), a training-free method for improving within-prompt diversity in text-to-image generation. Instead of relying solely on pairwise distances between predicted clean images, the method considers overlap between the corresponding denoising posteriors. The authors derive a conditional upper bound connecting posterior overlap to symmetrized KL divergence and use a state–posterior-mean identity to construct a tractable optimization objective.

Experiments on SD3 and FLUX.1-dev demonstrate improved group diversity with competitive image quality. The proposed formulation is conceptually interesting, and the comparison against a posterior-mean distance objective provides useful empirical support.

Critical strengths

1. Distributional perspective on group diversification. The paper provides a meaningful motivation for considering posterior overlap rather than relying solely on distances between clean predictions. The Gaussian-mixture examples further demonstrate that identical distances between posterior means can correspond to substantially different posterior overlaps.

2. Practical training-free optimization with the highest diversity gain. The proposed method can be applied to pretrained generative models by optimizing intermediate noisy states at a single sampling timestep, without updating the generator parameters. In addition, POR achieves the highest Vendi Score and lowest mean pairwise similarity among the compared methods across both SD3 and FLUX.1-dev on COCO and DPG-Bench. The objective comparison also shows improvements over a clean-prediction distance baseline under a matched optimization framework.

Critical weaknesses

1. Limited validation of the practical informativeness of the theoretical overlap bound. Proposition 1 establishes an upper bound on posterior overlap, Ωijt≤exp⁡(−κ04DsKL(qti,qtj))\Omega_{ij}^{t} \leq \exp\left(-\frac{\kappa_0}{4}D_{\mathrm{sKL}}(q_t^i,q_t^j)\right), under the relative posterior-variance condition mij/Mij≥κ0m_{ij}/M_{ij} \geq \kappa_0. However, the paper does not evaluate the validity or tightness of this bound in the relevant optimization regions. Although Appendix A.6 demonstrates overlap reduction in a controlled setting, it remains unclear how closely the bound tracks actual posterior overlap during optimization, particularly since the implemented softplus surrogate does not directly minimize the bound.

An analysis of the variance ratio and the gaps between these quantities during optimization would strengthen the connection between the theoretical result and the proposed objective.

2. Insufficient ablation of the factors contributing to the proposed objective's effiectiveness. The estimated symmetrized KL divergence (Eq. 14) can be expressed as D^sKL,ij,t=αt2σt2∣Δxt,ij∣∣Δx^0,c,ij∣cos⁡θij\widehat D_{\mathrm{sKL},ij}^{,t} = \frac{\alpha_t}{2\sigma_t^2}|\Delta x_{t,ij}||\Delta\hat x_{0,c,ij}|\cos\theta_{ij}, where θij\theta_{ij} denotes the angle between the noisy-state difference and the conditional clean-prediction difference. Consequently, the proposed objective depends on noisy-state separation, clean-prediction separation, and their directional alignment.

While POR outperforms the clean-prediction distance baseline, it remains unclear whether the improvement comes from the proposed posterior-divergence formulation or simply from incorporating noisy-state information. In particular, could a simpler repulsive objective based solely on noisy-state distances achieve comparable improvements? Comparing against such a baseline would help clarify the practical benefit of the proposed formulation.

Questions for authors

1. Reliability of the divergence estimator across sampling timesteps. How does the accuracy and stability of the estimated symmetrized KL divergence vary across sampling timesteps, particularly at the early high-noise stages? Does the estimator reliably reflect differences between the underlying posteriors at these stages?

2. Applicability to few-step sampling. How does POR perform with substantially fewer sampling steps, such as 4–8 steps, in terms of both diversity improvements and computational overhead? For separately distilled few-step generative models, does the conditional clean prediction retain the posterior-mean interpretation required by the proposed divergence estimator?

4 comments
AuthorsOpenReview ↗

Response to Reviewer K16H (1)

Thank you very much for the thoughtful review, we truly appreciate your time and effort. We responded below to all your comments and questions and adapted the manuscript to reflect them.

W1: “… Limited validation of the practical informativeness …”

Thank you for the constructive feedback. To address your concern about the bound's practical informativeness, we have extended the controlled Gaussian-mixture experiment in Appendix A.7 to examine how the bound expression relates to actual posterior overlap during optimization and to compare the gradients of the POR loss and actual posterior overlap. The results are summarized below.

σt\sigma_tGradient cosineActual overlap: InitialActual overlap: DirectActual overlap: POR
0.990.961 ± 0.0010.9810.9220.925
0.980.956 ± 0.0010.9620.8380.844
0.970.958 ± 0.0010.9420.7390.743
0.960.961 ± 0.0010.9230.6230.626
0.950.957 ± 0.0010.9030.5140.521

We observe closely aligned gradient directions at the same states along the POR trajectories, together with overlap reduction comparable to direct overlap minimization under the same state-update budget. Figure 8(a) shows that actual posterior overlap and the evaluated bound expression both decrease along the POR trajectory.

W2: “… Insufficient ablation of the factors contributing to the proposed objective's effectiveness …”

Thank you for this suggestion. We have extended our original comparison with the clean-prediction distance baseline (Mean-only) to include a Noisy-only baseline based solely on noisy-state distances. Figure 9 compares the quality–diversity curves of Noisy-only, Mean-only, and POR under the same optimization framework. Over the evaluated sweeps, Noisy-only achieves smaller diversity gains than Mean-only and POR. These results support combining noisy-state and clean-prediction differences to improve the quality–diversity tradeoff.

Q1: “… reliability of the divergence estimator across sampling timesteps …”

Thank you for raising this question. We understand your concern about the reliability of the divergence estimate, particularly at early high-noise stages. The state–mean identity is exact at every timestep with αt,σt>0\alpha_t,\sigma_t > 0. In practice, we replace the posterior means with conditional clean predictions from pretrained models. Since exact posterior means are not directly available for these models, evaluating estimation accuracy across timesteps is challenging.

Nevertheless, we can analyze how clean-prediction errors affect the estimate. Let ei=x^0,c(i)(t)−μt(xt(i),c)e_i=\hat{x}_{0,c}^{(i)}(t)-\mu_t(x_t^{(i)},c) denote the conditional clean-prediction error and Δxt,ij=xt(i)−xt(j)\Delta x_{t,ij}=x_t^{(i)}-x_t^{(j)}. For the exact symmetrized KL divergence DijD_{ij} and its estimate D^ij\widehat D_{ij}, we obtain

=\frac{\alpha_t}{2\sigma_t^2}\\, \Delta x_{t,ij}^{\top}(e_i-e_j).$$ This expression shows that error components shared by both predictions cancel. Only the component of the prediction-error difference along the noisy-state separation contributes to the estimation error. For fixed state differences and prediction errors, the smaller coefficient $\alpha_t/(2\sigma_t^2)$ at high noise reduces their absolute contribution to the estimate. Separately, Figure 5 shows that early intervention in SD3 achieves greater diversity at comparable image preference. While this experiment does not directly measure divergence estimation accuracy, it provides empirical support for our choice of intervention time.
AuthorsOpenReview ↗

Response to Reviewer K16H (2)

Q2: “… few-step sampling …”

Thank you for pointing this out. We have added experiments with SDXL-Lightning in the 8-step setting and SANA-Sprint in the 4-step setting. Their performance and inference time are reported in Tables 15 and 16 and summarized below. KK denotes the number of optimization updates.

SDXL-Lightning, 8-step

MethodCLIP ↑Pick ↑IR ↑HPSv2 ↑Vendi ↑MSS ↓LPIPS ↑Time (s/group) ↓
I.I.D.26.17222.5130.72628.4613.5780.2730.5771.9805 ± 0.0181
POR (K=1)26.16922.5480.74328.6983.5950.2660.5862.2843 ± 0.0173
POR (K=5)26.16922.5110.78129.1623.6450.2440.6243.5862 ± 0.0227

SANA-Sprint, 4-step

MethodCLIP ↑Pick ↑IR ↑HPSv2 ↑Vendi ↑MSS ↓LPIPS ↑Time (s/group) ↓
I.I.D.26.24222.7691.05230.3423.4160.3380.5340.6515 ± 0.0013
POR (K=1)26.26422.7571.05430.4283.4200.3370.5370.8365 ± 0.0003
POR (K=5)26.24622.7581.05730.5893.4590.3210.5441.5754 ± 0.0006

We would like to highlight that a single optimization update already improves group diversity over I.I.D. sampling on both models, and increasing KK from 1 to 5 further improves all three diversity metrics. POR retains comparable image quality and prompt alignment, while additional updates increase inference time.

Our divergence estimate relies on conditional clean predictions approximating the posterior means under the Gaussian forward path. SDXL-Lightning uses progressive adversarial distillation to reproduce multiple teacher sampling steps in a single student update. SANA-Sprint uses continuous-time consistency distillation to predict a common clean endpoint along a teacher ODE trajectory, together with latent adversarial training. Because these distillation objectives target sampling transitions or trajectory endpoints, they do not guarantee that the resulting clean predictions approximate the posterior means under the Gaussian forward path. Our results therefore provide empirical evidence for POR's applicability to the two evaluated few-step models.

Reviewer_K16HOpenReview ↗

Official Comment by Reviewer K16H

Thank you for the additional experiments. The Noisy-only comparison addresses W2, and the few-step results and qualifications are helpful. Appendix A.7 also strengthens the justification in the controlled setting.

One remaining concern is whether scores computed from learned predictions reliably indicate posterior separation on SD3/FLUX across timesteps. The Q1 error analysis explains how prediction errors enter the score, but does not measure their effect on real models. A small diagnostic comparing pairwise scores with the separation between repeated stochastic continuations from two distinct noisy states would help, explicitly treating this as a sampler-dependent proxy rather than a direct measurement of posterior overlap, consistent with Appendix A.1.

Appendix B.7 gives a clear rationale for using conditional predictions, but the practical effect of this choice within POR remains untested. The existing I.I.D. guidance sweeps do not isolate it. On SD3, comparing conditional versus CFG-combined predictions in the POR score, while fixing sampler guidance and the update budget and allowing comparable learning-rate sweeps, would directly address this question. For FLUX, the analogous comparison would use objective guidance inputs of 1.0 versus 3.5 under the same sampler. The resulting quality–diversity curves would clarify how this prediction choice affects POR’s performance under guided generation.

AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for these insightful and constructive suggestions. We have added stochastic-continuation experiments on SD3 and FLUX.1-dev. Table 17 examines the relationship between pairwise scores and branch-set similarity across early sampling steps 1–5, and Table 18 reports the changes before and after POR updates. Higher scores tend to correspond to lower branch-set similarity, and POR reduces mean similarity on both models. We explicitly treat branch-set similarity as a sampler-dependent proxy rather than a direct measurement of posterior overlap.

Further, these experiments compare conditional versus CFG-combined predictions on SD3 and objective guidance inputs of 1.0 versus 3.5 on FLUX.1-dev, while fixing sampler guidance and the update budget. Tables 17–18 show stronger associations between higher scores and lower branch-set similarity, together with larger mean similarity reductions, for POR-Conditional. Figure 10 additionally compares their quality–diversity curves through learning-rate sweeps. POR-Conditional achieves higher Vendi and LPIPS over most of the overlapping ImageReward and PickScore ranges on both models. These results provide empirical support for the prediction choices used by POR under guided generation.

Reviewer_V3iZOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the issue of text-to-image models producing visually similar images for the same prompt. It proposes Posterior Overlap Reduction, which jointly adjusts a group of candidates using an objective motivated by reducing overlap between their posterior distributions. The method requires no model training. Experiments on COCO and DPG-Bench with SD3 and FLUX.1-dev show improved group diversity while maintaining competitive image quality and prompt alignment.

Critical strengths
  • The method is well motivated and supported by a clear theoretical rationale. This allows the readers to understand its design beyond the empirical results.
  • The experiments are well designed, using appropriate benchmarks and contemporary text-to-image models. The qualitative examples show clear improvements in diversity.
Critical weaknesses
  • Applying the method requires generating multiple images jointly and needs additional backpropagation costs at inference time. These requirements lead to a practical deployment challenge. Incorporating the desired diversity behavior into the model through fine-tuning may be a more deployment-friendly alternative.
  • The diversity metrics are computed from only four or five images per prompt. The stability of these estimates and their statistical uncertainty remain unclear.
Questions for authors
  • Could the authors report results across multiple random seeds? The relatively small gains reported in Tables 1 and 2, particularly in Vendi Score, warrant further assessment of their robustness across random seeds, as their improvements may be smaller than the standard deviation across repeated runs.
  • Many text-to-image models already use prompt rewriting or expansion. In these settings, limited variation in ODE sampling trajectories may be less problematic, since diversity can come primarily from stochastic prompt expansion. This approach also avoids the requirement to generate multiple images for each request. What advantages does this method offer over this alternative?
4 comments
AuthorsOpenReview ↗

Response to Reviewer V3iZ

Thank you very much for the insightful comments and constructive suggestions. We truly appreciate your time and effort. We responded below to all your comments and questions and adapted the manuscript to reflect them.

W1: “… deployment challenge …”

We are glad to have the opportunity to discuss this direction. While fine-tuning can learn diversification behavior from data, our focus is on developing a theoretically motivated sampling objective that promotes group diversity without updating model parameters. We consider the connection between posterior overlap and this training-free sampling objective an important contribution.

From a practical deployment perspective, we see the fine-tuning approach you suggest as a complementary direction. In our SD3 experiments, POR requires about 20% more inference time than I.I.D. sampling due to additional backpropagation. Using POR's latent updates as training signals for fine-tuning, with the aim of reducing inference time and computational cost, could be an interesting research direction. We will explore this possibility in future work. Thank you for this suggestion.

W2: “… The stability of these estimates and their statistical uncertainty …”

We thank the reviewer for pointing this out. Our diversity metrics are computed within each prompt's image group and then averaged over prompts. To assess the stability of these dataset-averaged scores at the evaluated group sizes, we repeated POR with two additional generation seeds in all four model–dataset settings, keeping the other settings fixed. We have added the three-seed mean and standard deviation to Tables 1 and 2, with the Vendi results shown in our response to Q1 below.

Q1: “… report results across multiple random seeds …”

We thank the reviewer for this suggestion. We ran POR with two additional generation seeds in all four model–dataset settings, keeping the other generation settings fixed. We report the mean Vendi score ± sample standard deviation across the reference run and the two additional runs below.

MethodSD3 · COCOSD3 · DPGFLUX.1-dev · COCOFLUX.1-dev · DPG
I.I.D.4.1763.8463.3773.074
PG4.1783.8893.3893.072
CNO4.2393.9003.4443.136
SGI (M=32)4.2543.9573.4603.161
POR4.2863.9663.4723.178
POR (3 seeds)4.289 ± 0.0153.972 ± 0.0123.476 ± 0.0103.178 ± 0.010

All three POR runs exceed the reported SGI Vendi score in each setting. Results for the other metrics have been included in Tables 1 and 2.

Q2: “… prompt rewriting or expansion …”

We truly thank you for pointing this out. POR's advantage is diversifying a group while keeping the original conditioning prompt unchanged. To examine the alternative you suggest, we compared POR with prompt expansion on all DPG-Bench prompts using SD3. We instructed Gemma-3-12B-IT [1] to generate five expanded prompts for each original prompt, retaining all explicit scene constraints and varying only unspecified details. We then used the SD3 baseline to generate one image from each expanded prompt and evaluated all prompt-conditioned metrics against the original prompt. The results are summarized below.

MethodCLIP ↑Pick ↑IR ↑HPSv2 ↑Aesth. ↑Vendi ↑MSS ↓LPIPS ↑
I.I.D.29.73021.9000.84428.2543.6573.8460.3940.543
Expansion29.54221.8040.78327.9043.6473.9810.3600.561
POR (3 seeds)29.78821.9100.85328.2613.6593.9720.3620.591
Δ\Delta+0.246+0.106+0.070+0.357+0.012-0.009+0.002+0.030

POR scores are averaged over three generation seeds. Standard deviations are reported in Table 2. Δ\Delta denotes score differences relative to Expansion, using the three-seed mean for POR.

We observe that expansion scores lower than I.I.D. sampling on all image quality and prompt alignment metrics, with a clear gap relative to POR. In this setting, POR improves group diversity over I.I.D. sampling while retaining image quality and alignment with the original prompt. We added Figure 4 to illustrate how images generated from expanded prompts can depart from the original scene constraints. As you point out, expansion supports independent image generation, but we believe it is important to increase diversity while preserving alignment with the original prompt. We see value in both approaches, depending on the use case.

References

[1] Gemma 3 Technical Report, arXiv 2025.

Thank you for clarifying the inference overhead and addressing my concerns in the rebuttal.. For W1, my main concern was not about fine-tuning itself, but about the requirement to generate images jointly: even when a user requests only one image, the method requires generating an entire group.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for clarifying your concern. Following prior work on group diversification, including CNO [1], SGI [2], and PG [3], we consider the setting where the number of output images N is specified in advance, and the goal is to improve diversity among these N images under the same prompt. For N ≥ 2, POR jointly updates exactly these N candidates, without requiring additional candidates beyond the requested outputs.

For N = 1, the POR update can be omitted and the original sampler used directly, without generating auxiliary candidates. We also recognize that improving diversity without requiring joint sampling is an important research direction, and we plan to explore it in future work.

References

[1] Diverse Text-to-Image Generation via Contrastive Noise Optimization, ICLR 2026.

[2] Scaling Group Inference for Diverse and High-Quality Generation, ICLR 2026.

[3] Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models, ICLR 2024.

OK! Thank you again for your careful rebuttal.

Reviewer_KixxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes POR, a training-free method for improving diversity in text-to-image generation while maintaining image quality and prompt alignment. Unlike existing methods that encourage diversity by maximizing distances between predicted clean images, POR considers the overlap between posterior distributions over possible clean images, arguing that distinct posterior means can still share substantial probability mass. The authors derive an upper bound on posterior overlap using symmetrized KL divergence under assumptions on posterior variance and introduce a tractable surrogate estimated from noisy-state differences and conditional clean predictions. POR optimizes this surrogate by jointly updating a fixed group of latent states at a single early sampling timestep before resuming the original sampler. Experiments on COCO dataset and DPG-Bench demonstrate improved group diversity, achieving higher Vendi Scores and lower mean pairwise similarity than competing methods while maintaining competitive image quality and text alignment.

Critical strengths

S1. The proposed method introduces a theoretically motivated distribution-level diversity criterion that goes beyond conventional comparisons of predicted clean images. Controlled experiments demonstrate the effectiveness of the proposed surrogate in reducing posterior overlap.

Critical weaknesses

W1. Figure 5 raises concerns about output quality. In the museum gallery example, a skyblue window appears incongruous with the overall composition. In the lower-left landscape example, the sky abruptly changes to black. These examples raise the question of whether the proposed method maintains prompt alignment while producing variations that are coherent rather than arbitrary.

W2. CLIP, PickScore, ImageReward, and HPSv2 all condition on the prompt to assess alignment. The authors should also report an aesthetic quality metric that evaluates images independently of the prompt, allowing a more direct assessment of the quality of the generated images themselves.

W3. Computational costs need to be reported. Comparisons of latency, memory usage, and FLOPs against the baselines are essential.

W4. Qualitative results for FLUX are missing. These results are essential to substantiate the claim that the proposed method is model-agnostic. It would be great if the authors can demonstrate the performance of POR on diverse models beyond SD3 and FLUX, such as QWEN-Image 2.1.

W5. Only 2,000 prompts from COCO dataset are used. Why was the full set of prompts not evaluated? Were these prompts sampled without curation, or were they selected according to specific criteria that might favor the proposed method? The authors should clarify the prompt selection procedure.

W6. The proposed surrogate is theoretically motivated by an upper bound on posterior overlap. However, this bound relies on assumptions about posterior variance that are not verified in practical diffusion models, and increasing the surrogate divergence does not necessarily guarantee a reduction in actual posterior overlap. Additional analysis or a quantitative explanation of this relationship is needed.

Questions for authors

Please see the weaknesses section.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Kixx (1)

Thank you very much for the thoughtful review, we truly appreciate your time and effort. We responded below to all your comments and questions and adapted the manuscript to reflect them.

W1: “… concerns about output quality …”

We understand your concern about the visual coherence of the window and sky variations in Fig. 5 (Fig. 6 in the revised manuscript). Our evaluation includes ImageReward and HPSv2, which reflect human preferences encompassing visual fidelity and quality as well as text–image alignment. Across COCO and DPG-Bench, POR improves diversity while retaining competitive alignment and preference scores (Tables 1 and 2). Following your suggestion in W2, we have additionally evaluated prompt-independent aesthetics using Q-Align [1]. These scores are broadly comparable to those of the baselines, providing complementary evidence on image quality.

W2: “… report an aesthetic quality metric that evaluates images independently …”

We thank the reviewer for this suggestion. We have added prompt-independent aesthetic scores measured with Q-Align to Tables 1 and 2. The scores are shown below.

MethodSD3 · COCOSD3 · DPGFLUX.1-dev · COCOFLUX.1-dev · DPG
I.I.D.3.2913.6573.9343.960
PG3.2863.6513.9273.983
CNO3.2953.6573.9313.961
SGI (M=32)3.3013.6533.9043.935
POR (Ours)3.2983.6623.9183.942

Observe that POR achieves higher aesthetic scores than I.I.D. on SD3. On FLUX.1-dev, its scores are lower than I.I.D. but remain higher than those of SGI, a selection-based method that does not directly optimize latent states.

W3: “… computational costs …”

Inference time and peak allocated GPU memory are already reported in Appendix B.10 of the original submission. As requested, we have added FLOPs measurements on SD3. Inference time is reported as mean ± sample standard deviation over ten measured runs, excluding two warm-up runs. FLOPs cover tensor operations from text encoding through final image decoding. The results are summarized below.

MethodEager (s/group)Compiled (s/group)Peak memory (GiB, eager)TFLOPs/group (eager)
I.I.D.11.522 ± 0.0799.505 ± 0.02524.5332590
PG11.638 ± 0.0219.528 ± 0.02624.5332590
CNO12.555 ± 0.05410.311 ± 0.02724.5642859
SGI (M=32)19.037 ± 0.16316.532 ± 0.35625.4463686
POR (Ours)13.950 ± 0.02511.583 ± 0.03624.5333416
AuthorsOpenReview ↗

Response to Reviewer Kixx (2)

W4: “… substantiate the claim that the proposed method is model-agnostic …”

Thank you for this suggestion. We have added FLUX.1-dev qualitative results comparing I.I.D., SGI, and POR under short and detailed prompts in Figures 12–13. As you suggested, beyond SD3 and FLUX, we have also added qualitative comparisons between POR and the I.I.D. baseline on Qwen-Image 2.1 in Figure 14. These additional qualitative results allow comparison across the three models.

W5: “… clarify the prompt selection procedure …”

Thank you for pointing this out. For a fair comparison with CNO, we matched its evaluation scale by using 2,000 COCO prompts. These prompts were randomly sampled from COCO 2017 validation, without manual curation or selection based on generated outputs. All methods in our comparison were evaluated on this same sampled prompt set. We have clarified this evaluation setup and the sampling procedure in the manuscript.

W6: “… Additional analysis or a quantitative explanation …”

Thank you for pointing this out. On pretrained image models, we can estimate symmetrized KL divergence from noisy states and conditional clean predictions and evaluate the diversity of generated images. However, the exact posterior variances and actual overlap are not directly available, so our experiments do not directly verify the relative variance condition in these models.

As mentioned in the Introduction, we evaluate the surrogate in a Gaussian-mixture setting where posterior means, variances, and actual overlap can be computed. Since increasing the estimated symmetrized KL divergence does not necessarily guarantee a reduction in actual overlap, we extend this analysis in Appendix A.7 to compare POR and direct overlap minimization in terms of gradient directions and overlap reduction. The quantitative results are shown below.

σt\sigma_tGradient cosineActual overlap: InitialActual overlap: DirectActual overlap: POR
0.990.961 ± 0.0010.9810.9220.925
0.980.956 ± 0.0010.9620.8380.844
0.970.958 ± 0.0010.9420.7390.743
0.960.961 ± 0.0010.9230.6230.626
0.950.957 ± 0.0010.9030.5140.521

In this controlled setting, we observe that the surrogate's gradient directions are closely aligned on average with those of direct overlap minimization, and POR achieves comparable overlap reduction under the same update budget. We further evaluated POR empirically on pretrained SD3 and FLUX.1-dev, observing improved group diversity across multiple metrics on COCO and DPG-Bench.

References

[1] Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels, arXiv 2023.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you again for your thoughtful review. We would like to share additional experimental results that may help address your concern in W6. Both tables below report results for POR-Conditional, which uses conditional clean predictions on SD3 and a guidance input of 1.0 when computing the POR score on FLUX.1-dev.

We generated repeated stochastic continuations from different noisy states on pretrained SD3 and FLUX.1-dev, treating the outputs from each state as a branch set. For each pair of states, we compared the POR score computed from the states and their learned clean predictions with the similarity between their branch sets. At each of the early sampling steps 1–5, state pairs with higher scores tended to have less similar branch sets on both models (Table 17). The Spearman correlations between the negative pairwise score and branch-set similarity are summarized below.

ModelStep 1Step 2Step 3Step 4Step 5
SD30.5690.7460.8710.9320.958
FLUX.1-dev0.5030.6780.8110.8830.923

After POR updates at step 1, mean branch-set similarity decreased on both models. For most evaluated state pairs, the score increased while similarity decreased. The before-and-after results are summarized below (Table 18).

ModelMean similarity before PORMean similarity after PORPairs with score ↑ and similarity ↓
SD30.5600.406287/300 (95.7%)
FLUX.1-dev0.5710.385291/300 (97.0%)

These results complement our controlled Gaussian-mixture analysis with empirical evidence from pretrained image models. Please let us know if any further analysis or clarification would help address your concern.

Comments on the paper

2 comments

Dear reviewers,

This is a kind reminder that the discussion with the authors ends on 18th Nov. Please take a moment to leave a comment to acknowledge the authors’ responses and engage in the discussion if you have further questions, particularly on points that may affect your assessment.

Thank you!

Your AC

AuthorsOpenReview ↗

General Response

We thank the Area Chair for guiding the discussion and all three reviewers for their time and constructive feedback.

We are encouraged that the reviewers highlighted the theoretical motivation for using posterior overlap to improve group diversity (Kixx, V3iZ, K16H), the controlled experiments on posterior overlap (Kixx, K16H), and the empirical gains in group diversity (V3iZ, K16H).

We incorporated the additional experiments into the revised manuscript and summarize the main discussion outcomes below.

  • V3iZ. We addressed V3iZ’s concerns about stability across generation seeds and the advantages of POR over prompt expansion, and clarified when joint sampling is required.
  • K16H. We addressed K16H’s concerns about the effectiveness of POR’s objective and its connection to posterior overlap. For few-step sampling, we evaluated performance and cost and clarified the limits of the posterior-mean interpretation after distillation. We provided the requested stochastic-continuation analysis and a comparison of conditional and guided predictions in the POR score on SD3 and FLUX.1-dev.
  • Kixx. We addressed Kixx’s concerns about computational cost, the COCO prompt selection procedure, and whether optimizing the POR surrogate reduces posterior overlap. Remaining concerns focused on image quality and how clearly the FLUX examples demonstrate diversity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.