- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Posterior Overlap Reduction (POR), a training-free method for improving within-prompt diversity in text-to-image generation. Instead of relying solely on pairwise distances between predicted clean images, the method considers overlap between the corresponding denoising posteriors. The authors derive a conditional upper bound connecting posterior overlap to symmetrized KL divergence and use a state–posterior-mean identity to construct a tractable optimization objective.
Experiments on SD3 and FLUX.1-dev demonstrate improved group diversity with competitive image quality. The proposed formulation is conceptually interesting, and the comparison against a posterior-mean distance objective provides useful empirical support.
Critical strengths
1. Distributional perspective on group diversification. The paper provides a meaningful motivation for considering posterior overlap rather than relying solely on distances between clean predictions. The Gaussian-mixture examples further demonstrate that identical distances between posterior means can correspond to substantially different posterior overlaps.
2. Practical training-free optimization with the highest diversity gain. The proposed method can be applied to pretrained generative models by optimizing intermediate noisy states at a single sampling timestep, without updating the generator parameters. In addition, POR achieves the highest Vendi Score and lowest mean pairwise similarity among the compared methods across both SD3 and FLUX.1-dev on COCO and DPG-Bench. The objective comparison also shows improvements over a clean-prediction distance baseline under a matched optimization framework.
Critical weaknesses
1. Limited validation of the practical informativeness of the theoretical overlap bound. Proposition 1 establishes an upper bound on posterior overlap, , under the relative posterior-variance condition . However, the paper does not evaluate the validity or tightness of this bound in the relevant optimization regions. Although Appendix A.6 demonstrates overlap reduction in a controlled setting, it remains unclear how closely the bound tracks actual posterior overlap during optimization, particularly since the implemented softplus surrogate does not directly minimize the bound.
An analysis of the variance ratio and the gaps between these quantities during optimization would strengthen the connection between the theoretical result and the proposed objective.
2. Insufficient ablation of the factors contributing to the proposed objective's effiectiveness. The estimated symmetrized KL divergence (Eq. 14) can be expressed as , where denotes the angle between the noisy-state difference and the conditional clean-prediction difference. Consequently, the proposed objective depends on noisy-state separation, clean-prediction separation, and their directional alignment.
While POR outperforms the clean-prediction distance baseline, it remains unclear whether the improvement comes from the proposed posterior-divergence formulation or simply from incorporating noisy-state information. In particular, could a simpler repulsive objective based solely on noisy-state distances achieve comparable improvements? Comparing against such a baseline would help clarify the practical benefit of the proposed formulation.
Questions for authors
1. Reliability of the divergence estimator across sampling timesteps. How does the accuracy and stability of the estimated symmetrized KL divergence vary across sampling timesteps, particularly at the early high-noise stages? Does the estimator reliably reflect differences between the underlying posteriors at these stages?
2. Applicability to few-step sampling. How does POR perform with substantially fewer sampling steps, such as 4–8 steps, in terms of both diversity improvements and computational overhead? For separately distilled few-step generative models, does the conditional clean prediction retain the posterior-mean interpretation required by the proposed divergence estimator?
4 comments
Response to Reviewer K16H (2)
Q2: “… few-step sampling …”
Thank you for pointing this out. We have added experiments with SDXL-Lightning in the 8-step setting and SANA-Sprint in the 4-step setting. Their performance and inference time are reported in Tables 15 and 16 and summarized below. denotes the number of optimization updates.
SDXL-Lightning, 8-step
| Method | CLIP ↑ | Pick ↑ | IR ↑ | HPSv2 ↑ | Vendi ↑ | MSS ↓ | LPIPS ↑ | Time (s/group) ↓ |
|---|---|---|---|---|---|---|---|---|
| I.I.D. | 26.172 | 22.513 | 0.726 | 28.461 | 3.578 | 0.273 | 0.577 | 1.9805 ± 0.0181 |
| POR (K=1) | 26.169 | 22.548 | 0.743 | 28.698 | 3.595 | 0.266 | 0.586 | 2.2843 ± 0.0173 |
| POR (K=5) | 26.169 | 22.511 | 0.781 | 29.162 | 3.645 | 0.244 | 0.624 | 3.5862 ± 0.0227 |
SANA-Sprint, 4-step
| Method | CLIP ↑ | Pick ↑ | IR ↑ | HPSv2 ↑ | Vendi ↑ | MSS ↓ | LPIPS ↑ | Time (s/group) ↓ |
|---|---|---|---|---|---|---|---|---|
| I.I.D. | 26.242 | 22.769 | 1.052 | 30.342 | 3.416 | 0.338 | 0.534 | 0.6515 ± 0.0013 |
| POR (K=1) | 26.264 | 22.757 | 1.054 | 30.428 | 3.420 | 0.337 | 0.537 | 0.8365 ± 0.0003 |
| POR (K=5) | 26.246 | 22.758 | 1.057 | 30.589 | 3.459 | 0.321 | 0.544 | 1.5754 ± 0.0006 |
We would like to highlight that a single optimization update already improves group diversity over I.I.D. sampling on both models, and increasing from 1 to 5 further improves all three diversity metrics. POR retains comparable image quality and prompt alignment, while additional updates increase inference time.
Our divergence estimate relies on conditional clean predictions approximating the posterior means under the Gaussian forward path. SDXL-Lightning uses progressive adversarial distillation to reproduce multiple teacher sampling steps in a single student update. SANA-Sprint uses continuous-time consistency distillation to predict a common clean endpoint along a teacher ODE trajectory, together with latent adversarial training. Because these distillation objectives target sampling transitions or trajectory endpoints, they do not guarantee that the resulting clean predictions approximate the posterior means under the Gaussian forward path. Our results therefore provide empirical evidence for POR's applicability to the two evaluated few-step models.
Official Comment by Reviewer K16H
Thank you for the additional experiments. The Noisy-only comparison addresses W2, and the few-step results and qualifications are helpful. Appendix A.7 also strengthens the justification in the controlled setting.
One remaining concern is whether scores computed from learned predictions reliably indicate posterior separation on SD3/FLUX across timesteps. The Q1 error analysis explains how prediction errors enter the score, but does not measure their effect on real models. A small diagnostic comparing pairwise scores with the separation between repeated stochastic continuations from two distinct noisy states would help, explicitly treating this as a sampler-dependent proxy rather than a direct measurement of posterior overlap, consistent with Appendix A.1.
Appendix B.7 gives a clear rationale for using conditional predictions, but the practical effect of this choice within POR remains untested. The existing I.I.D. guidance sweeps do not isolate it. On SD3, comparing conditional versus CFG-combined predictions in the POR score, while fixing sampler guidance and the update budget and allowing comparable learning-rate sweeps, would directly address this question. For FLUX, the analogous comparison would use objective guidance inputs of 1.0 versus 3.5 under the same sampler. The resulting quality–diversity curves would clarify how this prediction choice affects POR’s performance under guided generation.
Official Comment by Authors
We thank the reviewer for these insightful and constructive suggestions. We have added stochastic-continuation experiments on SD3 and FLUX.1-dev. Table 17 examines the relationship between pairwise scores and branch-set similarity across early sampling steps 1–5, and Table 18 reports the changes before and after POR updates. Higher scores tend to correspond to lower branch-set similarity, and POR reduces mean similarity on both models. We explicitly treat branch-set similarity as a sampler-dependent proxy rather than a direct measurement of posterior overlap.
Further, these experiments compare conditional versus CFG-combined predictions on SD3 and objective guidance inputs of 1.0 versus 3.5 on FLUX.1-dev, while fixing sampler guidance and the update budget. Tables 17–18 show stronger associations between higher scores and lower branch-set similarity, together with larger mean similarity reductions, for POR-Conditional. Figure 10 additionally compares their quality–diversity curves through learning-rate sweeps. POR-Conditional achieves higher Vendi and LPIPS over most of the overlapping ImageReward and PickScore ranges on both models. These results provide empirical support for the prediction choices used by POR under guided generation.
Response to Reviewer K16H (1)
Thank you very much for the thoughtful review, we truly appreciate your time and effort. We responded below to all your comments and questions and adapted the manuscript to reflect them.
W1: “… Limited validation of the practical informativeness …”
Thank you for the constructive feedback. To address your concern about the bound's practical informativeness, we have extended the controlled Gaussian-mixture experiment in Appendix A.7 to examine how the bound expression relates to actual posterior overlap during optimization and to compare the gradients of the POR loss and actual posterior overlap. The results are summarized below.
We observe closely aligned gradient directions at the same states along the POR trajectories, together with overlap reduction comparable to direct overlap minimization under the same state-update budget. Figure 8(a) shows that actual posterior overlap and the evaluated bound expression both decrease along the POR trajectory.
W2: “… Insufficient ablation of the factors contributing to the proposed objective's effectiveness …”
Thank you for this suggestion. We have extended our original comparison with the clean-prediction distance baseline (Mean-only) to include a Noisy-only baseline based solely on noisy-state distances. Figure 9 compares the quality–diversity curves of Noisy-only, Mean-only, and POR under the same optimization framework. Over the evaluated sweeps, Noisy-only achieves smaller diversity gains than Mean-only and POR. These results support combining noisy-state and clean-prediction differences to improve the quality–diversity tradeoff.
Q1: “… reliability of the divergence estimator across sampling timesteps …”
Thank you for raising this question. We understand your concern about the reliability of the divergence estimate, particularly at early high-noise stages. The state–mean identity is exact at every timestep with αt,σt>0. In practice, we replace the posterior means with conditional clean predictions from pretrained models. Since exact posterior means are not directly available for these models, evaluating estimation accuracy across timesteps is challenging.
Nevertheless, we can analyze how clean-prediction errors affect the estimate. Let ei=x^0,c(i)(t)−μt(xt(i),c) denote the conditional clean-prediction error and Δxt,ij=xt(i)−xt(j). For the exact symmetrized KL divergence Dij and its estimate Dij, we obtain
=\frac{\alpha_t}{2\sigma_t^2}\\, \Delta x_{t,ij}^{\top}(e_i-e_j).$$ This expression shows that error components shared by both predictions cancel. Only the component of the prediction-error difference along the noisy-state separation contributes to the estimation error. For fixed state differences and prediction errors, the smaller coefficient $\alpha_t/(2\sigma_t^2)$ at high noise reduces their absolute contribution to the estimate. Separately, Figure 5 shows that early intervention in SD3 achieves greater diversity at comparable image preference. While this experiment does not directly measure divergence estimation accuracy, it provides empirical support for our choice of intervention time.