acceptodds
Under review as a conference paper at ICLR 2027

Understanding Intensity Saturation in Diffusion Models through Data Covariance

Abstract

Diffusion models exhibit approximately linear behavior at moderate to high noise levels, but how this linearity manifests in final generated samples remains less understood. We show that projections of initial noise and generated samples onto the same data covariance eigenvector generally exhibit a more linear relationship for eigenvectors with larger eigenvalues. In natural image datasets, leading eigenvectors with larger eigenvalues represent broad brightness variations. Large-magnitude initial noise projections along these directions can therefore contribute to spatially coherent regions becoming extremely bright or dark, a phenomenon we call *intensity saturation*. Classifier-free guidance (CFG) changes both the type of intensity saturation and the saturated area. Without guidance, saturation is more commonly dominated by either dark or bright regions, while their coexistence becomes more frequent as the CFG weight increases. Overall, increasing the CFG weight increases intensity saturation. However, it can increase or decrease the saturated area of individual samples. Our linear analysis suggests that these changes depend on the class and initial noise projections through changes in coefficient magnitudes along the leading eigenvectors. Motivated by these findings, we propose a method that selectively damps large coefficients along these directions to reduce intensity saturation, thereby improving image quality under guidance.

Primary area:
generative models
Keywords:
diffusion model, classifier free guidance

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_wqAAOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper tries to relate intensity saturation in diffusion-generated images to large coefficients along leading covariance eigenvectors. It measures initial-to-final coefficient linearity, and uses an existing Gaussian CFG analysis to interpret guidance-induced changes. It then proposes magnitude-dependent damping of leading covariance coefficients during sampling.

Critical strengths
  1. The paper goes beyond qualitative examples; Table 7 reports mean initial-to-final R2=0.9689R^2 = 0.9689 for the leading eigenvector, and removing that initial-noise component reduces one-sided saturated area by 65.1% (Fig. 5). The distinction between one-sided and two-sided saturation connects observable image behavior to different covariance components.

  2. It introduces no additional denoiser evaluations and consistently reduces ECA@10 while increasing HPSv2.1 in Table 2. Appendix R also evaluates matched recall, with improvements across the reported metrics for EDM2-S/CFG over 2.25≤w≤5.52.25 \leq w \leq 5.5. These results support a practical benefit beyond the fixed-guidance comparisons alone.

Critical weaknesses
  1. Section 3 introduces training-data statistics, but Appendix A estimates them from 50,000 model-generated images per class. The final-linearity experiment in Appendix B.6 uses seeds 0–9,999, which overlap that estimation set. This does not make the observed relationship automatic, but leaves its robustness to actual training-data statistics and independent evaluation insufficiently established. Appendix B.2 does use separate seeds, which partially addresses this concern. Separately, Appendix B.4 does not specify how the held-out eigenvector’s transition time is obtained. If it is extracted using Eq. (18), it depends on the final R2R^2 being predicted. This uncertainty matters for interpreting Table 5 as independent endpoint prediction.

  2. The Gaussian trajectory and CFG decomposition follow Wang and Vastola (2024) and Li et al. (2025), respectively. The incremental contribution therefore depends substantially on the saturation analysis. However, Table 2 omits frequency-decoupled guidance (Sadat et al., 2025), a closely related approach to controlling oversaturation, and there is no comparison with the same damping rule applied to a fixed low-frequency basis.

  • Section 4.2. An opposite-sign additive shift need not decrease coefficient magnitude because it can overshoot through zero. Writing Eq. (7) as acfg=Sa+ba^{\mathrm{cfg}} = Sa+b, the additive shift decreases magnitude precisely when ∣Sa+b∣2−∣Sa∣2=b(2Sa+b)<0|Sa+b|^2-|Sa|^2 = b(2Sa+b)<0. For example, Sa=−0.1Sa=-0.1 and b=1b=1 increase the magnitude from 0.10.1 to 0.90.9. Please qualify the corresponding bullet.

  • Appendix H/Fig. 22 versus Table 11 : Appendix H includes w=1w=1 and reports one-sided u1u_1 contributions of 47.3–53.1%. Table 11 gives an unguided sample-mean contribution of 41.41%, outside this range, whereas its pooled value of 50.67% falls inside.

[1] Wang and Vastola (2024). The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications. TMLR. Relevant locations: Eqs. (13), (15), and (17); Section 4.3 on prediction of coarse image structure. https://arxiv.org/abs/2412.09726

[2] Li, Wang, and Qu (2025). Towards Understanding the Mechanisms of Classifier-Free Guidance. NeurIPS. Relevant locations: Section 3.2, Theorem 1; Section 4.1 on the effects of mean-shift and covariance guidance. https://arxiv.org/abs/2505.19210

[3] Sadat, Vontobel, Salehi, and Weber (2025). Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low CFG Scales. Relevant locations: Section 4, Eqs. (3)–(6); Section 5 on EDM2 and DiT experiments. https://arxiv.org/abs/2506.19713

Questions for authors
  1. Please repeat the principal coefficient-linearity and saturation-prediction results using covariance estimated from actual ImageNet images, with evaluation seeds disjoint from any generated-sample calibration set. How do the leading eigenspaces, R2R^2?

  2. For Tables 5–6, is the held-out transition time predicted from its eigenvalue and the training modes, or measured from its own trajectory? Equation (18) uses the final R2R^2 through Lc,iL_{c,i}. Please specify the transition estimation and factorization fitting procedures, and report endpoint prediction.

  3. On EDM2-S, please compare the proposed method with frequency-decoupled guidance and with the same damping rule applied to a fixed, rank-matched low-frequency basis. Use identical seeds and sampling budgets, sweep regulation strength, and report ECA@10, HPSv2.1, FID, FDDINOv2\mathrm{FD}_{\mathrm{DINOv2}}.

8 comments

Thank you for the detailed review.

W1, Q1. Robustness of our analysis to actual training-data statistics and independent evaluation.

Our additional experiments sufficiently establish the robustness of the Section 3 results to actual training-data statistics and independent evaluation. We repeated the principal coefficient-linearity and saturation-prediction experiments using covariance from real ImageNet images and evaluation seeds (50,000–59,999) disjoint from all calibration seeds.

The generated-statistics baseline reproduces the paper’s results. With real statistics, coefficient linearity remains high and saturation remains partially predictable, as shown below. The top-5 eigenspaces also nearly coincide (Ec[Ei[∥Qreal,c⊤qgen,c,i∥22]]=0.97\mathbb{E}_{c}[\mathbb{E}_{i}[\lVert Q_{\mathrm{real},c}^{\top} q_{\mathrm{gen},c,i}\rVert_{2}^{2}]]=0.97).

MetricGeneratedReal
Leading eigenvectors
R2R^2 ↑ (top-1)0.970.97
Mean R2R^2 (top-5)0.910.91
Mean NRMSE ↓ (top-5)0.300.30
All images
Precision ↑0.220.19
Recall ↑0.330.23
Saturated area ≥10%\ge 10\%
Precision ↑0.800.77
Recall ↑0.320.28
We will add these results to the appendix.

W1, Q2. Does Table 5 need to be interpreted as independent endpoint prediction?

It is not necessary to interpret Table 5 as independent endpoint prediction, because its purpose is to validate whether ℓc\ell_c and kck_c are shared across eigenvectors within each class, conditional on their observed transition times. Additional experiments show that the results remain highly consistent whether or not transition-time estimation uses the held-out eigenvector’s final R2R^2.

Eq. (25) describes each eigenvector’s decorrelation dynamics using the same two class-specific functions, ℓc\ell_c and kck_c, with an eigenvector-specific transition time. In Table 5, we fit these functions using nine eigenvectors, then predict the remaining eigenvector’s trajectory and endpoint given its observed transition time. Thus, held-out refers to excluding the target eigenvector from fitting ℓc\ell_c and kck_c.

The original transition time was measured from the target’s own trajectory as H50H_{50}, the interpolated 50% crossing of endpoint-normalized cumulative-max HH. This calculation uses endpoint information, which is part of the conditioning in this experiment. Our description should have explicitly stated how transition time was obtained and what was held out.

To additionally address the concern about endpoint-dependent timing, we repeated the evaluation using eigenvalue-based transition times, without using the target’s trajectory to estimate its timing. The procedures are described below.

  • Transition estimation: Following Eqs. (20)–(21), we assign all modes the eigenvalue-based relative timing τ~j=−12(log⁡λj−log⁡λ‾train)\tilde{\tau}_j=-\frac12(\log\lambda_j-\overline{\log\lambda}_{\mathrm{train}}), without using the held-out mode’s final R2R^2.
  • Factorization fitting: Except for transition estimation, all factorization fitting procedures, including spline parameterization, the weighted ridge regression objective, and regularization, are identical between the original and additional experiments.

The results are shown below, with changes relative to the original results (new − original).

EvaluationTrajectory NRMSE↓Endpoint rel. error↓Endpoint corr.↑
Table 5: factorization0.1002 (−0.0047)0.1386 (−0.0128)0.9231 (−0.0001)
Table 6: baseline0.1092 (−0.0021)0.1573 (−0.0036)-

We will clarify the purpose, transition-time calculation, and conditioning in Table 5’s caption and App. B.4, and provide fitting details in App. B.4–B.5.


W2. Can an opposite-sign additive shift fail to decrease coefficient magnitude when intensity saturation decreases in practice?

An opposite-sign additive shift decreases coefficient magnitude in almost all of evaluated opposite-sign cases with reduced intensity saturation under CFG, at each evaluated CFG weight.

The statement “decreasing it for those in the opposite direction” was intended as a qualitative explanation of how mean shift contributes to the observed reductions in intensity saturation, rather than a guarantee for every initial projection and CFG weight.

To assess this interpretation, we evaluated each CFG weight separately using the results in Figures 6 and 27. Among samples whose contribution to ECA@10 decreases and whose additive shift has the opposite sign to the variance-scaled coefficient, coefficient magnitude decreases in 98.8%, 98.0%, 97.6%, and 97.0% of cases, respectively.

Although an opposite-sign additive shift can increase coefficient magnitude through overshoot, this rarely occurs in the evaluated cases where saturation decreases. We will revise the wording to make clearer that the bullet provides a qualitative explanation of the mean shift mechanism.

W2. Does frequency-decoupled guidance’s control of oversaturation diminish our contribution?

Our contribution concerns intensity saturation, which differs from the oversaturation addressed by frequency-decoupled guidance (FDG) [1] and related methods [2]. The metric in [2] measures oversaturation through the difference between the maximum and minimum RGB values relative to the maximum, whereas our intensity saturation concerns near-white or near-black pixels. Neither measure reliably captures the phenomenon measured by the other.

Moreover, [1,2] focus on oversaturation induced by strong guidance, whereas intensity saturation occurs even without guidance. The two phenomena also differ in how they appear across samples.

The methods also differ in their sample-wise selectivity. FDG modifies guidance strengths for low- and high-frequency components across samples, whereas our method selectively regulates samples prone to excessive intensity saturation, with weaker corrections to other samples. This sample-wise adjustment aims to reduce excessive intensity saturation while preserving other samples and the performance of the underlying guidance method as much as possible. Our method can also be combined with multiple guidance methods to control intensity saturation while preserving or even improving their image-quality benefits.

Thus, FDG’s control of oversaturation does not diminish our contribution.


W2, Q3. Comparisons with FDG and a fixed, rank-matched low-frequency basis.

Using Table 2’s EDM2-S setup with 50k samples per method, we compared our method with frequency-decoupled guidance. For FDG, we set (wlow,whigh)=(1,4)(w_{\mathrm{low}},w_{\mathrm{high}})=(1,4).

MethodECA@10 ↓HPSv2.1 ↑FID ↓FD_DINOv2 ↓
CFG8.8324.746.6894.53
CFG + ours6.2624.946.7692.29
FDG3.6723.993.6695.89
FDG + ours2.0724.143.8795.10

Thus, even when FDG is chosen as the underlying guidance method, our correction provides an additional benefit by reducing the intensity saturation that remains. FDG’s ability to reduce intensity saturation therefore does not eliminate the contribution of our complementary correction.

Fixed, rank-matched low-frequency basis. We use the same CFG setup and App. O damping, with variances v⊤Σvv^\top\Sigma v estimated from the same data. Using normalized Hadamard channel vectors, r=1r=1 pairs DCT (0,0)(0,0) with the first vector; r=5r=5 pairs DCT (0,0)(0,0) with all four vectors and DCT (0,1)(0,1) with the first. All metrics use 10k matched samples. Each cell reports Ours / DCT.

r=1r=1

zmax⁡z_{\max}ECA@10 ↓HPSv2.1 ↑FID ↓FD-DINOv2 ↓
0.54.50 / 9.2724.80 / 24.758.59 / 8.69115.04 / 115.37
15.35 / 9.2324.78 / 24.768.60 / 8.67114.98 / 115.10
26.80 / 9.0024.78 / 24.768.68 / 8.66115.07 / 115.47

r=5r=5

zmax⁡z_{\max}ECA@10 ↓HPSv2.1 ↑FID ↓FD-DINOv2 ↓
0.53.17 / 6.2925.07 / 24.998.86 / 8.68114.03 / 113.16
14.30 / 6.7925.02 / 24.968.77 / 8.67113.01 / 112.93
26.22 / 7.5724.94 / 24.918.75 / 8.66113.46 / 113.27

Under the same damping rule, at matched rank and zmax⁡z_{\max}, ours consistently achieves lower ECA@10 than the fixed low-frequency basis. While damping low frequencies can also reduce intensity saturation, frequency alone offers no clear rule for selecting an effective basis for a given diffusion model, requiring repeated sampling and evaluation for different candidate bases. In contrast, our approach does not require this repeated evaluation, as it directly derives an appropriate basis for selectively reducing intensity saturation from the model’s sample covariance, as described in the paper.


W2. Fig. 22/Table 11.

Table 11 reports both sample-mean and pooled contributions (App. I, Eqs. (38)–(39)), whereas Fig. 22(b) shows only pooled results. Both identify u1u_1 as the largest contributor among the leading eigenvectors for one-sided saturation. The comparison below also includes the results for Fig. 22(b) using sample-mean.

AggregationFig. 22Table 11
Sample mean30.9–41.4%41.41%
Pooled47.3–53.1%50.67%

At w=1w=1, the sample mean of 41.4% matches Table 11's 41.41%. We will clarify this and add the sample mean results.


[1] Sadat et al. (2025). Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low CFG Scales.

[2] Sadat et al. (2025). Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models.

Thank you for the response and the additional experiments. Some of my earlier concerns (in Questions and Weakness section) have been addressed, and I appreciate the authors' efforts.

However, I am still not satisfied with the author's response on W2 (incremental contribution + dependence on the saturation analysis). Therefore, I would like to keep my original rating for now.

Reviewer_tZFhOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper uses the leading directions of the data covariance to explain intensity saturation in diffusion models and proposes a sampling-time method to reduce it. Building on a linear CFG decomposition, the authors analyze how mean shift and variance scaling change these coefficients, then damp large coefficients along the five leading directions according to their magnitude. The method lowers ECA@10 in all 12 settings, for example from 8.83 to 6.26 for EDM2-S with CFG, while precision and HPSv2.1 improve slightly in most settings.

Critical strengths

To explain the large over bright and over-dark regions in diffusion samples, the authors link them to initial noise projections on leading covariance eigenvectors. The experiments show a stronger linear relationship between initial projections and final coefficients along directions with larger covariance eigenvalues. Prediction based on this relationship also clearly outperforms the random Gaussian baseline.

The authors separate one sided and two sided saturation and remove individual initial noise components. The experiments report that removing u1 reduces one-sided saturated area by 65.1%, while the share of two-sided samples grows faster as the CFG weight increases.

To reduce saturation under strong guidance without extra denoiser evaluations, the authors propose a magnitude-dependent damping step. Table 2 reports lower ECA@10 in all 12 settings and higher HPSv2.1 in every setting.

Critical weaknesses
  1. The abstract and conclusion claim that the method improves image quality, but the experiments mainly establish a reduction in saturation. FD_DINOv2, FID and recall do not improve in many settings, and the quality gain is small when comparing the best setting for each model. This shows that a lower ECA@10 does not necessarily lead to better overall generation quality. I suggest reporting matched-recall results for APG and guidance interval, together with confidence intervals for the metrics.

  2. The central claim is that saturation mainly comes from a few leading covariance directions, which motivates damping only those directions. However, the experiments do not compare against random orthonormal directions under the same gain rule or against a global variance correction such as CFG rescaling. Without these controls, it is unclear whether the gain comes from selecting the leading directions or from generic shrinkage of latent magnitudes.

Questions for authors

With the same gain rule applied to five random orthonormal directions, how do ECA@10, precision and FD_DINOv2 change?

3 comments

Thank you for the detailed review.

W1. A lower ECA@10 does not necessarily lead to better overall generation quality.

Reducing ECA@10 improves the quality of particular saturated samples rather than overall generation quality because intensity saturation is concentrated in these samples. Figures 1 and 6 show that intensity saturation is concentrated in some samples. Our method therefore aims to reduce saturation in these samples while preserving samples without excessive saturation as much as possible, thereby maintaining the performance of the underlying guidance method.

Appendix Q (Fig. 29) shows that larger reductions in excess saturated area generally accompany greater improvements in precision and HPSv2.1.

SamplesHPSv2.1 gain ↑Precision gain ↑
Top 10%+0.54+2.90
All+0.20+1.01

The top 10% comprises samples with the largest saturation reductions. Improving the generation quality of these saturated samples is our contribution and the intended meaning of the image-quality claims in the abstract and conclusion. The small quality gains for APG and guidance interval are consistent with their low baseline intensity saturation, which leaves little room for improvement through saturation reduction. We will clarify this intended scope in the abstract and conclusion.


W1. Reporting matched-recall results.

We already report a related matched-recall comparison for CFG on EDM2-S in Appendix R.

For 2.25≤w≤5.52.25 \leq w \leq 5.5, our method improves ECA@10, precision, HPSv2.1, and FD_DINOv2 (Fig. 31), suggesting that the gains are not solely explained by the quality–diversity trade-off. We will extend the matched-recall evaluation to APG and guidance interval, report confidence intervals for the metrics, and include these additional results in a revised appendix within the next few days.

MethodECA@10HPSv2.1PrecisionFD_DINOv2Recall (matched)
CFG (w=5.5w=5.5)11.4524.350.68149.350.42
CFG + ours (w=4w=4)6.2624.920.80101.860.42

W2. Five random orthonormal directions under the same gain rule.

Damping five random orthonormal directions does not reproduce the saturation reduction obtained by selecting the leading covariance directions. Random 1–3 are three independently sampled sets of five orthonormal directions (r=5r=5).

We apply the same gain rule with its strength calibrated on 5,000 separate samples to produce approximately the same overall latent correction as ours, measured by cumulative normalized squared correction. The CFG setup follows Table 2, with evaluation on 10,000 matched samples per setting. Ours uses r=5r=5, zmax⁡=2z_{\max}=2.

DirectionsECA@10 ↓Precision ↑FD_DINOv2 ↓
CFG without damping8.950.79118.17
Leading covariance (ours)6.360.80115.42
Random 18.970.79117.76
Random 28.940.79118.18
Random 38.910.79118.11

W2. Generic shrinkage of latent magnitudes.

Appendix N shows that generic shrinkage of latent magnitudes through DT reduces saturation but does not reproduce our image-quality improvements, even at nearly identical intensity saturation levels. DT clips the predicted clean latent and divides all its coordinates by a common sample-dependent scale. We compare DT at q=0.95q=0.95 (ECA@10 of 6.30) with ours (6.26), using EDM2-S with CFG w=4w=4 and 50,000 images per setting (Tables 2 and 12).

MethodECA@10 ↓Precision ↑HPSv2.1 ↑FD_DINOv2 ↓
CFG8.830.7924.7494.5
CFG + DT (q=0.95q=0.95)6.300.7624.31111.6
CFG + ours6.26 (matched ECA)0.8024.9492.3

Figure 28 also shows that applying DT directly in latent space substantially degrades the quality of generated images, with muted colors, reduced contrast, and degraded fine detail.

Thank you for the additional experiments. I look forward to the promised matched-recall results for APG and guidance interval, together with the confidence intervals for the metrics. Once they are posted I will reassess the soundness score.

Thank you for your response. We used the EDM2-S setup in Table 2 and swept the APG and guidance interval baseline weights from 4 to 5.5 in increments of 0.25. We selected comparisons with an absolute recall difference within 0.003. Ours remains at w=4w=4 with r=5r=5 and zmax⁡=2z_{\max}=2. All other generation settings are the same as in Table 2 except for the number of generated images.

We evaluated the selected pairs using five independent initial noise seed sets with 10,000 images per condition per repeat. Each entry shows the mean ± half the width of its 95% confidence interval computed using the Student t distribution. Bold values indicate improved displayed means for Ours.

MethodECA@10 ↓Saturation ↓HPSv2.1 ↑Precision ↑FD_DINOv2 ↓FID ↓Recall (matched)
APG (w=5w=5)2.12 ± 0.070.32 ± 0.0024.02 ± 0.020.84 ± 0.0191.66 ± 0.856.54 ± 0.150.56 ± 0.00
APG + Ours (w=4w=4)0.80 ± 0.080.29 ± 0.0023.93 ± 0.020.84 ± 0.0093.48 ± 0.976.27 ± 0.110.56 ± 0.00
Guidance interval (w=4.75w=4.75)3.90 ± 0.100.39 ± 0.0024.69 ± 0.020.79 ± 0.0085.55 ± 0.436.67 ± 0.060.58 ± 0.00
Guidance interval + Ours (w=4w=4)2.06 ± 0.070.36 ± 0.0024.66 ± 0.020.81 ± 0.0080.30 ± 0.366.44 ± 0.040.58 ± 0.00

Values are rounded to two decimal places. An entry of ± 0.00 reflects rounding and does not indicate zero uncertainty.

Our method substantially reduces ECA at matched recall while broadly maintaining guidance performance.

Reviewer_qV8iOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper explains why intensity saturation, which frequently occurs when classifier-free guidance (CFG) commonly applied in diffusion models is applied with a high scale, occurs and presents a method to reduce it through this analysis. Building on studies showing that the early stage of sampling in diffusion models can be approximated as a linear denoiser, this paper shows that the principal eigenvector directions of the dataset covariance are closely related to brightness changes in the final image. Through this, it shows that as the similarity between the noise direction in early denoising steps and this eigenvector direction increases, intensity saturation occurs more frequently. Especially, it explains that when CFG is applied, it can sometimes amplify this effect, and based on this, it solves the saturation problem in the final image through a correction that projects the noisy image during sampling onto the eigenvector directions and reduces the coefficient when it has an excessively large value, verifying this in combination with various backbone diffusion models and guidance methodologies at inference time.

Critical strengths

Whereas works such as CFG++ claimed that the saturation problem caused by a strong extrapolation coefficient of CFG is an off-manifold issue and proposed manifold-constrained guidance, this paper has a strength in that it presents a method to solve this from a new perspective and shows that the top eigenvector directions of the dataset covariance can affect saturation. Also, it is another strength that it showed brightness correction is possible by additionally applying it on top of various guidance methods.

Critical weaknesses

A. As far as I know, the dynamic thresholding (clipping) methodology [a] in ComfyUI, which is widely used in the community, resolves the oversaturation issue to some extent simply by clipping in the latent space. Compared to a method solved in such a simple manner, I wonder how much more advantage this method (which requires computing the covariance in advance and performing projection in early inference steps) has. Furthermore, although the authors additionally applied this method on top of CFG++ (Chung et al., 2025) or Guidance Interval (Kynkaanniemi et al., 2024) as an add-on to existing guidance to obtain performance improvements and compared them in the main text, since the inherent advantages of these two methods also have the effect of preventing oversaturation, I think it is necessary to apply the authors' proposed method standalone (e.g., CFG + ours) and directly compare its performance against CFG++ and Guidance Interval in order to evaluate the effect of the proposed method itself.

Moreover, I am concerned whether there is a degradation in image quality. The FD(DINOv2) score, which mainly measures how well the image distribution is modeled, is not consistently improved, and looking at Table 15, except for one case in CFG++, FID tends to consistently increase (worsen) when the authors' methodology is applied. Only in one case of CFG++ does FID slightly decrease (which is presumed to be because the improvement effect of CFG was reduced in the process of reducing its influence for the manifold constrained guidance claimed in CFG++), and I wonder if a trade-off has occurred where saturation is reduced while diminishing the modeling capability of the conditional distribution, which is the original effect of CFG.

B. To sample according to the method claimed in this paper, statistics of the trained dataset are required, and instead of computing this on the ImageNet 1.3M dataset, the authors generate 50,000 images each with a pretrained diffusion model and compute the mean and covariance on top of this generated dataset. However, first, although diffusion models have developed recently, data generated without guidance cannot be said to have the exact same distribution as the original data, so there is a question of whether the statistics computed on this generated data can be considered similar to the actual dataset. Next, since 50,000 images for 50 selected classes needed for analysis correspond to a total of 2.5M images, plus an additional 50k unconditional images have to be generated, I am concerned that the cost required for generating these seems considerable. I wonder whether the authors have tried the proposed method with statistics obtained on the ImageNet real dataset, and how much difference there is between the statistics obtained on such generated data and the statistics obtained on the real dataset.

Questions for authors
  • In works intended to complement CFG, such as PAG [b] and Autoguidance [c], CFG-like extrapolation is also used, and it is known that saturation generally occurs in these methods as well when a high guidance scale is used. I wonder whether the same benefit of preventing saturation can be obtained in these methods if the method proposed in this paper is additionally applied.

[a] https://github.com/mcmonkeyprojects/sd-dynamic-thresholding

[b] Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance

[c] Guiding a Diffusion Model with a Bad Version of Itself

2 comments

Thank you for the detailed review.

A. Advantage over the ComfyUI dynamic thresholding

Our method’s advantage over ComfyUI DT lies in higher HPSv2.1 and precision scores, whereas DT primarily improves recall and FID. We evaluate the two DT variants from the repository you provided using Table 2's EDM2-S setup with mimic scale m=1m=1 and percentile p=0.99p=0.99.

MethodECA@10↓HPSv2.1↑Precision↑FID↓FD_DINOv2↓Recall↑
CFG8.8324.740.796.6894.530.37
CFG + DT Simple7.4224.710.796.3592.680.39
CFG + DT Full6.4124.670.786.1792.860.39
CFG + ours6.2624.940.806.7692.280.36

Are our image-quality gains merely due to CFG’s original trade-off?

No, our image-quality gains are not merely a consequence of CFG’s original trade-off. At matched recall, our method reduces intensity saturation while improving image quality (Appendix R). These results argue against interpreting saturation reduction as merely diminishing CFG's ability to model the conditional distribution.

MethodECA@10HPSv2.1PrecisionFD_DINOv2Recall (matched)
CFG (w=5.5w=5.5)11.4524.350.68149.350.42
CFG + ours (w=4w=4)6.2624.920.80101.860.42

A. Necessity of a standalone comparison.

Standalone comparisons across different guidance methods do not isolate “the effect of the proposed method itself”, because they change both the underlying guidance method and whether our correction is applied. For example, comparing “CFG + ours” against standalone CFG++ cannot distinguish the effect of our correction from differences between CFG and CFG++. We propose a saturation correction applicable across guidance methods, not a new guidance method built on CFG. CFG is used in Section 4 just to analyze how guidance affects intensity saturation, and “CFG + ours” is simply one application of our correction.

As shown in Figures 1 and 6, intensity saturation becomes excessive in particular samples. Regardless of the guidance method to which it is applied, our correction aims to reduce it in affected samples while preserving the underlying guidance method’s performance and leaving other samples as unchanged as possible.

Even if CFG++ and Guidance Interval already mitigate intensity saturation, our contribution is to further reduce the excessive saturation that remains. Therefore, to isolate the effect of our correction, we compare each guidance method used alone with the same method augmented by our correction, as in Table 2. This comparison keeps the underlying guidance method fixed and directly measures the additional benefit of our correction.


B. Similarity between generated-data and real-data statistics.

Generated-data and real-data statistics yield closely aligned leading covariance subspaces. The top-5 subspace overlap is 0.970 averaged across 50 class-conditional covariance pairs and 0.999 for the unconditional covariance pair.


B. Is the cost required for our method considerable?

No. As already stated in Section 5, “to reduce the cost of estimating statistics for every class, we approximate the mean and covariance of each class with the unconditional mean and covariance.” Thus, our method uses only 50k images, not 2.5M. This approximation is supported by conditional–unconditional eigenspace overlaps of 0.936 for the top-1 and 0.820 for the top-5 eigenspaces, consistent with the eigenvector alignment observed in Appendix D.


B. Proposed method with real-data statistics.

There is almost no difference in performance between using generated-data and real-data statistics in Table 2’s EDM2-S CFG experiment.

sourceECA@10HPSv2.1PrecisionFD_DINOv2Recall
Generated6.2624.940.8092.280.36
Real ImageNet6.1524.950.8092.010.36

Q1. Applicability to other guidance methods.

Our method further reduces intensity saturation when additionally applied to FDG, CFG-rescale, and EP-CFG, where CFG-like extrapolation is also used. We use r=5r=5 and zmax⁡=2.5z_{\max}=2.5 to reduce the regulation strength. All remaining experimental settings follow Table 2's EDM2-S setup. We will extend this evaluation to PAG and Autoguidance within the next few days.

MethodECA@10HPSv2.1PrecisionFIDFD_DINOv2
FDG (wlow,whigh)=(1,4)(w_{\mathrm{low}},w_{\mathrm{high}})=(1,4)3.6723.990.733.6595.89
FDG + ours2.0724.140.743.8795.09
CFG-rescale ϕ=0.5\phi=0.53.5624.640.805.9288.23
CFG-rescale + ours2.2424.690.816.0088.74
EP-CFG (plow,phigh)=(45,55)(p_{\mathrm{low}},p_{\mathrm{high}})=(45,55)0.7423.870.785.1494.46
EP-CFG + ours0.2023.810.785.3497.24

Following our earlier response, we have now extended the evaluation to PAG and Autoguidance on EDM2-S. For PAG, we use a guidance scale of s=2s=2 and perturb the bottleneck self-attention layer (8x8_in0). For Autoguidance, we use the official FID preset with w=2.1w=2.1, an EDM2-S main model (EMA 0.070), and an EDM2-XS guiding model trained for T/16T/16 (EMA 0.125). We apply both methods without additional classifier-free guidance. All other settings follow Table 2, including 50,000 samples per configuration and our regulation parameters (r=5r=5, zmax⁡=2z_{\max}=2).

MethodECA@10 ↓HSV saturation ↓Precision ↑Recall ↑HPS ↑FD_DINOv2 ↓FID ↓
PAG4.670.370.860.4324.9050.786.33
PAG + ours2.470.340.880.4125.0851.206.68
Autoguidance3.320.360.710.6623.3248.131.29
Autoguidance + ours1.330.330.730.6423.4948.531.44

Our method reduces intensity saturation (ECA@10) while largely preserving guidance performance, with small improvements in precision, HPS, and HSV saturation, and small trade-offs in recall, FD_DINOv2, and FID.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.