- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
Classifier-free guidance (CFG) is an important technique for conditional generation in diffusion models, but selecting an appropriate guidance strength remains challenging. Stronger guidance typically improves condition consistency at the cost of reduced distributional coverage. Existing studies have explored adaptive guidance schedules across different denoising stages to improve this trade-off, but a principled optimization framework is still lacking. To address this issue, this paper proposes an information-theoretic framework for adaptive CFG schedule optimization based on a clean endpoint reference distribution. By minimizing the discrepancy between the actual distribution induced by the guided sampler and the target reference distribution, the proposed method aims to achieve a better balance between condition consistency and distributional coverage. Experiments across multiple diffusion backbones demonstrate improved consistency–FID trade-offs, achieving higher condition consistency at comparable FID.
Critical strengths
-
Principled information-theoretic framework. The paper addresses the trade-off between condition consistency and distributional coverage in CFG, as well as the limitation that fixed-time tilted distributions do not necessarily characterize the actual distribution induced by the guided sampler. To address these issues, it proposes a theoretical framework based on a clean endpoint reference distribution.
-
Practical guidance schedule optimization. Building on this framework, the paper develops a practical guidance schedule optimization method that directly optimizes the guidance weights at each sampling step, without training an additional network or requiring real-image supervision.
-
Empirical improvements. The method is evaluated on both class-conditional and text-to-image generation tasks across four diffusion backbones. Experimental results demonstrate improved condition consistency–FID trade-offs, achieving the lowest FID on DiT-XL/2 and higher precision on both backbones.
Critical weaknesses
-
Misalignment between motivation and proposed method. The paper aims to balance condition consistency and distributional coverage, but the proposed method primarily addresses how to optimize the CFG schedule to match a given reference distribution. It remains insufficiently justified whether this reference distribution corresponds to the desired consistency–coverage trade-off in practice. To some extent, the method shifts the problem of selecting an appropriate guidance schedule to that of selecting a reference distribution, without providing a way to choose the reference parameter .
-
Lack of descent guarantees in the practical optimization algorithm. Under idealized assumptions, the trajectory-level identities provide an exact reformulation of the endpoint KL objective. However, as the authors explicitly acknowledge, "This calculation does not account for how the new weights change the sampler's later states." Consequently, the resulting update direction is not necessarily a descent direction for the actual endpoint KL objective, which can be demonstrated using a simple Gaussian example. Although the acceptance/rejection mechanism based on rerunning the sampler may prevent some unfavorable updates, it does not replace the need to validate the update directions and the convergence behavior of the optimization algorithm.
-
Insufficient comparison with related methods and baselines. The paper lacks sufficient discussion and experimental comparisons with recent related methods. For example, Adaptive Diffusion Guidance via Stochastic Optimal Control (https://arxiv.org/abs/2505.19367) optimizes guidance weights from a stochastic optimal control perspective and is closely related to this work, but it is neither cited nor discussed or included in the experimental comparisons. Furthermore, is only evaluated on SD 1.5 and SDXL, but not on DiT-XL/2 or EDM2-S. The paper also does not clarify what is meant by "where applicable" in its baseline selection.
-
Insufficient analysis of recall degradation and distributional coverage. The paper aims to balance condition consistency and distributional coverage, but its main empirical conclusions rely heavily on FID, which alone cannot establish whether distributional coverage has improved. Moreover, Table 1 shows that the proposed method achieves lower recall than Interval Guidance on both backbones. This raises the question of whether improvements in other metrics come at the expense of distributional coverage, which requires further investigation and explanation. Although precision–recall curves are already provided in the appendix, additional trade-off curves between recall and other evaluation metrics are still needed. Furthermore, in the precision–recall comparisons (Figure 6), the methods perform very similarly on DiT-XL/2, while Interval Guidance exhibits a better precision–recall trade-off on EDM2-S.
Questions for authors
Please see the weaknesses.
2 comments
W2. Empirical validation (continued)
Gaussian Validation: Update Directions and Convergence Behavior (15 Trials)
| λ | Initial cosine ↑ | Minimum cosine ↑ | Accepted / Trials | Exact gradient norm (initial → final) ↓ | Endpoint KL reduction ↑ |
|---|---|---|---|---|---|
| 2 | 0.987 | 0.894 | 15/15 | 0.045 → 0.007 | 62% |
| 3 | 0.981 | 0.915 | 15/15 | 0.081 → 0.020 | 69% |
| 5 | 0.977 | 0.899 | 15/15 | 0.154 → 0.039 | 75% |
Gaussian example from Section 2.2 with 400 Euler steps. Cosine similarity measures the alignment between the fixed-trajectory derivative and the exact gradient of the finite-grid trajectory objective. Endpoint KL is computed independently from the densities of the generated samples.
Gaussian result. All 45 proposals are accepted, and all update directions are descent directions for the finite-grid objective (minimum cosine ≥ 0.894). Across the three runs, endpoint KL decreases by 62–75% within 15 trials.
SD1.5 result. We evaluated each schedule by rerunning the full sampler on 64 samples. As shown in Figure 6 of the revised manuscript, the objective decreases over the first five trials and then approximately plateaus.
W3. Insufficient comparison with related methods and baselines
Our intention was to ensure a fair comparison, rather than exclude β-CFG. β-CFG requires additional schedule hyperparameters. Its original paper only evaluates text-to-image generation and does not provide validated configurations for class-conditional generation. Applying those settings directly to DiT-XL/2 or EDM2-S could therefore underestimate its performance. We included β-CFG on SD 1.5 and SDXL using the configurations supported by the original work. This is what we meant by “where applicable.”
We thank the reviewer for pointing out Adaptive Diffusion Guidance via Stochastic Optimal Control (SOC). We also evaluated SOC on EDM2-S. As shown in Figure 9 of the revised manuscript, our method achieves a better classification accuracy–FID trade-off than SOC. Moreover, our method uses substantially fewer samples for optimization: SOC uses 20,000 samples over 200 iterations (100 per iteration), whereas ours uses a pool of 512 trajectories over 60 iterations (8 per iteration). On the same GPU model (RTX PRO 6000), optimization for a single setting of each method takes 17 min 14 s for Ours versus approximately 12 h 59 min for SOC. This corresponds to an approximately 45× speedup under their respective optimization settings.
W4. Insufficient analysis of recall degradation and distributional coverage
Table 1 reports the lowest-FID operating point for each method, not the highest-recall point. Lower recall at these selected points therefore does not necessarily imply a worse consistency–coverage trade-off across operating points.
Our primary evaluation focuses on consistency–FID. We use accuracy or CLIP score to assess condition consistency and FID, a widely used metric, to assess overall distributional quality. FID reflects both fidelity and coverage, but is not a direct measure of recall. Precision–recall does not directly correspond to our consistency–coverage objective, because precision measures sample fidelity rather than condition consistency.
To directly address the reviewer's concern, we additionally provide accuracy–recall trade-off curves for DiT-XL/2 and EDM2-S using the same operating points, as shown in Figure 10 of the revised manuscript.
W1. Misalignment between motivation and proposed method
Matching the clean reference distribution is mathematically equivalent to optimizing the consistency–coverage trade-off; it does not introduce a separate objective. Lemma 1 (Eqs. 15–16) establishes this equivalence and shows that the clean reference is the global minimizer of the consistency–coverage objective in Eq. (16). Eq. (52) further shows that λ determines the slope of the optimal trade-off frontier.
Our goal with λ is to improve the consistency–coverage trade-off frontier, rather than find a single λ that optimizes one metric (e.g., achieves the lowest FID). Figures 1 and 3 show that our method achieves higher consistency at comparable FID, or lower FID at comparable consistency, than constant CFG and the other scheduling baselines. Therefore, we do not prescribe a single optimal λ. As with guidance strength in conventional CFG, users can choose λ from the trade-off curve based on the consistency–coverage balance they prefer.
W2. Lack of descent guarantees in the practical optimization algorithm
We agree that the fixed-trajectory derivative is not the full gradient of the endpoint objective. We already derive the full trajectory objective in Eq. (109), but differentiating through the entire sampling trajectory incurs substantially higher computational and memory costs than fixed-trajectory updates. We therefore use the fixed-trajectory derivative to propose updates, then rerun the sampler to measure the actual objective change. We now explicitly characterize the omitted trajectory dependence and establish descent and finite-trial convergence guarantees, with theoretical analysis in Appendix H.3 and Gaussian and SD1.5 validations in Appendix H.4.
Our goal is to optimize the CFG guidance schedule w by minimizing the consistency–coverage objective in Eq. (16):
L0(w;λ)=−λEp(y)q0w[logp0(y∣x)]+Ep(y)DKL(q0w(⋅∣y)∥p0).
The first term measures consistency and the second coverage. Because the density of q0w is not explicitly available, we use the trajectory formulation in Eq. (109).
At each iteration, we generate trajectories under the current schedule and differentiate the objective while holding these trajectories fixed. The full gradient decomposes exactly as
∇Ltraj(w;λ)=G(w)+e(w).
Here G(w) is the fixed-trajectory derivative and e(w) is the omitted trajectory dependence, with components
Gj(w)=ajE[−Aj+(1−λ)Rj].
ej(w)=∑k<jakE[∇xϕk(Xk,wk)⊤∂wj∂Xkw].
We define ϕk(x,wk)=(λ−wk)Ak(x)+wk(1−λ)Rk(x). Lemma A characterizes e(w) through a discrete adjoint, and Proposition B bounds it using score differences and downstream Jacobians.
Theorem (Descent and Finite-Trial Convergence)
Suppose the regularity conditions in Appendix H.3 hold, the full trajectory objective is L-smooth and bounded below, and the TR optimizer uses exact evaluations and untruncated proposals.
Descent guarantee. The untruncated fixed-trajectory update decreases the full-rerun objective whenever
∥G(w)∥>∥e(w)∥+2LrTR(m).
Here rTR(m) is the trust-region radius at optimization trial m.
Finite-trial convergence. After M≥11 TR trials, including rejected proposals, the optimizer satisfies
minm<M∥∇Ltraj(wm;λ)∥≤εTR+MCTR.
Here εTR depends on the omitted trajectory dependence, minimum TR radius, and numerical acceptance threshold, while CTR is independent of M. Both constants are characterized explicitly in Appendix H.3.
Under these assumptions, despite the biased update direction, at least one of the first M iterates satisfies an approximate stationarity bound, with an O(1/M) optimization term and a nonzero limiting approximation error.