acceptodds
Under review as a conference paper at ICLR 2027

Information-Theoretic Classifier-Free Guidance with Adaptive Schedule Optimization

Abstract

Diffusion models have achieved strong performance in image, text-to-image, and video generation, where conditional generation is often controlled by classifier-free guidance (CFG). CFG improves condition consistency by increasing a guidance weight, but stronger guidance typically reduces diversity and distributional coverage. It remains unclear how this consistency-coverage trade-off should be controlled across the reverse trajectory, since the distribution induced by CFG is not simply the fixed-time tilted distribution given by the guided score field. To address this issue, we propose an information-theoretic framework for CFG schedule optimization. Our approach uses a clean endpoint reference to specify the desired consistency-coverage trade-off, while optimizing the actual distribution induced by the guided sampler toward this reference. We derive trajectory-level formulas to estimate the objective from samples and score evaluations, avoiding explicit density estimation. Across four diffusion backbones on ImageNet-512 and COCO, the learned schedules achieve improved consistency–FID trade-offs, with higher condition consistency at comparable FID, than constant and non-uniform guidance baselines.

Primary area:
generative models
Keywords:
Classifier-Free Guidance, Diffusion Models, Adaptive Guidance Scheduling, Conditional Generation, Information Theory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DpePOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

Classifier-free guidance (CFG) is an important technique for conditional generation in diffusion models, but selecting an appropriate guidance strength remains challenging. Stronger guidance typically improves condition consistency at the cost of reduced distributional coverage. Existing studies have explored adaptive guidance schedules across different denoising stages to improve this trade-off, but a principled optimization framework is still lacking. To address this issue, this paper proposes an information-theoretic framework for adaptive CFG schedule optimization based on a clean endpoint reference distribution. By minimizing the discrepancy between the actual distribution induced by the guided sampler and the target reference distribution, the proposed method aims to achieve a better balance between condition consistency and distributional coverage. Experiments across multiple diffusion backbones demonstrate improved consistency–FID trade-offs, achieving higher condition consistency at comparable FID.

Critical strengths
  1. Principled information-theoretic framework. The paper addresses the trade-off between condition consistency and distributional coverage in CFG, as well as the limitation that fixed-time tilted distributions do not necessarily characterize the actual distribution induced by the guided sampler. To address these issues, it proposes a theoretical framework based on a clean endpoint reference distribution.

  2. Practical guidance schedule optimization. Building on this framework, the paper develops a practical guidance schedule optimization method that directly optimizes the guidance weights at each sampling step, without training an additional network or requiring real-image supervision.

  3. Empirical improvements. The method is evaluated on both class-conditional and text-to-image generation tasks across four diffusion backbones. Experimental results demonstrate improved condition consistency–FID trade-offs, achieving the lowest FID on DiT-XL/2 and higher precision on both backbones.

Critical weaknesses
  1. Misalignment between motivation and proposed method. The paper aims to balance condition consistency and distributional coverage, but the proposed method primarily addresses how to optimize the CFG schedule to match a given reference distribution. It remains insufficiently justified whether this reference distribution corresponds to the desired consistency–coverage trade-off in practice. To some extent, the method shifts the problem of selecting an appropriate guidance schedule to that of selecting a reference distribution, without providing a way to choose the reference parameter λ\lambda.

  2. Lack of descent guarantees in the practical optimization algorithm. Under idealized assumptions, the trajectory-level identities provide an exact reformulation of the endpoint KL objective. However, as the authors explicitly acknowledge, "This calculation does not account for how the new weights change the sampler's later states." Consequently, the resulting update direction is not necessarily a descent direction for the actual endpoint KL objective, which can be demonstrated using a simple Gaussian example. Although the acceptance/rejection mechanism based on rerunning the sampler may prevent some unfavorable updates, it does not replace the need to validate the update directions and the convergence behavior of the optimization algorithm.

  3. Insufficient comparison with related methods and baselines. The paper lacks sufficient discussion and experimental comparisons with recent related methods. For example, Adaptive Diffusion Guidance via Stochastic Optimal Control (https://arxiv.org/abs/2505.19367) optimizes guidance weights from a stochastic optimal control perspective and is closely related to this work, but it is neither cited nor discussed or included in the experimental comparisons. Furthermore, β−CFG\beta-CFG is only evaluated on SD 1.5 and SDXL, but not on DiT-XL/2 or EDM2-S. The paper also does not clarify what is meant by "where applicable" in its baseline selection.

  4. Insufficient analysis of recall degradation and distributional coverage. The paper aims to balance condition consistency and distributional coverage, but its main empirical conclusions rely heavily on FID, which alone cannot establish whether distributional coverage has improved. Moreover, Table 1 shows that the proposed method achieves lower recall than Interval Guidance on both backbones. This raises the question of whether improvements in other metrics come at the expense of distributional coverage, which requires further investigation and explanation. Although precision–recall curves are already provided in the appendix, additional trade-off curves between recall and other evaluation metrics are still needed. Furthermore, in the precision–recall comparisons (Figure 6), the methods perform very similarly on DiT-XL/2, while Interval Guidance exhibits a better precision–recall trade-off on EDM2-S.

Questions for authors

Please see the weaknesses.

2 comments

W1. Misalignment between motivation and proposed method

Matching the clean reference distribution is mathematically equivalent to optimizing the consistency–coverage trade-off; it does not introduce a separate objective. Lemma 1 (Eqs. 15–16) establishes this equivalence and shows that the clean reference is the global minimizer of the consistency–coverage objective in Eq. (16). Eq. (52) further shows that λ\lambda determines the slope of the optimal trade-off frontier.

Our goal with λ\lambda is to improve the consistency–coverage trade-off frontier, rather than find a single λ\lambda that optimizes one metric (e.g., achieves the lowest FID). Figures 1 and 3 show that our method achieves higher consistency at comparable FID, or lower FID at comparable consistency, than constant CFG and the other scheduling baselines. Therefore, we do not prescribe a single optimal λ\lambda. As with guidance strength in conventional CFG, users can choose λ\lambda from the trade-off curve based on the consistency–coverage balance they prefer.

 

W2. Lack of descent guarantees in the practical optimization algorithm

We agree that the fixed-trajectory derivative is not the full gradient of the endpoint objective. We already derive the full trajectory objective in Eq. (109), but differentiating through the entire sampling trajectory incurs substantially higher computational and memory costs than fixed-trajectory updates. We therefore use the fixed-trajectory derivative to propose updates, then rerun the sampler to measure the actual objective change. We now explicitly characterize the omitted trajectory dependence and establish descent and finite-trial convergence guarantees, with theoretical analysis in Appendix H.3 and Gaussian and SD1.5 validations in Appendix H.4.

Our goal is to optimize the CFG guidance schedule ww by minimizing the consistency–coverage objective in Eq. (16):

L0(w;λ)=−λEp(y)q0w[log⁡p0(y∣x)]+Ep(y)DKL(q0w(⋅∣y)∥p0).\mathcal{L}_0(w;\lambda)=-\lambda\mathbb{E}_{p(y)q_0^w}[\log p_0(y\mid x)]+\mathbb{E}_{p(y)}D_{\mathrm{KL}}(q_0^w(\cdot\mid y)\Vert p_0).

The first term measures consistency and the second coverage. Because the density of q0wq_0^w is not explicitly available, we use the trajectory formulation in Eq. (109).

At each iteration, we generate trajectories under the current schedule and differentiate the objective while holding these trajectories fixed. The full gradient decomposes exactly as

∇Ltraj(w;λ)=G(w)+e(w).\nabla\mathcal{L}_{\mathrm{traj}}(w;\lambda)=G(w)+e(w).

Here G(w)G(w) is the fixed-trajectory derivative and e(w)e(w) is the omitted trajectory dependence, with components

Gj(w)=ajE[−Aj+(1−λ)Rj].G_j(w)=a_j\mathbb{E}[-A_j+(1-\lambda)R_j].

ej(w)=∑k<jakE[∇xϕk(Xk,wk)⊤∂Xkw∂wj].e_j(w)=\sum_{k < j}a_k\mathbb{E}[\nabla_x\phi_k(X_k,w_k)^\top\frac{\partial X_k^w}{\partial w_j}].

We define ϕk(x,wk)=(λ−wk)Ak(x)+wk(1−λ)Rk(x)\phi_k(x,w_k)=(\lambda-w_k)A_k(x)+w_k(1-\lambda)R_k(x). Lemma A characterizes e(w)e(w) through a discrete adjoint, and Proposition B bounds it using score differences and downstream Jacobians.

Theorem (Descent and Finite-Trial Convergence)

Suppose the regularity conditions in Appendix H.3 hold, the full trajectory objective is LL-smooth and bounded below, and the TR optimizer uses exact evaluations and untruncated proposals.

Descent guarantee. The untruncated fixed-trajectory update decreases the full-rerun objective whenever

∥G(w)∥>∥e(w)∥+LrTR(m)2.\lVert G(w)\rVert>\lVert e(w)\rVert+\frac{Lr_{\mathrm{TR}}^{(m)}}{2}.

Here rTR(m)r_{\mathrm{TR}}^{(m)} is the trust-region radius at optimization trial mm.

Finite-trial convergence. After M≥11M\geq 11 TR trials, including rejected proposals, the optimizer satisfies

min⁡m<M∥∇Ltraj(wm;λ)∥≤εTR+CTRM.\min_{m < M}\lVert\nabla\mathcal{L}_{\mathrm{traj}}(w_m;\lambda)\rVert\leq\varepsilon_{\mathrm{TR}}+\frac{C_{\mathrm{TR}}}{M}.

Here εTR\varepsilon_{\mathrm{TR}} depends on the omitted trajectory dependence, minimum TR radius, and numerical acceptance threshold, while CTRC_{\mathrm{TR}} is independent of MM. Both constants are characterized explicitly in Appendix H.3.

Under these assumptions, despite the biased update direction, at least one of the first MM iterates satisfies an approximate stationarity bound, with an O(1/M)O(1/M) optimization term and a nonzero limiting approximation error.

W2. Empirical validation (continued)

Gaussian Validation: Update Directions and Convergence Behavior (15 Trials)

λInitial cosine ↑Minimum cosine ↑Accepted / TrialsExact gradient norm (initial → final) ↓Endpoint KL reduction ↑
20.9870.89415/150.045 → 0.00762%
30.9810.91515/150.081 → 0.02069%
50.9770.89915/150.154 → 0.03975%

Gaussian example from Section 2.2 with 400 Euler steps. Cosine similarity measures the alignment between the fixed-trajectory derivative G(w)G(w) and the exact gradient of the finite-grid trajectory objective. Endpoint KL is computed independently from the densities of the generated samples.

Gaussian result. All 45 proposals are accepted, and all update directions are descent directions for the finite-grid objective (minimum cosine ≥ 0.894). Across the three runs, endpoint KL decreases by 62–75% within 15 trials.

SD1.5 result. We evaluated each schedule by rerunning the full sampler on 64 samples. As shown in Figure 6 of the revised manuscript, the objective decreases over the first five trials and then approximately plateaus.

 

W3. Insufficient comparison with related methods and baselines

Our intention was to ensure a fair comparison, rather than exclude β-CFG. β-CFG requires additional schedule hyperparameters. Its original paper only evaluates text-to-image generation and does not provide validated configurations for class-conditional generation. Applying those settings directly to DiT-XL/2 or EDM2-S could therefore underestimate its performance. We included β-CFG on SD 1.5 and SDXL using the configurations supported by the original work. This is what we meant by “where applicable.”

We thank the reviewer for pointing out Adaptive Diffusion Guidance via Stochastic Optimal Control (SOC). We also evaluated SOC on EDM2-S. As shown in Figure 9 of the revised manuscript, our method achieves a better classification accuracy–FID trade-off than SOC. Moreover, our method uses substantially fewer samples for optimization: SOC uses 20,000 samples over 200 iterations (100 per iteration), whereas ours uses a pool of 512 trajectories over 60 iterations (8 per iteration). On the same GPU model (RTX PRO 6000), optimization for a single setting of each method takes 17 min 14 s for Ours versus approximately 12 h 59 min for SOC. This corresponds to an approximately 45× speedup under their respective optimization settings.

 

W4. Insufficient analysis of recall degradation and distributional coverage

Table 1 reports the lowest-FID operating point for each method, not the highest-recall point. Lower recall at these selected points therefore does not necessarily imply a worse consistency–coverage trade-off across operating points.

Our primary evaluation focuses on consistency–FID. We use accuracy or CLIP score to assess condition consistency and FID, a widely used metric, to assess overall distributional quality. FID reflects both fidelity and coverage, but is not a direct measure of recall. Precision–recall does not directly correspond to our consistency–coverage objective, because precision measures sample fidelity rather than condition consistency.

To directly address the reviewer's concern, we additionally provide accuracy–recall trade-off curves for DiT-XL/2 and EDM2-S using the same operating points, as shown in Figure 10 of the revised manuscript.

Reviewer_oDxBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper analyzes CFG under the lens of a discrepancy between the distribution obtained by following the guided diffusion and a tilted clean distribution. It proposes an approximation of that discrepancy that can be computed by sampling trajectories with corrected weights and details an algorithm for updating the guidance weights so as to match the tilted clean distribution. This leads to a trade-off between condition adherence and uncondotional distribution following. Experiments are performed using SD1.5 and DiT-XL and show improvements over other guidance scheduling.

Critical strengths

The paper identifies a gap between the distribution produced by following guided diffusion and a tilted clean distribution, and proposes a way to compute it by integrating known quantities along the diffusion path.

Critical weaknesses

The paper is very difficult to follow, with an abundance of unnecessary complex notations. The 3 lemmas could probably have been condensed into a simpler objective and then explain where it comes from rather than the bottom up, although all derivations seem correct (I verified only the first one and I believe it is correct). Also, the motivation behind the clean tilted distribution is not clear. Yes, at every step, CFG corresponds to sampling from a tilted distribution. Yes the endpoint distribution of such trajectories does not correspond to tilting the clean distribution. But why would that matter? Why would one want to sample from the tilted clean distribution? Because it allows the user to tune a trade-off between condition adherence and fitting the unconditional distribution by introducing an additional parameter? I think I'm starting to make sense of the idea while writing this, but the presentation is very convoluted.

The proposed schedule requires additional training ("unsupervised") that competing methods do not. Each schedule w is trained specifically for a task and a model. This gives so many degrees of freedom that I am conflicted on whether the results are better because the method is genuinely interesting or if it's because of over-fitting these tasks. It was shown in "Analysis of Classifier-Free Guidance Weight Schedulers" by Wang et al. (TMLR 2024) that dynamic guidance schedules can easily overfit.

Questions for authors

I don't see any reason why the method would not work with flow matching (just reconstruct the score from the velocity). The experiments are using really dated models, which totally limit the practical use of the method. Since the training is not particularily intensive (a few hours as per the paper), I would like to see more modern architectures like SD3/Flux and SiT/JiT that also use CFG and smaller number of steps, because the experiments seem very limited to older models that nobody uses anymore.

1 comment

W1. Motivation for the clean reference distribution

As discussed in Section 2, we choose the clean reference at t=0t=0 because we want to control the consistency–coverage trade-off of the final generated distribution. The mismatch matters because setting the CFG tilt at each noise level does not tell us what trade-off the final samples will achieve. Figure 2 and Appendix Figure 5 illustrate the problem: at comparable consistency, CFG can produce overly concentrated samples with higher coverage KL than the clean reference. Lemma 1 also shows that the clean reference is the global minimizer of our consistency–coverage objective, providing a theoretical justification for using it as the optimization target.

 

W2. Potential overfitting of learned schedules

Wang et al. (2024) do not demonstrate overfitting to optimization samples. They show that the best schedule parameters depend on the model and task, stating that “specialized tuning is required for each model and task” (Sec. 6.3), and such tuning can improve generation performance. This supports our motivation for optimizing schedules for each model.

Our improvements are not simply due to overfitting. First, all guidance schedulers determine weights across all timesteps, not just ours. Our schedule is optimized through Eq. (19), with only λ\lambda as the tunable trade-off parameter. Second, we do not optimize evaluation metrics: FID, CLIP, and classification accuracy are never used in our information-theoretic optimization (Eq. 19). Third, the improvements generalize beyond the optimization samples: schedules learned using only 64 prompts for SDXL or 512 trajectories for EDM2-S improve trade-offs on 5K COCO and 10K ImageNet evaluations.

 

W3. Evaluation on modern architectures and flow matching

We thank the reviewer for suggesting flow-matching models. We have now evaluated our method on SiT-XL/2 and SD3 Medium, both flow-matching backbones. Figures 3(c) and 3(d) in the revised manuscript show that our method achieves better accuracy–FID and CLIP–FID trade-offs, respectively, than Constant CFG. These results extend our empirical evaluation beyond the diffusion backbones used in the original experiments.

We chose the backbones for our initial experiments to allow fair comparisons with existing guidance schedulers. Our method optimizes a model-specific information-theoretic objective, while the baselines use predefined curves or intervals with manually chosen hyperparameters. Using these baselines on new architectures without validated settings could make the comparisons unfair. We therefore started with backbones that had established baseline configurations.

Reviewer_uRUoOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an objective function to adjust the guidance weights of classifier-free guidance. The main problem is rooted from the trade-off between consistency and coverage, and its challenge in selecting the weights for all the steps. The authors select out a batch of samples, called as reference. They use this clean reference set to optimize the guidance weight through an trajectory objective approximated by Riemann sum. The results show some improvement

Critical strengths
  1. The paper's propose is simple
  2. there are some slight consistent improvement over several baselines
  3. The problem might be real.
Critical weaknesses
  1. The improvement is too slight. 5.33 vs 5.29 is not noticeable on qualitative. The improvement here might come from statistics bias instead of real improvement. Please run with different seeds.

  2. The baselines are too outdates. DiT and EDM have been several years ago, newest models such as Flow-matching models which is becoming more and more popular is not getting enough attention.

  3. How does the model work on very large number of steps or very few number of steps?

  4. The authors fixed the guidance weight to 1 at the start and the end of the sampling, this one is questionable because it is very close to IntervalGuidance. The main improvement might really come from Interval guidance.

  5. The presentation is very hard to understand. The most of the algorithm is not presented in the paper with lacking a lot of details.

  6. Figure 2 is not reproducable due to the lack of hyper-parameters, while this Figure 2 is the main motivation of the paper.

Questions for authors

See weaknesses

2 comments

Clarification of the Paper Summary

We do not select real data or generated samples to form a reference set. The clean reference in Eq. (14) is a mathematically defined target distribution, not a dataset. The samples in Eq. (19) are noisy or latent states generated by the current CFG sampler and used only to estimate the objective. Updating the guidance schedule also changes these trajectories, so they do not form a fixed reference set. Our optimization therefore requires neither real image supervision nor samples from the reference distribution.

 

W1. Robustness across random seeds

Our main empirical contribution is a better consistency–FID trade-off, not just a small reduction in the minimum FID. The 5.33 vs. 5.29 comparison captures only each method's lowest-FID operating point, while Figure 1 shows improvements across the accuracy–FID trade-off curve.

Random seedsOurs FID ↓Interval FID ↓
0–9,9995.295.33
10,000–19,9995.245.29
20,000–29,9995.355.37
Mean (three 10K groups)5.305.33

To assess variation across random seeds, we additionally evaluated DiT-XL/2 using three disjoint random seed groups (30K generated samples per method in total). Our method achieves lower FID in all three seed groups, providing additional evidence that the improvement is not limited to a single seed range.

 

W2. Evaluation on modern architectures and flow matching

We thank the reviewer for suggesting flow-matching models. We have now evaluated our method on SiT-XL/2 and SD3 Medium, both flow-matching backbones. Figures 3(c)3(c) and 3(d)3(d) in the revised manuscript show that our method achieves better accuracy–FID and CLIP–FID trade-offs than Constant CFG, respectively. These results extend our empirical evaluation beyond the diffusion backbones used in the original experiments.

We chose the backbones for our initial experiments to allow fair comparisons with existing guidance schedulers. Our method optimizes a model-specific information-theoretic objective, while the baselines use predefined curves or intervals with manually chosen hyperparameters. Using these baselines on new architectures without validated settings could make the comparisons unfair. We therefore started with backbones that had established baseline configurations.

 

W3. Performance with different sampling steps

To assess performance with different numbers of sampling steps, we evaluated EDM2-S with 8, 32, and 128 steps, generating 10K images per method at each step count.

StepsMethodFID ↓Accuracy (%) ↑IS ↑
8Constant CFG10.3980.25152.60
8Ours10.1983.88159.31
32Constant CFG5.2788.59209.66
32Ours4.8189.10215.02
128Constant CFG5.2588.66210.83
128Ours4.7989.33216.40

Our method outperforms Constant CFG at all three step counts, achieving lower FID, higher classification accuracy, and higher IS. The improvements persist when the number of sampling steps is reduced to 8 or increased to 128.

 

W4. Endpoint freezing vs. Interval Guidance

Fixing the endpoint weights at w=1w=1 does not identify an effective guidance interval, nor does arbitrarily adjusting the remaining weights guarantee better generation. Wang et al. (2024) further show that guidance schedule shape matters and that optimal scheduling parameters vary across models and tasks, challenging the assumption of a universally optimal guidance interval. Endpoint freezing only excludes steps that are ineffective or numerically sensitive, as explained in Appendix I.1. The effective guidance allocation must still be determined through optimization.

Our method determines both where and how strongly to apply guidance by optimizing the consistency–coverage objective, rather than prescribing a guidance interval and a constant weight as in Interval Guidance. Specifically, Eq. (19) optimizes the timestep-wise weights using the actual CFG trajectories. Figures 4 and 11 show that the resulting schedules differ from Interval Guidance in both their effective guidance regions and weight profiles, while Figures 1 and 3 demonstrate better consistency–FID trade-offs than Interval Guidance itself. These results indicate that the gains are not simply due to restricting guidance to an interval.

W5. Algorithm presentation and implementation details

We thank the reviewer for raising this concern. The complete optimization procedure was already included in our original submission (Appendix H.2, Algorithm 1), covering the update direction, constrained optimization, and trust-region acceptance/rejection rules. The main text (Sections 3–4) presents the motivation, derives the trajectory objective, and explains the key optimization principles, while the appendix provides the detailed procedure. We will make the main text reference to this existing algorithm more explicit so readers can find the full procedure more easily.

 

W6. Reproducibility of Figure 2

We thank the reviewer for pointing out the missing experimental details. We omitted several simulation parameters when presenting the Gaussian analysis. The full configuration for reproducing Figure 2 is: Gaussian means μ±=±(0.85,0.55)\mu_{\pm}=\pm(0.85,0.55), covariance Σ=(0.160.140.140.42)\Sigma=\begin{pmatrix}0.16 & 0.14 \\ 0.14 & 0.42\end{pmatrix}, terminal noise σT=5\sigma_T=5, an RK4 solver with 400 steps, 6,000 samples per class, and random seed 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.