- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Curvature-Aware Classifier-Free Guidance (CACFG), which reformulates CFG as a continuous-time optimal control problem using the Hamilton–Jacobi–Bellman (HJB) equation. The authors show that conventional CFG can be recovered under specific path costs and an unconstrained control set. Based on this formulation, they introduce a hyperspherical constraint derived from the Gaussian regularization of VAE latent representations to mitigate high-guidance degradation. Experiments on SD1.5 and SDXL demonstrate improvements in image quality and diversity, particularly at high guidance scales.
Critical strengths
S1. Optimal control interpretation of CFG
Although prior studies have explored diffusion guidance from an optimal control perspective [1,2], this paper presents a formulation that recovers conventional CFG through a deterministic HJB framework and provides a unified interpretation of existing guidance scheduling methods.
S2. Improved generation quality with negligible computational overhead
The proposed CACFG can be applied without additional training or significant computational overhead. Experimental results demonstrate improvements in FID, perceptual quality, and diversity-related metrics over standard CFG, particularly at high guidance scales.
[1] K. Pandey, F. M. Sofian, F. Draxler, T. Karaletsos, and S. Mandt, "Variational Control for Guidance in Diffusion Models," Proceedings of the 42nd International Conference on Machine Learning (ICML), https://arxiv.org/pdf/2502.03686.
[2] I. Azangulov, P. Potaptchik, Q. Li, E. Aamari, G. Deligiannidis, and J. Rousseau, "Adaptive Diffusion Guidance via Stochastic Optimal Control," Proceedings of the 29th International Conference on Artificial Intelligence and Statistics (AISTATS), 2026, https://arxiv.org/pdf/2505.19367.
Critical weaknesses
W1. Insufficient validation of the proposed failure mechanism
The authors argue that unconstrained control is a primary cause of high-guidance degradation. However, CACFG still exhibits increasing FID as the guidance scale increases (Appendix Tables 4–5). While the results demonstrate mitigation of degradation compared to standard CFG, they do not sufficiently establish that the proposed Gaussian norm constraint addresses the underlying failure mechanism. In particular, the relationship between norm-bound violations and actual generation degradation remains unclear.
W2. Unclear quality–alignment tradeoff
Although CACFG improves FID over standard CFG at high guidance scales, it remains unclear whether these improvements are achieved without sacrificing prompt alignment. The paper reports both FID and CLIP Score, but does not directly establish whether CACFG achieves better generation quality at comparable levels of text alignment. This is important for distinguishing a genuine improvement in the quality–alignment tradeoff from simply reducing the effective guidance strength.
W3. Limited evaluation on modern generative models
The experiments are limited to SD1.5 and SDXL. Since the proposed projection-based method does not directly depend on a specific UNet architecture, its effectiveness on modern diffusion transformers and flow-matching models, such as FLUX, remains unexplored. Furthermore, the validity of the Gaussian norm constraint across different latent representations is unclear.
Overall Assessment: Borderline Reject
The paper provides an optimal control interpretation of CFG and proposes a computationally inexpensive guidance correction. However, the experimental evidence primarily demonstrates mitigation rather than elimination of high-guidance degradation. The relationship between the proposed constraint and the underlying failure mechanism is not sufficiently validated, and the practical quality–alignment advantage and generalizability of the method remain unclear.
I would consider raising my rating to Borderline Accept if the authors could address the following points:
-
Failure mechanism validation: Provide an analysis relating Gaussian norm-bound violations to generation quality degradation, ideally including comparisons with simpler norm clipping or guidance rescaling baselines, to clarify whether the proposed constraint specifically addresses the claimed failure mechanism.
-
Quality–alignment tradeoff: Present an FID–CLIP Score Pareto frontier across guidance scales to demonstrate whether CACFG achieves better generation quality at comparable levels of prompt alignment than standard CFG.
-
Generalization: Provide evidence of applicability to a modern diffusion transformer or flow-matching model, or clarify the assumptions and limitations that prevent such an extension.
Questions for authors
Please refer to the Weaknesses section for my questions and concerns.
7 comments
(2/2)
(W3): The reviewer notes that the experiments are limited to SD1.5 and SDXL, and questions the validity of the Gaussian norm constraint across different latent representations.
Our derivation assumes the variance-preserving flow ODE dynamics of Eq. 2, so extending the optimal control result to flow-matching models requires re-deriving the HJB solution under flow-matching dynamics. Furthermore, FLUX.1 [dev] is guidance-distilled [1], and does not use CFG at inference. However, we agree that evaluating CACFG on a diffusion transformer would strengthen the paper, and we are currently running experiments to this end. Due to hardware constraints these will be provided later in the week, and we will post an additional comment alongside them.
Regarding the validity of the Gaussian norm constraint, we agree that SD latents are not isotropic Gaussian, and our method does not require them to be strictly Gaussian. As reviewer GJw1 notes, the VAE is trained so that its latents have approximately unit variance. This forces most encodings to lie within a shell, and the Gaussian hypothesis provides an estimate of where this shell is located. In Appendix C.2 we compare the empirical distribution of SD1.5 and SDXL encodings with the theoretical chi density: although the empirical distribution is broader and right-skewed, falls just past its mode for both models, i.e., the estimated shell captures a large proportion of embeddings. Furthermore, we ablate against the empirical maximum observed encoding magnitude (CACFG* in Appendix C.2), and find that the Gaussian estimate outperforms the empirical bound on FID, NIQE, BRISQUE, TReS, Density, and Coverage at moderate-to-high guidance strengths for both models. We hypothesise this is a result of the Gaussian bound constraining the sampling trajectory to high-density embedding regions.
References
[1] Black Forest Labs. FLUX. https://github.com/black-forest-labs/flux, 2024.
[2] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems 35, NeurIPS, 2022.
[3] Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV, 2024.
[4] Candi Zheng and Yuan Lan. Characteristic guidance: Non-linear correction for diffusion model at large guidance scale. In International Conference on Machine Learning, ICML, 2024.
[5] Seyedmorteza Sadat, Otmar Hilliges, and Romann M. Weber. Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In 13th International Conference on Learning Representations, ICLR, 2025.
Response to Authors
Thank you for the thoughtful response and the substantial revisions, which have clarified several aspects of the work.
W2 has been addressed by the additional experiments and comments. The following remain partially addressed or unresolved:
-
W1: The current evidence does not yet fully establish that bound violations are the primary cause of degradation, or that the proposed constraint specifically targets this cause. Moreover, the CACFG* results (Table 1) suggest that the gains depend strongly on how tightly the radius is set, leaving open the possibility that the effect stems from norm-shrinking regularization rather than from preventing violations.
-
W3: Thank you for pointing out that FLUX.1-dev is guidance-distilled. Models such as DiT-XL/2 or PixArt-α/Σ use VP diffusion with SD-family VAEs, so CACFG should apply to them without re-deriving the HJB solution.
I look forward to further discussion on these points.
(1/2)
We appreciate the reviewer's feedback and thank them for their comments. We are pleased that the reviewer found our formulation to provide "a unified interpretation of existing guidance scheduling methods", and we are glad they appreciated that CACFG "can be applied without additional training or significant computational overhead". We also thank the reviewer for clearly outlining the three points that would address their concerns, and we respond to each below.
Weaknesses
(W1): The reviewer notes that the relationship between norm-bound violations and generation degradation remains unclear.
We thank the reviewer for this suggestion, and have improved Appendix C.1 to address it. Our optimal control formulation shows that CFG places no constraint on the control (the Tweedie estimate), which permits the sampling path to move arbitrarily far from the current image estimate (Section 4). We then note that the VAE of a latent diffusion model is trained to regularise its encodings towards an isotropic Gaussian, which gives a theoretical upper bound on the magnitude of valid encodings, and hence of the Tweedie estimate (Section 5). In Appendix C.1 we show that as guidance strength increases, the proportion of SD1.5 CFG sampling paths that violate this bound rises from under 10% to over 90% (Figure 4), and that FID, NIQE, BRISQUE, TReS, Density, and Coverage all degrade as the proportion of violating paths grows (Figure 5). Furthermore, CACFG only alters a sampling path when the Tweedie estimate leaves the curvature-aware control set. Thus, at a fixed guidance strength, the improvements of CACFG arise solely from correcting violating paths, and these improvements are clear both qualitatively (Figure 3 and Appendix D.3) and quantitatively (Section 6.1, Figure 2b, and Appendices D.1 and D.2).
Regarding comparisons with norm clipping and guidance rescaling baselines, clipping methods such as the dynamic thresholding of Saharia et al. [2] bound the x0-prediction by the known range of the data ([−1,1] in pixel space). Latent spaces have no such known range, so these methods do not directly transfer to latent diffusion [3, 4]. Supplying this range for latent models is what drives our VAE-informed control set. We therefore compare against the two closest pre-existing latent-space analogues, CFG-Rescale (RECFG) [3] and APG [5], in Appendix D.5 (Figure 19). The curvature-aware correction can be applied on top of both methods, and it improves the FID-CLIP tradeoff of each at high guidance strengths (Figure 19b). In particular, applying the correction to CFG-Rescale (RECACFG) achieves the best FID-CLIP tradeoff of all methods tested. This further supports that bound violations are a source of degradation shared across guidance methods.
We agree that CACFG mitigates, rather than eliminates, degradation at high guidance strengths. Increasing guidance strength trades diversity for prompt alignment (which degrades FID at high guidance strengths). However, CACFG removes the additional degradation caused by controls that violate the upper bound.
(W2): The reviewer notes that it is unclear whether CACFG improves the quality-alignment tradeoff, or simply reduces the effective guidance strength.
We thank the reviewer for this suggestion. We have added FID-CLIP Pareto plots to the main paper (Figure 2b, lines 512-515) and Appendix D.2. At low CLIP scores, the curvature-aware and uncorrected curves coincide; however, as guidance increases, the uncorrected schedules incur a sharp FID penalty for minimal CLIP gains, whereas the curvature-aware variants achieve higher CLIP scores with a smaller FID penalty. Thus, at matched prompt alignment CACFG achieves better FID, and a reduction in effective guidance strength alone would not reproduce this result.