acceptodds
Under review as a conference paper at ICLR 2027

Contrast-of-Contrasts: On the Second Order of Diffusion and Bridge Guidance

Abstract

Diffusion and bridge models are developed to capture a complex data distribution from known prior, while learning errors and potential mismatch between training and desired target distribution limit their generation quality. Recent guidance methods address this issue by extrapolating a primary denoiser away from a reference one with quality differences at each sampling step, while a single reference contrast provides only a first-order estimate of the desired sampling direction and may leave residual bias, leading to sub-optimal performance. Here, we introduce Contrast-of-Contrasts Guidance (), a general second-order framework that extends existing guidance with multi-level reference predictions along its underlying degradation trajectories. By contrasting adjacent reference differences, captures the variation of the first-order steering direction along the degradation trajectory, aiming to recover the residual guidance signals missed by single-level guidance. Moreover, we introduce , which can inherit the tuned first-order guidance scale while decomposing and dynamically modulating the second-order correction along the directions parallel and orthogonal to the first-order guidance, exploiting their complementary effects throughout sampling. Extensive experiments show that improves the primary metrics of classifier-free guidance, auto-guidance, self-guidance, and prior guidance across diffusion and bridge models for image generation, image-to-image translation, and image-to-video generation. Complete implementation will be publicly released.

Primary area:
generative models
Keywords:
Guidance Method, Diffusion Models, Bridge Models, Error Correction, Image Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ZgYpOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces C²G, an inference-time guidance framework that enhances existing diffusion and bridge guidance methods by leveraging multiple reference predictions. The method interprets the difference between guidance contrasts as a second-order correction and further proposes ProjC²G to adaptively project and modulate this correction. The approach is evaluated on several guidance mechanisms, including CFG, SG, AG, and PG, across image generation, image translation, and image-to-video tasks.

Critical strengths
  • The paper proposes a simple and broadly applicable inference-time guidance refinement framework, extending conventional first-order - guidance to multi-level reference contrasts and providing a new perspective for designing diffusion and bridge guidance methods.
  • The experiments cover a wide range of guidance mechanisms, including CFG, SG, AG, and PG, as well as diverse generation tasks, demonstrating the general applicability of the proposed framework.
  • The motivation of exploiting additional reference differences for guidance refinement is interesting and has potential practical value.
Critical weaknesses
  • Compared with CFG, where the guidance direction has a clear interpretation as the difference between conditional and unconditional predictions, the role and physical meaning of the proposed second-order correction in the denoising process remain less intuitive. A more detailed explanation of why the contrast between multiple reference predictions provides a meaningful higher-order guidance signal would strengthen the paper.

  • The paper combines C²G with multiple existing guidance mechanisms (e.g., CFG, SG, PAG, and PG) in its experiments. Although this demonstrates broad applicability, the involvement of multiple guidance signals makes the underlying design principle and theoretical interpretation of the proposed second-order correction less clear.

Questions for authors

I have two main concerns.

  • First, the choice of the intermediate reference prediction D1D_1 seems critical, but the paper does not sufficiently analyze how different reference construction strategies affect the resulting correction. For example, different condition degradation strategies in CFG or different intermediate model capacities in AG may lead to substantially different second-order guidance signals.

  • Second, the relationship between first-order and second-order guidance coefficients remains unclear. Since the second-order term is intended to correct the residual error after first-order guidance, it seems intuitive that the optimal second-order correction should depend on the first-order guidance strength. However, the paper mainly fixes the first-order coefficient and tunes the second-order coefficient independently, without analyzing their coupling.

If these concerns can be adequately addressed, I would consider increasing my score and supporting acceptance of the paper.

12 comments
AuthorsOpenReview ↗

Reply to Reviewer ZgYp (1/3)

We appreciate your insightful comments and important questions. We address your concerns below.


Response to Q1: Choice of Intermediate Reference We thank the reviewer for raising this important point. We agree that reference construction matters: since u2=D0−2D1+D2\mathbf{u}_2=D_0-2D_1+D_2, changing an intermediate reference directly changes the second-order correction. This dependence is not unique to C$^2$G. For example, the first-order AG and PG rely on the choice of a weak model and a perturbation method, respectively. C$^2$G extends these method-specific reference choices to a multi-level contrast.

Across CFG, AG, SG, and PG, our experiments suggest two practical design principles for the reference construction strategy.

  • Mechanism Consistent Extension: D0D_0, D1D_1, and D2D_2 should follow the same method-specific degradation mechanism, forming multi-level references under the same mechanism.
  • Clear Quality Differences: Given the tuned first-order guidance term, the quality differences between these denoising results, namely D0D_0, D1D_1, and D2D_2, should be distinct, rather than nearly identical.

In our evaluated settings, following these two principles has usually yielded effective refinements over the corresponding first-order guidance. For CFG, the unconditional prediction Dθ(xtt,ϕ)D_{\theta}(\mathbf{x}_t t, \phi) removes the condition in Dθ(xt,t,c)D_{\theta}(\mathbf{x}_t, t, \mathbf{c}) entirely. we therefore use a vague caption H(c)\mathcal{H(\mathbf{c})} as an intermediate reference that weakens the condition while retaining semantic cues, as shown in Appendix E and examples in Appendix F.

For AG, we compare alternative constructions of the second order reference on EDM2/ImageNet-512 with S+XS at 32 sampling steps. The first order weak model is degraded in both training budget and model capacity. The consistent construction further reduces both factors for D2D_2, whereas the inconsistent construction reduces training budget alone while keeping the capacity unchanged. With identical hyperparameters for the two variants within each metric, the consistent construction performs best under both class sampling protocols.

Ablation of second-order branch construction on EDM2/ImageNet-512 (S+XS, 32 steps). All results are obtained at 32 sampling steps. The consistent and inconsistent variants use identical hyperparameters for each metric.

MetricMethodOld ↓New ↓
FIDAG (First-order)1.3901.248
FIDInconsistent C²G1.3681.234
FIDConsistent C²G (Ours)1.3591.225
FD-DINOv2AG (First-order)37.3535.17
FD-DINOv2Inconsistent C²G37.0734.92
FD-DINOv2Consistent C²G (Ours)36.8334.78

For PG, we compare repeated noise degradation using the same noise twice with degradation using independent noise draws. On 10,000 E2H samples with 10 sampling steps and no projection, both constructions improve over first order noise PG. The independent noise construction performs best. C²G also improves blur based PG.

  • Double: H(H(xt))=xt+2ϵ1H(H(x_t))=x_t+2\epsilon_1, where the same noise is applied twice.
  • Independent (used in our main experiments): H(H(xt))=xt+ϵ1+ϵ2H(H(x_t))=x_t+\epsilon_1+\epsilon_2, where ϵ2\epsilon_2 is an additional independent noise. The results are shown below. The guidance parameters are tuned separately for each construction to achieve the best performance.
Prediction StepsDegradation PathCorruptionFID ↓
10First-order ComparisonNoise3.216
10DoubleNoise3.115
10IndependentNoise3.094
10BlurFirst-order3.534
10BlurC²G3.497

Together, these results support the proposed guidelines: extending the established reference mechanism with meaningfully distinct levels can yield useful second order corrections, while the appropriate degradation levels remain method dependent. We will clarify these guidelines in revision.

AuthorsOpenReview ↗

Reply to Reviewer ZgYp (2/3)


Response to Q2: Guidance Coefficients Tuning

We thank the reviewer for highlighting the interaction between the first and second order guidance strengths.

We agree that the two coefficients need not be universally independent. ProjC$^2$G explicitly controls their interaction by decomposing u2=κtu1+v2,t\mathbf{u}2=\kappa_t\mathbf{u}1+\mathbf{v}{2,t}, where v2,t⊥u1\mathbf{v}{2,t}\perp\mathbf{u}_1, and using u~2=(1−ct)κtu1+v2,t.\tilde{\mathbf{u}}_2=(1-c_t)\kappa_t\mathbf{u}1+\mathbf{v}{2,t}. The resulting update has an aligned coefficient w1+w2(1−ct)κtw_1+w_2(1-c_t)\kappa_t and an orthogonal component scaled by w2w_2. Thus, full orthogonalization preserves the tuned baseline scale w1w_1, while partial projection provides explicit control over the aligned contribution of the second order term.

We further perform joint coefficient sweeps. On E2H, using 10,000 samples and 30 NFEs with cs=0.28c_s=0.28 and projection coefficient c=1c=1, w2=8w_2=8 achieves the row minimum or falls within 0.01 FID of it for every tested w1w_1.

w1w_1 \\ w2w_278910
93.453.443.443.46
103.453.453.463.48
113.463.463.483.51
123.483.493.513.55

On COCO-5000 text-to-image generation, with 84 NFEs and c=0.25c=0.25, w2=0.5w_2=0.5 remains within 0.01 CLIP of the row maximum across all tested w1w_1 values.

w1\w2w_1 \backslash w_200.51.01.52.0
2.031.612731.665931.615931.533931.5133
2.531.699831.728231.716431.655031.6147
3.031.714731.722231.703631.676731.6392
3.531.725231.716731.708731.701331.6944

These results show that, in the tested ranges, a single w2w_2 provides near optimal performance across multiple first order scales. We therefore retain the baseline tuned w1w_1 and tune w2w_2 as a practical choice, without assuming universal coefficient independence. We will include both joint sweeps in the revision.


Response to W1

Feature of Guidance Methods: Quality Differences: As claimed by AG and mentioned in PG, guidance methods construct denoising terms with quality differences, namely a high-quality (HQ) term and a low-quality (LQ) one, and contrast them to emphasize the desired direction, e.g., score accuracy in AG and condition alignment in CFG.

From this perspective, for CFG, the conditional distribution is easier to learn and the denoising term Dθ(xt,t,c)D_{\theta}(\mathbf{x}_t, t, \mathbf{c}) is trained with more resources, thereby serving as the HQ term. The unconditional distribution is more difficult to learn and the denoising term Dθ(xt,t,ϕ)D_{\theta}(\mathbf{x}_t, t, \phi) is trained with less resources, thereby serving as the LQ term. Contrasting their quality differences with a scaling factor, we can emphasize condition alignment at each inference step.

A more detailed explanation of C$^2$G: C$^2$G extends this contrastive view along the same method-specific degradation trajectory. Let δ01=D0−D1\mathbf{\delta}_{01}=D_0-D_1, δ12=D1−D2\mathbf{\delta}_{12}=D_1-D_2, and the second-order term $[\mathbf{u_2}=\mathbf{\delta}_{01}-\mathbf{\delta}_{12} =D_0-2D_1+D_2 ]$. Then, in comparison with first-order guidance, second-order guidance measures how the first-order contrast changes as the reference is further degraded (i.e., Contrast-of-Contrasts). With incremental degradation, it acts as a discrete curvature of the denoiser response. More generally, it measures the discrepancy between adjacent contrasts. Namely, C$^2$G refines the existing steering mechanism using additional information along its own reference trajectory.

For CFG, D0D_0, D1D_1, and D2D_2 correspond to the full, vague, and null conditions. The standard CFG direction is D0−D2=δ01+δ12D_0-D_2=\mathbf{\delta}_{01}+\mathbf{\delta}_{12}, whereas u2\mathbf{u_2} compares the condition-induced changes across the full-to-vague and vague-to-null transitions. It therefore captures variation in the denoiser’s response to condition weakening that a single reference contrast cannot express.

Since all predictions are evaluated at the same sampling state, adding w2u2w_2\mathbf{u_2} directly refines the denoising results used by the reverse process. Our anisotropic Gaussian analysis illustrates this added flexibility: while first-order guidance applies a shared scalar correction across eigendirections, the second-order term permits direction-dependent refinement. We do not claim that u2\mathbf{u_2} is guaranteed to equal or align with the true target residual, while it supplies a complementary higher-order signal whose practical value is assessed across guidance methods of diffusion and bridge models.

AuthorsOpenReview ↗

Reply to Reviewer ZgYp (3/3)


Response to W2: Involvement of Multiple Guidance Signals

We thank the reviewer for noting that the role of C$^2$G could be clearer when multiple guidance mechanisms are involved. We agree that the two types of experiments serve distinct purposes.

In the standalone experiments, C$^2$G is applied to one guidance mechanism at a time, with its references constructed along that mechanism’s own degradation trajectory. These results isolate how the same second-order guidance refines different guidance mechanisms.

As discussed in Section 3.2 of our submission, C$^2$G is conceptually orthogonal to guidance composition: composition specifies which guidance branches are combined, whereas C$^2$G specifies how a selected branch is further refined using multi-level references. Consistent with this distinction, in the CFG+SG and CFG+PAG+SG experiments of the SG baseline method, C$^2$G is applied only to the SG branch, and the CFG and PAG branches retain their baseline settings. These experiments test our compatibility with existing guidance combinations verified by baseline method, rather than define or motivate the second-order guidance. Our anisotropic analysis separately illustrates the role of this correction.

We will make this distinction and the purpose of each experiment explicit in the revision.


If these clarifications and results address your concerns, we respectfully invite you to reconsider your assessment. We would be happy to provide any additional experiments or analyses that you believe would be helpful.

Reviewer_aZcnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Contrast-of-Contrasts Guidance (C²G), which augments existing diffusion guidance with a second-order difference across three progressively degraded predictions. Its projected variant, ProjC²G, controls the correction’s components parallel and orthogonal to the first-order guidance direction. Experiments demonstrate improvements across classifier-free, auto-, self-, and prior guidance for image generation, image-to-image translation, and image-to-video generation.

Critical strengths
  1. The framework is evaluated across four guidance mechanisms and both diffusion and bridge models, demonstrating applicability beyond a single architecture or task.
  2. The method operates with frozen pretrained models and retains the baseline first-order guidance coefficient, making it straightforward to incorporate into existing sampling pipelines when suitable references are available.
  3. Meaningful improvements in several settings, e.g, SD3’s COCO FID improves from 27.00 to 24.73, while 50-step Edges2Handbags FID improves from 0.91 to 0.78.
  4. Separating the additional correction into parallel and orthogonal components provides explicit control over its overlap with first-order guidance. Partial projection offers flexibility between retaining the raw correction and enforcing orthogonality.
  5. Detailed implementation documentation, e.g., the appendices specify hyperparameter searches, projection schedules, model checkpoints, caption preprocessing, and sampling pseudocode, supporting reproducibility.
Critical weaknesses
  1. Novelty requires clearer differentiation. The paper acknowledges ICFG’s second-order extension and APG’s parallel–orthogonal decomposition, but does not provide direct experimental comparisons with these closely related approaches. Such comparisons are important for establishing the distinct contribution.
  2. The theoretical justification is restrictive. Theorem 1 assumes a target correction already expressible through the first-order and orthogonal second-order directions. It therefore explains projection under a particular structure, rather than establishing that actual degraded predictions recover useful residual information.
  3. The degradation trajectory is insufficiently calibrated. Weakened captions, weaker checkpoints, and repeated perturbations do not automatically define consistently spaced quality levels. The paper needs stronger evidence that these constructions capture useful second-order variation rather than merely provide another prediction to combine.
  4. The quality–compute trade-off remains unclear. C²G requires an additional prediction and tuning of correction/projection parameters. Fixed-step comparisons do not establish whether it outperforms baselines given equivalent computation or wall-clock time.
  5. Some incremental gains need stronger statistical support. Projection improves CFG+SG FID only from 23.96 to 23.93. ImageNet results report minima across three seed sets, with approximate variability provided separately; paired improvements and confidence intervals would make small gains easier to assess.
Questions for authors
  1. How does C²G compare directly with ICFG and APG on the same backbone, with comparable tuning and computational budgets?
  2. Can you demonstrate that the second-order correction aligns with the remaining prediction error, and evaluate sensitivity to alternative weakened captions, checkpoints, and perturbation strengths?
  3. Do the improvements persist under matched model-evaluation counts or wall-clock time, with paired multi-seed confidence intervals and clearly separated hyperparameter-selection and evaluation data?
  4. Also, address other weakness
4 comments
AuthorsOpenReview ↗

Reply to Reviewer aZcn (1/2)

Thank you for your careful review. We provide our reply below.


Response to Q1&W1: Comparison with ICFG and APG

As you suggested, we compare our method with both ICFG and APG. Both of them are evaluated on the COCO dataset same as our submission.

MethodNFEStepsFID ↓CLIP ↑
APG844225.9131.45
ICFG842826.5331.67
ProjC$^2$G (Ours)842824.7331.72

Beyond the empirical improvements, we believe that C$^2$G is fundamentally different from these methods.

ICFG: It relies on a specific structural assumption about the continuous condition space and constructs its second-order correction through interpolation of condition embeddings. Consequently, its formulation is inherently tied to CFG and cannot be directly extended to guidance mechanisms such as AG and PG.

APG: It primarily addresses oversaturation by modifying the existing first-order CFG direction through projection, rescaling, and momentum. It does not introduce additional higher-order information from model predictions at different quality levels. Importantly, while both APG and ProjC$^2$G employ projection, they serve fundamentally different purposes: APG suppresses the oversaturating component of CFG, whereas ProjC$^2$G explicitly controls the component of the second-order correction aligned with the original first-order guidance direction, reducing redundancy while preserving complementary second-order information.


Response to Q2&W2&W3

We thank the reviewer for these questions. We agree that Theorem 1 is conditional: it analyzes projection under an assumed residual structure. We therefore view it as a structured analysis, complemented by empirical evidence. We conduct additional analysis for the remaining error of guided generation.

Under S+XS+XXS setting of AG , we report the MSE of first-order and second-order guidance terms in the frequency domain. We use these parameters: ω1=1.45\omega_1=1.45(following the original AG paper), ω2=0.5\omega_2=0.5, c=0c=0 for the second-order guidance. We add noise using forward SDE to t=0.9Tt=0.9T, and using AG and ProjC$^2$G to restore the image.

The results are shown in the table below, where the relative reduction is computed as (MSEfirst−MSEsecond)/MSEfirst(\mathrm{MSE}_{\mathrm{first}}-\mathrm{MSE}_{\mathrm{second}})/\mathrm{MSE}_{\mathrm{first}}.

We randomly select 128 images to compute.

Frequency BandBinFirst-order MSESecond-order MSERelative Reduction
Low0–208.220728.107771.374%
Mid21–530.001151940.001110873.565%
High54–630.0001347970.0001297883.716%

For PG, under DIODE dataset using DBIM sampler, we select corresponding parameters: cs=0.3cs=0.3, ω1=20,cutoff=0.7,startratio=0.3\omega_1=20, cutoff=0.7, start_ratio=0.3 for the first-order guidance, and ω2=−15\omega_2=-15, c=0c=0 for the second-order guidance. Both of which use 10 sampling steps. We also use random 128 images to compute the MSE in the frequency domain.

Frequency BandBinFirst-order MSESecond-order MSERelative Reduction
Low0–200.015820.0125520.66%
Mid21–530.006690.00600110.25%
High54–630.0000750.0000689.27%

Our experiments suggest two practical principles for constructing references: (1) use the same method specific degradation mechanism for the denoising terms (D0D_0,D1D_1,D2D_2); (2) ensure that adjacent reference predictions are meaningfully distinct along that mechanism, yielding a nontrivial second order contrast. With these two design principles, C$^2$G has usually improved the generation quality of first-order guidance, without the necessity of assuming uniformly calibrated degradation intervals. For additional evidences, please kindly visit our Response to Q1 of reviewer ZgYp.


Response to Q3&W4: Inference Efficiency

We thank the reviewer for highlighting this tradeoff.

C$^2$G adds one denoiser evaluation per sampling step and aims to improve generation quality, not reduce inference cost. Our conclusion of submission acknowledges the added computation and memory, and identifies distillation of the guided sampling trajectory as future direction.

As reported in our Response to Q1&W1 of Reviewer 69bn, matched NFE comparisons at sufficiently large budgets show lower FID for PG on E2H (3.18 to 3.15) and DIODE (2.12 to 2.02), and for CFG on COCO (26.44 to 24.73). CFG+SG also improves from 24.19 to 23.93 with fewer evaluations (112 versus 114). At very small budgets, the second order term may offer limited benefit because it refines, rather than replaces, first order guidance, and the budget may be better spent on additional sampling steps. For AG, it should be noted that the added XXS branch incurs an estimated compute overhead of 7.7% for S+XS and 1.5% for XXL+M. We will clarify this tradeoff and report the matched NFE results.

AuthorsOpenReview ↗

Reply to Reviewer aZcn(2/2)


Response to W5: Evidence of Gains

To further demonstrate the effectiveness of projection and second-order guidance, we provide two additional experimental comparisons.

Projection on E2H. On E2H (10K, NFE=30), the FID decreases from 3.62 without projection (c=0c=0) to 3.54 with partial projection (c=0.5c=0.5) and 3.56 with full projection (c=1c=1). The observed FID standard deviation is approximately 0.03, suggesting that the improvements are meaningful relative to the evaluation variability.

Second-Order AutoGuidance. We additionally compare first- and second-order AutoGuidance under the same first-order parameter configuration. We use three EDM2-S checkpoints trained for TT, T/4T/4, and T/16T/16 iterations as D0D_0, D1D_1, and D2D_2, respectively, all with an EMA setting of 0.130. FID decreases from 2.903 to 2.540, while FD-DINO decreases from 90.160 to 68.624. The reported FID standard deviation is approximately 0.015, and the FD-DINO standard deviation is below 1. These improvements are substantially larger than the standard deviations.

Finally, we use the Table below to report all three ImageNet FID evaluations for each setting, rather than only the best run. The same seed sets are used across methods, allowing corresponding evaluations to be compared directly.

ImageNet-512 FID across three paired, disjoint seed sets under the old and new class-sampling protocols. Lower is better; values are listed in seed-set order.

Model pairMethodOld (three runs)New (three runs)
S+XSAG1.390, 1.398, 1.4171.248, 1.256, 1.261
S+XSAG-ProjC$^2$G†1.368, 1.374, 1.3941.234, 1.239, 1.247
S+XSAG-ProjC$^2$G1.359, 1.369, 1.3861.225, 1.234, 1.238
XXL+MAG1.329, 1.330, 1.3381.171, 1.176, 1.178
XXL+MAG-ProjC$^2$G1.313, 1.320, 1.3221.157, 1.158, 1.164

AG-ProjC$^2$G improves FID over AG on every paired seed set. Its mean paired FID reductions (95% tt intervals, n=3n=3) are 0.03030.0303 [0.02750.0275, 0.03320.0332] and 0.02270.0227 [0.02120.0212, 0.02410.0241] for S+XS under the old and new protocols, respectively, and 0.01400.0140 [0.00540.0054, 0.02260.0226] and 0.01530.0153 [0.00960.0096, 0.02110.0211] for XXL+M.


We hope these clarifications and results are helpful in addressing your concerns. We would be glad to provide any further experiments or analyses that may be useful.

Reviewer_aZcnOpenReview ↗

Thanks for the clarification!

Thanks for the clarifications and experiments. The authors have addressed my concerns. I'm keeping my positive rating.

AuthorsOpenReview ↗

Thank You for Your Positive Assessment

Dear Reviewer,

Thank you for your encouraging feedback and for maintaining your positive rating. We are glad that our clarifications and additional experiments addressed your concerns. We will incorporate the relevant clarifications and findings into the revised manuscript.

Best, The Authors

Reviewer_69bnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Contrast-of-Contrasts Guidance (C2GC^2G), a second-order guidance framework for diffusion and bridge models. It addresses the residual bias in existing first-order guidance methods (e.g., CFG, AG, SG, PG) by introducing a third reference prediction along the degradation trajectory to capture the variation of the guidance direction. Furthermore, it introduces ProjC2GProjC^2G, which utilizes orthogonal projection and dynamic modulation to apply second-order corrections without disrupting the tuned first-order guidance scale. Experiments across various generation tasks validate its effectiveness.

Critical strengths
  1. The framework seamlessly unifies diverse guidance methods (CFG, AG, SG, PG) across both diffusion and bridge models. The terminal-marginal interpretation provides a solid theoretical foundation.
  2. It effectively decouples higher-order and first-order terms. By projecting the second-order correction into an orthogonal subspace, it ensures stability and preserves the well-tuned first-order guidance scale.
Critical weaknesses
  1. Evaluating D2D_2 requires an extra forward pass per step (e.g., increasing CFG evaluations from 2 to 3, a 50% overhead).

  2. The design of D2D_2​ varies significantly across methods and relies heavily on manual heuristics. There is a lack of sensitivity analysis on the degradation degree, which could lead to noisy or collinear corrections if chosen improperly.

Questions for authors
  1. Can the authors provide performance comparisons under strictly aligned NFE? (e.g., Baseline CFG with 30 steps vs. C2GC^2G with 20 steps, both totaling 60 NFEs).

  2. How does the choice of the degradation interval affect performance? Are there guidelines to prevent u1u_1​ and u2u_2 from becoming highly collinear?

  3. Appendix B mentions K>2K>2. In the preliminary explorations, does third-order guidance yield further improvements, or does it lead to saturation/degradation due to error accumulation?

2 comments
AuthorsOpenReview ↗

Reply to Reviewer 69bn(1/2)

Thank you for your recognition and important questions. We provide our reply below.


Response to Q1&W1: Inference Efficiency

We thank the reviewer for highlighting the additional inference cost. We agree that C$^2$G adds one denoiser evaluation per step and is not intended to reduce the sampling cost. Its goal is to further improve the generation quality by refining an existing first-order guidance direction with a general second-order guidance framework. As demonstrated in our original submission, given the same experiment setting, our methods have further improved the best results reported by the corresponding first-order baseline guidance methods. Specifically, for CFG, SG, PAG given the COCO dataset on Stable-Diffusion3, our method achieves both lower FID(24.83$\rightarrow$23.93) and higher HPS(30.37$\rightarrow$30.48) than the best results reported across all Stable-Diffusion3 configurations in the original paper. Similarly, our method outperforms the best reported AG results in both FID(1.34$\rightarrow$1.31) and FD-DINOv2(24.18$\rightarrow$22.42) given Imagenet dataset, and the best reported PG results in FID given DIODE dataset(2.06$\rightarrow$2.02).

The quality-compute trade-off is acknowledged in the Conclusion section of our submission, which notes that C$^2$G adds computation and memory and identifies distilling its three branches into a single denoiser as future work.
As you suggested, we evaluate C$^2$G for both diffusion and bridge models under matched NFE budgets.

FrameworkDatasetSamplesGuidance NameNFEVanilla FID ↓Ours FID ↓
DiffusionCOCO5,000CFG8426.4424.73
DiffusionCOCO5,000CFG+SG114 (112)†24.1923.93
BridgeE2H10,000PG1503.183.15
BridgeDIODE16,502PG1502.122.02

For CFG+SG, Vanilla method uses 114 NFE and Ours uses 112 NFE. Ours achieves a lower FID (23.93 vs. 24.19) with fewer function evaluations.

Because the second-order term refines rather than replaces the primary first-order guidance, its benefit may be limited given a very small NFE budget. As shown above, given sufficient NFE, C$^2$G improves generation quality without sacrificing inference speed. Additionally, for AG, NFE also does not capture the different model inference costs: adding the smaller XXS branch incurs an estimated compute overhead of only 7.7% for S+XS and 1.5% for XXL+M. We will add these results and clarify this quality–compute trade-off in the paper.

AuthorsOpenReview ↗

Reply to Reviewer 69bn(2/2)


Response to Q2&W2: Choice of D2

We thank the reviewer for highlighting the importance of reference construction. We agree that the choice of D2D_2 affects the quality of the second-order correction. In our experiments across CFG, AG, SG, and PG, we find two practical principles for constructing D2D_2 in C$^2$G.

  • Mechanism Consistent Extension: D0D_0, D1D_1, and D2D_2 should follow the same method-specific degradation mechanism, forming multi-level references under the same mechanism.
  • Clear Quality Differences: Given the tuned first-order guidance term, the quality differences between these denoising results, namely D0D_0, D1D_1, and D2D_2, should be distinct, rather than nearly identical.

For CFG, for example, the unconditional denoising term in first-order guidance entirely drops the condition. According to principls above, we develop a vague caption as an intermediate reference, which is less informative than the original text descriptions but retains coarse semantic cues, as described in Appendix E.5 and shown in Appendix F. We detail our the method-specific constructions in Section 3.3 and Section 4 (Experiments), with further implementation details in Appendix E. The AG ablation (also included in Table 2 of the original submission) compares alternative reference constructions and provides targeted empirical support for a coherent degradation progression. For clarity, we extract the relevant ablation results from the original table and present them in a dedicated table in Response to Q1 of reviewer ZgYp.

Another controlled experiment ablates the effect of different degradation paths on PG-C$^2$G. Specifically, we compare two degradation paths for PG:

  • Double: H(H(xt))=xt+2ϵ1H(H(x_t))=x_t+2\epsilon_1, where the same noise is applied twice.
  • Independent (used in our main experiments): H(H(xt))=xt+ϵ1+ϵ2H(H(x_t))=x_t+\epsilon_1+\epsilon_2, where ϵ2\epsilon_2 is an additional independent noise. As experiment setting referred in the Response to Q1 of Reviewer ZgYp, we have tuned each parameter to achieve the best performance under each construction. We evaluate both constructions on 10,000 E2H samples with 10 predtion steps, and the reference are also using 10,000 samples, respectively, without projection.

We will make these design principles and the scope of the supporting evidence explicit in the revision. Also, Figure 5 of Appendix B shows that the cosine similarity between the first- and second-order directions varies across methods and tasks, and may be relatively high. In practice, ProjC$^2$G decomposes the second-order term into components parallel and orthogonal to the first-order direction and dynamically controls their contributions.


Response to Q3: Third-order Guidance

We thank the reviewer for pointing out the potential of third-order guidance. We conducted additional experiments of third-order guidance on SG for diffusion models and PG for bridge models. The results are shown below. We adopt the tuned hyperparameters of second-order guidance and apply full projection via Gram-Schmidt orthogonalization to remove the components of the third-order correction parallel to the first- and second-order guidance directions. By tuning only the third-order guidance scale, we observe that third-order guidance does not lead to degradation and may provide a modest additional improvement.

Below is third-order Prior Guidance on Edges-to-Handbags (10K images) with noise degradation cs=0.28c_s=0.28, g=17g=17, w2=−15w_2=-15, and cproj=1c_{proj}=1. We found that the third-order guidance scale w3w_3 can be tuned to further improve performance.

Third-order scale w3w_3-6-2034681012.5
FID ↓\downarrow3.503.503.493.473.473.463.473.493.57

Below is third-order Self-Guidance on COCO (5K images) (CFG+3-order-SG) with first-order scale w1=2.5w_1=2.5 , second-order scale w2=−0.5w_2=-0.5, and projection coefficient cproj=0.75c_{proj}=0.75.

Third-order scale w3w_3-1.0-0.50+0.5
FID ↓\downarrow35.3035.2335.0935.02

We hope these clarifications and results address your questions. Please kindly let us know if any further experiment or analyses would be helpful. If our reply and evidence could resolve your concerns, we would be grateful if you would consider improving your assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.