- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces C²G, an inference-time guidance framework that enhances existing diffusion and bridge guidance methods by leveraging multiple reference predictions. The method interprets the difference between guidance contrasts as a second-order correction and further proposes ProjC²G to adaptively project and modulate this correction. The approach is evaluated on several guidance mechanisms, including CFG, SG, AG, and PG, across image generation, image translation, and image-to-video tasks.
Critical strengths
- The paper proposes a simple and broadly applicable inference-time guidance refinement framework, extending conventional first-order - guidance to multi-level reference contrasts and providing a new perspective for designing diffusion and bridge guidance methods.
- The experiments cover a wide range of guidance mechanisms, including CFG, SG, AG, and PG, as well as diverse generation tasks, demonstrating the general applicability of the proposed framework.
- The motivation of exploiting additional reference differences for guidance refinement is interesting and has potential practical value.
Critical weaknesses
-
Compared with CFG, where the guidance direction has a clear interpretation as the difference between conditional and unconditional predictions, the role and physical meaning of the proposed second-order correction in the denoising process remain less intuitive. A more detailed explanation of why the contrast between multiple reference predictions provides a meaningful higher-order guidance signal would strengthen the paper.
-
The paper combines C²G with multiple existing guidance mechanisms (e.g., CFG, SG, PAG, and PG) in its experiments. Although this demonstrates broad applicability, the involvement of multiple guidance signals makes the underlying design principle and theoretical interpretation of the proposed second-order correction less clear.
Questions for authors
I have two main concerns.
-
First, the choice of the intermediate reference prediction seems critical, but the paper does not sufficiently analyze how different reference construction strategies affect the resulting correction. For example, different condition degradation strategies in CFG or different intermediate model capacities in AG may lead to substantially different second-order guidance signals.
-
Second, the relationship between first-order and second-order guidance coefficients remains unclear. Since the second-order term is intended to correct the residual error after first-order guidance, it seems intuitive that the optimal second-order correction should depend on the first-order guidance strength. However, the paper mainly fixes the first-order coefficient and tunes the second-order coefficient independently, without analyzing their coupling.
If these concerns can be adequately addressed, I would consider increasing my score and supporting acceptance of the paper.
12 comments
Reply to Reviewer ZgYp (2/3)
Response to Q2: Guidance Coefficients Tuning
We thank the reviewer for highlighting the interaction between the first and second order guidance strengths.
We agree that the two coefficients need not be universally independent. ProjC$^2$G explicitly controls their interaction by decomposing , where , and using The resulting update has an aligned coefficient and an orthogonal component scaled by . Thus, full orthogonalization preserves the tuned baseline scale , while partial projection provides explicit control over the aligned contribution of the second order term.
We further perform joint coefficient sweeps. On E2H, using 10,000 samples and 30 NFEs with and projection coefficient , achieves the row minimum or falls within 0.01 FID of it for every tested .
| \\ | 7 | 8 | 9 | 10 |
|---|---|---|---|---|
| 9 | 3.45 | 3.44 | 3.44 | 3.46 |
| 10 | 3.45 | 3.45 | 3.46 | 3.48 |
| 11 | 3.46 | 3.46 | 3.48 | 3.51 |
| 12 | 3.48 | 3.49 | 3.51 | 3.55 |
On COCO-5000 text-to-image generation, with 84 NFEs and , remains within 0.01 CLIP of the row maximum across all tested values.
| 0 | 0.5 | 1.0 | 1.5 | 2.0 | |
|---|---|---|---|---|---|
| 2.0 | 31.6127 | 31.6659 | 31.6159 | 31.5339 | 31.5133 |
| 2.5 | 31.6998 | 31.7282 | 31.7164 | 31.6550 | 31.6147 |
| 3.0 | 31.7147 | 31.7222 | 31.7036 | 31.6767 | 31.6392 |
| 3.5 | 31.7252 | 31.7167 | 31.7087 | 31.7013 | 31.6944 |
These results show that, in the tested ranges, a single provides near optimal performance across multiple first order scales. We therefore retain the baseline tuned and tune as a practical choice, without assuming universal coefficient independence. We will include both joint sweeps in the revision.
Response to W1
Feature of Guidance Methods: Quality Differences: As claimed by AG and mentioned in PG, guidance methods construct denoising terms with quality differences, namely a high-quality (HQ) term and a low-quality (LQ) one, and contrast them to emphasize the desired direction, e.g., score accuracy in AG and condition alignment in CFG.
From this perspective, for CFG, the conditional distribution is easier to learn and the denoising term is trained with more resources, thereby serving as the HQ term. The unconditional distribution is more difficult to learn and the denoising term is trained with less resources, thereby serving as the LQ term. Contrasting their quality differences with a scaling factor, we can emphasize condition alignment at each inference step.
A more detailed explanation of C$^2$G: C$^2$G extends this contrastive view along the same method-specific degradation trajectory. Let , , and the second-order term $[\mathbf{u_2}=\mathbf{\delta}_{01}-\mathbf{\delta}_{12} =D_0-2D_1+D_2 ]$. Then, in comparison with first-order guidance, second-order guidance measures how the first-order contrast changes as the reference is further degraded (i.e., Contrast-of-Contrasts). With incremental degradation, it acts as a discrete curvature of the denoiser response. More generally, it measures the discrepancy between adjacent contrasts. Namely, C$^2$G refines the existing steering mechanism using additional information along its own reference trajectory.
For CFG, , , and correspond to the full, vague, and null conditions. The standard CFG direction is , whereas compares the condition-induced changes across the full-to-vague and vague-to-null transitions. It therefore captures variation in the denoiser’s response to condition weakening that a single reference contrast cannot express.
Since all predictions are evaluated at the same sampling state, adding directly refines the denoising results used by the reverse process. Our anisotropic Gaussian analysis illustrates this added flexibility: while first-order guidance applies a shared scalar correction across eigendirections, the second-order term permits direction-dependent refinement. We do not claim that is guaranteed to equal or align with the true target residual, while it supplies a complementary higher-order signal whose practical value is assessed across guidance methods of diffusion and bridge models.
Reply to Reviewer ZgYp (3/3)
Response to W2: Involvement of Multiple Guidance Signals
We thank the reviewer for noting that the role of C$^2$G could be clearer when multiple guidance mechanisms are involved. We agree that the two types of experiments serve distinct purposes.
In the standalone experiments, C$^2$G is applied to one guidance mechanism at a time, with its references constructed along that mechanism’s own degradation trajectory. These results isolate how the same second-order guidance refines different guidance mechanisms.
As discussed in Section 3.2 of our submission, C$^2$G is conceptually orthogonal to guidance composition: composition specifies which guidance branches are combined, whereas C$^2$G specifies how a selected branch is further refined using multi-level references. Consistent with this distinction, in the CFG+SG and CFG+PAG+SG experiments of the SG baseline method, C$^2$G is applied only to the SG branch, and the CFG and PAG branches retain their baseline settings. These experiments test our compatibility with existing guidance combinations verified by baseline method, rather than define or motivate the second-order guidance. Our anisotropic analysis separately illustrates the role of this correction.
We will make this distinction and the purpose of each experiment explicit in the revision.
If these clarifications and results address your concerns, we respectfully invite you to reconsider your assessment. We would be happy to provide any additional experiments or analyses that you believe would be helpful.
Reply to Reviewer ZgYp (1/3)
We appreciate your insightful comments and important questions. We address your concerns below.
Response to Q1: Choice of Intermediate Reference We thank the reviewer for raising this important point. We agree that reference construction matters: since u2=D0−2D1+D2, changing an intermediate reference directly changes the second-order correction. This dependence is not unique to C$^2$G. For example, the first-order AG and PG rely on the choice of a weak model and a perturbation method, respectively. C$^2$G extends these method-specific reference choices to a multi-level contrast.
Across CFG, AG, SG, and PG, our experiments suggest two practical design principles for the reference construction strategy.
In our evaluated settings, following these two principles has usually yielded effective refinements over the corresponding first-order guidance. For CFG, the unconditional prediction Dθ(xtt,ϕ) removes the condition in Dθ(xt,t,c) entirely. we therefore use a vague caption H(c) as an intermediate reference that weakens the condition while retaining semantic cues, as shown in Appendix E and examples in Appendix F.
For AG, we compare alternative constructions of the second order reference on EDM2/ImageNet-512 with S+XS at 32 sampling steps. The first order weak model is degraded in both training budget and model capacity. The consistent construction further reduces both factors for D2, whereas the inconsistent construction reduces training budget alone while keeping the capacity unchanged. With identical hyperparameters for the two variants within each metric, the consistent construction performs best under both class sampling protocols.
Ablation of second-order branch construction on EDM2/ImageNet-512 (S+XS, 32 steps). All results are obtained at 32 sampling steps. The consistent and inconsistent variants use identical hyperparameters for each metric.
For PG, we compare repeated noise degradation using the same noise twice with degradation using independent noise draws. On 10,000 E2H samples with 10 sampling steps and no projection, both constructions improve over first order noise PG. The independent noise construction performs best. C²G also improves blur based PG.
Together, these results support the proposed guidelines: extending the established reference mechanism with meaningfully distinct levels can yield useful second order corrections, while the appropriate degradation levels remain method dependent. We will clarify these guidelines in revision.