Contrast-of-Contrasts: On the Second Order of Diffusion and Bridge Guidance

Diffusion and bridge models are developed to capture a complex data distribution from known prior, while learning errors and potential mismatch between training and desired target distribution limit their generation quality. Recent guidance methods address this issue by extrapolating a primary denoiser away from a reference one with quality differences at each sampling step, while a single reference contrast provides only a first-order estimate of the desired sampling direction and may leave residual bias, leading to sub-optimal performance. Here, we introduce Contrast-of-Contrasts Guidance (${\mathrm{C}}^2\mathrm{G}$), a general second-order framework that extends existing guidance with multi-level reference predictions along its underlying degradation trajectories. By contrasting adjacent reference differences, ${\mathrm{C}}^2\mathrm{G}$ captures the variation of the first-order steering direction along the degradation trajectory, aiming to recover the residual guidance signals missed by single-level guidance. Moreover, we introduce ${\mathrm{ProjC}}^2\mathrm{G}$, which can inherit the tuned first-order guidance scale while decomposing and dynamically modulating the second-order correction along the directions parallel and orthogonal to the first-order guidance, exploiting their complementary effects throughout sampling. Extensive experiments show that ${\mathrm{C}}^2\mathrm{G}$ improves the primary metrics of classifier-free guidance, auto-guidance, self-guidance, and prior guidance across diffusion and bridge models for image generation, image-to-image translation, and image-to-video generation. Complete implementation will be publicly released.