Dynamic Drift-Aware Guidance for Accelerated Generative Models

Classifier-free guidance (CFG) enables high-fidelity generation in diffusion and flow-matching models but requires two inference passes in each sampling step because of the additional unconditional network pass. While recent acceleration heuristics skip unconditional passes using fixed timestep schedules or early termination, they ignore prompt-dependent trajectory curvature, leading to trajectory deviation, object omission, and semantic collapse. To resolve this problem, we propose Dynamic Drift-Aware Guidance (DrAG), a training-free framework that selectively skips redundant unconditional evaluations while closely approximating the original sampling trajectory. DrAG is developed on the key intuition that exact unconditional computation is not required at every step. Our first variant called DrAG-Spatial, continuously tracks the directional and magnitude drift of the conditional velocity, to bypass the unconditional pass whenever the trajectory remains stable. Instead of freezing unconditional predictions, DrAG-Spatial tracks the relative difference between the conditional and unconditional passes. By pairing this preserved difference with the freshly computed conditional velocity, DrAG-Spatial accurately extrapolates guidance at skipped steps, refreshing the unconditional state only when significant trajectory drift occurs. Further, to prevent high-frequency noise from triggering unnecessary unconditional evaluations, we introduce a second variant called DrAG-Spectral, which applies a Gaussian low-pass Fourier filter to isolate structural shift and guide compute allocation purely by meaningful trajectory evolution. Through extensive experiments across three T2I backbones (FLUX.1-dev, Qwen-Image, SD3) and two T2V models (Wan 2.1, HunyuanVideo), we demonstrate substantially reduced inference computation using DrAG-Spatial and DrAG-Spectral, while faithfully matching and being comparable to the full-guidance quality across established benchmarks (e.g. GenEval2, VBench).