acceptodds
Withdrawn from ICLR 2027

Dynamic Drift-Aware Guidance for Accelerated Generative Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Classifier-free guidance (CFG) enables high-fidelity generation in diffusion and flow-matching models but requires two inference passes in each sampling step because of the additional unconditional network pass. While recent acceleration heuristics skip unconditional passes using fixed timestep schedules or early termination, they ignore prompt-dependent trajectory curvature, leading to trajectory deviation, object omission, and semantic collapse. To resolve this problem, we propose Dynamic Drift-Aware Guidance (DrAG), a training-free framework that selectively skips redundant unconditional evaluations while closely approximating the original sampling trajectory. DrAG is developed on the key intuition that exact unconditional computation is not required at every step. Our first variant called DrAG-Spatial, continuously tracks the directional and magnitude drift of the conditional velocity, to bypass the unconditional pass whenever the trajectory remains stable. Instead of freezing unconditional predictions, DrAG-Spatial tracks the relative difference between the conditional and unconditional passes. By pairing this preserved difference with the freshly computed conditional velocity, DrAG-Spatial accurately extrapolates guidance at skipped steps, refreshing the unconditional state only when significant trajectory drift occurs. Further, to prevent high-frequency noise from triggering unnecessary unconditional evaluations, we introduce a second variant called DrAG-Spectral, which applies a Gaussian low-pass Fourier filter to isolate structural shift and guide compute allocation purely by meaningful trajectory evolution. Through extensive experiments across three T2I backbones (FLUX.1-dev, Qwen-Image, SD3) and two T2V models (Wan 2.1, HunyuanVideo), we demonstrate substantially reduced inference computation using DrAG-Spatial and DrAG-Spectral, while faithfully matching and being comparable to the full-guidance quality across established benchmarks (e.g. GenEval2, VBench).

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Classifier-Free Guidance, Efficient Inference, Text-to-Image Generation, Text-to-Video Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XVJKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Dynamic Drift-Aware Guidance (DrAG), a training-free approach that accelerates classifier-free guidance (CFG) by caching and dynamically reusing the displacement between conditional and unconditional predictions (the guidance-gap) across sampling steps. To decide when the cached guidance-gap becomes stale and requires recomputation, the authors develop two variants: DrAG-Spatial, which tracks directional alignment and magnitude divergence of the conditional velocity in the spatial domain, and DrAG-Spectral, which leverages low-pass Fourier filtering to isolate low-frequency structural drift from high-frequency fluctuations. Extensive experiments on T2I generation demonstrate that DrAG significantly cuts down redundant unconditional evaluations without compromising sample fidelity or prompt alignment. In addition, the authors validate the transferability of this mechanism across text-to-video, few-step, and score-based diffusion models, as well as training-time acceleration for one-step Drifting generative models.

Critical strengths
  1. Clear Presentation and Solid Theoretical Grounding: The paper is well-written and easy to follow. The proposed method is simple, elegant, and intuitive, supported by rigorous theoretical error bounds (Proposition 1 and Corollaries 1–2) that link guidance-gap approximation error to observable conditional velocity drift.

  2. Insightful Guidance-Gap Caching Mechanism: Formulating CFG acceleration as dynamic guidance-gap reuse—rather than freezing stale unconditional predictions or relying on rigid step schedules—is insightful. Both the spatial and spectral drift-aware refresh criteria are well-motivated and reasonable.

  3. Comprehensive Evaluation Across Modern Backbones and Baselines: The experiments cover diverse mainstream T2I architectures (FLUX.1-dev, Qwen-Image, and SD3) and compare against recent CFG-acceleration baselines (Step-AG, THG, and AG), achieving strong performance across most evaluation settings.

  4. Extension to One-Step Model Training: Beyond multi-step inference acceleration, the authors successfully extend the core spectral drift principle to skip redundant negative feature-encoder evaluations during the training of one-step Drifting models, demonstrating strong versatility.

Critical weaknesses
  1. Marginal Advantage Over AG on Qwen-Image: In Table 1, DrAG does not demonstrate a clear advantage over Adaptive Guidance (AG) on the Qwen-Image backbone. Specifically, AG achieves a higher compute reduction (41.6% vs. 36.8% for DrAG-Spatial and 38.5% for DrAG-Spectral) while outperforming DrAG-Spatial on the overall GenEval2 score (37.86 vs. 37.35) and trailing DrAG-Spectral by only 0.22 points (37.86 vs. 38.08). The authors should provide a more detailed explanation for this behavior.

  2. Unaligned NFE Budgets in Quantitative Comparisons: In Table 1, different acceleration methods are evaluated at noticeably different average NFEs (e.g., ranging from 33.0 to 42.0 NFE on SD3, and 36.6 to 45.3 NFE on FLUX). Without strictly aligning the NFE budgets across methods, it is difficult to establish a completely fair, apples-to-apples quality comparison.

  3. Ad-Hoc Quality–Efficiency Ranking vs. Pareto Frontier: The custom Quality–Efficiency Ranking in Table 2 relies on ordinal ranks and a manually selected weight of λ=0.40\lambda = 0.40, which is not a standard evaluation metric and feels somewhat arbitrary. For a fairer and more informative comparison, the authors should provide Quality vs. NFE Pareto frontier curves by sweeping the thresholds/hyperparameters of each method.

  4. Lack of a Conceptual Method Diagram: Section 3 lacks an illustrative overview figure comparing the pipelines of standard CFG, fixed-schedule skipping, DrAG-Spatial, and DrAG-Spectral. Including a clear schematic diagram of the guidance-gap caching and refresh workflow would improve the paper's readability.

  5. Missing Wall-Clock Latency and Batched Inference Analysis: The main text reports theoretical NFE reductions in Table 1 without comparing actual end-to-end wall-clock latency. First, DrAG-Spectral performs discrete Fourier transforms (FFT) and low-pass filtering on high-dimensional velocity tensors at every step, and this computational overhead should be explicitly quantified. Second, because refresh steps are triggered dynamically per prompt, during batched inference (batch size>1\text{batch size} > 1), different samples within a batch will require unconditional evaluations at asynchronous steps. It is unclear how the method avoids this synchronization bottleneck ("straggler effect") to achieve actual wall-clock speedups in batched deployment.

Questions for authors

Please refer to the weaknesses detailed above. I am open to raising my score if the authors can satisfactorily address these concerns during the rebuttal.

Reviewer_piKiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose Dynamic Drift-Aware Guidance (DrAG), a training-free framework that reduces the computational cost of classifier-free guidance (CFG) by dynamically skipping redundant unconditional evaluations. DrAG separates the conditional and unconditional velocities and defines their difference as the guidance gap, which represents the conditional correction during the sampling process. To determine which timesteps can skip unconditional evaluations, the authors propose two strategies, DrAG-Spatial and DrAG-Spectral, based on different metrics. Experimental results demonstrate the computational efficiency of the proposed methods. The authors also investigate the transferability of the guidance-gap reuse strategy.

Critical strengths
  1. Guidance-gap estimation using conditional drift. The authors reformulate the iterative sampling scheme and assume that the unconditional branch does not drift significantly faster than the conditional branch. Based on this assumption, they propose estimating changes in the guidance gap using information from the conditional branch, reducing the need for unconditional evaluations.
  2. Two adaptive thresholding strategies. The authors propose two different metrics, DrAG-Spatial and DrAG-Spectral, to determine when unconditional evaluations can be skipped.
Critical weaknesses
  1. Limited theoretical guidance for threshold selection. Corollary 1 appears to have limited practical value, as it does not provide useful guidance for selecting the threshold hyperparameters. Instead, these parameters seem to be determined mainly through exhaustive empirical search. This weakens the theoretical contribution of the paper and makes the proposed approach appear more empirical than theoretically motivated.
  2. Unclear generalization of threshold parameters across different sampling steps. It is unclear whether the selected threshold parameters remain effective when the total number of sampling steps changes. Since the proposed method relies on thresholding to determine when unconditional evaluations can be skipped, the sensitivity of these thresholds to the number of sampling steps should be investigated. The authors should clarify whether the same thresholds can be applied to different sampling schedules.
Questions for authors

See the weaknesses.

Reviewer_r18cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DrAG reduces the cost of classifier-free guidance by caching the difference between conditional and unconditional predictions and refreshing it when the conditional prediction changes sufficiently. The paper proposes spatial and frequency-weighted criteria and derives local error bounds under branch-coupling assumptions. Experiments across image and video models report lower NFE with broadly similar generation quality.

Critical strengths
  1. The refresh decision uses a prediction that is already computed, so it needs neither an extra network nor an unconditional evaluation to decide whether to skip. This makes the method simple to integrate.

  2. The SD3 results are encouraging: DrAG-Spectral nearly matches full CFG with about 31% fewer evaluations and improves over THG at a similar NFE budget. The additional 50K-sample evaluation supports the quality–compute trade-off.

Critical weaknesses
  1. Guidance-residual reuse already appears in FasterCache (CFG-Cache) and THG, so the main contribution seems to be the online refresh rule and its frequency weighting. The experiments do not sufficiently isolate these additions at comparable NFE and tuning effort. The gains are clearest on SD3; on FLUX and Qwen, the reported results show mixed trade-offs.

  2. The local bound follows from the triangle inequality once branch coupling is assumed, so its practical value depends on how well that assumption holds. Appendix B.1 provides useful fitted slopes and percentile envelopes, but these do not calibrate the chosen thresholds to a stated error tolerance. For the spectral variant, the bound also applies only to frequency-weighted error.

Questions for authors
  1. Could you clarify how final settings are selected from the 50-prompt validation results and how the baselines are tuned? On one representative model, a comparison at similar NFE that keeps the gap update fixed while varying the refresh schedule and frequency weighting separately would help establish the added benefit.

  2. Given the coupling statistics in Appendix B.1, what guidance-gap errors are observed at the chosen thresholds, particularly in the upper tail? For the spectral variant, how do weighted and unweighted errors compare?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.