acceptodds
Withdrawn from ICLR 2027

One Threshold Is Not Enough: Prompt-Invariant Caching Schedules for Video Diffusion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video diffusion models remain expensive because denoising repeatedly evaluates a costly model backbone. Dynamic caching methods like TeaCache, EasyCache, and DiCache reduce this cost by reusing prior outputs while an accumulated drift signal remains below a threshold . Although existing methods improve the drift signal, they hold fixed throughout denoising. We show that fixed thresholds are structurally suboptimal: varying across denoising achieves quality-latency tradeoffs unattainable by any fixed threshold across three caching methods with distinct drift signals. We further show that, surprisingly, each method's drift trajectory is nearly prompt-invariant across seven method-model combinations. The shape of the drift trajectory is determined by the model and caching method; therefore, a threshold schedule can be calibrated offline and reused across generations. Based on these findings, we present ACID (Adaptive Caching for vIDeo generation), which uses a low threshold during critical regions where drift changes rapidly and a high threshold across stable regions. ACID identifies these regions offline from the drift signal's second derivative, requires no training, and adds negligible runtime overhead. Across TeaCache, EasyCache, and DiCache on HunyuanVideo, Wan 2.1, and CogVideoX, ACID pushes the speed-quality Pareto frontier beyond fixed thresholds. On TeaCache with HunyuanVideo, it achieves speedup over no caching and 38% additional speedup over a conservative fixed threshold, with less than 0.3 dB PSNR, 0.01 SSIM, and 0.01 LPIPS degradation relative to that fixed-threshold configuration.

Primary area:
generative models
Keywords:
video diffusion models, diffusion model acceleration, feature caching, training-free acceleration, inference efficiency, diffusion transformers

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2tgwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

ACID proposes scheduling cache thresholds over the denoising process to accelerate video diffusion models. It uses drift trajectories to identify critical regions offline, then applies conservative thresholds in those regions and more aggressive thresholds elsewhere. The schedule can be added to existing caching methods without retraining. The limited experiments across several methods suggest better quality–latency tradeoffs, but further ablations and analysis to explain the gains are lacking.

Critical strengths
  1. Threshold scheduling is a useful and simple design choice. ACID leaves the underlying drift estimator and reuse mechanism intact, making it easy to add to existing methods with little inference overhead.

  2. The evaluation covers seven method–model combinations and three different drift signals. Appendix C includes intermediate fixed thresholds, which makes the reported tradeoff gains more convincing than a comparison against only low and high endpoints.

Critical weaknesses
  1. The presentation is poor, and the core calibration procedure is difficult to follow. Algorithm 1 defines Rcrit={t:sˉ′′ has not yet stabilized}\mathcal R_{\mathrm{crit}}=\{t:\bar s''\text{ has not yet stabilized}\} without a numerical stability rule, while Appendix C omits the transition indices needed to reproduce the schedules. These central details need a much clearer explanation. More importantly, the contribution feels somewhat dated and reads like a small engineering trick. A small change can be valuable with careful analysis; limited novelty alone is not my concern. Here, however, the analysis remains shallow. The drift plots and proxy check in Figures 2–6 do not isolate whether the gains come from the curvature rule, warmup/end protection, or extra tuning, nor sufficiently explain how drift relates to accumulated caching error. Existing analyses of output sensitivity (Haghighi, 2026) and path-dependent caching error (Cui et al., 2026) also deserve discussion. Matched mechanism ablations are needed to explain what threshold scheduling adds within these broader caching frameworks.

  2. The evaluation is limited to PSNR, SSIM, and LPIPS against uncached outputs on 33 prompts with a fixed seed. There is no human evaluation or direct assessment of temporal consistency and prompt alignment. The many appendix curves add operating points but do not answer the underlying mechanism question. I would kindly encourage the authors to examine error propagation and when the threshold rule succeeds or fails across samplers, step counts, guidance settings, and motion complexity, beyond changing the model. A deeper analysis and discussion would make this more interesting than stopping at “we changed the threshold and it worked.”

Reference

Haghighi, Yasaman, and Alexandre Alahi. "SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching." arXiv preprint arXiv:2602.24208 (2026).

Cui, Bowen, et al. "Denoising as Path Planning: Training-Free Acceleration of Diffusion Models with DPCache." arXiv preprint arXiv:2602.22654 (2026).

Questions for authors
  1. What numerical rule determines stabilization of sˉ′′\bar s''? Please provide transition indices, threshold values, search ranges, and calibration cost for every reported configuration.
  2. At matched compute and tuning cost, what improvement comes from the curvature rule over simple schedules and warmup/end protection? Can per-step reuse errors and their propagation explain the difference?
  3. How does ACID relate to sensitivity-aware and path-dependent caching approaches, and what distinct insight or benefit does its threshold rule provide?
  4. With calibration frozen, does the rule transfer across seeds, samplers, step counts, guidance settings, and motion complexity? Can human judgments and video-specific metrics confirm the quality tradeoff?
  5. Could the readability of algorithm 2 be improved?
Reviewer_FHBwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates dynamic feature caching methods, and reveals that holding a single caching threshold constant throughout denoising is structurally suboptimal. The authors propose ACID, a training-free framework that applies a low threshold during critical regions where drift changes rapidly and a high threshold across stable regions. Experiments across Hunyuan Video, Wan 2.1, and CogVideoX demonstrate that ACID shows boosted performance.

Critical strengths
  1. The paper demonstrates that the limitation in current dynamic caching methods lies not just in how drift is estimated, but in the structural reliance on a single, fixed threshold across the entire denoising timeline.
  2. ACID could be integrated across three dynamic caching baselines (TeaCache, EasyCache, DiCache) and three major video diffusion architectures.
  3. ACID is plug-and-play, and is simple.
Critical weaknesses
  1. While using the second derivative of the drift signal to find critical regions is intuitive, the precise criteria for selecting the exact values of τlow\tau_{low} and τhigh\tau_{high} appear heavily reliant on empirical grid searches or calibration studies rather than a formal theoretical guarantee.
  2. The prompt-invariance assumption relies on a fixed number of sampling steps and standard schedules. If users alter solver types, step counts dynamically, or use aggressive few-step distillation solvers, the pre-computed offline schedule may lose its alignment.
  3. The paper evaluates across standard benchmarks (T2V-CompBench, VBench), but does not thoroughly analyze whether extreme variations in video aspect ratios, frame lengths, or spatial resolutions shift the temporal location of the second-derivative inflection points.
Questions for authors
  1. How sensitive is the offline-calibrated schedule to changes in the total number of denoising steps (e.g., reducing from 50 steps to 20 steps for fast sampling)? Does the second-derivative inflection point scale linearly, or must a new schedule be calibrated for every step-count configuration?
  2. How difficult is it to tune the optimal pair of (τlow,τhigh)(\tau_{low}, \tau_{high}) for a new model? Is there an automated way to derive them alongside the critical region boundaries from the second derivative?
Reviewer_RCXxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper points out that using a single, fixed threshold in dynamic caching methods is suboptimal. The model’s dynamics shift throughout the denoising process; applying a high threshold during critical steps—where the drift signal changes rapidly—leads to significant degradation in image quality. Conversely, using a low threshold during phases where the signal is stable wastes the potential for acceleration. To address this, the paper proposes a "Threshold Scheduling" scheme. Since the trajectory of the drift signal is independent of the prompt—determined instead by the model and the caching method itself—the optimal timing for switching between low and high thresholds can be precomputed offline and then applied directly during inference.

Critical strengths
  1. This paper breaks past research that only focused on improving the accuracy of drift signal estimation, pointing out that fixed thresholds are the bottleneck that limits the upper limit of acceleration, providing a new perspective for video generation acceleration.

  2. This paper proposes a plug-and-play method that is training-free, signal-agnostic, and incurs negligible runtime overhead. It can be integrated directly into existing video denoising models.

Critical weaknesses
  1. The author’s core assumption is that the "drift signal" remains essentially invariant to changes in the prompt. Since this conclusion is based on a limited set of prompts and fixed generation settings, does it still hold true under a wider range of conditions?

  2. Does the need to recalibrate whenever the specific model and caching method are changed entail significant additional overhead?

  3. There is no precise definition for the criterion used to determine when the second derivative has stabilized; is this assessment made manually? Furthermore, how many consecutive steps are required for it to be considered "stable"?

  4. The authors assert that ACID can reach operating points that are unattainable with any fixed threshold. This is a rather strong claim; is it possible that there exists some as-yet-unexplored fixed threshold that could yield even better results?

Questions for authors

same as the weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.