acceptodds
Withdrawn from ICLR 2027

VeloCache: Frequency-Decoupled Velocity Caching for Training-Free Video Diffusion Acceleration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Training-free step caches accelerate video diffusion by reusing cached computation at selected denoising steps. Prior work mostly refines when to reuse; what is reused is typically the whole cached signal, which can wash out texture, or per-block feature forecasts whose cost grows with model size. Analyzing the per-step spectrum of the denoiser output, we find a volatility crossover: about one fifth into the trajectory, changes in the low spatial band that carries coarse structure decline, while the high band that carries fine texture keeps changing. This motivates two decisions. What to reuse: VeloCache retains the cached low band and advances the high band by a first-order trend from the two most recent computed outputs. Only true evaluations refresh this history, so skipped steps never feed predictions back into the cache. A local error analysis gives the condition under which the correction improves on replay, and shows how unequal sampling intervals limit it. When to evaluate: an early segment set by the pooled crossover is computed in full and the remaining budget is spread uniformly, yielding one shared rule across backbones. VeloCache reads no internal features and stores two outputs per guidance stream. At comparable evaluation budgets, it reaches the lowest LPIPS to the full-step reference on Wan2.1 and CogVideoX ( vs. on Wan2.1), competitive LPIPS on Cosmos3-Nano, and the highest VBench quality of the compared caches on all three backbones. Code will be released.

Primary area:
generative models
Keywords:
video diffusion models, cache-based acceleration, training-free inference, inference efficiency, video generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_dKrVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes VeloCache, a training-free caching method for video diffusion models acceleration. It reuses low-frequency velocity while extrapolating high-frequency components from previous steps to improve the generation quality. The authors conduct extensive experiments to demonstrate its efficiency and generation fidelity.

Critical strengths
  1. The paper is well organized and easy to read, and the motivation is clear.
  2. The paper provides a direct and lightweight way to utilize the frequency information in video generation to accelerate inference. Decoupling the low and high components shows some insights for the following works.
Critical weaknesses
  1. The benefits of frequency-aware caching are still unclear. As shown in Table 3, the "whole output" achieves a significant improvement in LPIPS over "direct replay" in Wan 2.1 results, which calls into question the benefits of frequency-aware extrapolation. The "interval-scaled" setting beats VeloCache in the Cogvideo. These results raise questions about the general effectiveness of the proposed fixed-gain frequency-aware design.
  2. Lacking important metrics for video quality: SSIM and PSNR. The paper heavily relies on LPIPS for ablation studies. However, LPIPS only indicates the frame-wise perceptual distance between the target and the original video. Almost all previous works cited by the paper utilize SSIM and PSNR as important metrics. A more comprehensive evaluation would help clarify.
Questions for authors

The proposed method only applies frequency filtering along the spatial dimensions. Could incorporating temporal-frequency information further improve the caching strategy, particularly in terms of motion consistency and video quality?

Reviewer_eQFLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes VeloCache, a training-free caching method for video diffusion. It retains the low-frequency output, adds a high-frequency correction from two recent computed outputs, and uses a fixed schedule that protects early denoising steps.

Critical strengths
  • The method is simple, requires only two cached outputs per guidance stream, and needs no access to internal layers.

  • Experiments cover several video backbones. Fixed-schedule comparisons show useful reference-LPIPS improvements on Wan2.1 and CogVideoX.

Critical weaknesses

Limited novelty is my main concern. FreqCa already combines low-frequency reuse with high-frequency prediction and caches a single cumulative residual tensor rather than per-layer features. VeloCache mainly moves this rule to the denoiser output, simplifies the predictor, and adds an early-step schedule. Table 7 shows better results than the FreqCa port, but changes the cache representation, filter, and predictor together. It therefore does not isolate the benefit of the output-level change. I find the advance over FreqCa too limited.

The frequency analysis does not sufficiently support the chosen rule. A crossover in relative band changes does not show that the previous high-frequency increment predicts the next one. The local error identity holds for any fixed linear filter and depends on the unknown target output; it does not justify the fixed gain or protection fraction. Table 3 also shows no clear improvement over direct replay on the tested Cosmos subset, while interval-scaled correction performs better on CogVideoX. The results support a useful empirical heuristic, but provide limited additional insight beyond the existing frequency-aware caching principle.

Questions for authors

See weaknesses.

Reviewer_ua9jOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VeloCache is a training-free step cache for video diffusion. At a skipped step it keeps the cached low-frequency band of the denoiser output and advances the high-frequency band by a first-order extrapolation from the two most recent true outputs, with a fixed gain of 1 and a 5×5 binomial low-pass filter. It also always evaluates the first 22% of steps, motivated by a reported "volatility crossover" where high-band changes overtake low-band changes, and spreads the remaining evaluation budget uniformly. On Wan2.1-1.3B, CogVideoX-5B and Cosmos3-Nano, it reaches lower LPIPS to the full-step reference than MagCache, EasyCache, TeaCache and TaylorSeer at comparable cost.

Critical strengths
  1. The method is simple and lightweight, acting only at the output level with two cached tensors per guidance stream, independent of network depth and with no access to internal features.
  2. The ablations are reasonably careful, including a same-anchor, same-history whole-output control that isolates the band split, and paired bootstrap CIs.
Critical weaknesses
  1. The novelty is incremental over FasterCache and FreqCa, and Proposition 1 is a generic expansion that holds for any linear filter and does not justify the frequency split.
  2. Speedups are not measured under a unified end-to-end protocol, and the quality evidence is thin: VBench gains are within noise, there is no human study, and the key Table 3 ablation covers only a prefix of the prompts and is not significant on Cosmos.
  3. The most relevant baselines (FasterCache, SeaCache, official FreqCa) are missing or only ported, and the evaluation covers just three models without high-motion, I2V or long-video settings.
Questions for authors
  1. Can you report results under a unified end-to-end protocol (same machine and pipeline, including VAE decoding and text encoding), with real speedups and peak memory versus MagCache and EasyCache? Does the "same cost" comparison still hold?
  2. Why does removing the band split make little difference on Cosmos (Table 3: direct replay −0.0012, interval-scaled −0.0023, CIs include zero)? Does the gain there come mainly from the schedule rather than the reuse rule?
  3. Can you compare at equal budget against the official implementations of FasterCache and FreqCa, and against SeaCache?
  4. Can you directly measure whether the correction helps, e.g. how often 2⟨q, D⟩ > ‖D‖² holds on each backbone, per step and per spatial band, compared with whole-output extrapolation? This would test whether the crossover actually predicts the alignment of D.
Reviewer_q8zjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies training-free caching for video diffusion models. VeloCache retains the low-frequency part of a cached denoiser output and corrects its high-frequency part using the two most recent model evaluations. Based on measured frequency dynamics, it uses an early fully computed segment followed by uniformly spaced evaluations. Experiments on Wan2.1, CogVideoX, and Cosmos3-Nano report improved fidelity to full-step outputs.

Critical strengths

1.The method is simple and lightweight. It requires no model training or internal feature access, and stores only two outputs per guidance stream.

2.The experiments cover several video backbones and show useful LPIPS improvements on Wan2.1 and CogVideoX.

Critical weaknesses

1.I find the novelty limited. Similar frequency behavior has been discussed in diffusion/flow-based visual generation, and FreqCa already reuses low frequencies and predicts high frequencies. The output-level correction is heuristic, and the local analysis does not sufficiently justify the fixed gain or recomputation schedule.

2.Table 1 does not show a clear VBench advantage. The gains are small, and some scores are below competing methods. LPIPS improves on Wan2.1 and CogVideoX, but closer agreement with the full-step reference does not necessarily mean better video quality.

Questions for authors

1.What is the main contribution over FreqCa, and how are the fixed gain and recomputation schedule justified?

2.Are the VBench gains statistically significant in paired comparisons at comparable latency?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.