acceptodds
Under review as a conference paper at ICLR 2027

In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video Diffusion

Abstract

Few-step autoregressive video diffusion generates a long video by splitting the video into temporal chunks and generating chunk-by-chunk, each through a short sequence of denoising stages. To memorize chunks that are already generated, previous methods reconstruct a clean or less-noisy key–value (KV) cache by additional forwards to build the cache without advancing an output latent. However, every denoising forward itself already computes the in-flight KV of the current chunk. We introduce FlashForward, which directly reuses this cache to avoid the heavy cache-update-only model forwards. After the current chunk completes one denoising stage, its stage-specific cache is already available for the next chunk. Assigning one GPU to each stage therefore lets different chunks occupy different stages concurrently. This early availability has a quality cost: the resulting stage-matched history is noisy, causing appearance and motion drift among chunks. To complement it, FlashForward produces sparse auxiliary clean anchor latents before the corresponding region is generated so the generation trajectories can be stabilized by this two-sided conditioning. The two memories operate at different temporal scales: sparse clean anchor KV supplies coarse, long-range two-sided structural guidance, while dense stage-matched history preserves fine, recent evolution. With up to four GPUs, FlashForward runs – faster than HiAR and – faster than Self-Forcing for 16 FPS videos of 20 seconds or longer across 1.3B and 14B backbone scales at 480p and 720p. On VBench, for the 1.3B model at 480p, it achieves higher scores and remains stable at longer durations, demonstrating that FlashForward generates high-quality and temporally consistent videos across durations of 20s, 35s and 65s at a much faster generation speed.

Primary area:
generative models
Keywords:
video generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_wdcKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FlashForward is an algorithm designed for the parallel acceleration of few-step autoregressive video diffusion models. It modifies the availability timing of cross-chunk memory in few-step autoregressive video diffusion: during each forward pass, the renderer uses key-value (KV) caches generated from the current noisy input to subsequent video chunks at the same denoising stage. This enables cross-chunk execution without requiring redundant forward passes solely dedicated to updating the cache.

For video outputs ranging from 20 to 65 seconds—under the constraint of using up to four GPUs with each baseline configured for maximum speed—the paper reports denoising-path speedups of 1.16× to 1.69× relative to HiAR, and 1.42× to 2.92× relative to Self-Forcing. Quality evaluations exclusively cover the trained 1.3B/480p model: achieving an overall VBench-1.0 score of 0.838 (compared to 0.821 and 0.805 for the two baselines), while maintaining an overall VBench-Long score of approximately 0.840 across 20-second, 35-second, and 65-second outputs.

Critical strengths

Clear Motivation: Existing approaches often run redundant, auxiliary model passes solely to construct historical KV caches—even though the standard denoising process already computes KV states internally. Exploring how to reuse these states directly and propagate them earlier to subsequent chunks is therefore backed by clear computational logic.

Practical method and good efficiency gains: By reusing same-stage KV caches generated during the renderer's standard forward passes, this method eliminates the overhead of dedicated cache construction and enables parallel execution.

Comprehensive theoretical analysis: The paper includes a meaningful theoretical analysis of the efficiency gains, deriving both forward-count reductions and wall-clock break-even conditions, and relating these predictions to the measured speedups.

Critical weaknesses

Insufficient Ablation:

  • Conditioning Variants: While Table 3 demonstrates that the full method outperforms the evaluated conditioning variants, "Clean history and future" is described as a Self-Forcing variant that alters the temporal generation order. It remains unclear whether this serves as a rigorous control that solely removes short-term historical KVs within the same planner–renderer framework. Furthermore, controlled comparisons isolating clean versus noisy anchors, as well as past-only versus bidirectional anchors, are missing. Consequently, it is difficult to determine whether the quality gains stem from the clean nature of the anchors, the incorporation of future context, or their complementary interplay with dense historical conditioning.
  • Training Ablations: A similar limitation appears in the training ablations. Role embeddings and LoRA adapters are introduced simultaneously, obscuring their individual contributions.
  • Sensitivity to Hyperparameters: Although Appendix A.1 justifies the fixed anchor stride from the standpoint of speed matching between the planner and renderer, the paper lacks a quality–latency sensitivity analysis across varying anchor strides and historical window lengths.

The gap between the theoretical and measured speedups is not fully explained: The forward-count analysis suggests a substantial computational advantage over HiAR (e.g., 1.72$\times$) at 81 latents), while the measured speedup over HiAR can be as low as about (1.16$\times$) in some settings. Although the paper mentions pipeline fill/drain, synchronization, communication, and sublinear parallel efficiency as possible sources of overhead, it does not quantitatively decompose this gap. Could the authors provide a more detailed latency breakdown, e.g., separating planner cost, renderer computation, cache operations, inter-stage communication, synchronization, and pipeline bubbles, to clarify why the practical speedup can fall substantially below the forward-count prediction and which component dominates the loss?

Questions for authors

see weakness.

Reviewer_pBwYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work focuses on autoregressive video generation. The work uses multiple GPUs to simultaneously denoise chunks and reuses the KV cache from every timestep of the previous chunk. Since the reused KV caches are noisier than those used in previous work, the model also introduces an additional planner (basically another generator) to generate clean anchors.

Critical strengths

The key difference between this work and previous work is that it reuses the KV caches produced at every denoising step of previous chunks.

Since KV caches from early denoising steps contain more noise and may provide less reliable information, the method introduces an additional planner (basically another generator that shares the same base weight with different lora) to generate clean anchors at sparse temporal positions, providing high-quality and clean guidance for the renderer.

This complementary design, combining dense noisy KV caches with sparse clean anchors, is the key contribution of this work.

Critical weaknesses

The proposed framework relies on an additional planner to generate sparse clean anchors, which provide the renderer with long-range structural guidance. This design is reasonable, as the method reuses KV caches from early denoising stages, which are much noisier than those from later stages.

However, this raises a concern about whether the overall generation quality, particularly long-term temporal consistency, is heavily dependent on the quality of the planner. If the generated anchors contain inconsistencies in appearance, identity, or motion, it is unclear whether the renderer can effectively correct these errors or simply propagates them into the final video.

I raise this concern because in ablation study (Table 3), when using only stage-matched noisy KV caches, the model suffers from severe temporal flickering, resulting in failed evaluation for two of three VBench metrics. However, after introducing clean anchors, the model achieves the best results. This suggests that the clean anchors play a critical role in maintaining generation quality, raising questions about how much the stage-matched noisy KV caches actually contribute.

The paper does not provide an ablation in which the renderer only uses clean anchor KV caches without stage-matched historical KV caches. This is a critical missing experiment for evaluating the necessity of the proposed KV reuse strategy. If the model can achieve comparable generation quality using only clean anchor KV caches, the contribution and necessity of the proposed stage-matched noisy KV reuse would become questionable.

Questions for authors
  • Please discuss the concerns mentioned in the critical weaknesses part.
  • Do you plan to open-source the code? Code availability is important to me for evaluating the reproducibility of this submission.
  • FlashForward and HiAR use different training data and supervision lengths. I am not whether the quality improvements come from the proposed design or the data used in this paper. If possible, could you retrain HiAR using your collected data?
  • In HiAR, they identify motion collapse as a major issue in self-rollout DMD used in this submission and introduces Forward-KL Regularization to preserve motion diversity. FlashForward only reports aggregated VBench scores in Table 2 without Dynamic Degree or other detailed motion-related metrics. Could you provide related metric results?
  • HiAR proposes fusing historical KV updates and current-chunk denoising into a single forward pass. Do you retains this optimization when calculating latency for HiAR?
  • The 14B experiments are limited to latency measurements, without actual generation quality evaluation. This point is not so important, just for your consideration to improve the evaluation.

I cannot recommend acceptance without satisfactory responses to the concerns raised in the Critical Weaknesses section. Nevertheless, the overall quality of this submission is fairly good, apart from these technical issues. If the authors address these concerns, I would be willing to recommend acceptance.

Reviewer_7HzROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FlashForward, a generation pipeline for few-step autoregressive video diffusion that removes forward passes used only to build the KV cache. Each chunk reuses the KV that earlier chunks computed at the same denoising stage during their ordinary denoising forwards. Successive chunks can therefore run concurrently across GPUs, with one GPU per denoising stage. This reused history is noisy, so a planner first generates sparse, clean anchor latents along the timeline. A renderer then generates every output frame, conditioned on the nearby past and future anchors as well as the reused history. The two roles share one backbone with role-specific adapters. They are trained with supervised fine-tuning on real videos, followed by self-rollout distillation. The paper evaluates multi-GPU generation latency across model sizes and resolutions, and evaluates generation quality with VBench on 20–65 s videos.

Critical strengths
  1. The paper pinpoints a real bottleneck in few-step autoregressive video diffusion: forward passes spent only on building memory. It then observes that the needed KV is already computed during ordinary denoising. Reusing that KV is simple, and it naturally turns generation into a multi-GPU pipeline.
  2. The paper pairs dense but noisy recent history with sparse, clean anchors that constrain the video from both sides. This sensibly divides the work across temporal scales, and the ablations show that both components are needed.
  3. Quality stays stable as videos grow longer. The supplementary videos show noticeably less color and appearance drift than the baselines while still keeping visible motion.
  4. The interactive supplementary demo, detailed algorithms, and implementation details make the method easy to understand and help reproducibility.
Critical weaknesses
  1. Limited novelty relative to pipelined and keyframe-based generation. Each of the two ingredients has clear precedents. Chunk-level pipelining over partially denoised predecessors appears in MAGI-1 and HiAR. Planning sparse keyframes and then filling the frames between them appears in plan-then-infill methods such as MMPL. The genuinely new piece is conditioning on same-stage history, which the paper frames as a new “cross-chunk state contract”. Its positioning against the closest prior work is thin: MAGI-1 appears only as a VBench row, and MMPL only as future work.
  2. The speedup does not come mainly from where the paper says. The paper presents removing “heavy” cache-update-only forwards as the key gain. Against clean-history methods, however, the compute saving is small: on a single GPU, FlashForward is slightly slower than Self-Forcing. Its speedup there comes mostly from spreading the work over more GPUs, which trades per-GPU throughput for latency. The reduction in forward passes is real against HiAR, but the paper should state this distinction plainly.
  3. The core design lacks its most important ablation and baseline. There is no anchors-only variant without stage-matched history, and no comparison with a plan-then-parallel-infill pipeline under the same GPU budget. Without them, the paper does not show that the wavefront over noisy history is the right way to exploit the anchors.
Questions for authors
  1. (Clarification; this affects how I weigh the speed claims.) Which configurations in Tab. 4 correspond to trained FlashForward models with verified quality? Is there a trained 1.3B 720p model, or any 14B model? If those rows are systems-only, will you restrict the abstract’s ranges to the validated setting or label them explicitly? Please also report per-GPU throughput, and compare with Self-Forcing at an equal GPU count (e.g., four independent one-GPU Self-Forcing streams)..
  2. Please add an anchors-only renderer: same training, no stage-matched history. Report its quality, and its latency when the intervals between anchors are rendered in parallel. Ideally, also compare with MMPL under the same GPU budget. This would show whether the wavefront over noisy history is needed, or whether the anchors do most of the work..
  3. To separate the data and recipe from the state contract, can you report a plain causal clean-history (Self-Forcing-style) model trained with your data and your SFT+DMD recipe, without future anchors? Please also report per-dimension VBench scores (especially dynamic degree, temporal flickering, and subject/background consistency) over more than one seed..
  4. Renderer chunks read anchors roughly 4–5 s ahead, and each planner block commits about 7.5 s of anchors at once. How would FlashForward handle prompt switching or action-conditioned generation, which motivates several AR baselines, and what is the response delay?.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.