- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies long-horizon streaming video generation and argues that existing methods either produce only few-step distilled generators or manage history in ways that discard intermediate context. The paper presents MovieRoll, a framework that first trains a multi-step causal base with Window-wise Teacher Forcing, which jointly denoises rolling windows at progressive noise levels. To bound memory, HieraKV keeps the full history only in shallow layers and retains just the sink and recent chunk in deep layers. The base is then distilled into a few-step generator with CuLT-DMD, which averages clip-level DMD gradients over a long self-rollout into a single update, and inference is further stabilized with KV cache recomputation and CFG annealing. Experiments on 30-second videos using HeliosBench metrics show that MovieRoll-Distilled achieves the best overall score among the compared methods in both single and multi-event settings.
Critical strengths
- Simple and memory-efficient KV design. HieraKV keeps the full history only in the first few layers and retains just the sink and recent chunk in deep layers. This simple layer-wise retention reduces peak KV cache memory and gives a speedup over full-history caching.
- Lightweight modification to long-horizon DMD. CuLT-DMD only changes the update schedule of LongLive-style long tuning, averaging clip-level gradients over a rollout before a single update. It is easy to implement and shows improved long-horizon results over the LongLive in the experiments.
Critical weaknesses
- Lack of novelty. Although the modifications are efficient, the framework largely combines existing methods. The rolling denoising window follows Rolling Forcing, CuLT-DMD only changes the generator update schedule of LongLive-style long tuning, and HieraKV’s layer-wise KV allocation closely resembles PyramidKV [1] in the LLM domain, which is not discussed.
- Insufficient qualitative results. Prior work [2] has shown that VBench-like automated metrics do not fully reflect long video quality. Therefore, verifying whether the generated videos are actually better requires actual videos or human evaluation, yet the paper provides neither supplementary videos nor a human preference study.
- Limited validation of HieraKV. Although HieraKV improves inference speed, the attention analysis and ablations are conducted only on HunyuanVideo-1.5. It remains unverified whether the observed layer-wise pattern and the layer split transfer to other video models such as Wan 2.1 or Wan 2.2.
- Interactivity latency is not evaluated. With a rolling denoising window, a new prompt can be fully reflected after several chunks, since partially denoised chunks from the previous event remain in the window. However, the paper does not report prompt switch latency or compare it with chunk-wise baselines such as LongLive.
[1]: Cai et al, “PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling”, COLM 2025
[2]: Cui et al, "Self-Forcing++: Towards Minute-Scale High-Quality Video Generation", ICLR 2026
Questions for authors
- How long does it take for a prompt switch to be reflected in the generated video? A comparison with chunk-wise methods such as LongLive would help assess the time cost of rolling denoising for interactive use.
- Could the authors provide qualitative videos or a human preference study for verification?
- Could the authors show whether the layer-wise attention pattern motivating HieraKV is also observed in other video backbones, such as Wan 2.1 or Wan 2.2, and whether HieraKV remains effective when applied to them?
5 comments
Response to Reviewer b4af (Part 2/2)
3. Attention patterns and HieraKV transfer to Wan
We have repeated the attention analysis on Wan2.1-1.3B, using an all-global, 30-layer model. Following the paper’s definition of history, we compute the statistics only over cached historical chunks preceding the active window, excluding the entire active window.
We analyze the last round before the first KV recomputation, when the cache contains 10 historical chunks, matching the configured recomputation threshold. Attention is renormalized within these ten chunks, entropy is normalized by log(10), and sink + recent coverage measures the combined attention to the first and most recent cached historical chunks. Results are averaged over four prompts and ten denoising passes:
| Layer group | Normalized historical attention entropy | Sink + recent coverage |
|---|---|---|
| First 4 layers | 0.983 | 22.5% |
| Remaining 26 layers | 0.952 | 30.7% |
The deeper-layer group exhibits a modest decrease in historical attention entropy and higher sink + recent coverage, consistent with the direction observed in HunyuanVideo. The pattern is not strictly monotonic across individual layers, and we do not assume a universal layer split across backbones.
We also implement HieraKV in the Wan port using four global layers and 26 local layers. The model uses public Wan2.1 weights and public OpenVid/VidProM data. Representative HeliosBench results are below; complete method scores are provided in the main text.
| Method | Duration | Aes. | Dyn. | Smooth. | Sem. | Nat. | D-Aes. | D-Smooth. | D-Sem. | D-Nat. | Total |
|---|---|---|---|---|---|---|---|---|---|---|---|
| LongLive | 60s | 8 | 3 | 10 | 5 | 4 | 9 | 10 | 10 | 3 | 6.093 |
| Helios-Base | 60s | 8 | 4 | 10 | 6 | 6 | 6 | 10 | 9 | 2 | 6.393 |
| Helios-Distilled | 60s | 7 | 6 | 9 | 4 | 3 | 1 | 10 | 5 | 1 | 4.128 |
| MemFlow | 60s | 8 | 4 | 10 | 5 | 4 | 8 | 10 | 10 | 2 | 5.925 |
| MovieRoll-Distilled | 60s | 7 | 6 | 10 | 4 | 10 | 6 | 10 | 8 | 4 | 7.032 |
| MovieRoll-Wan-Base | 30s | 8 | 2 | 10 | 5 | 6 | 7 | 10 | 9 | 2 | 6.177 |
| MovieRoll-Wan-Distilled | 60s | 8 | 6 | 9 | 6 | 7 | 7 | 10 | 9 | 2 | 6.777 |
All normalized ratings are higher-is-better. Aes., Dyn., Smooth., Sem., and Nat. denote aesthetics, dynamic degree, motion smoothness, semantic consistency, and naturalness; D-* denotes the corresponding temporal degradation rating.
MovieRoll-Wan-Distilled achieves the highest 60s overall score among the compared methods except the original MovieRoll-Distilled. Together with the attention analysis, this supports transferring the design to Wan. The generation results evaluate the complete framework, rather than independently isolating HieraKV’s contribution on Wan.
4. Prompt-switch latency versus LongLive
The reviewer correctly identifies the latency introduced by rolling denoising. MovieRoll does not clear its active window when the prompt changes. Existing partially denoised chunks continue through the window, while newly entering chunks undergo their complete denoising process under the new prompt.
Consequently, the first output can use the new prompt after one window round, but the first chunk denoised entirely under the new prompt requires W rounds, where W is the window size. LongLive instead applies the new prompt throughout the next chunk’s complete denoising process, requiring its denoising steps plus cache recaching.
We measured both methods on one H20 GPU, batch size 1, using the same six-event prompt with five switches and CUDA-synchronized timing:
| Method | Backbone | Chunk size (latent frames) | First output using the new prompt | First output denoised entirely under the new prompt |
|---|---|---|---|---|
| MovieRoll-Distilled | HunyuanVideo-1.5, 8.3B | 4 | 8.72–9.65s | 42.53s |
| MovieRoll-Wan-Distilled | Wan2.1, 1.3B | 3 | 2.04s | 10.25s |
| LongLive | Wan2.1, 1.3B | 3 | 2.27s | 2.27s |
For MovieRoll, W = 5. The Wan comparison uses the same backbone size, resolution (480 × 832), and chunk size. LongLive’s 2.27s includes recaching and four denoising steps; including its subsequent KV update gives 2.57s. These measurements exclude VAE decoding and any residual wait when a prompt arrives during an ongoing round. They measure conditioning and computation latency, rather than visually judged semantic response.
We acknowledge that MovieRoll has higher full-switch latency than LongLive even with the same backbone and chunk size. The rolling window introduces a quality–latency tradeoff: output begins incorporating the new conditioning after one round, while fully new-prompt-conditioned output requires the chunk to traverse the window.
Dear Reviewer b4af,
We would like to kindly ask whether our responses and additional evidence have addressed your main concerns about MovieRoll. If so, would you consider revising your score upward? We would be happy to provide any further clarification.
Thank you again for your time and consideration.
Response to Reviewer b4af (Part 1/2)
Thank you for the detailed review. We address the weaknesses and questions below.
1. Novelty and connections to Rolling Forcing, LongLive, and PyramidKV
Window-wise TF. Our contribution is to extend rolling denoising to supervised streaming fine-tuning of a multi-step base model. Rolling Forcing primarily uses rolling denoising for few-step distillation. Window-wise TF combines bidirectional attention within the active window with causal access to clean history, and isolates noisy training windows to support parallel supervision. This enables both historical KV reuse during streaming inference and parallel window supervision during base-model training. Appendix D, Table 11 shows the long-horizon benefit: at 721 frames, Window-wise TF scores 6.321 versus 5.001 for chunk-wise TF.
HieraKV and PyramidKV. PyramidKV studies layer-dependent KV compression in LLMs through attention-based token selection and layer-wise cache budgets. HieraKV shares the principle that different layers need different amounts of history. Our contribution is to establish and exploit this property in streaming video generation, where existing cache compression primarily operates along the temporal dimension. HieraKV complements temporal compression by retaining full historical video chunks in shallow layers and sink + recent chunks in deeper layers, following our video-attention analysis. The key finding is that not every layer needs full historical KV: this structured retention improves quality from 6.321 to 6.873 while reducing KV memory by 78%. We do not claim that allocating cache by depth, or outperforming full-cache inference, is unique to our work.
CuLT-DMD. CuLT is a specific use of gradient accumulation across temporal positions within one long self-rollout. Its contribution concerns the rollout-level training schedule. LongLive-style training updates the generator within a rollout while carrying forward previously generated history; CuLT holds the generator fixed throughout the rollout and averages locally truncated gradients before updating. Accumulating gradients from independent short clips does not provide the histories encountered at late positions in a long self-rollout. With matched initialization, rollout lengths, and optimizer update counts, CuLT scores 7.260 versus 7.131 for the LongLive-style schedule.
2. Human evaluation
We have conducted a human preference study with 59 participants, 350 video pairs, and 2,616 judgments. Each pair contains videos generated from the same prompt, and participants select their preferred video or a tie. The study covers 10s, 30s, and 60s single-event videos and 30s and 60s multi-event videos.
The table reports MovieRoll-Distilled’s preference win rates, aggregated across the evaluated durations. Ties remain in the denominator.
MovieRoll-Distilled is preferred over every evaluated baseline, providing human evidence beyond the automated metrics.