acceptodds
Under review as a conference paper at ICLR 2027

MovieRoll: Rolling Out Long-Form Videos with Hierarchical KV Caching and Cumulative Long Tuning

Abstract

Streaming video generation seeks to extend high-quality short-video synthesis to continuous, interactive generation. However, the mismatch between training on short videos with ground-truth context and long-horizon inference conditioned on generated content makes quality difficult to sustain. Meanwhile, full-history KV caching incurs memory costs that grow with generation length. To address these challenges, we present MovieRoll, a base-to-distilled framework that first establishes a multi-step streaming base through supervised training and then distills it into a few-step generator through long self-rollouts. At the base stage, Window-wise Teacher Forcing jointly denoises multiple chunks with progressive noise levels within each window, preserving within-window temporal interactions and improving resistance to long-horizon degradation compared with chunk-wise training. To retain historical context efficiently, hierarchical KV-cache compression (HieraKV) exploits layer-wise attention sparsity in streaming DiTs, retaining the global available history in shallow layers but only the sink chunk and the most recent history chunk in deep layers, which reduces KV-cache memory by 78% and improves degradation resistance. Building on this base, cumulative long tuning with distribution matching distillation (CuLT-DMD) accumulates and averages gradients from different local clips throughout an entire long self-rollout. The resulting gradient spans the full sample and provides a rollout-level estimate, improving stability during long-horizon generation. Finally, at inference time, threshold-triggered history selection and KV recomputation bound the retained context, while classifier-free guidance annealing (AnnealCFG) progressively reduces guidance strength toward the clean endpoint during denoising to further stabilize MovieRoll-Base. In 30-second HeliosBench evaluations, MovieRoll-Base achieves competitive generation quality. MovieRoll-Distilled achieves state-of-the-art (SOTA) overall scores in both single- and multi-event settings, combining the highest naturalness with sufficient dynamics. Model weights and code will be publicly released after the review process.

Primary area:
generative models
Keywords:
Autoregressive Streaming Video Generation, Hierarchical Context Management, Long-Horizon Generation Stability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_b4afOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies long-horizon streaming video generation and argues that existing methods either produce only few-step distilled generators or manage history in ways that discard intermediate context. The paper presents MovieRoll, a framework that first trains a multi-step causal base with Window-wise Teacher Forcing, which jointly denoises rolling windows at progressive noise levels. To bound memory, HieraKV keeps the full history only in shallow layers and retains just the sink and recent chunk in deep layers. The base is then distilled into a few-step generator with CuLT-DMD, which averages clip-level DMD gradients over a long self-rollout into a single update, and inference is further stabilized with KV cache recomputation and CFG annealing. Experiments on 30-second videos using HeliosBench metrics show that MovieRoll-Distilled achieves the best overall score among the compared methods in both single and multi-event settings.

Critical strengths
  1. Simple and memory-efficient KV design. HieraKV keeps the full history only in the first few layers and retains just the sink and recent chunk in deep layers. This simple layer-wise retention reduces peak KV cache memory and gives a speedup over full-history caching.
  2. Lightweight modification to long-horizon DMD. CuLT-DMD only changes the update schedule of LongLive-style long tuning, averaging clip-level gradients over a rollout before a single update. It is easy to implement and shows improved long-horizon results over the LongLive in the experiments.
Critical weaknesses
  1. Lack of novelty. Although the modifications are efficient, the framework largely combines existing methods. The rolling denoising window follows Rolling Forcing, CuLT-DMD only changes the generator update schedule of LongLive-style long tuning, and HieraKV’s layer-wise KV allocation closely resembles PyramidKV [1] in the LLM domain, which is not discussed.
  2. Insufficient qualitative results. Prior work [2] has shown that VBench-like automated metrics do not fully reflect long video quality. Therefore, verifying whether the generated videos are actually better requires actual videos or human evaluation, yet the paper provides neither supplementary videos nor a human preference study.
  3. Limited validation of HieraKV. Although HieraKV improves inference speed, the attention analysis and ablations are conducted only on HunyuanVideo-1.5. It remains unverified whether the observed layer-wise pattern and the layer split transfer to other video models such as Wan 2.1 or Wan 2.2.
  4. Interactivity latency is not evaluated. With a rolling denoising window, a new prompt can be fully reflected after several chunks, since partially denoised chunks from the previous event remain in the window. However, the paper does not report prompt switch latency or compare it with chunk-wise baselines such as LongLive.

[1]: Cai et al, “PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling”, COLM 2025

[2]: Cui et al, "Self-Forcing++: Towards Minute-Scale High-Quality Video Generation", ICLR 2026

Questions for authors
  1. How long does it take for a prompt switch to be reflected in the generated video? A comparison with chunk-wise methods such as LongLive would help assess the time cost of rolling denoising for interactive use.
  2. Could the authors provide qualitative videos or a human preference study for verification?
  3. Could the authors show whether the layer-wise attention pattern motivating HieraKV is also observed in other video backbones, such as Wan 2.1 or Wan 2.2, and whether HieraKV remains effective when applied to them?
5 comments

Response to Reviewer b4af (Part 1/2)

Thank you for the detailed review. We address the weaknesses and questions below.

1. Novelty and connections to Rolling Forcing, LongLive, and PyramidKV

Window-wise TF. Our contribution is to extend rolling denoising to supervised streaming fine-tuning of a multi-step base model. Rolling Forcing primarily uses rolling denoising for few-step distillation. Window-wise TF combines bidirectional attention within the active window with causal access to clean history, and isolates noisy training windows to support parallel supervision. This enables both historical KV reuse during streaming inference and parallel window supervision during base-model training. Appendix D, Table 11 shows the long-horizon benefit: at 721 frames, Window-wise TF scores 6.321 versus 5.001 for chunk-wise TF.

HieraKV and PyramidKV. PyramidKV studies layer-dependent KV compression in LLMs through attention-based token selection and layer-wise cache budgets. HieraKV shares the principle that different layers need different amounts of history. Our contribution is to establish and exploit this property in streaming video generation, where existing cache compression primarily operates along the temporal dimension. HieraKV complements temporal compression by retaining full historical video chunks in shallow layers and sink + recent chunks in deeper layers, following our video-attention analysis. The key finding is that not every layer needs full historical KV: this structured retention improves quality from 6.321 to 6.873 while reducing KV memory by 78%. We do not claim that allocating cache by depth, or outperforming full-cache inference, is unique to our work.

CuLT-DMD. CuLT is a specific use of gradient accumulation across temporal positions within one long self-rollout. Its contribution concerns the rollout-level training schedule. LongLive-style training updates the generator within a rollout while carrying forward previously generated history; CuLT holds the generator fixed throughout the rollout and averages locally truncated gradients before updating. Accumulating gradients from independent short clips does not provide the histories encountered at late positions in a long self-rollout. With matched initialization, rollout lengths, and optimizer update counts, CuLT scores 7.260 versus 7.131 for the LongLive-style schedule.

2. Human evaluation

We have conducted a human preference study with 59 participants, 350 video pairs, and 2,616 judgments. Each pair contains videos generated from the same prompt, and participants select their preferred video or a tie. The study covers 10s, 30s, and 60s single-event videos and 30s and 60s multi-event videos.

The table reports MovieRoll-Distilled’s preference win rates, aggregated across the evaluated durations. Ties remain in the denominator.

Compared methodSingle-event win rateMulti-event win rate
CausVid90.3%—
Self Forcing81.9%—
Causal Forcing78.3%—
Rolling Forcing72.8%—
MemFlow72.6%62.3%
LongLive68.4%66.7%
Echo-Infinity77.2%64.3%
Krea73.0%—
Helios-Base67.0%67.1%
Helios-Distilled73.0%81.9%

MovieRoll-Distilled is preferred over every evaluated baseline, providing human evidence beyond the automated metrics.

Response to Reviewer b4af (Part 2/2)

3. Attention patterns and HieraKV transfer to Wan

We have repeated the attention analysis on Wan2.1-1.3B, using an all-global, 30-layer model. Following the paper’s definition of history, we compute the statistics only over cached historical chunks preceding the active window, excluding the entire active window.

We analyze the last round before the first KV recomputation, when the cache contains 10 historical chunks, matching the configured recomputation threshold. Attention is renormalized within these ten chunks, entropy is normalized by log(10), and sink + recent coverage measures the combined attention to the first and most recent cached historical chunks. Results are averaged over four prompts and ten denoising passes:

Layer groupNormalized historical attention entropySink + recent coverage
First 4 layers0.98322.5%
Remaining 26 layers0.95230.7%

The deeper-layer group exhibits a modest decrease in historical attention entropy and higher sink + recent coverage, consistent with the direction observed in HunyuanVideo. The pattern is not strictly monotonic across individual layers, and we do not assume a universal layer split across backbones.

We also implement HieraKV in the Wan port using four global layers and 26 local layers. The model uses public Wan2.1 weights and public OpenVid/VidProM data. Representative HeliosBench results are below; complete method scores are provided in the main text.

MethodDurationAes.Dyn.Smooth.Sem.Nat.D-Aes.D-Smooth.D-Sem.D-Nat.Total
LongLive60s8310549101036.093
Helios-Base60s841066610926.393
Helios-Distilled60s76943110514.128
MemFlow60s8410548101025.925
MovieRoll-Distilled60s7610410610847.032
MovieRoll-Wan-Base30s821056710926.177
MovieRoll-Wan-Distilled60s86967710926.777

All normalized ratings are higher-is-better. Aes., Dyn., Smooth., Sem., and Nat. denote aesthetics, dynamic degree, motion smoothness, semantic consistency, and naturalness; D-* denotes the corresponding temporal degradation rating.

MovieRoll-Wan-Distilled achieves the highest 60s overall score among the compared methods except the original MovieRoll-Distilled. Together with the attention analysis, this supports transferring the design to Wan. The generation results evaluate the complete framework, rather than independently isolating HieraKV’s contribution on Wan.

4. Prompt-switch latency versus LongLive

The reviewer correctly identifies the latency introduced by rolling denoising. MovieRoll does not clear its active window when the prompt changes. Existing partially denoised chunks continue through the window, while newly entering chunks undergo their complete denoising process under the new prompt.

Consequently, the first output can use the new prompt after one window round, but the first chunk denoised entirely under the new prompt requires W rounds, where W is the window size. LongLive instead applies the new prompt throughout the next chunk’s complete denoising process, requiring its denoising steps plus cache recaching.

We measured both methods on one H20 GPU, batch size 1, using the same six-event prompt with five switches and CUDA-synchronized timing:

MethodBackboneChunk size (latent frames)First output using the new promptFirst output denoised entirely under the new prompt
MovieRoll-DistilledHunyuanVideo-1.5, 8.3B48.72–9.65s42.53s
MovieRoll-Wan-DistilledWan2.1, 1.3B32.04s10.25s
LongLiveWan2.1, 1.3B32.27s2.27s

For MovieRoll, W = 5. The Wan comparison uses the same backbone size, resolution (480 × 832), and chunk size. LongLive’s 2.27s includes recaching and four denoising steps; including its subsequent KV update gives 2.57s. These measurements exclude VAE decoding and any residual wait when a prompt arrives during an ongoing round. They measure conditioning and computation latency, rather than visually judged semantic response.

We acknowledge that MovieRoll has higher full-switch latency than LongLive even with the same backbone and chunk size. The rolling window introduces a quality–latency tradeoff: output begins incorporating the new conditioning after one round, while fully new-prompt-conditioned output requires the chunk to traverse the window.

Dear Reviewer b4af,

We would like to kindly ask whether our responses and additional evidence have addressed your main concerns about MovieRoll. If so, would you consider revising your score upward? We would be happy to provide any further clarification.

Thank you again for your time and consideration.

Reviewer_ZQ19OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MovieRoll, a streaming long-video generation framework that employs a "train a multi-step base model, then distill it into a few-step generator" approach, built upon the HunyuanVideo-1.5 (8.3B) backbone. It consists of four key components:

Window-wise Teacher Forcing: A method for training the base model, where multiple chunks within a window are denoised jointly using progressive noise levels. HieraKV: A KV-caching strategy that retains the full history of KV pairs for the first 6 (shallow) blocks, while keeping only the "sink" chunk and the most recent chunk for the subsequent 48 (deeper) blocks. CuLT-DMD: A technique applied during a single long self-rollout; model parameters remain frozen, while gradients from DMD (Distribution Matching Distillation) across various clips are averaged to perform a single, consolidated parameter update. Inference Strategy: Includes a threshold-triggered KV-recomputation mechanism, combined with "AnnealCFG"—a strategy where the Classifier-Free Guidance (CFG) scale is linearly decayed from 4 to 1.

Critical strengths
  1. The system is comprehensive, with targeted designs for the training, distillation, and inference phases. The appendices provide highly detailed hyperparameters (Tables 3–6), offering significant practical value for implementation.

  2. The design of HieraKV is empirically motivated: the analysis of layer-wise attention entropy and "sink + recent" token coverage (Figure 3) supports the observation that attention in deeper layers tends to be more localized. The approach yields notable improvements in both memory efficiency and inference speed, while simultaneously enhancing quality (from 6.321 to 6.873).

  3. Table 11 clearly demonstrates that the advantage of "Window-wise TF" (Temporal Fusion) lies primarily in its ability to extrapolate beyond the training sequence length: while performance is comparable across methods at 241 frames, it reaches 6.321 vs. 5.001 at 721 frames—a result that directly aligns with the core problem the paper aims to address.

  4. The evaluation covers 241, 721, and 1441 frames, as well as single-event and multi-event scenarios, benchmarking against more than 10 other methods. The authors also transparently report the limitations regarding inference speed (1.48 FPS; Table 12).

Critical weaknesses
  1. There is a confounding of the backbone model and the training data. Most baselines are based on the 1.3B-parameter "Wan" series models, whereas this paper employs the 8.3B-parameter "HunyuanVideo-1.5" model, supplemented by approximately 1.9 million internal video clips (Table 5). Consequently, the reported state-of-the-art (SOTA) performance cannot be attributed solely to the proposed methodology. The choice of evaluator appears biased in favor of the authors' work.

  2. For the "Naturalness" metric—which carries a weight of 0.255—the authors replaced the original GPT-5.2 evaluator used in HeliosBench with Gemini-2.5-Pro, justifying this change by claiming it offers "greater discriminative power" (Table 9). The authors' advantage in the overall score stems primarily from this "Naturalness" metric (scoring 10, compared to 4–8 for other models). Such a methodological choice should have been established a priori, or, at the very least, results from both evaluation setups should have been reported.

  3. The metrics lack sufficient resolution. All metrics are discretized into integer scores ranging from 1 to 10, with no raw scores, variance across multiple random seeds, or confidence intervals provided. In Table 10, the results for "Full History" and "Sliding Window" are identical (6.519), which indicates limited discriminative capacity. Furthermore, the paper contains no human evaluation.

  4. The novelty of CuLT-DMD and the fairness of the comparisons are questionable. The method is essentially equivalent to "gradient accumulation with frozen parameters." In comparisons with LongLive-style approaches, there are discrepancies in both the number of generator updates and the sample size used per update. The performance gain is marginal (7.131 → 7.260), and the lack of variance data makes it difficult to rule out noise.

  5. The ablation study is incomplete. For HieraKV, only global values ​​of 6, 12, 18, and 54 were tested; the optimal value (6) happens to fall at the very edge of the tested range. Values ​​such as 0 (pure sink + recent) or 3 were not evaluated. Ablation studies for HieraKV were conducted only on the "Base" model, not on the "Distilled" version. The discussion of related work is insufficient. Regarding AnnealCFG, the paper fails to address existing research on timestep-dependent CFG scheduling (e.g., limited-interval guidance). Furthermore, the distinctions between "Window-wise TF" and methods like Rolling Diffusion, Rolling Forcing, and MAGI-1 are not clearly articulated.

  6. Issues regarding inference settings and fairness: The inference strategy—retaining intermediate chunks for each event—relies on prior knowledge of event boundaries. All prompts were rewritten using the authors' own techniques, which may have inadvertently favored their specific model.

  7. The training data is not publicly available. There is a tension between the reported performance of 1.48 FPS and the paper's stated positioning of the model for "streaming/interactive" applications.

Questions for authors
  1. Could you provide a comparison using the same backbone (e.g., by porting components to Wan-1.3B, or by reproducing LongLive/Self-Forcing on HunyuanVideo-1.5) to disentangle the contributions of the proposed method from those of the backbone or the training data?

  2. Could you provide the full Table 1 using the original HeliosBench evaluator (GPT-5.2), including the raw scores for each metric and the variance across multiple seeds?

  3. How does HieraKV perform when global is set to 0 or 3? Is it equally effective when applied to distilled models?

  4. Could the comparison between CuLT-DMD and LongLive-style methods be conducted using the same number of generator updates and the same sample size? Furthermore, if the batch size for the LongLive-style approach is increased, can it achieve comparable results?

  5. How does AnnealCFG compare to other CFG scheduling strategies (such as cosine or interval-based guidance)? Why is it unnecessary for distilled models?

  6. Does the ranking remain consistent if prompt rewriting is omitted, or if the prompt formats recommended by the respective baselines are used instead?

  7. How can the "intermediate chunk per event" retention strategy—used during inference—be implemented in real-world interactive scenarios where event boundaries are unknown?

4 comments

Response to Reviewer ZQ19 (Part 1/3)

Thank you for the detailed review. We address the weaknesses and questions below.

1. Backbone and training-data differences

We have ported MovieRoll to Wan2.1-1.3B, using public pretrained weights and public datasets: OpenVid for supervised fine-tuning and VidProM prompts for distillation. This tests our approach on another architecture without relying on our confidential training corpus.

The Wan model is trained at 480 × 832 resolution and 16 FPS. Window-wise TF uses 98,064 OpenVid clips, 177-frame training sequences, and 10,000 training steps on 64 GPUs. We then perform ODE initialization using 1,000 VidProM prompts, followed by 600 short-rollout and 1,200 long-rollout DMD training steps. HieraKV uses four global layers and 26 local layers.

We select several representative methods below; complete method scores are provided in the main text. MovieRoll-Wan-Distilled achieves the highest 60s overall score among the compared methods except the original MovieRoll-Distilled. This supports the effectiveness of our approach beyond the original backbone and private data. These are same-backbone comparisons with Wan-based methods, rather than comparisons with identical training data and budgets.

MethodDurationAes.Dyn.Smooth.Sem.Nat.D-Aes.D-Smooth.D-Sem.D-Nat.Total
LongLive60s8310549101036.093
Helios-Base60s841066610926.393
Helios-Distilled60s76943110514.128
MemFlow60s8410548101025.925
MovieRoll-Distilled60s7610410610847.032
MovieRoll-Wan-Base30s821056710926.177
MovieRoll-Wan-Distilled60s86967710926.777

All entries except Total are HeliosBench normalized ratings; higher is better for every column. Aes., Dyn., Smooth., Sem., and Nat. denote aesthetics, dynamic degree, motion smoothness, semantic consistency, and naturalness. D-* denotes the corresponding temporal degradation rating.

2. Evaluation with the original GPT-5.2 evaluator

Below we report raw metric scores and the overall HeliosBench score for all 12 methods on the 30s single-event evaluation, using 200 videos per method and the original GPT-5.2 naturalness evaluator.

MethodAes. ↑Dyn. ↑Smooth. ↑Sem. ↑Nat. ↑D-Aes. ↓D-Smooth. ↓D-Sem. ↓D-Nat. ↓Total ↑
CausVid0.61700.15180.98800.24110.14460.04140.00340.02810.14424.776
Self Forcing0.59330.14660.98870.25360.15960.09610.00970.05490.28633.783
Causal Forcing0.59870.28220.98500.25460.25080.05820.00490.02850.24755.076
Rolling Forcing0.62620.05590.99060.25390.32170.03140.00280.01580.15675.964
MemFlow0.63010.20990.99160.25580.33250.02950.00190.00990.12006.450
LongLive0.63800.10210.99240.25470.35750.02220.00140.00830.09506.489
Echo-Infinity0.61200.24780.98770.25570.34130.05400.00320.02100.16135.628
Krea0.59060.74870.98550.25180.11170.10220.00560.05530.18884.191
Helios-Base0.62240.17330.99100.25920.42710.03880.00340.01770.14426.279
Helios-Distilled0.60100.25900.98660.25070.41790.06960.00770.03030.22385.418
MovieRoll-Base0.59840.17000.99150.25460.48250.04520.00350.01560.14006.504
MovieRoll-Distilled0.60080.27390.99500.24870.60960.04180.00130.02420.13087.005

MovieRoll-Distilled also achieves the highest overall score under GPT-5.2, with MovieRoll-Base ranking second. The two evaluators yield strongly correlated naturalness rankings (Spearman’s ρ = 0.930), with both MovieRoll models occupying the top two positions. We select Gemini 2.5 Pro because it provides greater discrimination between methods, as analyzed in Appendix Table 9.

Response to Reviewer ZQ19 (Part 2/3)

3. HieraKV allocation and the missing global = 0 ablation

We have evaluated the all-local configuration, global = 0:

Number of global layers06121854 (all-global)
Total ↑6.8046.8736.7746.6456.321

The results show that retaining full history in every layer is unnecessary and can reduce generation quality. Conversely, using only sink + recent chunks in every layer discards intermediate context. The attention analysis motivates a compromise: preserve full history in shallow layers and compress it in deeper layers.

Our contribution is this finding about historical KV retention, rather than identifying a universally optimal global/local ratio. The six-layer configuration is slightly better than the tested alternatives and substantially better than all-global caching, while reducing KV memory by 78%. The appropriate split may vary across backbones.

4. CuLT-DMD versus gradient accumulation and LongLive-style updates

CuLT uses gradient accumulation within a single long self-rollout. Its distinction lies in which rollout positions contribute to an update and whether the generator changes while that rollout is being produced. Generator parameters remain fixed throughout the rollout, and locally truncated gradients from different temporal positions are averaged before an update. Accumulating gradients from independent short clips does not expose the model to the histories encountered at later positions in a long self-rollout.

LongLive-style training instead updates the generator within the rollout while carrying forward previously generated history. CuLT therefore studies how to use supervision across one long sample under a consistent generator, rather than introducing a new gradient-accumulation operator.

Our comparison examines how different rollout-level schedules make use of supervision from a single long sample. We therefore match initialization, rollout lengths, and optimizer update counts. Under this setting, CuLT obtains 7.260 versus 7.131 for the LongLive-style schedule. Giving LongLive-style training additional optimizer steps would change the update budget and could introduce effects from prolonged optimization under DMD’s mode-seeking objective.

5. CFG scheduling and why distilled inference does not require CFG

Our contribution is to show that reducing CFG toward the end of denoising can improve streaming video generation by limiting guidance-induced errors that enter the generated history and propagate to subsequent chunks.

We add comparisons with increasing CFG and interval schedules that retain high CFG at both ends of denoising:

ScheduleAes.Dyn.Smooth.Sem.Nat.D-Aes.D-Smooth.D-Sem.D-Nat.Total
Constant 474958610916.489
Constant 67595538815.160
Increase 1 → 47210410510816.516
Increase 1 → 67310410610916.744
Interval 4 → 1 → 4741059510916.675
Interval 6 → 1 → 6761058510916.480
Decay 6 → 175957510926.264
Decay 4 → 1741059610926.873

Results are for 30s generation; all normalized ratings are higher-is-better. Interval schedules use high guidance during the first and last 20% of denoising, with CFG = 1 during the middle 60%.

Constant, increasing, and high-at-both-ends interval schedules retain strong guidance near the clean endpoint and show less stable long-horizon behavior. Decaying CFG from 4 to 1 performs better than these selected alternatives. We do not claim a globally optimal schedule: the schedule space is large, and suitable schedules may differ across backbones. Our main finding is the benefit of weakening late-stage guidance; linear decay is a simple implementation of this principle.

For distilled inference, guided behavior is learned from the teacher trajectories during ODE initialization and from the CFG-guided real-score teacher during DMD. The student therefore uses only the conditional branch at inference. The DMD teacher still uses CFG.

Response to Reviewer ZQ19 (Part 3/3)

6. Prompt rewriting and evaluation fairness

All methods receive the same rewritten prompts; rewriting is not applied exclusively to MovieRoll. Prompt expansion is also used in prior evaluation practice—for example, Self Forcing rewrites MovieGen prompts to better elicit model capabilities.

Our comparisons therefore measure performance under a common expanded-prompt protocol. We have not separately evaluated whether every ranking remains unchanged without rewriting or with method-specific prompt formats.

7. Event boundaries in online interaction

Event boundaries are determined by the timestamps at which the user changes the prompt. When a new prompt arrives, the interval since the previous prompt update becomes a completed event, and its intermediate chunk can be selected from the already generated history. The current event remains active until the next prompt update. This requires no advance knowledge of when the user will switch events.

8. Window-wise TF versus prior rolling-denoising methods

Window-wise TF extends rolling denoising to supervised streaming fine-tuning of a multi-step base model. Rolling Forcing primarily uses rolling denoising for few-step distillation. Rolling Diffusion couples historical and active content through bidirectional attention, preventing reuse of a fixed historical KV cache. MAGI-1 uses causal interactions across chunks, including within the active window.

Window-wise TF instead combines bidirectional attention within the active window with causal access to clean history. During training, noisy windows are isolated so that multiple windows can be supervised in parallel while preserving the intended history dependencies. These choices enable cached streaming inference and parallel window supervision during base-model training. Appendix D, Table 11 supports the resulting long-horizon benefit: at 721 frames, Window-wise TF scores 6.321 versus 5.001 for chunk-wise TF.

9. Human evaluation

We have conducted a human preference study with 59 participants, 350 video pairs, and 2,616 judgments. Each pair contains videos generated from the same prompt, and participants choose their preferred video or a tie. The study covers 10s, 30s, and 60s single-event videos and 30s and 60s multi-event videos.

The table reports MovieRoll-Distilled’s preference win rates, aggregated across the evaluated durations. Ties remain in the denominator.

Compared methodSingle-event win rateMulti-event win rate
CausVid90.3%—
Self Forcing81.9%—
Causal Forcing78.3%—
Rolling Forcing72.8%—
MemFlow72.6%62.3%
LongLive68.4%66.7%
Echo-Infinity77.2%64.3%
Krea73.0%—
Helios-Base67.0%67.1%
Helios-Distilled73.0%81.9%

These preferences provide human evidence for the quality improvements observed in the automated evaluation.

10. Streaming capability and inference speed

The reported 1.48 FPS on a single H20 GPU reflects the computational cost of our larger 8.3B backbone and full-history KV retention in shallow layers. Our focus is to establish stable long-horizon streaming generation that supports prompt changes during generation. Real-time interactive deployment additionally requires sufficient inference throughput, which can be pursued through more powerful accelerators or parallel inference across more GPUs.

11. Score resolution, identical Table 10 scores, and random seeds

The identical scores of 6.519 for Full History and Sliding Window in Table 10 are not typographical errors. Applying HeliosBench’s normalization and weighted aggregation yields the same total for both methods. Both are outperformed by KV recomputation (6.873) under the same model checkpoint.

Dear Reviewer ZQ19,

We would like to kindly follow up on our responses and ask whether the additional experiments and clarifications have resolved your main concerns. If so, we would appreciate it if you could consider raising your score. Please let us know if any questions remain.

Thank you again for your time and consideration.

Reviewer_v5LJOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MovieRoll, a framework that first trains a multi-step streaming video model and then distills it into a few-step generator. Window-wise Teacher Forcing jointly denoises chunks at progressive noise levels. HieraKV retains the available history in shallow layers and only sink and recent history in deeper layers. CuLT-DMD holds the generator fixed during a long self-rollout and averages locally computed gradients before updating it. Cache recomputation and guidance annealing provide additional inference-time stabilization. On the authors' HeliosBench evaluation, MovieRoll-Distilled achieves the highest reported aggregate scores among the compared methods for single- and multi-event generation at 30 and 60 seconds.

Critical strengths
  1. Simple and effective techniques for improving long-video quality. The proposed training and cache-management techniques are straightforward and improve reported long-video generation quality.
Critical weaknesses
  1. The technical novelty appears limited. Window-wise Teacher Forcing closely resembles existing rolling-diffusion approaches, while HieraKV is a straightforward layer-dependent KV-cache reduction strategy. CuLT-DMD primarily changes the student model's update schedule by accumulating gradients across local clips before an optimizer step. However, the paper does not vary batch size or compare against controls with matched effective batch sizes, leaving it unclear whether the benefit comes from the proposed rollout-level scheme or simply from gradient accumulation and less frequent updates. The three components address largely separate aspects of the pipeline, and the work currently reads as a collection of incremental engineering refinements to existing streaming video generation methods.

  2. Different backbones make the external comparisons insufficiently controlled. MovieRoll uses an 8.3B HunyuanVideo backbone, whereas several baselines use 1.3B Wan backbones, and Helios is also evaluated with a 14B model and generation configuration. Differences in model capacity, pretraining, and training data can affect quality independently of the proposed techniques. Matched-backbone experiments are needed to support a fairer assessment. Since the proposed methods can be easily implemented on WAN.

Questions for authors
  1. What distinguishes Window-wise Teacher Forcing from existing rolling-diffusion approaches and HieraKV from straightforward layer-wise cache reduction? For CuLT-DMD, could the authors vary batch size and update frequency independently, with effective batch size and training compute controlled, to establish the benefit of the proposed rollout-level update scheme?

  2. Could the authors evaluate the proposed components on the same Wan backbone used by the relevant baselines, or provide equivalent matched-backbone controls?

3 comments

Response to Reviewer v5LJ (Part 1/2)

We thank the reviewer for the constructive comments. We clarify the methodological contributions and provide additional Wan-backbone results below.

1. Overall novelty and contribution

MovieRoll studies how to establish a capable multi-step streaming base and use it to train a few-step generator that remains stable beyond the supervised horizon. The base supplies teacher trajectories and student initialization, while the rolling attention and history-retention structure carries through distillation and inference. Our contributions concern how this structure is trained, which historical information each layer retains, and how supervision is aggregated over long self-rollouts.

2. Window-wise Teacher Forcing

WinTF extends rolling-window denoising to supervised streaming fine-tuning. Rolling Forcing primarily targets few-step distillation, while Rolling Diffusion couples history and the active window through bidirectional attention, preventing historical KV reuse. MAGI-1 supports caching but imposes chunk-wise causality within the active window. WinTF combines bidirectional refinement within each window with causal access to clean history, enabling both KV caching and parallel window supervision. Under the same backbone and training data, it improves the 721-frame score from 5.001 to 6.321 over chunk-wise teacher forcing (Appendix D, Table 11).

3. HieraKV

HieraKV complements temporal history compression in streaming video by examining which layers require intermediate historical context. Keeping full history everywhere is costly and does not necessarily improve generation; keeping only sink–recent history everywhere removes intermediate information. Our attention analysis (Figure 3) reveals broader historical attention in shallow layers and a stronger sink–recent preference in deeper layers, motivating a structured retention policy across depth. This configuration scores 6.873, compared with 6.321 for all-global retention and 6.804 in an additional all-local ablation, while reducing KV-cache memory by 78%. Our contribution is identifying and validating this tradeoff: intermediate history can remain accessible through a subset of layers without storing it throughout the network. We do not claim a universally optimal global–local allocation.

4. CuLT-DMD and gradient accumulation

CuLT can indeed be viewed as gradient accumulation applied within a single long self-rollout. The distinction lies in the supervision being accumulated: independent short clips do not provide the self-generated histories encountered at later positions of a long rollout. CuLT averages local gradients across those positions under fixed parameters, whereas LongLive-style training updates within the rollout while carrying history generated under earlier parameter states forward.

Our comparison examines how different rollout-level schedules make use of supervision from a single long sample. We therefore match initialization, rollout lengths, and optimizer update counts, obtaining 7.260 versus 7.131 (Table 2; Appendix A). We have not independently swept batch size and update frequency, as the comparison focuses on organizing rollout supervision under the same update budget. Giving LongLive-style training additional optimizer steps would change this budget and could introduce effects from prolonged optimization under DMD’s mode-seeking objective.

Response to Reviewer v5LJ (Part 2/2)

5. Wan-backbone evaluation

We additionally implemented MovieRoll on Wan2.1-T2V-1.3B, providing a comparison with the backbone used by LongLive and MemFlow.

The Wan model uses 98,064 public OpenVid clips for WinTF and VidProM prompts for distillation, including 1,000 prompts for ODE initialization. Training uses 64 GPUs, sequence-parallel degree 2, global video batch size 32, BF16, and AdamW. The generator and critic are 1.3B models, while the frozen real-score teacher is Wan2.1-14B. This controls the generator backbone, although training data and budgets differ across independently trained methods.

On the 60-second single-event evaluation, MovieRoll-Wan-Distilled achieves 6.777, the highest score among the compared methods apart from the original MovieRoll-Distilled (7.032). This supports the effectiveness of our framework beyond HunyuanVideo, using a smaller backbone and public training data. Representative methods are shown below; complete baseline results are reported in the paper.

MethodDurationAes.Dyn.Smooth.Sem.Nat.D-Aes.D-Smooth.D-Sem.D-Nat.Total
LongLive60s8310549101036.093
Helios-Base60s841066610926.393
Helios-Distilled60s76943110514.128
MemFlow60s8410548101025.925
MovieRoll-Distilled60s7610410610847.032
MovieRoll-Wan-Base30s821056710926.177
MovieRoll-Wan-Distilled60s86967710926.777

All ratings are higher-is-better, including drift ratings. Each evaluation uses 200 single-event prompts. MovieRoll-Wan, LongLive, and MemFlow use 1.3B backbones; Helios uses 14B; the original MovieRoll uses 8.3B. Wan outputs are encoded at 24 FPS without interpolation.

Dear Reviewer v5LJ,

We would like to kindly follow up and ask whether our responses and additional results have addressed your main concerns about MovieRoll. If so, would you consider revising your score upward? We would be happy to clarify any remaining questions.

Thank you again for your time and consideration.

Comments on the paper

2 comments

Summary of Our Responses (Part 1/2)

We summarize the evidence and clarifications provided during rebuttal. MovieRoll addresses stable long-horizon streaming generation through a multi-step streaming base, few-step distillation, and structured history retention.

1. Methodological contributions and relation to prior work

Window-wise TF extends rolling denoising to supervised streaming fine-tuning. Rolling Forcing primarily targets few-step distillation; Rolling Diffusion couples history and the active window bidirectionally, preventing fixed historical KV reuse; MAGI-1 imposes chunk-wise causality within the active window. WinTF combines bidirectional window refinement with causal clean-history access and isolated noisy training windows, enabling KV reuse and parallel window supervision. Under the same backbone and training data, its 721-frame score is 6.321 versus 5.001 for chunk-wise TF.

HieraKV establishes that full historical KV is unnecessary at every layer. It complements temporal compression by retaining full available history in shallow layers and sink + recent history in deeper layers, guided by video-attention analysis. Compared with PyramidKV, we clarified the setting and retention mechanism: structured video-chunk retention for streaming generation versus attention-based LLM token selection and layer budgets. We do not claim that depth-dependent cache allocation itself is new. HieraKV improves quality while reducing KV memory by 78%.

CuLT-DMD organizes supervision across one long self-rollout under fixed generator parameters. It averages locally truncated gradients across temporal positions before updating; LongLive-style training updates within the rollout while carrying forward previously generated history. Independent short-clip accumulation does not expose the generator to late-rollout histories. With matched initialization, rollout lengths, and optimizer update counts:

Update scheduleTotal ↑
LongLive-style7.131
CuLT-DMD7.260

The contribution concerns rollout-level supervision, not a new gradient-accumulation operator. We have not independently isolated batch-size and update-frequency effects. Additional LongLive-style updates would change the update budget and could introduce effects from prolonged optimization under DMD's mode-seeking objective.

2. Public-data Wan experiments

We ported MovieRoll to public Wan2.1-T2V-1.3B weights, using OpenVid for supervised fine-tuning and VidProM prompts for distillation. WinTF uses 98,064 clips, 177-frame sequences, and 10,000 steps on 64 GPUs. Distillation uses 1,000 prompts for ODE initialization, followed by 600 short-rollout and 1,200 long-rollout DMD steps. The generator and critic are 1.3B; the frozen real-score teacher is Wan2.1-14B.

MethodDurationHeliosBench Total ↑
LongLive60s6.093
Helios-Base60s6.393
Helios-Distilled60s4.128
MemFlow60s5.925
Original MovieRoll-Distilled60s7.032
MovieRoll-Wan-Base30s6.177
MovieRoll-Wan-Distilled60s6.777

Among the compared methods, MovieRoll-Wan-Distilled achieves the highest 60s score except the original MovieRoll-Distilled. This supports effectiveness beyond HunyuanVideo and our confidential corpus. It controls the generator backbone against Wan-based baselines, not training data or budgets. Wan outputs follow the reported benchmark convention of encoding at 24 FPS without interpolation.

3. Additional HieraKV validation

The Base-model allocation results are:

Global layers06121854
Total ↑6.8046.8736.7746.6456.321

The finding is that not all layers require complete history, rather than a universally optimal split. HieraKV is used in Base and Distilled; we did not report global = 3 or a separate Distilled allocation sweep.

On an all-global, 30-layer Wan model, we analyzed only the ten cached historical chunks before the first recomputation, excluding the entire active window. Averaging over four prompts and ten denoising passes gives:

Layer groupNormalized history entropySink + recent coverage
First 4 layers0.98322.5%
Remaining 26 layers0.95230.7%

This supports the same directional pattern, without claiming monotonicity at every layer. The Wan port uses four global and 26 local layers. Its generation results evaluate the complete framework, not an isolated Wan HieraKV ablation.

Summary of Our Responses (Part 2/2)

4. Evaluator comparison and human evidence

We reported raw metrics for 12 methods, with 200 videos per method, in the 30s single-event setting, using the original GPT-5.2 naturalness evaluator.

MethodOverall score with GPT-5.2 ↑
LongLive6.489
MovieRoll-Base6.504
MovieRoll-Distilled7.005

MovieRoll-Distilled remains first overall and MovieRoll-Base second in this evaluator comparison. GPT-5.2 and Gemini produce strongly correlated naturalness method rankings, with Spearman's rho = 0.930. Gemini 2.5 Pro was selected for its greater discrimination, analyzed in Appendix Table 9. This comparison does not constitute a GPT-5.2 reevaluation of every duration and multi-event setting.

We also supplied a human preference study with 59 participants, 350 video pairs, and 2,616 judgments, covering 10s/30s/60s single-event and 30s/60s multi-event videos. MovieRoll-Distilled is preferred over every evaluated baseline:

Compared methodSingle-event win rateMulti-event win rate
CausVid90.3%—
Self Forcing81.9%—
Causal Forcing78.3%—
Rolling Forcing72.8%—
MemFlow72.6%62.3%
LongLive68.4%66.7%
Echo-Infinity77.2%64.3%
Krea73.0%—
Helios-Base67.0%67.1%
Helios-Distilled73.0%81.9%

Win rates aggregate judgments across evaluated durations; ties remain in the denominator.

5. CFG scheduling and distilled inference

We clarified that the contribution is reducing guidance-induced deviations near the clean endpoint, where they can enter generated history and propagate to later chunks. Additional comparisons include constant, increasing, and high-at-both-ends interval schedules:

Schedule30s Total ↑
Constant 46.489
Constant 65.160
Increase 1 → 46.516
Increase 1 → 66.744
Interval 4 → 1 → 46.675
Interval 6 → 1 → 66.480
Decay 6 → 16.264
Decay 4 → 16.873

The intervals use high CFG in the first/last 20% of denoising and CFG = 1 in the middle 60%. Decay 4 → 1 outperforms these selected alternatives; we do not claim a globally optimal schedule. The contribution is the benefit of weakening late-stage guidance, with suitable schedules potentially varying across models.

The distilled student learns guided behavior through ODE initialization and DMD, so inference uses only the conditional branch. The DMD real-score teacher still uses CFG.

6. Evaluation fairness and online cache handling

All methods use the same rewritten prompts, rather than rewriting only MovieRoll's inputs. Prompt expansion is also used in prior evaluation practice, including Self Forcing. We have not established ranking invariance under unrevised or method-specific prompt formats.

User prompt-update timestamps define event boundaries online. Once a new prompt arrives, the preceding interval is a completed event whose intermediate chunk can be selected; future switch times need not be known.

The identical 6.519 scores for Full History and Sliding Window in Table 10 are not typographical errors: HeliosBench normalization and aggregation produce the same total. Both are below KV recomputation (6.873) under the same checkpoint. We do not claim repeated-generation-seed variance or statistical significance.

7. Streaming capability and measured prompt-switch latency

The reported 1.48 FPS on one H20 reflects the larger 8.3B backbone and full-history retention in shallow layers. Our focus is stable streaming output with prompt updates. Real-time deployment requires greater inference throughput, which can be pursued with stronger accelerators or more GPUs.

MovieRoll retains its active window at a prompt switch. New conditioning first affects output after one round; a chunk denoised entirely under it requires W = 5 rounds. We measured the same six-event prompt with five switches on one H20, batch size 1:

MethodBackboneFirst output using new promptFirst output fully denoised under new prompt
MovieRoll-DistilledHunyuanVideo, 8.3B8.72–9.65s42.53s
MovieRoll-Wan-DistilledWan2.1, 1.3B2.04s10.25s
LongLiveWan2.1, 1.3B2.27s2.27s

The Wan comparison matches resolution (480 × 832) and chunk size (3 latent frames). LongLive's 2.27s includes recaching and denoising; including its subsequent KV update gives 2.57s. These are conditioning/computation timings, excluding VAE decoding and residual waiting for an ongoing round, not measured visual semantic response times. We acknowledge the higher full-switch latency.

We thank the reviewers for their feedback. We hope these results and clarifications help readers assess MovieRoll's methodological and empirical contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.