acceptodds
Under review as a conference paper at ICLR 2027

Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation

Abstract

Streaming video generation (SVG) distills a pretrained bidirectional video diffusion model into an autoregressive model equipped with sliding window attention (SWA). However, SWA inevitably loses distant history during long video generation, and its computational overhead remains a critical challenge to real-time deployment. In this work, we propose Hybrid Forcing, which jointly optimizes temporal information retention and computational efficiency through a hybrid attention design. First, we introduce lightweight linear temporal attention to preserve long-range dependencies beyond the sliding window. In particular, we maintain a compact key–value state to incrementally absorb evicted tokens, retaining temporal context with negligible memory and computational overhead. Second, we incorporate block-sparse attention into the local sliding window to reduce redundant computation within short-range modeling, reallocating computational capacity toward more critical dependencies. Finally, we introduce a decoupled distillation strategy tailored to the hybrid attention design. A few-step initial distillation is performed under dense attention, then the distillation of our proposed linear temporal and block-sparse attention is activated for streaming modeling, ensuring stable optimization. Extensive experiments on both short- and long-form video generation benchmarks demonstrate that Hybrid Forcing consistently achieves state-of-the-art performance. Notably, our model achieves real-time, long video generation at 29.5 FPS on a single NVIDIA H100 GPU without quantization or model compression. The source code and trained models will be released.

Primary area:
generative models
Keywords:
stream video generation, hybrid attentioin

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iaauOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Hybrid Forcing combines a persistent linear history state with sparse sliding-window attention for streaming video generation. A staged distillation procedure, bounded relative temporal RoPE, and teacher-guided regularization support stable generation. The method reports improved quality and throughput on short and 30-second video benchmarks.

Critical strengths
  1. The architecture directly addresses the complementary requirements of efficient local processing and retention of evicted history.

  2. Automatic evaluation is supplemented by a human preference study and qualitative longer-duration examples.

Critical weaknesses
  1. The strongest long-horizon claims exceed the quantitative evidence. The primary quantitative evaluation covers 30-second videos, whereas ten-minute stability is supported mainly by examples. Fixed-suite evaluations of segment-wise quality, identity retention, repetition, and failure frequency would better substantiate the broader claim.
  2. History-state usefulness does not fully establish detailed long-term recall. The interventions show that historical information affects generation, but do not determine whether the state preserves specific earlier events or mainly stabilizes aggregate appearance and motion. Controlled object-reappearance or delayed-attribute-recall tasks would be informative.
  3. The user-study statistics contain an inconsistency. The appendix describes complete rankings of seven methods. Under the standard 1–7 ranking convention, the seven average ranks must sum to 28. However, Table 5 sums to 29.00, which cannot be explained by rounding. The ranking convention and aggregation procedure should be clarified, or the table corrected.
  4. The throughput measurement boundary needs clarification. It should be explicit whether the reported 29.5 FPS includes VAE decoding, text encoding, and cache reconstruction. First-chunk latency should also be reported separately from sustained throughput.
Questions for authors
  1. Responses to the weaknesses.

  2. And also:

  • Can the authors provide anonymized ranking data and explain the average-rank inconsistency?
  • How do quality and consistency evolve beyond 30 seconds on a fixed evaluation suite?
  • What are the complete measurement boundaries and first-chunk latency for the throughput result?
2 comments
Reviewer_iaauOpenReview ↗

Responses to rebuttal

Thank the authors for the rebuttal.

There are substantive responsies and candidaly acknowledge the limiations of the long-term recall.

However, the statistical issues in user study remain unresolved, and the conclusion regarding long videos and real-time performance also need to be narrowed down.

Based on the rebuttal, the reviewer tend to keep weak accept atm.

Thank you for recognizing our substantive responses and our candid discussion of long-term recall. We have narrowed the conclusions about long videos and DiT generation throughput on an H100. The revised text specifies that the reported 29.5 FPS includes clean KV recaching and excludes text encoding, VAE decoding, and video encoding/writing. Thank you for helping us make the scope of these conclusions more precise. We have also explicitly acknowledged this limitation in the Limitations section, noting the constraints of the base model’s capabilities and the potential need for fine-tuning on high-quality exploration-and-return data.

Given the limited time remaining and the visual similarity of some generated videos, we cannot conduct a complete new user study during the rebuttal period. We have clarified in the manuscript that some visually similar outputs received tied ranks, which our original description of complete rankings did not make explicit. And we will conduct a more comprehensive user study with an explicit protocol for handling tied rankings and report the updated results in the final version.

Reviewer_sV4zOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Hybrid Forcing combines sparse attention over recent frames with a recurrent linear state for older history. Starting from a Self-Forcing checkpoint, it first trains with dense DMD and then switches to hybrid attention, with relative RoPE and teacher regularization. The paper reports improved 30-second video quality at 29.5 FPS on a single H100 GPU.

Critical strengths
  1. The speed–quality trade-off is useful. The method improves the reported long-video score while increasing throughput by roughly 35% over the fastest compared baseline.
  2. The training schedule is worth investigating. Dense-to-hybrid training gives a clear improvement in Table 3. This is a more convincing contribution than the claim of a new attention architecture.
  3. The ablations go beyond simply removing modules. Freezing, resetting, and shuffling the historical state provide useful evidence that the model actually uses its stored history.
Critical weaknesses
  1. The contribution relative to prior work is not sufficiently isolated. ReHyAt already combines local attention with recurrent historical compression, while SLA provides closely related sparse selection and additive fusion. These precedents do not establish the same few-step streaming pipeline, so the adaptation has value. However, stronger controls—particularly sparse-only attention under the complete training recipe—are needed to show what this specific combination adds.
  2. The long-horizon claims are stronger than the evidence. Quantitative evaluation stops at 30 seconds. The ten-minute examples are encouraging, but they do not establish reliable memory over that duration. I would like to see whether an identity or attribute is preserved after leaving the local window.
  3. The human-study numbers need checking. The seven average ranks in Table 5 sum to 29, although complete rankings from 1 to 7 should sum to 28. This may be a reporting error, but it needs to be resolved before the preference results can support the paper’s conclusions.
  4. The efficiency measurements need a clearer protocol. It is unclear whether the reported throughput includes KV recaching and VAE decoding. First-frame and per-chunk latency would also help assess whether the method is suitable for interactive applications.
Questions for authors
  1. How does sparse-only attention perform with the same positional handling, regularization, and staged training as the final model?
  2. Can the authors provide a longer quantitative evaluation or a targeted test of information retained beyond the local window?
  3. What explains the average-rank discrepancy in Table 5? Were ties or weighting used?
  4. As the sliding window advances, how is positional information handled in the compressed historical state?
  5. What motivates independently normalizing and adding the two attention branches rather than using ReHyAt-style joint normalization? Have these alternatives been compared under the same streaming and training setup?
Reviewer_1YNVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studys streaming video generation, where a bidirectional video diffusion model is distilled into a few-step autoregressive generator that uses sliding-window attention. The proposed method, Hybrid Forcing, splits attention into two parts. Frames evicted from the KV cache are folded into a fixed-size linear-attention state, so older history is kept at little cost. Block-sparse attention is used for speed inside the smaller local window. Training is done in two stages: ordinary DMD with dense attention first, then the hybrid attention is switched on. The authors also cap the temporal RoPE index during both training and inference, and add a loss that ties the student’s first denoising step to the teacher’s output, to limit drift and mode collapse in long rollouts. Experiments on VBench and VBench-Long show better scores than recent streaming video generation baselines and 29.5 FPS on a H100 GPU.

Critical strengths
  1. Speed and long-horizon quality improve together. Because the linear history state keeps older content, the window can shrink, and sparse local attention makes the remaining window cheaper.

  2. Clear evidence that the history state is used. Freezing, zeroing or shuffling the linear state lowers quality at inference, and shuffling hurts the most. This shows the model actually reads the history content, which supports the paper’s claim.

Critical weaknesses
  1. In Table 3, hybrid attention alone reaches an evaluation score of 81.56, which is below Rolling Forcing (82.66) and InfRoPE (82.18) in Table 2. The jump to 83.49 only comes after adding relative RoPE, the regularization loss and decoupled distillation, and the baselines may also benefit from these generic changes. This weakens the claim that the architecture is what leads to the SOTA results.

  2. Linear attention for history, block-sparse attention with top-K block selection, a capped RoPE index and teacher-rollout regularization each closely follow existing work (e.g., SLA, SANA-Video, VSA, InfRoPE). The paper’s contribution is mainly in combining them, so the technical novelty seems to be modest.

Questions for authors
  1. If relative RoPE, the regularization loss and the two-stage schedule were applied to a baseline such as Self-Forcing or Rolling Forcing with plain sliding-window attention, how would their performance change?

  2. Were all baselines trained with the same steps, data and prompt augmentation, or are their numbers taken from released checkpoints?

  3. How does the linear history branch compare with simply using a longer dense window (e.g., 21 frames) at a matched compute budget?

  4. Where is the main speedup coming from: the shorter window (21 → 9), the sparse attention, or both?

  5. Beyond VBench-Long, is there quantitative evidence of quality or drift for videos longer than 30 seconds?

4 comments

Thank the authors for the detailed response. My concerns about long-horizon evidence and novelty have been addressed, but the authors still haven’t applied the full training recipe (relative RoPE, regularization, decoupled distillation) to a plain sliding-window baseline, and the dense 21-frame control without history actually scores slightly higher than the 9-frame with history (81.71 vs 81.43). I remain unsure how much of the final gain comes from the architecture rather than training, thus I lean to keep my original score.

We apologize for omitting a relevant supplementary control from our earlier response. We also evaluated the plain dense 21-frame baseline with relative RoPE and regression regularization:

ConfigurationFPSTotalQualitySemanticDynamicConsistency
Dense window 9 with linear history (base)22.381.4382.5876.856625.65
Dense window 21, no history15.481.7182.9076.955925.95
Dense window 21 + relative RoPE + regression15.482.4083.6577.426525.98
Sparse window 9 + linear history + relative RoPE + regression29.582.3883.8776.416525.68
Our full pipeline, including decoupled distillation29.583.4984.6778.787426.02

RoPE and regularization improve the dense baseline’s Total from 81.71 to 82.40, confirming that these components also benefit plain sliding-window attention. With the same components, our hybrid model achieves a comparable Total of 82.38, with higher Quality but lower Semantic and Consistency scores. Our decoupled schedule further raises Total to 83.49. The base 21-frame control’s slight advantage over the 9-frame model with history should also be considered alongside inference speed: our reported H100 throughput is 22.3 FPS for the dense 9-frame model with history and 29.5 FPS for our final sparse-plus-linear model, compared with 15.4 FPS for the dense 21-frame baseline. We also consider latency a critical factor in streaming applications, as it directly affects responsiveness and the model’s practical usability.

Specifically, decoupled distillation separates dense-attention representation learning from the subsequent activation of sparse attention and linear history compression. The plain dense 21-frame baseline contains neither module and therefore has no corresponding dense-to-hybrid transition.

Following your suggestion, we have revised the manuscript to attribute our final performance to the combined contributions of the attention architecture and training recipe. The architecture improves the quality–speed trade-off, while the training procedure further improves generation quality.

Thanks for the additional control, my main concern is resolved and I will raise my score accordingly.

Thank you for reviewing the additional control and reconsidering your assessment. Your constructive feedback helped us clarify the contributions of the architecture and training pipeline and strengthen the paper. We appreciate your time and thoughtful engagement throughout the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.