acceptodds
Under review as a conference paper at ICLR 2027

Residual Forcing: Structure-Aware Sparse Attention for Autoregressive Video Diffusion

Abstract

Autoregressive video diffusion enables streaming long-video generation, but self-attention over cached visual history remains a major inference bottleneck. Training-free block-sparse methods can flexibly select key blocks for each query block, yet routing based on inexpensive mean-pooled scores can miss concentrated interactions amplified by softmax. These misses degrade generation quality, while compensating with more key blocks erodes the efficiency gains; independently changing the selected context across chunks can also introduce periodic visual discontinuities. In this paper, we show that many of the missed high-response interactions follow identifiable attention structures that can be protected at low cost. We therefore introduce , a training-free framework that separates structure preservation from residual attention selection. It protects spatially aligned key blocks, their axial halo tokens, and shared salient keys, then routes only the residual key space using reusable head-level priors and block-adaptive budgets. To improve temporal continuity, it shares candidate-frame choices across chunk boundaries without increasing the frame budget. Fused routing and dynamic sparse-attention kernels efficiently execute the resulting heterogeneous routes. On LongLive, Residual Forcing achieves a self-attention speedup in fixed-cache profiling and a end-to-end speedup for 30-second generation, while maintaining competitive video quality and mitigating chunk-boundary discontinuities.

Primary area:
generative models
Keywords:
autoregressive long video generation, efficient inference, block-sparse attention, KV cache

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VPxMOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

Residual Forcing is a training-free sparse attention framework for autoregressive video diffusion. It protects structured blocks accounting for ~96% of attention mass missed by mean-pooled routing, adaptively routes the remaining keys, and shares context across chunks to improve continuity. On LongLive (Wan2.1-1.3B), it achieves 4.12× self-attention speedup under fixed-cache profiling and 1.25× end-to-end speedup at 30 seconds, with the best quality and boundary continuity among evaluated sparse baselines, measured using the proposed PM-BLEx metric.

Critical strengths
  1. The paper is easy to read and follow.

  2. Well-supported motivation. The exponential response-gap bound and evidence that missed attention mass is highly structured provide a solid basis for the design. The decomposition into stable head-level and dynamic query-level budgets is particularly insightful.

  3. Transparent efficiency reporting. Table 2 includes routing overhead and useful controls—Dense FA2-6 and route-refresh frequency—showing when routing costs outweigh sparsity gains.

  4. Reasonable baseline coverage. Four sparse methods are evaluated on the same generator at matched token budgets across three video durations.

Critical weaknesses
  1. Modest practical gain. Dense FA2-6 already achieves 1.17× end-to-end speedup versus 1.25× for the full method—only ~7% additional gain. The headline 4.12× applies only to self-attention under fixed-cache profiling.

  2. Narrow evaluation. All experiments use LongLive/Wan2.1-1.3B at 480p, leaving high-resolution performance and cross-model calibration transfer untested. At 5 seconds, the method also trails sparse baselines on VBench Total.

  3. Missing relevant baselines. Baslines like FAST-AR and Light Interaction are cited as closely related but not compared experimentally.

  4. Unvalidated continuity metric. The central continuity claim relies on the authors’ PM-BLEx metric, without human or independent perceptual validation.

Questions for authors

My major concerns are the modest practicle gains and generalization for other backbones.

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer VPxM (Part 1/2)

Thank you for the thoughtful feedback. We respond point-by-point below.

W1: Limited Practical Gains over Dense FA2-6

The limited additional speedup of our method over FA2-6 at 480p is mainly due to the relatively small scale of self-attention computation, where the gains in attention execution are partially offset by routing and other overheads. We therefore extended the same method to higher-resolution experiments at 720p, with the results shown below:

ConfigurationControl / secondsResidual Forcing / secondsEnd-to-end speedup
480p, full Dense FA272.7258.411.245×
480p, Dense FA2-662.3058.411.067×
720p, full Dense FA2210.50130.321.615×
720p, Dense FA2-6171.11133.341.283×

The results show that at 720p, Residual Forcing achieves end-to-end inference speedups of 1.615× over full Dense FA2 and 1.283× over FA2-6 with six candidate frames. Compared with the 1.067× speedup over FA2-6 at 480p, the high-resolution experiments demonstrate more pronounced practical acceleration gains.

W2: High Resolution, Cross-Backbone Calibration Transfer

1. High-Resolution Evaluation

In addition to the timing results above, we completed quality evaluations of all five methods on the full set of 128 30-second videos at 720p.

MethodKeqK_{\mathrm{eq}}Quality ↑PM-BLEx ↓DiT speedup ↑
Dense650.3081.500.3931.00×
Light Forcing68.2380.780.6301.81×
DFSAttn75.0681.411.0671.64×
SpargeAttention71.1481.360.8861.60×
Residual Forcing63.6081.520.3172.04×

The results show that at 720p, Residual Forcing achieves the highest Quality, lowest PM-BLEx, and greatest DiT speedup with the smallest effective attention support, demonstrating its ability to balance generation quality, boundary continuity, and computational efficiency in high-resolution settings.

2. Cross-Model Threshold Transfer

We completed bidirectional transfer of frozen thresholds between LongLive and Self-Forcing, using all 946 prompts for 5-second generation in both directions. The two models have the same architecture, both with 30 layers × 12 heads. Self-Forcing is calibrated using 5-second trajectories. The results are shown below:

Target modelCalibration sourceQuality ↑Total ↑PM-BLEx ↓
LongLiveLongLive83.1782.660.569
LongLiveSelf-Forcing82.7982.390.560
Self-ForcingSelf-Forcing84.0483.470.246
Self-ForcingLongLive84.3483.750.266

The results show that using LongLive's calibration improves performance on Self-Forcing, whereas transferring Self-Forcing's calibration to LongLive performs worse than LongLive's original performance. Overall, performance remains comparable when calibration thresholds are transferred between the two backbones.

W3: Missing Comparisons with Other Baselines

Since FAST-AR has not yet been open-sourced, we additionally adapted and evaluated Light Interaction. We integrated only its 3D sparse attention module, adjusted the temporal chunk size from 4 frames to 3 frames as in LongLive, used the same candidate-frame allocation as DFSAttn and SpargeAttention, and retained its Top-17.5% selection of historical blocks along with full protection of the current chunk. We used 128 MovieGen prompts for a complete 30-second quality evaluation, with the results shown below:

MethodKeqK_{\mathrm{eq}}Total ↑PM-BLEx ↓DiT speedup ↑
Light Interaction90.5579.640.9391.28×
Residual Forcing29.7679.310.2841.49×

The results show that Light Interaction has a higher Total score than Residual Forcing. This is mainly because its original method requires the current chunk's KV to participate fully in attention computation, resulting in 3 times as much effective KV participating in attention as in Residual Forcing. In contrast, Residual Forcing achieves better boundary continuity and practical acceleration with less cached KV.

AuthorsOpenReview ↗

Author Response to Reviewer VPxM (Part 2/2)

W4: PM-BLEx Lacks Human or Independent Perceptual Validation

We added human perceptual validation involving three evaluators. All three agreed that Residual Forcing, with Cross-Chunk Context Sharing, can effectively mitigate visual jumps at boundaries between adjacent chunks in autoregressive video generation, making cross-chunk transitions more continuous and natural. We also updated the supplementary material to include generation results from both LongLive 2.0 and the original LongLive, presenting 12 videos in total, with 6 each for Light Forcing and Residual Forcing. Please refer to the supplementary videos for a direct visual comparison of the two methods at chunk boundaries.

Q1: Practical Gains and Generalization to Other Backbones

We added an evaluation on LongLive 2.0 5B. Quality results cover 128 30-second videos. Dense retains the native 32-frame KV context (8 sink + 16 recent + 8 current frames), while sparse methods use 12 candidate frames within the same physical cache and retain their respective scoring and block-selection rules. The results are shown below:

MethodKeqK_{\mathrm{eq}}Quality ↑Total ↑PM-BLEx ↓DiT speedup ↑
Dense411.3082.5579.310.1741.00×
Light Forcing44.8482.0679.461.8821.52×
DFSAttn38.3681.9879.785.6151.36×
SpargeAttention36.3581.2878.872.5651.39×
Residual Forcing40.0982.6679.921.4801.55×

The results show that Residual Forcing achieves the highest Quality, Total, and DiT speedup on this backbone, as well as the lowest PM-BLEx among the sparse methods, further supporting its effectiveness across generative backbones. For additional evidence on practical gains, please refer to W1 & W2.

Thank you for the response and additional experiments. The 720p and 5B results address my main concerns about practical gains and generalization. Some limitations remain, including modest gains over sparse baselines, limited calibration transfer, and unclear PM-BLEx validation. Nevertheless, I am raising my score. Thank you.

AuthorsOpenReview ↗

Thanks to Reviewer VPxM for Raising the Score

Thank you for your update. We are glad that our response has addressed your main concerns regarding practical gains and generalization. We sincerely appreciate your constructive feedback.

Reviewer_yaNyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Residual Forcing is a training-free sparse attention method for autoregressive video diffusion. It combines structured token protection, adaptive residual routing, and cross-chunk context sharing to accelerate generation while preserving quality and temporal continuity.

Critical strengths
  1. The paper is well written and easy to follow, with clear motivation and an accessible presentation of the system.
  2. The paper studies sparse attention for autoregressive video generation and provides operator-level latency analyses, which are valuable for practical deployment.
  3. The method protects local and salient tokens before allocating an adaptive residual attention budget, while sharing historical context across chunk boundaries.
Critical weaknesses
  1. The novelty is limited. The core design combines predefined spatial protection (aligned blocks and axial halos), score-based shared salient-token protection, and dynamic selection of residual blocks. In my view, this is primarily a hybrid of static and dynamic sparse attention.
  2. The motivation(mean-based retrieval can miss strong attention responses)does not seem closely connected to the proposed method. In my view, the method does not fully address the limitations of mean-based scoring.
  3. Both shared salient-token selection and residual routing select tokens based on attention scores. Would using either mechanism alone, or combining both, lead to substantial differences in final generation quality?
  4. The experiments are limited to LongLive. Could the authors also evaluate other autoregressive generators, such as Self Forcing, or larger autoregressive models?
  5. Are all latency measurements, including those of the baselines, obtained using FlashInfer? Since the proposed attention masks are non-contiguous, does the method require FlashInfer kernels? Would other kernels, such as FlashAttention-4, be unsuitable for these masks?
  6. Quantitative quality evaluation primarily relies on VBench. Could the authors report additional assessments, such as VisionReward [1] scores or human preference results, to better substantiate the claimed long-video generation quality?

[1] Jiazheng Xu et al. VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(13): 11269–11277, 2026.

Questions for authors

Refer to Weaknesses.

5 comments
AuthorsOpenReview ↗

Author Response to Reviewer yaNy (Part 1/2)

Thank you for the thoughtful feedback. We respond point-by-point below.

W1 & W2: Core Contributions and Motivation

Our method does combine static and dynamic sparse attention, but its core objective is to address a key challenge in block-sparse attention: how to select the most suitable KV blocks for each Q block at low overhead. Methods such as Light Forcing and VSA[1] use mean pooling to estimate the match between Q–K block pairs. Such proxies are usually effective when the Q and K representations within each block are relatively consistent. However, strong responses from a small number of outlier Q–K token pairs can be diluted by numerous weak interactions in the mean, yet be exponentially amplified by softmax and carry substantial attention weight. Consequently, whole-block means may underestimate KV blocks containing critical interactions, rank them lower, and cause them to be missed.

To address scoring errors caused by within-block heterogeneity, DFSAttn uses 3D Hilbert reordering to improve within-block consistency, while SpargeAttention selectively applies mean compression based on within-block self-similarity. We take a different approach: protect support that may carry strong interactions in advance, ensuring that it participates in the final attention computation, and then perform query-adaptive mean-pooling block selection over the residual region. This reduces the risk of missing critical interactions because they are underestimated by whole-block means. We observe that strong Q–K token interactions exhibit structural characteristics that generalize across samples, summarized as three main high-response structures: spatially corresponding positions, neighboring spatial positions, and shared salient keys. Based on these observations, we design a routing method that efficiently constructs a protected set for each query block.

We do not claim that the protected set covers all strong interactions, nor that residual mean pooling can fully recover the true ranking. However, Fig. 2(a) clearly shows that isolated peaks with high true attention mass remain near the end of the whole-block mean ranking, and that these peaks are substantially reduced after removing protected support and recomputing the scores. Fig. 2(b) also shows that spatially corresponding blocks, neighboring regions, and salient-key columns together account for 95.78% of the missed attention mass in this diagnostic, providing empirical support for the protection design. In addition, the experimental results show that our method achieves higher VBench Total, lower PM-BLEx, and greater DiT speedups than Light Forcing, DFSAttn, and SpargeAttention in long-video generation, demonstrating practical benefits in generation quality, boundary continuity, and inference efficiency.

W3: Independent Contributions and Combined Effects of Salient-Token Selection and Residual Routing

The two components differ in resolution and coverage objectives. Salient-token selection protects a small number of shared high-response keys and forms part of our protected set. Residual routing uses mean pooling to select the corresponding KV blocks for each query block from the residual set to meet the target attention mass. We consider both components indispensable. The ablation results in Table 3 show that removing salient-token selection increases the total amount of selected KV while reducing overall generation quality. Residual routing cannot simply be removed, since relying only on the protected set would lose substantial attention mass and thereby substantially degrade generation quality.

AuthorsOpenReview ↗

Author Response to Reviewer yaNy (Part 2/2)

W4: Effectiveness on Other Generative Backbones

We added a complete evaluation on LongLive 2.0 5B, covering 128 30-second videos. Dense retains the native maximum of 32 visible KV frames (8 sink + 16 recent + 8 current frames), while sparse methods use 12 candidate frames within the same physical cache and retain their respective scoring and block-selection rules.

MethodKeqK_{\mathrm{eq}}Quality ↑Total ↑PM-BLEx ↓DiT speedup ↑
Dense411.3082.5579.310.1741.00×
Light Forcing44.8482.0679.461.8821.52×
DFSAttn38.3681.9879.785.6151.36×
SpargeAttention36.3581.2878.872.5651.39×
Residual Forcing40.0982.6679.921.4801.55×

Residual Forcing achieves the highest Quality, Total, and DiT speedup in this setting, together with the lowest PM-BLEx among the sparse methods, supporting its effectiveness on a larger generative backbone.

W5: Selection and Adaptation of Execution Backends

For the original LongLive evaluation on A100/A800, the methods use different execution backends: FlashAttention-2 for Dense, FlashInfer FA2 indexed execution for Residual Forcing, the author-provided SM80 Triton sparse operator for Light Forcing, the Block-Sparse-Attention operator for DFSAttn, and the official SM80 quantized sparse operator for SpargeAttention.

FlashAttention-4 can be combined with paged KV, and the FA4 branch in vLLM passes page_table. However, in the vLLM version we examined, the hardware check for this FA4 backend permits only compute capabilities 9.x, 10.x, and 11.x, excluding the 8.0 capability of the A100/A800 used in this paper. Therefore, Residual Forcing uses the FA2 backend of FlashInfer to access non-contiguous K/V through compact page tables and applies a unified softmax over the protected and residual sets.

W6: Additional Evaluation Metrics

We added VisionReward[2] and VideoReward[3] evaluations, covering all 946 5-second videos, as well as 128 30-second videos and 128 60-second videos. The results are shown below:

VideoReward ↑

Method5s30s60s
Dense2.21036.28077.8048
Light Forcing2.16215.50926.8515
DFSAttn1.99495.38936.7355
SpargeAttention2.14225.64037.0174
Residual Forcing2.26046.18477.5287

VisionReward ↑

Method5s30s60s
Dense0.17410.13670.1422
Light Forcing0.17480.12390.1215
DFSAttn0.17450.12900.1221
SpargeAttention0.17530.12950.1314
Residual Forcing0.17440.13760.1413

The results show that Residual Forcing achieves higher scores than the other sparse baselines in most settings, particularly leading on both metrics for 30-second and 60-second long videos. This provides further evidence of its generation-quality advantage beyond VBench.

References

[1] Zhang P, et al. Faster Video Diffusion with Trainable Sparse Attention. NeurIPS, 2025.

[2] Xu J, et al. VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation. AAAI, 2026.

[3] Liu J, et al. Improving Video Generation with Human Feedback. NeurIPS, 2025.

AuthorsOpenReview ↗

Looking forward to your feedback

Dear Reviewer yaNy,

We would like to kindly follow up on our previous response. We hope that our clarifications and additional results have addressed your main concerns. We would greatly appreciate any further feedback you may have and would be glad to provide additional clarifications if needed.

Thank you again for your time and consideration.

Best regards,

The Authors

Reviewer_NcDmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free sparse-attention method for autoregressive video diffusion. It protects spatially aligned and salient tokens, selects additional tokens through residual routing, and incorporates adaptive budgets, cross-chunk context sharing, and routing reuse. Experiments report inference speedups with competitive generation quality and improved boundary continuity.

Critical strengths
  • The method addresses a practically relevant inference bottleneck and achieves measurable end-to-end acceleration without retraining.

  • The paper includes component ablations and a breakdown of routing, attention, and end-to-end latency, providing useful evidence about the quality–efficiency trade-offs.

Critical weaknesses
  • While the paper combines various improvements for training-free acceleration of autoregressive video diffusion, its contribution appears somewhat incremental. Block-sparse attention for video diffusion has been studied in VSA and VMoBA. Localized attention based on observed attention patterns has been explored in Sliding Tile Attention (STA), while cross-step pattern reuse has been explored in AdaSpa. Building on these ideas, incorporating salient-token protection and residual routing is a reasonable design, but the paper does not clearly establish a substantial technical contribution beyond their combination.

  • In addition, the benefits of the proposed method over simpler configurations appear modest. In Table 3, spatial alignment alone provides most of the improvement in VBench Total. Adding halo and saliency protection increases Total from 78.97 to 79.31 while increasing attention latency from 21.94 to 23.45 ms. Figure 5(a) also shows only a small overall coverage-hit improvement from query-adaptive budgeting, from 70.11% to 70.70%. Although the improvement on difficult queries is larger, its effect on generation quality or runtime is not demonstrated. Furthermore, Table 3 progressively adds different types of protected support but does not compare residual routing against ordinary mean-pooled routing with the same protected tokens. The practical benefits of residualization and query-adaptive budgeting therefore remain insufficiently established, making it difficult to assess the contribution of these components.

  • The evaluation is limited to LongLive with the relatively small Wan2.1-T2V-1.3B backbone. The videos in the supplementary material exhibit noticeable artifacts both with and without the proposed method. Evaluation on a stronger backbone would help establish whether the reported quality–efficiency benefits generalize to higher-quality video generation.

Questions for authors
  • Which baselines reuse their routing decisions across denoising calls, and at what frequency are their routes refreshed? Does the quality–efficiency advantage persist when the strongest baselines use the same route-refresh frequency and cache allocation?

  • Does query-adaptive budgeting improve generation quality or end-to-end latency relative to static per-head budgets at comparable computational cost? Figure 5(a) reports attention-coverage statistics, but evidence of a practical benefit would materially strengthen this component’s contribution.

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer NcDm (Part 1/2)

Thank you for the thoughtful feedback. We respond point-by-point below.

W1: What Is the Technical Contribution Relative to VSA, VMoBA, STA, and AdaSpa?

Our method makes two main contributions:

1. Structure-Aware Sparse Attention

  • VSA/VMoBA: Although both VSA and VMoBA use block-sparse attention, their block selection primarily relies on mean pooling. Strong Q–K interactions can be diluted by averaging yet carry substantial weight after softmax, causing critical interactions to be missed. In contrast, Residual Forcing introduces structural protection to address this risk. Applying mean pooling only to the residual set can substantially reduce high-response omissions (Fig. 2(a)).
  • STA: STA allows each query block to attend only to key blocks within its neighboring spatiotemporal window. Our method instead protects spatially corresponding positions, halos, and salient keys within candidate frames, then selects residual blocks from the remaining region. STA emphasizes three-dimensional spatiotemporal locality to construct hardware-friendly sparse windows. Our method emphasizes protecting high-response interactions and supplementing nonlocal dependencies through residual routing.
  • AdaSpa: Although both AdaSpa and Residual Forcing reuse sparse patterns across denoising steps, AdaSpa requires token-level dense attention statistics, incurring substantially higher routing overhead than our method. In a mechanism-level reproduction on the same A100 with the same six candidate frames, routing takes approximately 1.93 ms for AdaSpa and 1.01 ms for Residual Forcing at 480p. At 720p, these times are approximately 8.15 ms and 1.88 ms.

2. Cross-Chunk Context Sharing

  • The methods above primarily optimize sparse computation within a given context or reuse across denoising steps. However, in autoregressive video generation, sparse attention also faces the issue of historical-context consistency across chunks. If adjacent chunks retrieve historical frames independently, abrupt changes in historical conditioning can cause noticeable boundary discontinuities even when cache capacity and candidate count remain the same. These discontinuities are visually apparent but difficult to capture with existing quality metrics, and the methods above do not specifically address this issue.
  • In contrast, Residual Forcing combines mean-based proxies with spatially aligned interactions for historical retrieval. The first query frame of each new chunk inherits the historical frames selected for the last query frame of the preceding chunk. Without increasing cache size or candidate-frame count, queries across the boundary reuse the same selected historical frames while retaining content adaptivity. Fig. 3 illustrates how our method can substantially improve visual coherence at adjacent chunk boundaries.

W2: Modest Component Gains and the Lack of an Ordinary Mean-Routing Control with the Same Protected Set

  • In Table 3, the alignment configuration achieves a Total of 78.97, which increases to 79.31 after adding halo and saliency protection. This is already close to the Dense result of 79.47 with the full 12-frame KV context, leaving relatively limited room for further quality recovery. Meanwhile, the approximately 9.6% reduction in KeqK_{eq} reduces attention execution time from 18.66 ms to 17.03 ms. This will offer greater benefits in settings with more expensive attention computation (the reduction in attention execution time can offset the additional routing overhead). Our current trade-off prioritizes preserving generation quality while maintaining overall acceleration.
  • Our query-adaptive budgeting achieves a higher hit rate on difficult queries, but this advantage is diluted when averaged over all queries. Supplementary evaluations also show that static and dynamic budgets yield similar overall generation quality and latency. One possible explanation is that protected-set attention priors vary across queries and should inform dynamic budgets alongside residual-set information. We will pursue this as future work to further improve query-adaptive budgeting.
  • Under the protection rules, we added a control using whole-block mean scores and obtained offline head-wise thresholds using the same calibration set. At equal block counts (five selection fractions from 5% to 30%), residual-set mean scoring (Residual Forcing) recovers 3.07%–24.19% more true residual attention mass than whole-block mean scoring in the control. We further evaluated 128 30-second videos under the same τ\tau setting. The VBench Quality scores of Residual Forcing and the control are 81.50 / 81.24, respectively, and their Total scores are 79.31 / 79.10. Both scores exceed those of the control using whole-block mean routing.
AuthorsOpenReview ↗

Author Response to Reviewer NcDm (Part 2/2)

W3: A Stronger Generative Backbone Is Needed, and the Original Videos Already Contain Artifacts

We adapted each method to LongLive 2.0 5B and completed an evaluation on 128 30-second videos. This model uses the Wan2.2 VAE, 24 heads, and 8 latent frames per chunk. Dense retains the native maximum of 32 visible KV frames (8 sink + 16 recent + 8 current frames), while sparse methods use 12 candidate frames within this physical cache and retain their respective scoring and block-selection rules.

MethodKeqK_{\mathrm{eq}}Quality ↑Total ↑PM-BLEx ↓DiT speedup ↑
Dense411.3082.5579.310.1741.00×
Light Forcing44.8482.0679.461.8821.52×
DFSAttn38.3681.9879.785.6151.36×
SpargeAttention36.3581.2878.872.5651.39×
Residual Forcing40.0982.6679.921.4801.55×

Residual Forcing achieves the highest quality scores in this setting and the lowest PM-BLEx among the sparse methods, while maintaining the highest speedup. These results further support the effectiveness and applicability of Residual Forcing on a larger generative backbone.

In addition, we updated the supplementary material to include generation results from both LongLive 2.0 and the original LongLive, presenting 12 videos in total, with 6 each for Light Forcing and Residual Forcing. Please refer to the supplementary videos for a direct visual comparison of the two methods at chunk boundaries.

Q1: Which Baselines Reuse Routes, and Does the Advantage Persist with Matched Refresh Frequency and Cache Allocation?

In the current implementations, Residual Forcing constructs only 1 route per chunk, while Light Forcing, DFSAttn, and SpargeAttention refresh their routes 5 times (four denoising calls and one context write-back). We selected SpargeAttention, the strongest baseline in the 30s setting, and matched its physical cache, candidate-frame rules, and route-refresh frequency to those of Residual Forcing while retaining its original block-selection algorithm. We completed an evaluation on 128 30-second videos, with the results shown below:

ConfigurationKeqK_{\mathrm{eq}}Quality ↑Total ↑PM-BLEx ↓DiT / seconds ↓End-to-end / seconds ↓
SpargeAttention / per call32.07681.0579.220.76542.1565.55
SpargeAttention / per chunk30.32077.2875.810.80941.2764.69
Residual Forcing / per chunk30.23981.5079.310.28433.1656.54

The results show that Residual Forcing still achieves higher quality and lower latency under the matched setting. SpargeAttention, when reusing the blocks selected by the first routing call, reduces latency but suffers a noticeable loss in video quality.

Q2: Can Dynamic Budgets Improve Generation Quality or End-to-End Latency at Comparable Cost?

Please see our response to Weakness 2.

AuthorsOpenReview ↗

Looking forward to your feedback

Dear Reviewer NcDm,

We would like to kindly follow up on our previous response. We hope that our clarifications and additional results have addressed your main concerns. We would greatly appreciate any further feedback you may have and would be glad to provide additional clarifications if needed.

Thank you again for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.