- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Residual Forcing is a training-free sparse attention framework for autoregressive video diffusion. It protects structured blocks accounting for ~96% of attention mass missed by mean-pooled routing, adaptively routes the remaining keys, and shares context across chunks to improve continuity. On LongLive (Wan2.1-1.3B), it achieves 4.12× self-attention speedup under fixed-cache profiling and 1.25× end-to-end speedup at 30 seconds, with the best quality and boundary continuity among evaluated sparse baselines, measured using the proposed PM-BLEx metric.
Critical strengths
-
The paper is easy to read and follow.
-
Well-supported motivation. The exponential response-gap bound and evidence that missed attention mass is highly structured provide a solid basis for the design. The decomposition into stable head-level and dynamic query-level budgets is particularly insightful.
-
Transparent efficiency reporting. Table 2 includes routing overhead and useful controls—Dense FA2-6 and route-refresh frequency—showing when routing costs outweigh sparsity gains.
-
Reasonable baseline coverage. Four sparse methods are evaluated on the same generator at matched token budgets across three video durations.
Critical weaknesses
-
Modest practical gain. Dense FA2-6 already achieves 1.17× end-to-end speedup versus 1.25× for the full method—only ~7% additional gain. The headline 4.12× applies only to self-attention under fixed-cache profiling.
-
Narrow evaluation. All experiments use LongLive/Wan2.1-1.3B at 480p, leaving high-resolution performance and cross-model calibration transfer untested. At 5 seconds, the method also trails sparse baselines on VBench Total.
-
Missing relevant baselines. Baslines like FAST-AR and Light Interaction are cited as closely related but not compared experimentally.
-
Unvalidated continuity metric. The central continuity claim relies on the authors’ PM-BLEx metric, without human or independent perceptual validation.
Questions for authors
My major concerns are the modest practicle gains and generalization for other backbones.
4 comments
Author Response to Reviewer VPxM (Part 2/2)
W4: PM-BLEx Lacks Human or Independent Perceptual Validation
We added human perceptual validation involving three evaluators. All three agreed that Residual Forcing, with Cross-Chunk Context Sharing, can effectively mitigate visual jumps at boundaries between adjacent chunks in autoregressive video generation, making cross-chunk transitions more continuous and natural. We also updated the supplementary material to include generation results from both LongLive 2.0 and the original LongLive, presenting 12 videos in total, with 6 each for Light Forcing and Residual Forcing. Please refer to the supplementary videos for a direct visual comparison of the two methods at chunk boundaries.
Q1: Practical Gains and Generalization to Other Backbones
We added an evaluation on LongLive 2.0 5B. Quality results cover 128 30-second videos. Dense retains the native 32-frame KV context (8 sink + 16 recent + 8 current frames), while sparse methods use 12 candidate frames within the same physical cache and retain their respective scoring and block-selection rules. The results are shown below:
| Method | Quality ↑ | Total ↑ | PM-BLEx ↓ | DiT speedup ↑ | |
|---|---|---|---|---|---|
| Dense | 411.30 | 82.55 | 79.31 | 0.174 | 1.00× |
| Light Forcing | 44.84 | 82.06 | 79.46 | 1.882 | 1.52× |
| DFSAttn | 38.36 | 81.98 | 79.78 | 5.615 | 1.36× |
| SpargeAttention | 36.35 | 81.28 | 78.87 | 2.565 | 1.39× |
| Residual Forcing | 40.09 | 82.66 | 79.92 | 1.480 | 1.55× |
The results show that Residual Forcing achieves the highest Quality, Total, and DiT speedup on this backbone, as well as the lowest PM-BLEx among the sparse methods, further supporting its effectiveness across generative backbones. For additional evidence on practical gains, please refer to W1 & W2.
Thank you for the response and additional experiments. The 720p and 5B results address my main concerns about practical gains and generalization. Some limitations remain, including modest gains over sparse baselines, limited calibration transfer, and unclear PM-BLEx validation. Nevertheless, I am raising my score. Thank you.
Thanks to Reviewer VPxM for Raising the Score
Thank you for your update. We are glad that our response has addressed your main concerns regarding practical gains and generalization. We sincerely appreciate your constructive feedback.
Author Response to Reviewer VPxM (Part 1/2)
Thank you for the thoughtful feedback. We respond point-by-point below.
W1: Limited Practical Gains over Dense FA2-6
The limited additional speedup of our method over FA2-6 at 480p is mainly due to the relatively small scale of self-attention computation, where the gains in attention execution are partially offset by routing and other overheads. We therefore extended the same method to higher-resolution experiments at 720p, with the results shown below:
The results show that at 720p, Residual Forcing achieves end-to-end inference speedups of 1.615× over full Dense FA2 and 1.283× over FA2-6 with six candidate frames. Compared with the 1.067× speedup over FA2-6 at 480p, the high-resolution experiments demonstrate more pronounced practical acceleration gains.
W2: High Resolution, Cross-Backbone Calibration Transfer
1. High-Resolution Evaluation
In addition to the timing results above, we completed quality evaluations of all five methods on the full set of 128 30-second videos at 720p.
The results show that at 720p, Residual Forcing achieves the highest Quality, lowest PM-BLEx, and greatest DiT speedup with the smallest effective attention support, demonstrating its ability to balance generation quality, boundary continuity, and computational efficiency in high-resolution settings.
2. Cross-Model Threshold Transfer
We completed bidirectional transfer of frozen thresholds between LongLive and Self-Forcing, using all 946 prompts for 5-second generation in both directions. The two models have the same architecture, both with 30 layers × 12 heads. Self-Forcing is calibrated using 5-second trajectories. The results are shown below:
The results show that using LongLive's calibration improves performance on Self-Forcing, whereas transferring Self-Forcing's calibration to LongLive performs worse than LongLive's original performance. Overall, performance remains comparable when calibration thresholds are transferred between the two backbones.
W3: Missing Comparisons with Other Baselines
Since FAST-AR has not yet been open-sourced, we additionally adapted and evaluated Light Interaction. We integrated only its 3D sparse attention module, adjusted the temporal chunk size from 4 frames to 3 frames as in LongLive, used the same candidate-frame allocation as DFSAttn and SpargeAttention, and retained its Top-17.5% selection of historical blocks along with full protection of the current chunk. We used 128 MovieGen prompts for a complete 30-second quality evaluation, with the results shown below:
The results show that Light Interaction has a higher Total score than Residual Forcing. This is mainly because its original method requires the current chunk's KV to participate fully in attention computation, resulting in 3 times as much effective KV participating in attention as in Residual Forcing. In contrast, Residual Forcing achieves better boundary continuity and practical acceleration with less cached KV.