- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper measures how much an autoregressive video model's output depends on a past frame at each temporal distance and denoising step. Based on the pattern, it adds a step-dependent bias on attention to past frames (TRB), implemented through a FlashAttention-compatible rewrite (BAR). Used training-free or with short retraining, it improves long-video quality over prior methods at little extra cost.
Critical strengths
- A simple diagnostic that directly motivates the method: older frames matter less, and later denoising steps rely more on older frames.
- Practical: works with or without retraining, generates far beyond the training length, and BAR is much faster than adding the bias directly.
Critical weaknesses
- Key effects are not isolated. On short videos the gain is not separated from retraining, and the paper says the bias barely matters within the training window while also claiming it improves generation. There is no ablation with the same bias at every step, although step dependence is the core design.
- Limited generality and a weak user study. Only one small backbone is used. The user study reports only percentages, without significance or agreement.
Questions for authors
- How does an equally retrained model without TRB, and a step-independent bias of the same strength, perform?
- Can you report significance and per-pair counts for the user study? In Figure 7, color consistency and subject consistency have exactly the same preference rates against both Self Forcing and LongLive; is this correct?
- Does the pattern hold on other backbones?
7 comments
W2 / Q3. One small backbone was used and Temporal Response Analysis on other backbones
W2 / Q3. One small backbone was used and Temporal Response Analysis on other backbones
Justification on one backbone. We acknowledge the limited backbone coverage in Appendix G. Limitations. However, Wan2.1-1.3B is the common testbed used by all baselines in this line of work (Self-Forcing, Causal Forcing, Rolling Forcing, LongLive, Reward Forcing), primarily for compute feasibility and fair comparison.
We further conduct our temporal response analysis on the Wan2.2-5B few-step autoregressive model released by LongLive2.0 [1], extending the analysis beyond Wan2.1-1.3B. To the best of our knowledge, this is the only available few-step autoregressive distilled student based on a backbone other than Wan2.1-1.3B. The released model uses a 32-frame attention context window, generates 8 frames per chunk, and runs 4 denoising steps, meaning only 4 chunks in attention window. Thus, we intentionally extend the model's attention window to 64 frames (8 chunks) for measuring .
The normalized positional response plot: LongLive2.0 5B's R plot
The results exhibit the same pattern reported in the main paper: monotonic ordering, steep decay, and denoising-step dependence, providing additional evidence that these trends extend beyond Wan2.1-1.3B. Interestingly, we observe a slight upward rebound at the tail of the curve () for the LongLive2.0 5B model. We hypothesize that this behavior may be attributed to the use of Attention Sinks along with the model's training on long-horizon videos.
We have added this new analysis in Appendix B. Positional Response on Wan2.2-5B in our revised paper.
References
[1] Chen, Yukang, et al. "LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation." arXiv preprint arXiv:2605.18739 (2026).
W1 / Q1. An equally retrained model without TRB and step-independent bias
W1 / Q1.1. "The bias barely matters within the training window"
Clarify: Adding TRB improves short video generation. As shown in Table 3 of the main paper, our method achieves the highest overall score on VBench for short videos. Thus, the bias does not "barely matter" within the training window.
We acknowledge that our original phrasing contradicted the results, and we have revised the paper in Section 4.1:
Old: "Our method achieves the best overall score on VBench, demonstrating that integrating TRB introduces no degradation in short-video quality - the bias has negligible effect when all context frames fit within the training window."
New: "Our method achieves the best overall score on VBench, demonstrating that integrating TRB actively benefits short-video generation."
W1 / Q1.2. An equally retrained model without TRB and step-independent bias
The model without TRB is Causal Forcing baseline, reported in Tables 3 and 4 in the Main paper for short- and long-video generation. Both methods use the same training dataset and initialization from the Causal Forcing stage-2 causal ODE checkpoint, as detailed in Appendix D Implementation Details.
The only differences between our training settings and original baseline's settings are training iterations and batch size. To additionally control the settings, we retrain the Causal Forcing baseline (without TRB) for 3,500 iterations and batch size 4 - matches the training setting of Recency Forcing (same dataset, same initialization,...). The evaluation protocol is the same on both VBench and VBench long reported in the main paper.
Short-video generation: VBench, 5 seconds
| Model | Total ↑ | Quality ↑ | Semantic ↑ |
|---|---|---|---|
| Causal Forcing (original) | 84.04 | 84.59 | 81.84 |
| Causal Forcing (retrained, 3,500 iterations, batch size 4) | 81.84 | 82.82 | 77.90 |
| Recency Forcing | 85.08 | 86.20 | 80.59 |
Long-video generation: VBench-Long, 60 seconds
| Model | Quality ↑ | Background ↑ | Subject ↑ | Motion ↑ | Dynamic ↑ | Aesthetic ↑ | Imaging ↑ | Temporal ↑ |
|---|---|---|---|---|---|---|---|---|
| Causal Forcing (original) | 78.98 | 95.51 | 95.39 | 96.94 | 65.19 | 51.35 | 63.44 | 95.21 |
| Causal Forcing (retrained, 3,500 iterations, batch size 4) | 79.28 | 94.80 | 94.34 | 96.06 | 68.34 | 57.40 | 66.95 | 93.75 |
| Recency Forcing | 84.02 (+4.74) | 96.49 | 97.68 | 98.00 | 75.48 | 61.31 | 70.50 | 96.32 |
Result: Under the same training setting, adding TRB improves every short-video score, and Total rises from 81.84 to 85.08 (+3.24). On 60-second videos, TRB consistently improves all dimensions, and Quality rises from 79.28 to 84.02 (+4.74). Since every other factor is held fixed, these improvements come from TRB and not from differences in training.
To isolate denoising-step dependence, we provide a comparison between step-dependent (Recency Forcing) and step-independent variants, using our best-performing exponential bias by fixing throughout training and inference, while keeping the other settings unchanged.
Short-video generation: VBench, 5 seconds
| Bias configuration | Total ↑ | Quality ↑ | Semantic ↑ |
|---|---|---|---|
| Step-independent | 83.76 | 84.68 | 80.06 |
| Step-dependent (original Recency Forcing) | 85.08 | 86.20 | 80.59 |
Long-video generation: VBench-Long, 60 seconds
| Bias configuration | Quality ↑ | Background ↑ | Subject ↑ | Motion ↑ | Dynamic ↑ | Aesthetic ↑ | Imaging ↑ | Temporal ↑ |
|---|---|---|---|---|---|---|---|---|
| Step-independent | 82.75 | 96.41 | 97.82 | 98.08 | 61.96 | 59.87 | 69.90 | 96.38 |
| Step-dependent (original Recency Forcing) | 84.02 | 96.49 | 97.68 | 98.00 | 75.48 | 61.31 | 70.50 | 96.32 |
Result: Our original step-dependent TRB design form outperforms the step-independent variant on both short and long horizon VBench. On short VBench, Total Score improves by +1.32, Quality by +1.52, and Semantic by +0.53. On 60-second videos, it improves Quality by +1.27 (84.02 vs. 82.75), and the gap comes from Dynamic (75.48 vs. 61.96, +13.52).
W2 / Q2. Weak user study and significant test - Same preference rates in color consistency and subject consistency
W2 / Q2. Weak user study and significant test - Same preference rates in color consistency and subject consistency
Justification on user study. We conduct a user study with 45 participants, collected 3384 responses. For context, prior work in this field typically uses 20–30 participants, so our user study is at or above the common scale.
We ran a two-sided binomial test on the collected responses. We note that statistical stablility in a 2AFC study is driven by the number of pairwise comparisons rather than the number of participants. Across all four aspects, Recency Forcing is preferred at a statistically significant rate:
Recency-Forcing vs Self-Forcing
Recency-Forcing vs Rolling-Forcing
Recency-Forcing vs LongLive
Same preference rates in color consistency and subject consistency.
This is true, the preference rates for color consistency and subject consistency are identical in the comparisons against Self Forcing and LongLive. The complete preference rates are reported above. One possible explanation is that these two criteria may be correlated, as color drift can coincide with changes in subject appearance.