- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Step Forcing, a training-free method for mitigating model drift in autoregressive video diffusion. The method stores KV caches at different denoising steps, reuses step-matched noisy KV caches from previous chunks, and retains a subset of clean KV tokens as anchors for temporal consistency. Experiments on several autoregressive video generation baselines show noticeable improvements on CausVid and Causal Forcing, although the effectiveness is less clear on stronger training-free baselines.
Critical strengths
The paper identifies a potentially important mismatch between the noisy state of the current chunk and the clean historical KV cache reused throughout the denoising trajectory. Based on this observation, it proposes to store KV caches at different denoising steps, reuse step-matched noisy KV caches, and retain clean anchor tokens to preserve temporal consistency.
Critical weaknesses
-
The comparison between Step Forcing and other training-free methods is not sufficiently fair or clearly defined. The official Deep Forcing implementation uses the Self-Forcing DMD checkpoint and functions as a training-free inference and KV-cache policy. Therefore, the appropriate comparison should apply Deep Forcing and Step Forcing separately to the same frozen Self-Forcing checkpoint, using the same prompts, seeds, sampling schedule, generation horizon, and memory budget. However, the paper presents Deep Forcing as a base framework and reports “Deep Forcing + Step Forcing” without clearly explaining this relationship or isolating the standalone effect of the two inference policies. The paper should also compare Step Forcing with closely related training-free methods such as TokenTrim or Relax Forcing. The small additional gain on Deep Forcing, only 0.09 in the VBench-Long Total score, further makes the claimed general effectiveness unclear.
-
The effectiveness of the clean-anchor mixing strategy is still insufficiently established. Table 3 includes the r_max = r_min = 0 setting, which appears to remove clean-anchor replacement and therefore provides a noisy-only ablation. However, the paper does not provide a matched comparison among full clean KV, noisy-only Step KV, and the proposed noisy-clean mixed KV under the same configuration. It also does not validate the proposed clean-token selection rule, which selects tokens using the norm of clean keys and applies the same positions to both keys and values. The compatibility of clean and noisy representations, as well as the effect of possible anchor-selection errors, therefore remains unclear.
Questions for authors
-
The official Deep Forcing implementation appears to use the Self-Forcing DMD checkpoint as its frozen base model. Could the authors clarify whether the Deep Forcing and “Deep Forcing + Ours” results in Table 2 are obtained on the same Self-Forcing checkpoint? For a fair comparison, could the authors compare Deep Forcing applied to the Self-Forcing checkpoint against Step Forcing applied to the same checkpoint, under identical prompts, seeds, sampling schedules, generation horizons, and memory budgets? If applicable, please also report the result of applying both policies together.
-
Could the authors compare Step Forcing with other closely related training-free inference methods, such as TokenTrim or Relax Forcing, using the same frozen base model and evaluation protocol?
-
Table 3 includes the r_max = r_min = 0 setting. Could authors provide a matched comparison among the original full-clean KV cache, noisy-only Step KV, and the proposed noisy-clean mixed KV strategy?
-
Could the authors clarify why clean-token importance is estimated using the norm of the clean keys, and why the same selected positions are used to replace both keys and values? How sensitive is the method to random anchor selection or other importance criteria, such as attention-based selection?
-
Could the authors report the peak GPU memory and inference latency of Step Forcing compared with Deep Forcing and other training-free methods under the same model, generation horizon?
3 comments
Response to Reviewer DZ1r (2/2)
W2.2&Q4: Explanation of the Clean-Token Selection
Video tokens often share visual content across space and time, which can lead to correlations among their key representations. Inspired by prior work on low-rank approximations of attention representations (Saxena et al., 2024), we assume that the clean-key matrix is approximately low-rank. We adopt a row-sampling strategy inspired by (Frieze et al., 2004), using the squared norm of each clean key as a query-independent proxy for token importance. The ratio measures the fraction of the clean-key matrix’s energy contributed by token (i). Eq.(5) uses this ratio to give higher-energy clean keys a greater probability of being selected when the anchor budget is limited.
We select anchor positions based on the clean keys based on the Eq.(5) and replace both the key and the value at each selected position with their clean counterparts. This preserves positional embedding alignment between keys and values, ensuring that the attention weight computed from a clean key is applied to the clean value from the same token position.
Furthermore, we use clean-key norms because they allow us to select anchors as soon as a chunk finishes denoising. An attention-based score depends on queries from the next chunk, which become available only during its denoising. Norm-based selection therefore lets us prepare the mixed KV cache in advance and release the temporary clean and noisy KV buffers before generating the next chunk.
Q1: Experimental Setup for the Comparison.
All four configurations in Table 2 use the same frozen Self Forcing DMD checkpoint. For the 30-second comparison, we use the GPT-4o-extended prompts released by Self Forcing (https://github.com/guandeh17/Self-Forcing/blob/main/prompts/vbench/all_dimension_extended.txt) and follow the VBench-Long evaluation protocol (https://github.com/Vchitect/VBench/blob/master/prompts/README.md).
For each configuration, we generate five videos per prompt for Appearance Style (90 prompts), Color (85), Human Action (100), Multiple Objects (82), Object Class (79), Overall Consistency (93), Scene (86), Spatial Relationship (84), Subject Consistency (72), and Temporal Style (100). For Temporal Flickering, we generate 25 videos per prompt for 75 prompts. This yields 6,230 videos across 946 prompts per configuration.
We derive each seed deterministically using the following function. Each baseline and its Step Forcing counterpart use identical values for all four inputs, yielding the same prompt–seed pairs.
def stable_seed(base_seed, folder, prompt, sample_index):
payload = f"{base_seed}\0{folder}\0{prompt}\0{sample_index}".encode("utf-8")
return int.from_bytes(hashlib.sha256(payload).digest()[:4], "little") & 0x7FFFFFFF
The original Self Forcing configuration achieves a Total score of 80.84. Applying Step Forcing alone yields 81.94, and applying Deep Forcing alone yields 83.03. Applying both policies together yields 83.12, an observed gain of 0.09 points over Deep Forcing alone.
Q4.2: How sensitive is the method to random anchor selection?
We compare our anchor token selection with random selection. Replacing our anchor token selection with random selection reduces the Total score by 0.29 points. The drop is larger in Quality than in Semantic, indicating modest sensitivity to the selection rule in this setting.
| Method | Total ↑ | Quality ↑ | Semantic ↑ |
|---|---|---|---|
| Anchor Token Selection (Ours) | 82.79 | 84.33 | 76.62 |
| Random Anchor Selection | 82.50 | 83.99 | 76.53 |
Q5: Peak GPU memory usage and inference latency.
We report peak GPU memory and inference latency for Self Forcing, Relax Forcing, Deep Forcing, and Step Forcing under the same model and generation horizon.
| Method | Peak GPU memory usage (GB) | Inference latency (s) |
|---|---|---|
| Self Forcing | 32.947 | 56.347 |
| Relax Forcing | 61.998 | 44.710 |
| Deep Forcing | 36.494 | 52.060 |
| Step Forcing | 42.936 | 52.543 |
Step Forcing has comparable latency to Deep Forcing (52.543 vs. 52.060 s) and uses 6.442 GB more peak GPU memory.
Follow-UP
Dear Reviewer DZ1r,
Thank you so much for your valuable feedback and the insightful discussions. They have truly helped improve our work!
In our responses, we compare Step Forcing with other training-free methods, explain the effects of clean and noisy KV representations and the role of Anchor Token Selection, provide details on the experimental setup, evaluate the method’s sensitivity to random anchor selection, and report peak GPU memory usage and inference latency.
As the discussion phase comes to a close on November 18, we would like to check in and see if we’ve addressed all of your questions. If you're satisfied with our responses, we would be grateful if you’d consider updating your score.
Thank you again for all your time and support!
Best regards, Authors of submission 2775
Response to Reviewer DZ1r (1/2)
We appreciate your constructive comments and suggestions. We respond to your concerns below to further improve the quality of our paper. In addition, we have revised the manuscript and highlighted the changes in blue for ease of reference.
All four 30-second configurations in Table 2 use the same frozen Self Forcing DMD checkpoint. Deep Forcing and Step Forcing are both applied at inference. Specifically, “Self Forcing + Ours” applies Step Forcing alone, “Deep Forcing” applies Deep Forcing alone, and “Deep Forcing + Ours” applies both inference policies to the same checkpoint.
Applied individually, Step Forcing and Deep Forcing increase the Total score over the original Self Forcing baseline by 1.10 and 2.19 points, respectively. Applying both policies yields a Total score of 83.12.
Deep Forcing already supports long-horizon video generation. At 30 seconds, integrating Step Forcing modestly increases the VBench-Long Quality, Semantic, and Total scores by 0.09, 0.13, and 0.09 points, respectively.
To address this concern, beyond the qualitative 120-second results, we further evaluated Deep Forcing with and without Step Forcing on 120-second video generation. As shown in the table below, integrating Step Forcing improves all three aggregate scores: Quality, Semantic, and Total. Together, these results show improved aggregate performance at long-horizen generation, such as 120 seconds.
TokenTrim’s implementation is currently unavailable, so we include its published 30-second VBench-Long results in the table below. Both TokenTrim and Step Forcing achieve higher Total scores than Self Forcing. Step Forcing also achieves higher Semantic and Total scores than those reported for TokenTrim.
To understand the effects of clean and noisy KV representations and anchor token selection, we provide a comparison among Full Clean KV, Noisy-only Step KV, and the proposed Noisy-Clean Mixed KV under identical experimental settings (rows 1–3 of the table below). As shown, Noisy-only Step KV outperforms Full Clean KV in Quality, Semantic, and Total scores. Although the proposed Noisy-Clean Mixed KV achieves a slightly lower Semantic score than Noisy-only Step KV, it delivers higher Quality and Total scores, demonstrating a better trade-off between Quality and Semantic performance.
Furthermore, to better understand the effect of anchor selection and justify our design choice, we conduct additional ablations using (rmax,rmin)=(0.8,0.4) from Table 3 as the reference configuration, corresponding to clean-anchor ratios of ([0.8, 0.5597, 0.4]) from the first to the last denoising step. Reversing the schedule to (0.4, 0.5597, 0.8) improves the Semantic score but substantially reduces the Quality score, resulting in a lower Total score. Similarly, using a fixed ratio of (0.5865667), which matches the average ratio of the original schedule, yields lower Quality, Semantic, and Total scores. These results indicate that allocating more clean anchors to earlier denoising steps achieves a better balance between Quality and Semantic performance.