- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces an inference-time acceleration method that interpolates the residual (changes) after the DiT block. Specifically, after block i, the delta of the anchor feature (net changes from the input of block i) is used to interpolate the non-anchor (skipped frames) delta. The paper analyzes when residual reconstruction reduces local error and why minimizing that error alone can suppress frame-specific variation. Experiments across several video diffusion models evaluate generation fidelity, efficiency, reconstruction choices, and implementation safeguards, including comparisons with step reduction and caching.
Critical strengths
-
Simple yet effective and well-motivated novel method. Preserving an already available input and reconstructing only the delta introduced by model block is a clear design choice. Table 1 isolates this change under the same FIS-DiT anchor, dense-block, and dense-step settings, with identical PFLOPs and nearly identical latency. The proposed method has minimal performance degradation compared with dense inference and strong speedup. It is also compatible with SVG to further speed up with similar, sometimes better, performance
-
Solid ablations and analysis. Equations and propositions provide solid analysis, and the experiments are thorough and complete in analyzing the effect of each component.
Critical weaknesses
- Weak motion may favor the evaluation benchmark. The issue is (and the author also acknowledged) evaluation benchmark can possibly tend to be better when motion is weaker. Table 7 and some visual examples show that the motion is sometimes weaker than the dense baseline. I would like to see evaluation metrics where the author can somehow cap the motion (filtering based on motion). This can rule out the effect of motion strength.
Questions for authors
See weakness. I would like to see an evaluation under a motion filter (generate a lot of videos, filter, and regenerate until a fixed number of videos, etc). I will preserve the WA rating if the evaluation is provided.
3 comments
Thanks the author for response. My concerns are mostly addressed (though I believe it is worth investigating on solving the motion degradation issue) and I will maintain the score of 3.
Thank you for your thoughtful feedback and continued support for our work!
We fully agree that mitigating motion loss warrants further investigation, and we have highlighted it as the primary open problem in the revised manuscript (Appendix K, Future Work). Specifically, the lost motion resides in the frame-specific component of the block residual, which cannot be recovered through interpolation or simply by minimizing squared error. We therefore plan to treat skipped-frame reconstruction as conditional generation rather than regression.
Response to Reviewer zmp1
W1: We thank the reviewer for this suggestion. We ran the evaluation under a motion filter on all VBench dimensions of the three models, comparing videos of the same prompt with nearly equal RAFT motion. At matched motion, residual reconstruction scores higher than FIS-DiT on all ten dimensions of both I2V models, and imaging and aesthetic quality on Wan2.2 show no drop whose 95% interval excludes 0, nor do the eight semantic and style dimensions of Wan2.2 T2V. The consistency gains of BlockDelta shrink, suggesting that part of them comes from weaker motion, as the reviewer suspected.
The table below gives the VBench scores of BlockDelta minus dense inference, over all videos / over the RAFT-matched pairs. The matched pairs come from a subset of the prompts, for example 82%, 54% and 77% of them for dynamic degree on the three models. * marks a 95% interval that excludes 0, and imaging quality is on a 0–100 scale.
Appendix I.2 of the revised paper adds these results for both motion measures and three tolerances, with the check on different seeds.