- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The main idea of this paper is to have the video model spend more compute where a simple average is not good enough. The proposed method also makes the history cache smaller by keeping an accurate summary and compressing the finer details. These changes help make inference cheaper with as little loss in quality as possible.
Critical strengths
- The paper addresses a fundamental problem with clear motivation and a well-grounded formulation. Rather than following conventional technical approaches, it offers a fresh perspective by focusing on new types of errors introduced by acceleration.
- The experimental results and qualitative visualizations provide sufficient evidence to support the effectiveness of the proposed method.
Critical weaknesses
- The paper would benefit from clearer and more accessible writing, particularly for readers who are not specialists in this area. Streamlining the mathematical exposition, providing more intuitive explanations, and explicitly highlighting the differences from existing methods would make the paper easier to follow.
- The proposed method is somewhat complex, and its generalizability warrants further investigation. Additional evaluations across different models and settings would help assess whether its benefits extend beyond the configurations considered in the paper.
Questions for authors
None.
4 comments
W2 Generalizability across models and settings
We have added two complementary evaluations.
First, we test longer autoregressive continuation using the 480p InfinityStar-Interact checkpoint. Each method generates five successive five-second continuations from a common reference and fixed continuation prompt, feeding its own latent history into the next round. This evaluates 25 newly generated seconds.
InfinityStar-Interact: complete 25-second continuation latency. Lower time and higher speedup and retained-history compression are better.
\text{Method} & \text{Time (s)} & \text{Speedup} & \text{History comp.} \\\\ \hline \text{Dense} & 141.92 & 1.00\times & 1.00\times \\\\ \text{OSR} & 97.90 & 1.45\times & 1.00\times \\\\ \text{Dense + RIQ} & 143.63 & 0.99\times & 5.37\times \\\\ \text{OSR + RIQ} & 99.49 & 1.43\times & 5.37\times \end{array}$$ The complete-continuation VBench dimensions are: InfinityStar-Interact: VBench dimensions for complete 25-second continuations (higher is better). $$\begin{array}{lrrrr} \text{Dimension} & \text{Dense} & \text{OSR} & \text{Dense + RIQ} & \text{OSR + RIQ} \\\\ \hline \text{Subject consistency} & 93.227 & 92.793 & 94.422 & 93.770 \\\\ \text{Background consistency} & 96.460 & 96.542 & 97.308 & 97.043 \\\\ \text{Motion smoothness} & 99.196 & 99.196 & 99.157 & 99.194 \\\\ \text{Aesthetic quality} & 67.181 & 67.353 & 69.463 & 70.069 \\\\ \text{Imaging quality} & 71.210 & 70.012 & 71.580 & 69.484 \\\\ \text{Text consistency} & 27.838 & 27.224 & 28.171 & 27.737 \end{array}$$ OSR + RIQ achieves 1.43× speedup and 5.37× retained-history compression, with similar subject/background consistency and motion smoothness to Dense. Aesthetic quality increases while imaging quality decreases, making the quality trade-off explicit. This evaluation is included in Appendix C.10, Tables 13–14. Second, we evaluate transfer to LongCat-Video, an independent Diffusion Transformer backbone with autoregressive video continuation. We use 20 paired inputs at 832 × 480 and 15 fps. Both methods share a Dense-generated 93-frame initial clip. Each of ten continuations uses the most recent 73 frames as conditioning and generates 20 more frames, for 200 new frames and 293 total frames (approximately 13.33 and 19.53 seconds, respectively). The inference configuration uses CFG 4.0 and nominally 50 steps, with high-frequency enhancement resulting in 56 denoising calls per continuation. LongCat-Video: latency per continuation. Lower time and higher speedup and KV-cache compression are better. $$\begin{array}{lrrr} \text{Method} & \text{Time (s)} & \text{Speedup} & \text{KV comp.} \\\\ \hline \text{Dense} & 166.95 & 1.00\times & 1.00\times \\\\ \text{OSR + RIQ} & 118.78 & 1.41\times & 5.54\times \end{array}$$ LongCat-Video VBench dimensions (higher is better). $$\begin{array}{lrr} \text{Dimension} & \text{Dense} & \text{OSR + RIQ} \\\\ \hline \text{Subject consistency} & 96.09 & 95.26 \\\\ \text{Background consistency} & 96.11 & 94.95 \\\\ \text{Motion smoothness} & 99.29 & 99.16 \\\\ \text{Aesthetic quality} & 65.31 & 63.38 \\\\ \text{Imaging quality} & 72.10 & 72.51 \\\\ \text{Text consistency} & 29.37 & 29.35 \end{array}$$ This independent-backbone comparison yields 1.41× speedup and 5.54× KV-cache compression on the tested continuations. Motion smoothness and text consistency change by only -0.13 and -0.02 points; imaging quality increases by 0.41 points. Subject consistency, background consistency, and aesthetic quality decrease by 0.83, 1.16, and 1.93 points. We present this as evidence of transfer feasibility under a distinct generation architecture, while keeping the observed quality trade-offs explicit. The revised PDF contains the longer InfinityStar-Interact continuation experiment.I thank the authors for their rebuttal. They have addressed my concerns, and I have decided to maintain my original score.
Thank you for following up and for taking the time to review our responses. We are glad that our rebuttal has addressed your concerns, and we appreciate your constructive feedback throughout the discussion.
Thank you for the suggestions on clarity and generality. We have expanded the intuitive explanations in the main text and added longer autoregressive continuation experiments in Appendix C.10. We additionally report a transfer test on LongCat-Video in this response.
W1 Clearer and more accessible writing
We have added explanations at the points where the mathematical quantities and design choices are introduced, while preserving the original formulation.
Introduction: We now explain pooled correction directly: an omitted block is represented by its mean key and value, retaining an inexpensive estimate of its contribution despite skipping token-level computation.
Method overview and Section 3.1: We add a short roadmap linking visibility, residual decomposition, routing, and cache representation. We also distinguish SSA’s native visibility mask from the further reduction in computation performed by OSR.
Section 3.2: We explain the block mean and token-count factor, then interpret the two residual terms separately: missing value detail and excess normalization mass. A simple example shows that equal logits within a block make pooling exact even when the block receives substantial attention. This illustrates why attention mass alone is insufficient.
Section 3.3: We clarify that the content and excess-mass terms estimate what pooling misses, while the mass guard provides conservative protection against underestimation by the compact proxy.
Section 3.4: We explain why cached blocks must support both pooled and token-level reads and clarify the complementary roles of reconstruction and interaction-aware key fitting. Key reconstruction error indirectly bounds perturbations of both the K–V cross-moment and key covariance, whereas interaction fitting weights key errors according to their interaction with value residuals to more directly target cross-moment distortion. The reconstruction regularizer retains indirect control of covariance perturbations.
Related Work: We add SparSTAR and clarify the distinction between its scale-specific block selection and our combination of post-correction routing with history representation.
These changes are incorporated into the revised ten-page main paper and marked in blue. They provide intuition alongside the equations without expanding the technical formulation.