- Rating
- 3
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates quality degradation caused by low-bit KV-cache quantization in autoregressive video models. It finds that quantization errors shift attention mass toward cached keys, which the paper calls Jensen bias. Based on this observation and a mathematical derivation, the authors propose a bias correction method. Experiments on three video models demonstrate consistent quality improvements under INT2 quantization.
Critical strengths
- The core observation is intuitive and insightful. Starting from the mathematical formulation of attention, the paper identifies a systematic bias introduced by key quantization and derives a corresponding correction. The progression from observation to analysis and solution is concise and elegant.
- The method demonstrates consistent quality gains across three autoregressive video models and multiple quantization schemes. It requires no retraining, making it easy to integrate.
Critical weaknesses
- A key motivation for KV-cache compression is to retain more historical context under a fixed memory budget, which is particularly important for long-video generation. However, the evaluation is limited to 7–10-second videos, leaving the method’s effectiveness over substantially longer generation horizons unclear.
- The reported end-to-end overhead is estimated rather than directly measured in the full generation pipeline. A direct end-to-end latency comparison would strengthen the practical efficiency claims.
Questions for authors
- Could the authors report results on substantially longer video sequences?
- Could the authors report directly measured end-to-end generation latency comparison?
3 comments
Response to Reviewer sPWd (Additional results)
As promised in our initial response, we have evaluated our correction on substantially longer videos.
Setup. We generated 30 s videos (720 frames, three times the longest videos in the paper) with MAGI-1 for all 40 long-form prompts of VBench’s long-prompt list (competitions/long_prompt_list.txt), none of which were part of our original evaluation. All variants use identical prompts and seeds, so comparisons are paired per prompt. Quantization is INT2 RTN with group size 32, with and without our correction. We also report a BF16 reference and an uncorrected INT4 baseline with the same RTN quantizer and group size. We use MAGI-1's default KV-range schedule.
Over long horizons, quantization slightly perturbs the autoregressive trajectory so frames drift out of alignment with the BF16 reference. This drift compounds over the video length, so frame-aligned PSNR, SSIM and LPIPS become increasingly unreliable, and can even favor blurry frames over sharp ones that are slightly displaced, a known limitation of pixel-aligned metrics for generated video (Mathieu et al., 2016; Denton & Fergus, 2018; Unterthiner et al., 2018). We therefore use VBench-Long’s reference-free evaluation protocol. Over the full 30 s videos, we evaluate frame-wise imaging and aesthetic quality, motion smoothness, dynamic degree, and subject- and background-consistency, which VBench-Long scores as a blend of within-clip and across-clip (long-range) consistency.
These are the dimensions VBench-Long supports for prompts outside its standard prompt suite: its semantic dimensions (e.g. object class, color, spatial relationship) rely on per-prompt annotations that exist only for the standard suite, not for the long-prompt list.
Mean ± standard error over the 40 prompts (VBench scale, 0–100):
| Variant | Imaging | Aesthetic | Motion smoothness | Dynamic degree | Subject cons. | Background cons. |
|---|---|---|---|---|---|---|
| BF16 | 62.32 ± 1.93 | 61.97 ± 1.39 | 99.33 ± 0.08 | 20.68 ± 5.22 | 98.32 ± 0.30 | 97.59 ± 0.23 |
| INT2 | 52.97 ± 1.81 | 53.23 ± 1.34 | 98.91 ± 0.20 | 5.81 ± 2.17 | 98.55 ± 0.21 | 97.36 ± 0.15 |
| INT2 + corr. | 60.82 ± 2.09 | 60.63 ± 1.21 | 99.37 ± 0.07 | 16.41 ± 4.47 | 98.33 ± 0.27 | 97.49 ± 0.21 |
| INT4 | 57.00 ± 2.06 | 53.54 ± 1.38 | 98.79 ± 0.17 | 5.30 ± 2.40 | 98.38 ± 0.26 | 98.14 ± 0.14 |
At 30 s, our correction recovers most of the quality lost to INT2 quantization: about 84% of the imaging-quality gap and 85% of the aesthetic-quality gap to BF16. It also restores motion: uncorrected INT2 and INT4 videos frequently degrade into heavily distorted frames that often remain largely static, which lowers dynamic degree (5.8 and 5.3 vs. 20.7 for BF16). The correction largely recovers it (16.4), and motion smoothness returns to the BF16 level. Corrected INT2 also outperforms uncorrected INT4 on imaging and aesthetic quality while using about 42% less KV-cache memory (2.75 vs 4.75 effective bits per element). The table below gives per-prompt paired tests of corrected against uncorrected INT2 (exact two-sided Wilcoxon signed-rank test, Holm-corrected across the five dimensions).
| Dimension | Prompts improved | p | Holm-corrected p |
|---|---|---|---|
| Imaging | 32/40 | 1.7×10⁻⁵ | 6.7×10⁻⁵ |
| Aesthetic | 36/40 | 2.0×10⁻⁷ | 1.0×10⁻⁶ |
| Motion smoothness | 31/40 | 2.4×10⁻⁴ | 7.1×10⁻⁴ |
| Subject cons. | 24/40 | 0.88 | 0.88 |
| Background cons. | 23/40 | 0.35 | 0.70 |
Imaging quality, aesthetic quality and motion smoothness improve significantly. Subject and background consistency are saturated for all variants and show no significant difference. VBench reports a known trade-off between these consistency dimensions and dynamic degree, as less dynamic videos score higher on consistency (Huang et al., 2024). The slightly higher consistency of uncorrected INT2 and INT4 is in line with their low dynamic degree (5.8 and 5.3 vs. 20.7 for BF16).
We conclude that our approach remains effective for longer videos. We will add these results to the main manuscript in Section 5, together with qualitative frames from the 30 s videos (BF16, INT2, INT2 with correction, INT4) in the appendix.
I thank the authors for their detailed response and additional experiments.
The results on longer video sequences and the end-to-end latency measurements provide further support for the paper’s conclusions. I encourage the authors to include these results in the revised manuscript.
Overall, the authors have adequately addressed my main concerns. I maintain my rating of 3 (Weak Acceptance) and raise my confidence score.
Response to Reviewer sPWd
W1/Q1: Longer video sequences
Reply: We thank the reviewer for this point. We agree that KV-cache compression matters substantially for long-horizon generation.
"Longer videos." We are currently checking whether an evaluation on substantially longer videos is feasible with our compute resources. We will come back to the reviewer on this point within the next few days.
"More context at the same memory budget." KV-cache compression can also enable longer context windows without increasing memory usage. The cache footprint grows with the window size and the bits per element. At equal window size, INT2 with our correction matches or exceeds uncorrected INT4 in PSNR, SSIM and LPIPS (Section 5.3). INT2 can therefore use a window approximately twice as long as INT4 at the same footprint. The evaluated models limit how much past video they attend to: MAGI-1 and SkyReels-V2 use sliding windows, while HY-WorldPlay selects a fixed number of past latent frames. Although KV-cache compression could allow these models to retain more historical context at the same memory budget, evaluating whether this improves generation quality would require repeating the experiments with larger context windows, including the BF16 references. The computational cost makes such an evaluation impractical within the rebuttal period, so we will discuss it as future work in the revised paper.
W2/Q2: Measured end-to-end latency
Reply: Since our correction only modifies the attention computation, we deliberately measured the correction in isolation, at the attention kernel, to give a detailed picture of its cost across all attention shapes. The 1.8% stated in Section 4.3 is an estimate derived from this kernel measurement and the attention share of the baseline pipeline.
In response to the reviewer's request, we also measured the end-to-end generation latency directly. With our fused Triton kernels (Appendix C.4) integrated into the complete MAGI-1 pipeline, generating a 10 s video takes on average 702.9 s without and 708.8 s with the correction on an NVIDIA A100, an increase of 0.84%. We will add these measured numbers to Section 4.3.