- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CORAL-KV, a KV-cache compression method for autoregressive video generation. It proposes block-DPLR that utilizes the redundancy between layers to improve the quantization performance. It introduces low-bit residual quantization and cheap predictors to capture both intra-head and inter-head correlations. Experiments on HY-WorldPlay and MAGI-1 demonstrate improved memory–fidelity trade-offs over existing methods like QVG and QVG-Pro.
Critical strengths
-
The block-DPLR predictor efficiently exploits cross-layer redundancy without introducing too much overhead.
-
The method achieves better memory-quality trade-offs than existing baselines, particularly on MAGI-1.
Critical weaknesses
-
Experiments cover only two video models, leaving the applicability to other models unclear. Meanwhile, performance gains on HY-WorldPlay are relatively small compared to the huge gains on MAGI-1.
-
The video-adapted variant introduces additional latency (especially on Hy World Play), which is much larger than QVG or QVG Pro. However, on MAGI-1, the latency overhead is on par with QVG and QVG-Pro. Doing further ablation studies on why this happens and how to control the overhead will be critical.
Questions for authors
Please refer to the weakness section. I would recommend that the authors add experiments on at least two more model families to validate the effectiveness and generalizability of their method.
5 comments
[Answer 2] The HY-WorldPlay quality gains hold across input sources and longer contexts
Building on the HY-WorldPlay quality gains demonstrated in the paper at 48 retained latent frames, we show that these gains also hold at 72 and 256 retained latent frames, alongside substantial reductions in cache storage. At 72 retained latent frames, video-adapted CORAL-KV improves video PSNR over QVG-Pro by 0.17 dB while using 32.70% less cache storage. At 256 retained latent frames, this advantage increases to 0.31 dB with 35.20% less storage. Reusable CORAL-KV likewise improves over QVG-1 by 0.53 dB with 13.70% less storage at 72 frames, and 1.19 dB with 18.20% less storage at 256 frames. These results demonstrate a quality–storage Pareto improvement: reusable CORAL-KV outperforms QVG-1, and video-adapted CORAL-KV outperforms both QVG-4 and QVG-Pro, in fidelity and cache storage at both context lengths. We report results on 32 prompts with matched BF16 references below.
| Method | 72 latent frames: Video PSNR (dB) ↑ | 72 latent frames: Stored cache (GB) ↓ | 256 latent frames: Video PSNR (dB) ↑ | 256 latent frames: Stored cache (GB) ↓ |
|---|---|---|---|---|
| QVG-1 | 15.77 | 7.99 | 16.49 | 28.42 |
| QVG-4 | 16.30 | 13.36 | 17.91 | 47.52 |
| QVG-Pro | 16.57 | 14.46 | 18.37 | 51.41 |
| CORAL-KV reusable | 16.30 | 6.90 | 17.69 | 23.26 |
| CORAL-KV video-adapted | 16.74 | 9.74 | 18.68 | 33.34 |
Across these two context lengths, both CORAL-KV variants provide higher mean fidelity with less cache storage than their respective QVG baselines, with larger advantages at 256 retained latent frames.
Quality gains on a different input source. We also evaluate 32 inputs from VBench-I2V, reusing the same predictors trained on MovieGen. On HY-WorldPlay, video-adapted CORAL-KV achieves 16.69 dB PSNR versus 16.20 dB for QVG-Pro, a 0.49 dB gain, while using 4.79% less cache storage (9.71 GB versus 10.20 GB). It also improves SSIM from 0.42 to 0.45 and reduces LPIPS from 0.46 to 0.42. These results show that the quality gains also hold on a different input source.
[Answer 3] We explain the higher adaptation cost on HY-WorldPlay and show that fewer fitting tokens reduce adaptation overhead
Why adaptation is more costly on HY-WorldPlay. HY-Worldplay fits predictors across 24 heads with 256 channels per head, compared with eight heads with 128 channels per head on MAGI. To isolate the effect of this geometry, we tested the block-diagonal fitting implementation on synthetic KV inputs with 7,040 fitting tokens. Each measurement uses three warm-ups and nine repetitions; the reported cost is the sum of the median key and value update times for one layer, normalized to MAGI's geometry.
| Geometry | Attention heads | Channels per head | Time for a key-and-value adaptation update, relative to MAGI geometry |
|---|---|---|---|
| MAGI-1 | 8 | 128 | 1.00× |
| HY-WorldPlay | 24 | 256 | 2.81× |
At the same token count and rank, an adaptation update takes 2.81× as long with HY's geometry. This is consistent with the larger generation overhead reported in Table 1: 29.10% on HY versus 10.00% on MAGI. This shows that HY-Worldplay's adaptation cost comes from having to fit more head-aligned blocks, as well as larger dimension per block.
Reducing adaptation cost with fewer fitting tokens. One way to reduce the cost of updating the block-diagonal component is to use fewer KV tokens for each fitting update. We compare using all available KV tokens in the cache block with using 2,048 token positions spaced evenly across the block. At each layer, we use the KV vectors at these positions to refit the block-diagonal component, then apply the updated predictor to the entire cache block. The shared low-rank factors and residual quantization format remain unchanged.
We measured the associated fidelity trade-off on eight fixed inputs per model, and extended the comparison to QVG-1, QVG-4 and QVG-Pro using the same inputs and BF16 references. For QVG, we fit the centroids using the same 2,048 token positions, while still compressing all KV tokens in the block.
| Method | HY: all fitting tokens PSNR ↑ | HY: 2,048 fitting tokens PSNR ↑ | MAGI: all fitting tokens PSNR ↑ | MAGI: 2,048 fitting tokens PSNR ↑ |
|---|---|---|---|---|
| QVG-1 | 14.05 | 14.03 | 25.29 | 25.16 |
| QVG-4 | 14.24 | 14.08 | 27.13 | 25.90 |
| QVG-Pro | ||||
| CORAL-KV video-adapted | 15.27 | 14.94 | 34.81 | 34.80 |
We find that using only 2,048 tokens can reduce the online adaptation overhead by 64.90% on HY and 37.40% on MAGI. Additionally, the results above show that at the same token budget, CORAL-KV outperforms all three QVG baselines in mean PSNR on both models. We use evenly spaced tokens without optimizing which tokens to select.
On MAGI, mean PSNR changes by only 0.02 dB, compared with a 2.02 dB decrease for QVG-Pro. The 2,048-token CORAL-KV configuration retains a 4.65 dB gain over QVG-Pro fitted on all tokens (95% CI: [3.17, 5.99]), with better mean SSIM and LPIPS. On HY, using fewer fitting tokens reduces mean PSNR by 0.34 dB, compared with 0.46 dB for QVG-Pro, while lowering CORAL-KV's latency overhead. This simple experiment shows that fitting on fewer tokens can retain most of the fidelity while substantially reducing adaptation time; more sophisticated token selection may further improve this trade-off and is left as future work.
We will incorporate the additional model-family evaluations and HY-WorldPlay quality–storage findings into the revised manuscript, with the detailed adaptation-cost and fitting-token ablations in the appendix.
Summary of Response to Reviewer oQvF
We thank the reviewer for the constructive feedback and, in particular, for providing concrete suggestions for addressing your concerns. In response, we have carried out the requested additional experiments and analyses, including evaluation on two additional model families and further ablations of the HY-WorldPlay gains and video-adaptation overhead. These additions directly address each of the concerns raised in the review. We organize the new evidence as follows:
We hope the new evidence addresses the reviewer’s concerns and would appreciate a reassessment in light of these results.
[Answer 1] Video-adapted CORAL-KV outperforms QVG-Pro in both fidelity and cache storage on Self-Forcing and LongCat-Video
We add evaluations of CORAL-KV on Self-Forcing DMD 1.3B and LongCat-Video 13.6B. We fit one predictor per layer for each model using 32 calibration inputs, and evaluate both models on the same 32 held-out MovieGen prompts. We compare both reusable CORAL-KV and video-adapted CORAL-KV to QVG baselines, using the same model settings and BF16 references. Both CORAL-KV configurations use the block-DPLR design with rank r=128.
For Self-Forcing, we retain 144 and 288 latent frames as context, generating 573 and 1,149 decoded frames, respectively. We compress the KV cache history in blocks of 24 latent frames. For LongCat, we use a conditioning prefix of 49 and 97 decoded frames, and generate 80 new decoded frames at each setting. We compress the conditioning prefix’s KV cache once. Consistent with our accounting for HY-WorldPlay and MAGI-1, we report memory as including predictor weights, adaptation state, quantized residuals, quantizer metadata and any uncompressed KV retained during generation.
Self-Forcing DMD 1.3B
At 144 retained latent frames, reusable CORAL-KV achieves 26.60 dB with 11.71 GB of stored cache, using the least memory among all methods while outperforming QVG-1 and QVG-4 in PSNR. Video-adapted CORAL-KV achieves the highest mean PSNR of 27.18 dB, with 11.80% less storage than QVG-Pro.
At 288 retained latent frames, reusable CORAL-KV achieves 27.07 dB, outperforming all three QVG configurations in both mean PSNR and stored-cache memory, with 21.10% less storage than QVG-Pro. Video-adapted CORAL-KV further improves fidelity to 27.82 dB, gaining 0.95 dB over QVG-Pro with 16.80% less storage. Together, these results show favorable quality–storage scaling for CORAL-KV on Self-Forcing.
LongCat-Video 13.6B
At 49 conditioning frames, reusable CORAL-KV achieves 25.15 dB, outperforming all three QVG configurations in mean PSNR while using 36.90% less storage than QVG-Pro. Video-adapted CORAL-KV achieves the highest mean PSNR of 25.61 dB, with 30.10% less storage than QVG-Pro.
At 97 conditioning frames, reusable CORAL-KV becomes the most memory-efficient method, achieving 25.50 dB with 4.55 GB of stored cache. It improves over QVG-1 by 4.27 dB with 1.90% less storage. Video-adapted CORAL-KV achieves 26.62 dB, improving over QVG-Pro by 0.94 dB with 29.90% less storage.