- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes a training-free VQ-like (due to residual learning) method for compressing the KV-cache used by autoregressive video generators. Since the method is vector-quantization based, unlike traditional quantization, it by definition has the ability to provide sub-bit quantization capability. Although sounds like an interesting approach, the paper fails to properly evaluate and motivate the proposed approach, barring aside the robustness and the generalizability of the proposed approach.
A summary of the approach is:
a. For all the KV vectors in the cache, reconstruct them based on a persistent base + learned residuals.
b. The persistent bases are heuristically adaptive in order to maintain a budget Q.
c. The learned residuals are also heuristically adaptive in order to maintain the same budget Q.
d. Both b. and c. are chunkwise heuristically adapted depending on the recency of the chunks.
In short, it's a delicately engineered and handcrafted approach.
Critical strengths
None
Critical weaknesses
-
The biggest critical weakness, I believe, is that the main motivation of this paper is quite cloudy. First of all, KV-caching is not really a thing for the native Video Generation (text / image conditioned or otherwise) models. This is primarily because they are all Diffusion based and not Autoregressive, unlike LLMs where it's an indispensable component. Two of the main experiments that this paper does are video generation which are not very meaningful. The HY-WorldPlay 8B experiments make sense, being a camera, action conditioned world model, but again, in this setting the chosen baselines are biased. Overall, the baselines are biased towards: "Retain all KVs, and quantize them all", which would have been fair, if SuBitKV also did the same, but as mentioned earlier in the summary, they have a heuristic soft dropping of KV entries from further past. That's why two straightforward cache-compression baselines are quite important to compare against. First, Equal-memory BF16 recent-window; and second uniformly sampled historical-chunk baseline. Without this, the complexity and the trouble of incorporating SubitKV is not justified.
-
Highly underspecified setting of the Wan2.2 5B model, which is used quite heavily in all the experiments of the paper (also in motivation / analysis). The native generation of this model is bidirectional single chunk denoising for 31 latent / 121 RGB frames. How exactly was this native DiT transformed into your causal chunkwise generation setting of 11 latent chunk generation with 4 frames per chunk leading to 44 latent / 173 RGB frames? This setting is vastly different from the original model, and without significant post-training, similar to self-forcing, cannot lead to any reliable analysis and thus conclusions. It's imperative to specify this properly in the paper. If the model was indeed post-trained, then the claims of being a training-free method are not valid.
-
Apart from the Wan 2.2 5B setting, just everywhere the paper is so underspecified. No details are provided in so many places. For instance Figure 2. What exactly is being computed here? What is the BF16 baseline? If BF16 baseline is just native generation, then why does it even have any temporal jumps? Are you operating on a non-post-trained-causalified DiT backbone? If yes, then all this analysis is invalid. This should be done on something like Self-Forcing (or CausVid). Figure 3: Which model's which layer's KV vectors are being visualized here? Which calibration prompts? These are so cryptic, that it's impossible to distill meaningful information. Same with Figure 5, which residual energy is being referred to here? Where is that defined?
-
Barring aside all these problems of the paper, one of the main thing stands out is the actual SuBit compression quality is extremely bad. For instance Wan 2.25B for Q=0.2 ~0.747 bits/scalar and Q=0 ~0.532 bits/scalar, the PSNRs are 18.32 and 16.30, which is absolutely non-usable settings. For training-free methods the VBench is not really a correct metric, because it operates on semantics, and in this case the visual fidelity is what matters. What makes this even worse is that zero (I mean absolute zero) qualitative samples have been provided. Neither in the supplementary material nor in the paper to demonstrate the before / after generation quality through figures for instance. Without this signal, it is impossible to judge the validity of the proposed paper.
-
Finally, going full circle, the motivation of this paper is absolutely unclear. Of course for something like world models, we could understand their use, but world-models are evolving to preserve explicit state that renders techniques like KV-caching unusable. All the motivation related analysis that this paper builds on top of (which is very confusing and underspecified in itself), is based on Wan 2.2 5B text-to-video generation. Since, no robust long-video generation AR-rollout candidate exists for this model, there is no clarity on exactly what problem this paper is trying to solve, and where it wishes to apply itself.
Other Minor issues:
- Figure 4 is confusing. Stage 1 shows computation of residuals, this confuses with the residuals of Stage 2. would be suggested to clearly delineate the two separate operations.
- The explanation of the Time-Adaptive allocation (B(t)) would be better prosed through an algorithm or pseudocode, instead of the current math.
Justification: Given the current state the paper is in, it's unclear motivation, and underspecified experiments, I recommend a rejection. It might be possible to work on this idea, address the weaknesses mentioned above, have better grounded motivation, in order to bring it into a good impactful state for publication. But this effort is way out of scope of the current submission, and thus I recommend reject.
Questions for authors
Would be great if you could clarify the settings for the Wan2.2 5B and the Self-Forcing 1.3B model's causal chunkwise generation. Could help understand the soundness of the experiments a bit better.
16 comments
Weakness 2: Setting Clarafication
Clarification on experimental settings
We use the Wan2.2-5B (WorldPlay AR-distilled) checkpoint, with causal attention across chunks and persistent historical KV, instead of bidirectional attention of vanilla Wan2.2-5B. SuBitKV compresses the historical KV using offline codec calibration, with no additional generator fine-tuning.
| Setting | Wan2.2-5B | Self-Forcing 1.3B |
|---|---|---|
| Base | Wan2.2-TI2V-5B | Wan2.1-T2V-1.3B |
| Post-trained checkpoint | WorldPlay wan_distilled_model/model.pt | Self-Forcing self_forcing_dmd.pt, EMA weights |
| Attention / generation | Causal across chunks; persistent historical KV | Causal across chunks; persistent historical KV |
| Latent layout | 11 chunks × 4 frames = 44 | 10 chunks × 3 frames = 30 |
| RGB output | 173 frames, 1280 × 704 | 117 frames, 832 × 480 |
Weakness 1: Precision reduction is not KV eviction.
SuBitKV compresses the full historical KV context, retaining a representation of every historical chunk for attention. All original compression baselines use the same full-history KV retention policy, inputs, and generation settings.
We additionally evaluated the requested eviction baselines using Wan2.2-5B (WorldPlay AR-distilled) on ten matched scene/seed pairs across three revisit gaps, under comparable memory budgets. Eviction methods retain four BF16 chunks, while SuBitKV preserves compressed representations of up to ten historical chunks.
KV memory is measured as the average final cache size over 30 runs per method, including codec data and metadata but excluding workspace and model weights.
| Method | KV memory (GiB) ↓ | Gap 1: LPIPS ↓ / SSIM ↑ / DINO ↑ | Gap 2: LPIPS ↓ / SSIM ↑ / DINO ↑ | Gap 3: LPIPS ↓ / SSIM ↑ / DINO ↑ |
|---|---|---|---|---|
| BF16 Full | 12.09 | – / – / 0.805 | – / – / 0.782 | – / – / 0.737 |
| SuBitKV Q=4 | 3.92 | 0.212 / 0.7390 / 0.8054 | 0.194 / 0.770 / 0.786 | 0.1848 / 0.7884 / 0.7479 |
| BF16, recent-only | 4.83 | 0.303 / 0.607 / 0.753 | 0.300 / 0.619 / 0.710 | 0.308 / 0.617 / 0.659 |
| BF16, uniform-history | 4.83 | 0.280 / 0.635 / 0.774 | 0.276 / 0.657 / 0.786 | 0.242 / 0.689 / 0.746 |
SuBitKV achieves better LPIPS and SSIM across all three revisit gaps while using 19% less KV memory than the eviction baselines. Its DINO scores are also comparable to or slightly higher than full-history BF16, demonstrating that preserving compressed historical information can be more effective than retaining fewer chunks at a larger memory cost.
Weakness 1,5 Clarification on motivation
Weakness 1,5 Clarification on motivation
Thank you for your insights and suggestions.
Our motivation is that KV caching is widely used in modern autoregressive video generation and interactive world models. As generation length increases, growing KV caches increase memory usage and inference latency, motivating KV compression to improve efficiency while maintaining generation quality.