- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces DeCoPrune, a training-free KV-cache pruning method for autoregressive video diffusion that removes tokens whose intermediate clean predictions closely match final outputs, reusing denoising predictions at negligible cost. It also introduces CMBench, comprising 116 reference-grounded Reappear/Revisit tasks over minute-scale contexts. On LingBot World v2, DeCoPrune achieves near-FullKV recall (0.678 vs. 0.680 DINO) at ~86% pruning and a 4.1× continuation speedup, outperforming evaluated compression baselines at comparable budgets.
Critical strengths
-
The paper is easy to read and follow.
-
Novel, model-intrinsic criterion. Using step-to-final denoising discrepancy to identify tokens redundant given the retained context is an original and elegant idea. It reuses existing predictions without auxiliary models, retrieval indices, motion estimators, or additional scoring overhead.
-
Strong qualitative and quantitative results.
-
Well-motivated benchmark. CMBench directly measures contextual recall through reference-grounded evaluation. The paper demonstrates its value by showing that standard quality metrics remain saturated even as contextual recall deteriorates.
-
Convincing ablations. At matched budgets, DeCoPrune (0.670) clearly outperforms both Random (0.609) and the reverse criterion (0.451), supporting the proposed selection direction. The ablations also cleanly separate the contribution of RoPE re-indexing.
Critical weaknesses
-
Limited evidence of generalization. The main results rely on a single backbone, LingBot World v2, while validation on LongLive covers only 18 cases with 10-second contexts.
-
Benchmark and evaluation limitations. CMBench is largely synthetic: 50 of 58 episodes are generated by another video model, with only 14 real-world tasks, raising concerns about distribution bias. The OWL-ViT detection and max-over-frames DINO protocol also has limitations: a single matching frame can yield a high score, while detection failure forces the score to zero.
-
Unbounded memory and future relevance. The retained cache continues to grow with rollout length, and the criterion does not explicitly account for future relevance: a rare detail that is easy to denoise now may be discarded despite being needed later. Although both limitations are acknowledged, discussing bounded-memory or task-aware extensions would strengthen the paper.
Questions for authors
The paper presents an original and well-ablated training-free pruning criterion with strong results, but its reliance on a single backbone and a mostly synthetic self-constructed benchmark leaves generalization insufficiently established. Please refer to weaknesses above.
4 comments
Response to Reviewer viwi (2/2)
W2 — Synthetic-heavy benchmark and max-over-frames scoring.
Synthesis broadens CMBench's coverage, and most synthetic videos depict real-world settings. Generation lets us include stylized domains such as anime and game worlds and control exactly when each target appears; 38 of the 50 synthetic videos (76%) are photorealistic clips of real-world settings. The real subset checks that conclusions transfer: DeCoPrune ranks first among the pruning methods on both real and synthetic footage, and the two subsets agree on the ordering of 14 of 15 method pairs (Kendall's τ = 0.87, Table 6 in Appendix D).
The comparison holds when several frames must score well. We re-scored all methods with the mean of the top-3 and the top-5 of the 10 sampled frames (Table R2). With top-3, the ranking of all 12 methods is identical to the max ranking. With top-5, DeCoPrune ranks first among all methods, including FullKV, and its paired CIs against TempDiff, the context-free control and Random still exclude zero. We also manually reviewed all cases and found none in which OWL-ViT missed a successfully generated target, so a zero score corresponds to a genuine recall failure.
Table R2. DINO under three frame aggregations on 116 CMBench tasks (LingBot World v2). Δ = DeCoPrune − comparator (paired bootstrap 95% CI).
| Method | Max | Top-3 | Top-5 | Δ top-3 [95% CI] | Δ top-5 [95% CI] |
|---|---|---|---|---|---|
| FullKV | 0.6935 | 0.6583 | 0.6080 | −0.0069 [−0.0436, +0.0310] | +0.0027 [−0.0342, +0.0402] |
| DeCoPrune | 0.6865 | 0.6514 | 0.6107 | — | — |
| DensityKV | 0.6781 | 0.6427 | 0.6064 | +0.0087 [−0.0208, +0.0398] | +0.0043 [−0.0250, +0.0359] |
| TempDiff | 0.6328 | 0.5934 | 0.5503 | +0.0580 [+0.0226, +0.0946] | +0.0604 [+0.0262, +0.0961] |
| Context-free | 0.6267 | 0.5883 | 0.5548 | +0.0631 [+0.0362, +0.0917] | +0.0559 [+0.0294, +0.0842] |
| Random | 0.5976 | 0.5622 | 0.5188 | +0.0892 [+0.0558, +0.1233] | +0.0920 [+0.0605, +0.1251] |
| Future Forcing | 0.5969 | 0.5582 | 0.5242 | +0.0932 [+0.0637, +0.1243] | +0.0866 [+0.0574, +0.1175] |
| WorldKV | 0.5542 | 0.5209 | 0.4864 | +0.1305 [+0.0878, +0.1751] | +0.1243 [+0.0817, +0.1685] |
| SnapKV-style | 0.5533 | 0.5205 | 0.4931 | +0.1309 [+0.0959, +0.1672] | +0.1176 [+0.0828, +0.1537] |
| ForcingKV | 0.5461 | 0.5127 | 0.4786 | +0.1387 [+0.0921, +0.1884] | +0.1321 [+0.0877, +0.1798] |
| DummyForcing | 0.4538 | 0.4240 | 0.3960 | +0.2274 [+0.1876, +0.2679] | +0.2147 [+0.1768, +0.2543] |
| Streaming | 0.4488 | 0.4196 | 0.3968 | +0.2318 [+0.1925, +0.2729] | +0.2139 [+0.1765, +0.2533] |
W3 — Unbounded memory and future relevance.
Rare details tend to be retained rather than discarded. When the context provides little evidence about a rare detail, the model is uncertain about its appearance, so its tokens have high denoising discrepancy and are kept. A detail is easy to denoise mainly when the cache already contains it.
We agree that the current setting has no fixed memory budget. DeCoPrune reduces the KV retained per chunk, but the cache still grows with rollout length. Because every token has a score, a bounded variant is direct: keep a short recent window and evict the lowest-scoring older tokens whenever the cache reaches a fixed budget. Where task cues are available, such as a requested viewpoint in a camera-conditioned world model, the eviction policy can protect the tokens likely to be needed next, in the spirit of WorldMem (arXiv:2504.12369) and Context-as-Memory (arXiv:2506.03141). We will state this limitation and discuss both extensions in the revision.
Thank you for the rebuttal, which addressed my concerns and I belive most of the other reviewer's concerns. I have decided to maintain my score.
Thank you for taking the time to read our rebuttal and follow up. We’re glad it addressed your concerns and appreciate your constructive feedback.
Response to Reviewer viwi (1/2)
We are grateful for the reviewer’s positive assessment and helpful comments. We respond to the concerns below.
W1 / Q1 — Generalization beyond a single backbone.
We use LingBot World v2 because it is the only backbone we tested that actually recalls content from a one-minute context. Even with FullKV, LongLive, Self-Forcing and Causal Forcing score about 0.26 DINO, compared with about 0.68 for LingBot (Appendix C). With such a weak unpruned reference, the effect of pruning cannot be isolated. The denoising signal behind our criterion holds on all four backbones: in Fig. 2(b), each shows lower discrepancy when the relevant content is already in the history.
Following your suggestion, we expanded the 10-second LongLive evaluation from 18 cases to all 116 CMBench tasks. Each task uses the clip that contains its annotated reference frame, with the original continuation prompt, target, reference box and scoring. We widened the budgets of the four LongLive cache policies so that all methods run at a comparable pruning ratio (62–65%). At 62.38% pruning, DeCoPrune scores 0.5460 DINO versus 0.5486 for FullKV, and it scores 0.0558–0.0874 higher than all four policies, with every paired 95% CI excluding zero (Table R1). TempDiff is close to DeCoPrune here because, with only a 10-second context, the target lies a few chunks back and any budget-matched selector tends to keep it; on minute-scale LingBot contexts, DeCoPrune outperforms TempDiff by 0.0537 (paired 95% CI [+0.0181, +0.0902]).
Table R1. LongLive, CMBench (116 tasks, seed 2), 10 s context. All compression methods run at a comparable PR (62–65%). Δ = DeCoPrune − comparator (paired bootstrap 95% CI).