- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Forcing-KV, a training-free KV cache compression method for autoregressive video diffusion. It distinguishes static heads that emphasize local transitions from dynamic heads that use historical context for motion and consistency. The method combines offline head profiling, structured retention of the transition anchor frame, and similarity-based pruning of historical segments. Experiments across several models and generation lengths report reduced cache usage and faster inference while broadly maintaining generation quality.
Critical strengths
The paper connects attention patterns to functional behavior through targeted context-masking experiments, rather than relying solely on visualizations, which provide an interpretable motivation for assigning different compression strategies to different heads.
Critical weaknesses
-
One of my major concern is that the paper's novelty appears very limited. The approach largely extends existing head-wise KV cache compression with additional retention and pruning rules. These changes seem incremental, and the conceptual distinction from closely related work is not sufficiently clear. While the empirical gains are useful, they do not by themselves establish a substantial methodological contribution.
-
The sources of improvement over Dummy Forcing could be isolated more clearly. The comparisons at similar speeds in Tables 1–2 and the cache-removal ablations provide substantial evidence for the complete method. However, the explanation for its improvement over Dummy Forcing is less directly tested: Dummy Forcing already includes predecessor context for its dummy heads through Packed Attention Forward, while the two methods also differ in head classification and cache allocation. The profiling study does not isolate the effect of including transition-frame attention, and Figure 7 changes both selection criterion and pruning granularity. The paper explicitly credits both similarity and segment structure, but their separate contributions remain unclear. A profiling control at comparable cache budgets and pruning comparisons with the same segment granularity would strengthen this explanation.
-
The chunk-discontinuity metric needs stronger validation. It seems that Appendix B averages the K−1 largest adjacent-frame optical-flow quantities δ_t defined in Equation (5), where K is the number of chunks, and divides by the mean of δ_t over all adjacent-frame pairs, without using boundary locations. Although this ratio is invariant to uniform rescaling, naturally uneven motion can still affect it independently of boundary artifacts. Figure 8 provides illustrative examples, but does not establish this distinction across a broader set of videos. The user study offers useful supporting evidence, although its Visual Quality, Dynamic Degree, and Consistency criteria (Appendix D) do not directly validate the proposed metric. This matters because the metric supports both the functional interpretation of static heads and the claimed continuity advantage over Dummy Forcing.
-
The use of a single prompt for model-level offline head classification is insufficiently justified. Although Section 2.3 provides evidence of relatively stable attention patterns across prompts, it does not directly establish that the resulting head assignments and downstream performance are robust to the choice of calibration prompt. A prompt that underrepresents a head’s dependence on historical context could cause it to be classified as static, leading to excessive pruning in subsequent generations. A theoretical analysis clarifying when head assignments remain stable across prompts would strengthen this model-level assumption. Without such justification, the method’s generalizability to unseen prompts and effectiveness in practical applications remain insufficiently supported.
Questions for authors
-
Could the authors isolate the effect of including transition-frame attention in profiling by comparing settings with comparable cache budgets and otherwise unchanged pruning rules? For dynamic pruning, could similarity-based selection be compared with random and uniform selection using the same segment granularity? Reporting continuity, consistency, overall quality, and latency together would help establish which design accounts for the gains.
-
How often do the K−1 largest δ_t values selected over the entire video in Equation (6) occur near chunk boundaries? Could the authors provide a boundary-aligned evaluation or a controlled comparison that separates natural motion changes from injected boundary artifacts? Evidence that the resulting rankings agree with human judgments of transition quality would substantially strengthen this part of the evaluation.
6 comments
Initial Response to Reviewer mA2s (Part 2/3)
W2 & Q1: Separating transition-aware profiling, isolating the improvements over Dummy Forcing, and comparing the pruning methods with the same segment granularity.
- Transition frame attention in profiling.
Including transition attention in profiling has a significant effect. Specifically, we compare the profiling of Forcing-KV w/ and w/o the transition attention using the same prompt, and evaluate the results on 80 LongLive videos of 30-seconds.
R3-Table-1: Effect of including transition frame attention in profiling.
| Static Heads | Dynamic Heads | VBench Score ↑ | Dynamic Degree ↑ | Consistency ↑ | Chunk Discontinuity (Original) ↓ | Chunk Discontinuity (Boundary) ↓ | |
|---|---|---|---|---|---|---|---|
| w/ Transition Frame Attention | 101 | 259 | 81.05 | 37.50 | 97.83 | 2.53 | 1.41 |
| w/o Transition Frame Attention | 64 | 296 | 80.89 | 36.17 | 97.85 | 2.69 | 1.47 |
Including transition attention improves total score from 80.89 to 81.05 and dynamic degree from 36.17 to 37.50. Removing the transition anchor frame from profiling results in fewer heads being classified as static, and causes discontinuity to rise from 2.53 to 2.69. The continuity scores are close because dynamic heads may also retain anchor. Notably, under the current policy, excluding transition frame attention yields a larger KV cache budget because more heads are classified as dynamic and retain more cached tokens. Despite the larger budget, w/o transition frame achieves lower performance, supporting the effectiveness of the transition frame.
- Isolating the improvements over Dummy Forcing.
The improvements over Dummy Forcing stem primarily from three aspects.
-
Accurate head classification grounded in empirical observations. Dummy Forcing categorizes heads into sink, dummy, and global heads, each with a more narrow compression strategy. It also relies on a small subset of attention rows as proxy for online profiling. Through detailed empirical analysis, we identify two head categories and incorporate transition frame attention into profiling, improving performance as reflected in R3-Table-1. We also adopt offline profiling with full attention prior.
-
Explicitly preserving the transition anchor frame. Although Dummy Forcing uses Packed Attention Forward for dummy heads, its classification already overlooks some heads that requires transition context, and its sink head policy does not preserve the transition anchor frame. As a result, the transition frame is not sufficiently protected as our method.
-
Dynamic similarity pruning. Our pruning strategy for dynamic heads further exploits redundancy across the full history, freeing up additional KV cache budget.
- Comparison using the same segment granularity.
Our similarity-based selection consistently outperforms random and uniform segment selection at matching budgets. Specifically, we compare random, uniform, and similarity segment selection using the same segment partition and token budgets. Under each budget, the latency is nearly identical since similarity computation iccurs minimal overhead as shown in Figure 11 of our manuscript.
R3-Table-2: VBench score, dynamic degree (DD) and consistency at the same token budgets. Consistency is the mean of subject consistency and background consistency. Results are evaluated across 4720 video of LongLive, 5-second.
| Token Budget | 1,560 | 1,560 | 1,560 | 3,120 | 3,120 | 3,120 | 4,680 | 4,680 | 4,680 |
|---|---|---|---|---|---|---|---|---|---|
| Selection Method | VBench | DD | Consistency | VBench | DD | Consistency | VBench | DD | Consistency |
| Similarity Segment | 83.19 | 45.00 | 96.62 | 83.20 | 45.28 | 96.62 | 83.29 | 46.95 | 96.60 |
| Random Token | 82.92 | 40.56 | 96.59 | 82.91 | 40.28 | 96.60 | 82.89 | 40.00 | 96.59 |
| Uniform Token | 83.02 | 42.50 | 96.63 | 83.00 | 42.50 | 96.64 | 83.00 | 42.22 | 96.64 |
| Random Segment | 83.18 | 42.78 | 96.75 | 83.14 | 41.94 | 96.75 | 83.16 | 42.50 | 96.74 |
| Uniform Segment | 83.14 | 42.50 | 96.74 | 83.18 | 43.06 | 96.73 | 83.16 | 43.61 | 96.75 |
The results further isolate the effect of segment and similarity pruning. Specifically, the segment methods generally improve upon their token-level alternatives, supporting the usefulness of contiguous grouping. Crucially, similarity-based selection also outperforms random and uniform selection at the same segment granularity and different budgets. Our method improves dynamic degree and total score over random and uniform segments, and has the highest total score at all three budgets (up to 83.29). The consistency scores remain very close, differing by less than 0.16%. In conclusion, both the segment partition and similarity metric contribute to the effectiveness of our method.
Initial Response to Reviewer mA2s (Part 3/3)
W3 & Q2: Validation of the chunk discontinuity metric at known boundaries.
Following your suggestion, we add measurements evaluated using true boundaries to compute the optical difference, denoted as Chunk Discontinuity (Boundary). We also add two additional boundary perceptual metrics for further comparison. Specifically:
- Boundary tLPIPS Ratio compares the mean LPIPS distance between adjacent frames at chunk boundaries with the corresponding mean at non-boundary positions.
- Boundary CLS Ratio uses the same construction with cosine distance between DINOv2 CLS embeddings.
These measures assess whether perceptual changes are disproportionately concentrated at generation boundaries. All metrics are evaluated on 640 LongLive videos of 30 seconds.
R3-Table-4: Chunk transition measurements for Dummy Forcing and Forcing-KV at comparable speedups. All metrics are evaluated on 640 LongLive videos of 30 seconds.
| Method | Speedup ↑ | Chunk Discontinuity (Original) ↓ | Chunk Discontinuity (Boundary) ↓ | Boundary tLPIPS Ratio ↓ | Boundary CLS Ratio ↓ |
|---|---|---|---|---|---|
| Dummy Forcing | 1.24× | 3.0 | 1.76 | 2.72 | 2.71 |
| Forcing-KV | 1.27× | 2.4 | 1.53 | 1.88 | 1.96 |
-
Forcing-KV has lower discontinuity under all three boundary-aligned measurements, consistent with the ranking under the original metric. The boundary tLPIPS and CLS ratios decrease from 2.72 to 1.88 and from 2.71 to 1.96, respectively.
-
The resulting rankings agree with the original metric, and align with the perceptual observations in the demo videos. We believe these evaluations provide sufficient evidence of our method’s advantages and the overall validity of the metrics. We will also update
Appendix BandTables 1, 2with boundary-aligned metrics to further strengthen the empirical evidence.
W4: Robustness of model-level classification to the calibration prompt.
We find that our method are highly robust to prompt selection. Following your suggestion, we conduct offline classification with two different calibration prompts. The calibration prompts do not overlap with the test set, and we will provide the full prompt in Appendix Q.
R3-Table-5: Sensitivity to the calibration prompt on 80 videos of 30 seconds using LongLive. Head agreement is measured relative to the official prompt.
| Calibration Prompt | Head Agreement ↑ | VBench Score ↑ | Dynamic Degree ↑ |
|---|---|---|---|
| Official Prompt | 100.0% | 81.05 | 38.50 |
| Calibration Prompt 1 | 89.2% | 81.05 | 37.50 |
| Calibration Prompt 2 | 95.3% | 81.07 | 38.17 |
We find that head classifications are highly consistent across different calibration prompts on downstream performance, with agreement rates of 89.2% and 95.3%, consistent with Figure 3 (e) of our manuscript. Surprisingly, the small fraction of heads classified differently does not lead to appreciable differences in the total score. One possible explanation is that these heads exhibit ambiguous characteristics and perform similarly under the two pruning strategies. We will update these findings to Table 3 and Section 4.3. These results justify our model-level offline head classification.
In summary, we thank again for your valuable suggestions and consideration. We have further clarified the novelty of our method, demonstrated its improvements over Dummy Forcing, and provided further ablation studies. We hope our detailed responses have addressed your concerns and would sincerely appreciate any further feedback and your reconsideration of the rating.
Initial Response to Reviewer mA2s (Part 1/3)
Thank you for the detailed assessment and constructive suggestions. We clarify the contribution and novelty of our method, isolate the sources of improvement over Dummy Forcing, and provide additional experiments below. Due to space constraints, for all experiments that report the overall VBench score, the scores for all individual dimensions will be provided in the appendix.
W1: Methodological novelty.
While novelty is multifaceted in academic research, we believe our work contributes at two levels:
Our method introduces three new method innovations:
We present the first systematic study of KV cache utilization patterns in autoregressive video diffusion models, identifying static and dynamic head modes, that distinguish static local attention from dynamic context utilization. These findings constitute a central contribution and provide empirical guidance for video KV cache compression.
Lastly, we argue that a simple yet effective methodology that translates empirical insights into practical acceleration is a strength of our work, instead of a shortcoming. Given the structured nature of KV caches, many established compression methods [1][2][3] likewise build on empirical observations and combine basic retention and pruning rules. As an advantage of the simple yet effective design, our design is also fully compatible with modern optimized kernels (e.g., FlashAttention), making the algorithm both novel and practical for applications.
[1] Efficient Streaming Language Models with Attention Sinks. ICLR 2024.
[2] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads. ICLR 2025.
[3] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS 2023.