acceptodds
Under review as a conference paper at ICLR 2027

Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion

11 views

Abstract

Autoregressive (AR) video diffusion models adopt a streaming generation framework, enabling long-horizon video generation with real-time responsiveness, as exemplified by the Self Forcing training paradigm. However, existing autoregressive video diffusion models still incur substantial attention and memory overhead because they retain largely redundant key–value (KV) caches for historical frames, limiting their scalability. In this paper, we tackle this challenge by introducing KV cache compression into autoregressive video diffusion. We observe that attention heads in mainstream AR diffusion models exhibit markedly distinct attention patterns and functional roles that remain stable across samples and denoising steps. Building on our empirical study of head-wise functional specialization, we divide the attention heads into two categories: static heads, which focus on transitions across autoregressive chunks and intra-frame fidelity, and dynamic heads, which govern inter-frame motion and consistency. We then propose Forcing-KV, a hybrid KV cache compression strategy that performs structured static pruning for static heads and dynamic pruning based on segment-wise similarity for dynamic heads. While maintaining output quality, Forcing-KV compresses 53%-73% of the KV cache, achieves 29 FPS on a single NVIDIA H200 GPU with 30% peak cache memory reduction, and delivers up to 1.35x and 1.50x speedups on LongLive and Self Forcing at 480P resolution. The efficiency gains further scale with resolution and sliding window size. Code and demo videos are provided in the supplementary material.

Primary area:
generative models
Keywords:
Autoregressive Video Generation, KV Cache Compression

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_mA2sOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Forcing-KV, a training-free KV cache compression method for autoregressive video diffusion. It distinguishes static heads that emphasize local transitions from dynamic heads that use historical context for motion and consistency. The method combines offline head profiling, structured retention of the transition anchor frame, and similarity-based pruning of historical segments. Experiments across several models and generation lengths report reduced cache usage and faster inference while broadly maintaining generation quality.

Critical strengths

The paper connects attention patterns to functional behavior through targeted context-masking experiments, rather than relying solely on visualizations, which provide an interpretable motivation for assigning different compression strategies to different heads.

Critical weaknesses
  • One of my major concern is that the paper's novelty appears very limited. The approach largely extends existing head-wise KV cache compression with additional retention and pruning rules. These changes seem incremental, and the conceptual distinction from closely related work is not sufficiently clear. While the empirical gains are useful, they do not by themselves establish a substantial methodological contribution.

  • The sources of improvement over Dummy Forcing could be isolated more clearly. The comparisons at similar speeds in Tables 1–2 and the cache-removal ablations provide substantial evidence for the complete method. However, the explanation for its improvement over Dummy Forcing is less directly tested: Dummy Forcing already includes predecessor context for its dummy heads through Packed Attention Forward, while the two methods also differ in head classification and cache allocation. The profiling study does not isolate the effect of including transition-frame attention, and Figure 7 changes both selection criterion and pruning granularity. The paper explicitly credits both similarity and segment structure, but their separate contributions remain unclear. A profiling control at comparable cache budgets and pruning comparisons with the same segment granularity would strengthen this explanation.

  • The chunk-discontinuity metric needs stronger validation. It seems that Appendix B averages the K−1 largest adjacent-frame optical-flow quantities δ_t defined in Equation (5), where K is the number of chunks, and divides by the mean of δ_t over all adjacent-frame pairs, without using boundary locations. Although this ratio is invariant to uniform rescaling, naturally uneven motion can still affect it independently of boundary artifacts. Figure 8 provides illustrative examples, but does not establish this distinction across a broader set of videos. The user study offers useful supporting evidence, although its Visual Quality, Dynamic Degree, and Consistency criteria (Appendix D) do not directly validate the proposed metric. This matters because the metric supports both the functional interpretation of static heads and the claimed continuity advantage over Dummy Forcing.

  • The use of a single prompt for model-level offline head classification is insufficiently justified. Although Section 2.3 provides evidence of relatively stable attention patterns across prompts, it does not directly establish that the resulting head assignments and downstream performance are robust to the choice of calibration prompt. A prompt that underrepresents a head’s dependence on historical context could cause it to be classified as static, leading to excessive pruning in subsequent generations. A theoretical analysis clarifying when head assignments remain stable across prompts would strengthen this model-level assumption. Without such justification, the method’s generalizability to unseen prompts and effectiveness in practical applications remain insufficiently supported.

Questions for authors
  • Could the authors isolate the effect of including transition-frame attention in profiling by comparing settings with comparable cache budgets and otherwise unchanged pruning rules? For dynamic pruning, could similarity-based selection be compared with random and uniform selection using the same segment granularity? Reporting continuity, consistency, overall quality, and latency together would help establish which design accounts for the gains.

  • How often do the K−1 largest δ_t values selected over the entire video in Equation (6) occur near chunk boundaries? Could the authors provide a boundary-aligned evaluation or a controlled comparison that separates natural motion changes from injected boundary artifacts? Evidence that the resulting rankings agree with human judgments of transition quality would substantially strengthen this part of the evaluation.

6 comments
AuthorsOpenReview ↗

Initial Response to Reviewer mA2s (Part 1/3)

Thank you for the detailed assessment and constructive suggestions. We clarify the contribution and novelty of our method, isolate the sources of improvement over Dummy Forcing, and provide additional experiments below. Due to space constraints, for all experiments that report the overall VBench score, the scores for all individual dimensions will be provided in the appendix.


W1: Methodological novelty.

While novelty is multifaceted in academic research, we believe our work contributes at two levels:

  1. Methodological Innovation:

Our method introduces three new method innovations:

  • Observation-driven head profiling. We introduce a new classification criterion and algorithm that distinguishes static structural heads from dynamic heads, a first-time distinction not identified in prior work.
  • Static structural pruning with transition anchor preservation. Similar to StreamingLLM [1], our design builds on empirical observations to identify and explicitly preserve important anchors. We consider the first-time identification and structural preservation of transition anchors in autoregressive video generation a novel contribution.
  • Dynamic segment-wise pruning. We introduce both a similarity proxy to guide pruning, and a segment-wise pruning design. To our knowledge, this is the first such cache compression strategy for autoregressive video generation.
  1. Empirical Innovation

We present the first systematic study of KV cache utilization patterns in autoregressive video diffusion models, identifying static and dynamic head modes, that distinguish static local attention from dynamic context utilization. These findings constitute a central contribution and provide empirical guidance for video KV cache compression.

Lastly, we argue that a simple yet effective methodology that translates empirical insights into practical acceleration is a strength of our work, instead of a shortcoming. Given the structured nature of KV caches, many established compression methods [1][2][3] likewise build on empirical observations and combine basic retention and pruning rules. As an advantage of the simple yet effective design, our design is also fully compatible with modern optimized kernels (e.g., FlashAttention), making the algorithm both novel and practical for applications.

[1] Efficient Streaming Language Models with Attention Sinks. ICLR 2024.

[2] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads. ICLR 2025.

[3] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. NeurIPS 2023.

AuthorsOpenReview ↗

Initial Response to Reviewer mA2s (Part 2/3)

W2 & Q1: Separating transition-aware profiling, isolating the improvements over Dummy Forcing, and comparing the pruning methods with the same segment granularity.

  1. Transition frame attention in profiling.

Including transition attention in profiling has a significant effect. Specifically, we compare the profiling of Forcing-KV w/ and w/o the transition attention using the same prompt, and evaluate the results on 80 LongLive videos of 30-seconds.

R3-Table-1: Effect of including transition frame attention in profiling.

Static HeadsDynamic HeadsVBench Score ↑Dynamic Degree ↑Consistency ↑Chunk Discontinuity (Original) ↓Chunk Discontinuity (Boundary) ↓
w/ Transition Frame Attention10125981.0537.5097.832.531.41
w/o Transition Frame Attention6429680.8936.1797.852.691.47

Including transition attention improves total score from 80.89 to 81.05 and dynamic degree from 36.17 to 37.50. Removing the transition anchor frame from profiling results in fewer heads being classified as static, and causes discontinuity to rise from 2.53 to 2.69. The continuity scores are close because dynamic heads may also retain anchor. Notably, under the current policy, excluding transition frame attention yields a larger KV cache budget because more heads are classified as dynamic and retain more cached tokens. Despite the larger budget, w/o transition frame achieves lower performance, supporting the effectiveness of the transition frame.


  1. Isolating the improvements over Dummy Forcing.

The improvements over Dummy Forcing stem primarily from three aspects.

  • Accurate head classification grounded in empirical observations. Dummy Forcing categorizes heads into sink, dummy, and global heads, each with a more narrow compression strategy. It also relies on a small subset of attention rows as proxy for online profiling. Through detailed empirical analysis, we identify two head categories and incorporate transition frame attention into profiling, improving performance as reflected in R3-Table-1. We also adopt offline profiling with full attention prior.

  • Explicitly preserving the transition anchor frame. Although Dummy Forcing uses Packed Attention Forward for dummy heads, its classification already overlooks some heads that requires transition context, and its sink head policy does not preserve the transition anchor frame. As a result, the transition frame is not sufficiently protected as our method.

  • Dynamic similarity pruning. Our pruning strategy for dynamic heads further exploits redundancy across the full history, freeing up additional KV cache budget.


  1. Comparison using the same segment granularity.

Our similarity-based selection consistently outperforms random and uniform segment selection at matching budgets. Specifically, we compare random, uniform, and similarity segment selection using the same segment partition and token budgets. Under each budget, the latency is nearly identical since similarity computation iccurs minimal overhead as shown in Figure 11 of our manuscript.

R3-Table-2: VBench score, dynamic degree (DD) and consistency at the same token budgets. Consistency is the mean of subject consistency and background consistency. Results are evaluated across 4720 video of LongLive, 5-second.

Token Budget1,5601,5601,5603,1203,1203,1204,6804,6804,680
Selection MethodVBenchDDConsistencyVBenchDDConsistencyVBenchDDConsistency
Similarity Segment83.1945.0096.6283.2045.2896.6283.2946.9596.60
Random Token82.9240.5696.5982.9140.2896.6082.8940.0096.59
Uniform Token83.0242.5096.6383.0042.5096.6483.0042.2296.64
Random Segment83.1842.7896.7583.1441.9496.7583.1642.5096.74
Uniform Segment83.1442.5096.7483.1843.0696.7383.1643.6196.75

The results further isolate the effect of segment and similarity pruning. Specifically, the segment methods generally improve upon their token-level alternatives, supporting the usefulness of contiguous grouping. Crucially, similarity-based selection also outperforms random and uniform selection at the same segment granularity and different budgets. Our method improves dynamic degree and total score over random and uniform segments, and has the highest total score at all three budgets (up to 83.29). The consistency scores remain very close, differing by less than 0.16%. In conclusion, both the segment partition and similarity metric contribute to the effectiveness of our method.

AuthorsOpenReview ↗

Initial Response to Reviewer mA2s (Part 3/3)

W3 & Q2: Validation of the chunk discontinuity metric at known boundaries.

Following your suggestion, we add measurements evaluated using true boundaries to compute the optical difference, denoted as Chunk Discontinuity (Boundary). We also add two additional boundary perceptual metrics for further comparison. Specifically:

  • Boundary tLPIPS Ratio compares the mean LPIPS distance between adjacent frames at chunk boundaries with the corresponding mean at non-boundary positions.
  • Boundary CLS Ratio uses the same construction with cosine distance between DINOv2 CLS embeddings.

These measures assess whether perceptual changes are disproportionately concentrated at generation boundaries. All metrics are evaluated on 640 LongLive videos of 30 seconds.

R3-Table-4: Chunk transition measurements for Dummy Forcing and Forcing-KV at comparable speedups. All metrics are evaluated on 640 LongLive videos of 30 seconds.

MethodSpeedup ↑Chunk Discontinuity (Original) ↓Chunk Discontinuity (Boundary) ↓Boundary tLPIPS Ratio ↓Boundary CLS Ratio ↓
Dummy Forcing1.24×3.01.762.722.71
Forcing-KV1.27×2.41.531.881.96
  1. Forcing-KV has lower discontinuity under all three boundary-aligned measurements, consistent with the ranking under the original metric. The boundary tLPIPS and CLS ratios decrease from 2.72 to 1.88 and from 2.71 to 1.96, respectively.

  2. The resulting rankings agree with the original metric, and align with the perceptual observations in the demo videos. We believe these evaluations provide sufficient evidence of our method’s advantages and the overall validity of the metrics. We will also update Appendix B and Tables 1, 2 with boundary-aligned metrics to further strengthen the empirical evidence.


W4: Robustness of model-level classification to the calibration prompt.

We find that our method are highly robust to prompt selection. Following your suggestion, we conduct offline classification with two different calibration prompts. The calibration prompts do not overlap with the test set, and we will provide the full prompt in Appendix Q.

R3-Table-5: Sensitivity to the calibration prompt on 80 videos of 30 seconds using LongLive. Head agreement is measured relative to the official prompt.

Calibration PromptHead Agreement ↑VBench Score ↑Dynamic Degree ↑
Official Prompt100.0%81.0538.50
Calibration Prompt 189.2%81.0537.50
Calibration Prompt 295.3%81.0738.17

We find that head classifications are highly consistent across different calibration prompts on downstream performance, with agreement rates of 89.2% and 95.3%, consistent with Figure 3 (e) of our manuscript. Surprisingly, the small fraction of heads classified differently does not lead to appreciable differences in the total score. One possible explanation is that these heads exhibit ambiguous characteristics and perform similarly under the two pruning strategies. We will update these findings to Table 3 and Section 4.3. These results justify our model-level offline head classification.


In summary, we thank again for your valuable suggestions and consideration. We have further clarified the novelty of our method, demonstrated its improvements over Dummy Forcing, and provided further ablation studies. We hope our detailed responses have addressed your concerns and would sincerely appreciate any further feedback and your reconsideration of the rating.

Reviewer_yMsZOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies historical KV cache compression in autoregressive video diffusion models, with the goal of reducing computation and memory overhead while preserving the motion quality of videos and the continuity between clips.

The authors divide attention heads into static heads that depend on the current clip and the nearest transition anchor, and the common dynamic heads that exploit broad historical information. After identifying the heads offline, the authors apply structured pruning to the static heads and dynamic pruning to the dynamic heads. Through experiments, the authors characterize the historical dependence of different attention heads in greater detail and design the cache compression strategy accordingly.

Critical strengths

1.The authors make a meaningful distinction between long-range history and local transition information, which convincingly demonstrates the significance of protecting transition anchors. This is highly valuable for guiding video cache compression.

2.The experiments cover short videos, long videos, and interactive generation. This is a broad evaluation, and the results under most of the evaluated settings show significant speedups.

Critical weaknesses

1.The description of the method in the paper does not seem to correspond one-to-one with the code. For example, Eq. (3) describes selecting clips independently within each frame, whereas the dynamic compression in the code merges the clips of three frames before selection. In addition, the script shows a classification threshold of 0.5, while Appendix K gives 0.8. The authors should clarify the correspondence between each table and the run configuration. This would facilitate reproduction of the paper and assessment of its value.

2.The authors do not directly evaluate the known clip boundaries. Consequently, peaks and motion intensity in normal motion may affect the evaluation of the discontinuity metric.

3.Fig. 7 supports the overall effectiveness of the proposed pruning scheme, but it does not yet sufficiently distinguish the respective contributions of clip structure and similarity-based selection.

4.The method appears applicable online, but the current experiments are not sufficient to equate the DiT throughput gain directly with a user-perceivable improvement in real-time performance.

Questions for authors

1.The authors should explicitly state the correspondence between the experimental settings and procedures, and specify which acceleration measures were actually used for the results in each table.

2.Can the clip discontinuity metric reflect artifacts at true clip boundaries after excluding the influence of motion intensity? We suggest adding comparisons between boundary and non-boundary positions as well as controlled perturbation experiments. Would human-evaluation metrics be valuable?

3.Under the same retained-token budget and the same clip partition, is similarity-based clip selection better than random or uniform selection? Can the authors add an application pipeline for online generation? I attach particular importance to the practical application value of the method. A broader experimental evaluation and more detailed descriptions would help me further assess the value of this work.

5 comments
AuthorsOpenReview ↗

Initial Response to Reviewer yMsZ (Part 1/2)

Thank you for your detailed and practical suggestions, and for highlighting that our method is highly valuable for guiding video cache compression. We follow your suggestions to add clarifications, additional experiments, and an online serving inference pipeline. We are pleased to share the updated results with you below.


W1 & Q1: Correspondence between the method description, implementation, and experimental configurations.

Thank you for carefully examining the released implementation. We apologize for the description mistake in Section 3.3, and we will revise line 306 to : “We evict the top-k segments from historical frames with the highest similarity values.” And the method formulation remains unchanged. Notably, this correction aligns the textual description with the intended method and actual implementation, and does not affect our main claims or reported experimental results. We also proofread the manuscript and performed additional checks.

For the classification threshold of 0.5 in the script, this value was retained from an ablation experiment. Our experiments use a threshold of 0.8, as specified in the implementation details of our method. Other run configuratios are also originally reported in the implementation details section in Appendix K.


W2 & Q2: Boundary-aligned evaluation of the chunk discontinuity metric.

Forcing-KV has higher continuity under boundary-aligned evaluations. Following the suggestion, we add measurements evaluated using true boundaries to compute the optical difference, denoted as Chunk Discontinuity (Boundary). We also add two additional boundary perceptual metrics for further comparison. Specifically:

  • Boundary tLPIPS Ratio compares the mean LPIPS distance between adjacent frames at chunk boundaries with the corresponding mean at non-boundary positions.
  • Boundary CLS Ratio uses the same construction with cosine distance between DINOv2 CLS embeddings.

These measures assess whether perceptual changes are disproportionately concentrated at generation boundaries. All metrics are evaluated on 640 LongLive videos of 30 seconds.

R2-Table-1: Chunk-transition measurements for Dummy Forcing and Forcing-KV at comparable speedups. All metrics are evaluated on 640 LongLive videos of 30 seconds.

MethodSpeedup ↑Chunk Discontinuity (Original) ↓Chunk Discontinuity (Boundary) ↓Boundary tLPIPS Ratio ↓Boundary CLS Ratio ↓
Dummy Forcing1.24×3.01.762.722.71
Forcing-KV1.27×2.41.531.881.96
  1. Forcing-KV has lower discontinuity under all three boundary-aligned measurements, consistent with the ranking under the original metric. The boundary tLPIPS and CLS ratios decrease from 2.72 to 1.88 and from 2.71 to 1.96, respectively.

  2. Moreover, the metrics exhibit consistent trends that align with the perceptual observations in the demo videos. We believe these evaluations provide sufficient evidence of our method’s advantages and the overall validity of the metrics. We will also update Appendix B and Tables 1, 2 with boundary-aligned metrics to further strengthen the empirical evidence.

Human evaluation would ideally provide a more direct assessment. However, time constraints prevent us from conducting such a study during the rebuttal period. We will provide additional demo videos and the above results to supplement the evaluation.

AuthorsOpenReview ↗

Initial Response to Reviewer yMsZ (Part 2/2)

W3 & Q3: Similarity pruning vs. contiguous segment structure.

Our similarity-based selection consistently outperforms random and uniform segment selection at matching budgets. Specifically, we add comparison for the dynamic heads between similarity-based selection and random or uniform segment selection under the same token budgets and the same segment partition. Results are evaluated across 4720 video clips of LongLive, 5-second consistent with Figure 7 in our manuscript.

R2-Table-2: Pruning method comparisons on VBench and Dynamic Score (DD) at the same token budgets. Results are evaluated across 4720 video clips of LongLive, 5-second.

Token Budget1,5601,5603,1203,1204,6804,680
Pruning MethodVBench ↑DD ↑VBench ↑DD ↑VBench ↑DD ↑
Similarity Segment (Ours)83.1945.0083.2045.2883.2946.95
Random Token82.9240.5682.9140.2882.8940.00
Uniform Token83.0242.5083.0042.5083.0042.22
Random Segment83.1842.7883.1441.9483.1642.50
Uniform Segment83.1442.5083.1843.0683.1643.61

The results further isolate the effect of segment-pruning and similarity pruning. Specifically, the segment methods generally improve upon their token-level alternatives, supporting the usefulness of contiguous grouping. Crucially, under the same retained-token budget and the same clip partition, similarity-based clip selection is still better than random or uniform clip selection. Our method improves dynamic degree by 2.22–4.45 over random segments and 2.22–3.34 over uniform segments, and has the highest total score at all three budgets (up to 83.29).


W4 & Q3: Practical throughput and latency beyond DiT-only measurements.

Thank you for this valuable suggestion. It prompted us to examine practical deployment of Forcing-KV more closely to user-perceivable efficiency gains. We largely extend the current inference pipeline and construct two online serving protocols:

  • Online interactive generation: We evaluate online prompt-switch response when user provides a new prompt during the generation, and measure the response latency (time-to-first-chunk) under existing historical cache compression. This protocol evaluates user-perceived latency.
  • Batched serving for concurrent user requests: We increase the batch size on a single GPU to simulate a real-deployment scenario of user requests. This protocol evaluates throughput under concurrent requests from multiple users.

R2-Table-3: Online interactive generation on a single NVIDIA H200. We report multiple throughput, response time, TFLOPs, and KV cache memory metrics. This provides a comprehensive study of the efficiency gains of Forcing-KV.

MethodDiT Throughput (FPS) ↑DiT + VAE Throughput (FPS) ↑Prompt-Switch Chunk Response: DiT (ms) ↓Prompt-Switch Chunk Response: DiT + VAE (ms) ↓TFLOPs per Sampled Chunk ↓KV Cache Memory (GiB) ↓
Self Forcing17.7512.91706960202.0812.2
+Forcing-KV25.28 (1.42×)16.47 (1.28×)476 (−32.6%)730 (−24.0%)98.52 (−51.3%)8.7 (−28.7%)

R2-Table-4: DiT throughput under batched serving for concurrent user requests, on a single NVIDIA H200. Speedup compares Forcing-KV with Self Forcing at the same batch size and 30-second video generation.

Batch SizeSelf Forcing Throughput (FPS) ↑Forcing-KV Throughput (FPS) ↑Speedup ↑
117.8225.281.42×
219.7933.221.68×
420.3936.801.80×
821.0138.561.84×

We are encouraged that these deployment evaluations demonstrate additional benefits across multiple efficiency dimensions. By compressing the KV cache and reducing attention computation, our method improves both throughput and first-chunk latency. As concurrency increases, computation becomes a more pronounced compute-bound bottleneck on a single GPU for the base model, and reducing this cost through our method yields progressively greater speedups, from 1.42× at batch size 1 to 1.84× at batch size 8.

In short, our method delivers 1.42× DiT throughput, 1.28× end-to-end throughput, 32.6% lower DiT chunk response time, 24.0% lower end-to-end chunk response time, 51.3% fewer estimated FLOPs, and 28.7% lower KV cache memory usage, with increasing speedup gains under higher concurrency. We thank the reviewer for motivating this analysis, which reveals additional practical value for interactive responsiveness and concurrent generation beyond the original benchmark evaluation. We are excited to present these results, and recommend our method for real-world user application.

AuthorsOpenReview ↗

Follow up Response to Reviewer yMsZ

Dear Reviewer yMsZ,

Following up on our recent exchange regarding this paper, we wanted to kindly check if there are any further concerns or feedback from you. We especially thank you for motivating the experiment of online serving and practical deployment, which we share new exciting results and benefits of our method. Your insights are invaluable to us, and we greatly appreciate your time and consideration. Please feel free to share any thoughts you may have.

Best regards,

The Authors

Reviewer_1uHVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a training-free method to compress the KV cache of autoregressive video diffusion models. The authors divide attention heads into two groups using attention statistics collected from one calibration prompt. This classification is fixed for the model and reused across videos. Static heads keep sink frames, the latest historical latent frame, and the current chunk. Dynamic heads keep sink frames and the current chunk, but prune older history by comparing corresponding token segments in adjacent frames. Segments with lower key similarity are retained because they are considered less redundant. The selection is computed from first-layer keys and reused in deeper layers, which keep their own keys and values at the selected positions.

The main contribution is connecting different uses of historical context to different cache policies. The authors evaluate quality and speed on several models, short and long videos, and interactive generation. They report substantial cache compression and roughly 1.3–1.5× DiT speedups in the main experiments, with similar overall benchmark scores. They also study head stability, pruning settings, and the first-layer proxy.

Critical strengths

The transition-anchor finding gives a clear reason for the cache policy. The authors do not rely only on attention visualizations. They remove context from the two head groups and measure the effects. This makes the method more convincing than applying the same pruning rule to every head.

The method provides useful speed improvements without retraining, while broadly preserving benchmark quality. Results on additional models and interactive generation support its practical value. The authors also include cache-management overhead and provide end-to-end timing rather than reporting attention speed alone.

Critical weaknesses

The claim of preserved quality needs some qualification for human identity. In the poker-player comparison, I see the player's identity drift in the Forcing-KV video, while it stays stable in the LongLive video. This is one example, and it does not establish that Forcing-KV generally preserves identity less well than LongLive. Still, the reported subject-consistency scores do not explain this observation. Standard VBench includes first-frame similarity, and VBench-Long measures long-range consistency, but general image-feature similarity is not a dedicated test of whether a face still belongs to the same person. Averaging can also hide failures late in a video. I would like the authors to explain this case and clarify whether compression introduces a noticeable identity trade-off. This is an unresolved quality concern, not evidence that the whole evaluation is invalid.

Some ablations leave the quality trade-offs of individual components unclear. The full-method results support the practical value of the approach, but Figure 7 reports only dynamic degree, and Table 8 does not include a version with no dynamic-head pruning while keeping static-head pruning fixed. Appendix O supports the first-layer proxy through score correlations, but does not directly compare its output quality with separate per-layer selection. These comparisons would help explain how much each shortcut contributes to speed and whether it has a quality cost. Their absence limits that explanation; it does not by itself show that the design is unsound or that either component causes the observed identity drift.

Questions for authors

Identity preservation - explanation of the observed example: In the poker-player comparison, the player's identity drifts with Forcing-KV but stays stable with LongLive. Can you explain this difference? Is this an isolated example, or have you observed similar failures elsewhere? Face crops across time, compared with the original face, would help show when the drift starts. If available, an identity-focused evaluation would provide stronger evidence than general image-feature similarity alone.

Dynamic pruning - a controlled quality comparison: Do you have results for the same head classification and static-head policy, but with all dynamic-head history retained? Comparing this setting with the full method would help isolate the quality and speed effects of dynamic pruning. Please include subject consistency and overall quality, not only dynamic degree. A matched-budget random or uniform segment-selection baseline would also help separate the value of similarity ranking from the use of contiguous segments.

First-layer proxy - clarification and supporting evidence: How are the first-layer similarity scores combined across heads, and at which denoising step is selection performed? Is the resulting selection shared by all dynamic heads in deeper layers? Beyond the score correlations in Appendix O, do you have a direct quality comparison with separate per-layer selection, even on a small set? This would clarify the trade-off behind using a shared proxy.

Head classification - robustness and reproducibility: The cross-prompt analysis, random-classification ablations, and threshold study provide useful support for fixed head labels. Have you also checked whether using a different calibration prompt changes the final quality or speed? Please specify how the attention masses in Equation 1 are combined across query tokens, autoregressive chunks, and denoising steps.

5 comments
AuthorsOpenReview ↗

Initial Response to Reviewer 1uHV (Part 1/2)

Thank you for your careful reading and constructive suggestions. We appreciate your recognition of the empirical finding and the practical value of our method, which is a great encouragement to us! Following your suggestions, we provide additional identity-specific evaluation and broader ablation experiments as follows.


W1 & Q1: Identity preservation and the poker-player example.

We agree that general subject-consistency scores in VBench do not fully characterize facial identity. We additionally evaluate identity consistency using ArcFace [1] embeddings extracted with InsightFace at 2 FPS. The evaluation covers 11 face-containing, 60-second videos from all available official interactive prompt sets of LongLive, including the demo case. Two metrics are measured:

  • Initial Identity Similarity: comparing subsequent faces with a fixed initial reference for each tracked identity, excluding self-comparisons.
  • ID Similarity at Different Separation: comparing valid face pairs within the same identity track at the specified temporal separation. Average ID Similarity is the arithmetic mean.

R1-Table-1: Face identity consistency on 60-second interactive generation of LongLive.

MethodInitial Identity Similarity ↑ID Similarity (15s Apart) ↑ID Similarity (30s Apart) ↑Average ID Similarity ↑FPS ↑
LongLive0.33870.41880.30250.360723.07
+Forcing-KV0.32960.33930.30720.323326.35
+Dummy Forcing0.27450.25220.23170.241925.74

As shown, Forcing-KV maintains Initial Identity Similarity comparable to LongLive over a 60-second horizon (0.3296 vs. 0.3387), while exhibiting a modest decrease in Average ID Similarity (0.3233 vs. 0.3607), consistent with the observations in the poker-player case. Still, it substantially outperforms Dummy Forcing at comparable throughput (0.3233 vs. 0.2419). Two factors explain these results.

  1. Our method compresses more than 53% of the KV cache for efficiency gains, this inevitably entails subtle loss of details from distant context. We regard this as a reasonable efficiency–quality trade-off associated with the substantial efficiency gains from cache compression. And our method substantially exceeds previous method Dummy Forcing.

  2. Our method primarily targets general video generation following Self Forcing and LongLive, whereas dedicated facial identity preservation is a more specialized task beyond our primary scope. We acknowledge that face identity is an important dimension, and believe that incorporating an identity-aware mechanism could further enhance our method on this specific task in the future.

[1] ArcFace: Additive Angular Margin Loss for Deep Face Recognition. CVPR 2019.


W2 & Q2 (Part 1/2): Quality controlled dynamic pruning.

Our dynamic pruning largely increases the compression ratio while incurring only minor quality degradation. Following the suggestion, we retain all dynamic-head history while keeping the head classification and static-head pruning policy unchanged, and report the full quality breakdown with 640 videos of 30-second using LongLive as base model. Original Forcing-KV uses a dynamic compression ratio of 0.6.

R1-Table-2: Controlled ablation of dynamic head pruning on LongLive, 30-second generation. Both variants use the same head classification and static-head policy.

MethodKV Retention RatioVBench-Long Total ↑Dynamic Degree ↑Subject Consistency ↑Background Consistency ↑Imaging Quality ↑Motion Smoothness ↑Aesthetic Quality ↑
Forcing-KV46.44%80.6543.6598.00496.80970.2998.6361.98
w/o Dynamic Pruning65.51%80.7144.5598.00196.80270.2198.6361.96

Dynamic pruning reduces KV retention by 19.07 percentage points, with only a slight decrease in the total score (80.65 vs. 80.71) and dynamic degree (43.65 vs. 44.55). Other metrics remain close. This is consistent with the observation in our manuscript that dynamic pruning achieves acceleration primarily by compressing distant context. On VBench-Long, this effect is mainly reflected in the dynamic degree metric and total score. The high quality preservation in R1-Table-2 also demonstrates the effectiveness of dynamic pruning.

AuthorsOpenReview ↗

Initial Response to Reviewer 1uHV (Part 2/2)

W2 & Q2 (Part 2/2): Segment-wise pruning at matching budgets.

Our similarity-based selection consistently outperforms random and uniform segment selection at matching budgets. Specifically, we compare similarity-based selection for the dynamic heads with random and uniform segment selection under the same token budgets and the same segment partition. Results are evaluated across 4720 video clips of LongLive, 5-second consistent with Figure 7 in our manuscript.

R1-Table-3: Pruning method comparisons on VBench and Dynamic Score (DD) at the same token budgets.

Token Budget1,5601,5603,1203,1204,6804,680
Pruning MethodVBench ↑DD ↑VBench ↑DD ↑VBench ↑DD ↑
Similarity Segment (Ours)83.1945.0083.2045.2883.2946.95
Random Token82.9240.5682.9140.2882.8940.00
Uniform Token83.0242.5083.0042.5083.0042.22
Random Segment83.1842.7883.1441.9483.1642.50
Uniform Segment83.1442.5083.1843.0683.1643.61

The results further isolate the effect of segment-pruning and similarity pruning. Specifically, the segment methods improve upon their token-level alternatives, supporting the usefulness of contiguous grouping. Crucially, similarity-based selection also consistently outperforms random and uniform selection at the same segment granularity and different budgets. Our method improves dynamic degree by 2.22–4.45 over random segments and 2.22–3.34 over uniform segments, and has the highest total score at all three budgets (up to 83.29). Therefore, both the segment partition and similarity metric contribute to the effectiveness of our method.


W2 & Q3: First-layer proxy vs. independent per-layer selection.

First-layer proxy achieves comparable performance with less overhead than independent per-layer selection. For clarity, the similarity scores are averaged across all heads and computed at the t=0t=0 denoising step, when the clean cache is refreshed. These scores are then shared across all dynamic heads in deeper layers. We will clarify this more explicitly in Section 3.3. In addition, we compare quality score against independent per-layer selection, using a subset of 50 videos on Self Forcing under the same setting as Appendix O.

R1-Table-4: Quality and throughput comparison between shared first-layer proxy selection and independent per-layer selection on Self Forcing.

FPS ↑Speedup ↑VBench-Long ↑
Shared First-Layer Proxy25.71.44×80.193
Independent Per-Layer Selection24.71.39×80.177

Independent per-layer selection introduces additional computation and separate cache management for each layer, reducing the speedup from 1.44× to 1.39× while yielding nearly identical quality scores. These results support our use of a shared proxy to balance efficiency and generation quality.


Q4: Calibration prompt robustness for head classification.

We find that our method are highly robust to prompt selection. Following the suggestion, we conduct offline classification with two different calibration prompts. The full prompt will be provided in Appendix Q. For clarity, the attention masses in Equation (1) are averaged over all query tokens and computed at the t=0t=0 denoising step of a stable chunk, once the cache has been filled for the first time. We will also clarify this more explicitly in Section 3.1.

R1-Table-5: Sensitivity to the calibration prompt on 80 videos of 30 seconds using LongLive. Head agreement is measured relative to the main table prompt.

Calibration PromptHead Agreement ↑VBench Score ↑Dynamic Degree ↑
Official Prompt100.0%81.0538.50
Calibration Prompt 189.2%81.0537.50
Calibration Prompt 295.3%81.0738.17

We find that head classifications are highly consistent across different calibration prompts, with agreement rates of 89.2% and 95.3%, consistent with Figure 3 (e) of our manuscript. Surprisingly, the small fraction of heads classified differently does not lead to appreciable differences in the total score. One possible explanation is that these heads exhibit ambiguous characteristics and perform similarly under the current two pruning strategies. We will update these empirical findings to Table 3 and Section 4.3.


In summary, we thank again for your valuable suggestions on identity-specific evaluation and broader ablation studies, which have substantially strengthened the empirical rigor of our manuscript. We hope our detailed responses have addressed your concerns and would sincerely appreciate any further feedback and your reconsideration of the rating.

AuthorsOpenReview ↗

Response to Reviewer 1uHV

Thank you for reviewing our rebuttal and raising your score. We greatly appreciate your recognition of our work and will incorporate the latest experimental results into the final version. Your feedback has been instrumental in enhancing the quality of our manuscript, and we deeply appreciate your support!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank the Area Chair and all reviewers for their thoughtful and constructive feedback. We are encouraged by their recognition of the key contributions and practical value of our work. For clarity, we refer to Reviewers 1uHV, yMsZ, and mA2s as R1, R2, and R3, respectively.

We are grateful that R1 and R3 have confirmed that their main concerns were addressed and raised their ratings to 3. R2 maintained a positive rating of 3 with increased confidence score 3. In particular, we appreciate the acknowledgment of our empirical contribution (R1, R3), the recognition of the highly valuable for guiding video cache compression (R2), the appreciation of our efficiency gains on broad evaluation (R1, R2), and the practical value of our method (R2).

Below, we summarize the core contributions of our study, the additional experiments, and the important clarifications during the rebuttal. All modifications in the PDF are highlighted in red.


Core Contributions of Our Work

  1. Novel Pattern Discovery: We uncover a universal head specialization pattern shared by mainstream autoregressive video diffusion models: transitions across autoregressive chunks are mediated by static heads that concentrate on the transition anchor, whereas long-horizon consistency and dynamics are sustained by dynamic heads through sparse inter-frame attention.
  2. Observation-Driven Hybrid KV Cache Compression: We translate these findings into transition-aware head profiling, structured static pruning that explicitly preserves the transition anchor, and similarity-based segment pruning for dynamic heads. This design decouples static attention from dynamic context utilization.
  3. Practical Acceleration with Broad Quality Preservation: Across the evaluated models and generation settings, Forcing-KV compresses 53%–73% of the KV cache*and achieves up to 1.35× and 1.50× speedups on LongLive and Self Forcing at 480P, respectively. The rebuttal further demonstrates benefits for interactive response latency and batched serving.

Updates on Clarifications and Experiments During Rebuttal

  1. Interactive Responsiveness and Concurrent Serving (R2-W4/Q3): New serving experiments demonstrate additional efficiency gains of our method: lower responsiveness time and throughput speedups increasing from 1.42× to 1.84× as batch size grows. These deployment evaluations have been added to Section 4.3: Practical Deployment.

  2. Clarification on Method Novelty and Improvements over Dummy Forcing (R3-W1): We clarify our empirical findings on head specialization and the resulting methodological contributions. We also isolate the effect of transition-aware profiling and explain the benefits of head classification, anchor preservation, and dynamic similarity pruning. This discussion has been added to Appendix S.

  3. Method–Implementation Consistency (R2-W1/Q1): We correct the dynamic pruning description and clarify that the reported experiments hyperparameter. These corrections do not change the method or reported results. The clarification has been added to line 306 and line 1067 in Appendix K.

  4. Identity Quality Evaluation (R1-W1/Q1): Additional ArcFace evaluation quantifies the facial-identity trade-off relative to LongLive and shows substantially better identity preservation than Dummy Forcing at comparable throughput. These results have been added to Appendix Q.

  5. Pruning Ablations (R1-W2/Q2, R2-W3/Q3, R3-W2/Q1): At matching budgets and segment partitions, similarity-based selection achieves the best VBench and Dynamic Degree scores across all tested budgets. Ablation on dynamic pruning is also validated. These ablations have been added to line 514 in Section 4.4.

  6. Boundary-Aligned Continuity Evaluation (R2-W2/Q2, R3-W3/Q2): We add three measurements at true chunk boundaries to support the temporal-continuity advantage of Forcing-KV. These evaluations have been added to Appendix R.

  7. Proxy Validation and Calibration Robustness (R1-Q3/Q4, R3-W4): The shared first-layer proxy achieves nearly identical quality with higher throughput than per-layer selection. Alternative calibration prompts yield high head agreement. These results have been added to line 521 in Section 4.4.


We believe these experiments and clarifications comprehensively address the reviewers' concerns and enhance the quality of our manuscript. We are grateful for the reviewers' valuable suggestions and the Area Chair's diligent oversight.

Wishing you a pleasant day ahead.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.