acceptodds
Under review as a conference paper at ICLR 2027

NanoKV: High-Ratio KV Cache Compression for Autoregressive Video Generation

14 views

Abstract

Autoregressive video models rely on historical key–value (KV) caches to preserve scene consistency over extended rollouts, making KV storage and repeated attention over history a growing memory and computational bottleneck. Aggressive compression can degrade generation quality, while reconstructing full-dimensional KV leaves attention computation largely unchanged. We introduce NanoKV, a frozen-backbone framework for generation-aware high-ratio KV compression. NanoKV exploits redundancy across tokens and attention heads by storing shared content in cluster centers and encoding joint head–channel residuals into low-bit coefficients. Crucially, NanoKV uses different fitting objectives for Keys and Values according to their distinct roles in attention: Key decoders are optimized to preserve downstream block outputs, whereas Value encoders and decoders are fitted for faithful reconstruction through the deployed quantization path. Building on the same compressed representation, the optional NanoKV-Fast variant reuses the stored coefficients for mixed-dimensional attention, reducing computation without modifying the video backbone. Across three autoregressive video backbones, NanoKV consistently improves the quality–storage trade-off. On HY-WorldPlay, NanoKV achieves 26.09× effective KV compression—nearly twice that of KVTC—while improving PSNR by 1.0 dB. NanoKV-Fast retains 20.53× effective compression and achieves an approximately 1.15× end-to-end speedup over KVTC. Code will be released at https://anonymous.4open.science/r/NanoKV-ICLR2027-187D.

Primary area:
generative models
Keywords:
KV cache compression, autoregressive video generation, efficient inference, low-dimensional attention, quantization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vjCwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

NanoKV compresses historical KV caches for autoregressive video generation while keeping the backbone frozen. It combines shared-center residual coding, joint head–channel compression, and separate fitting objectives for Keys and Values. NanoKV-Fast adds mixed-dimensional attention using the same compressed representation. Experiments on three backbones show lower KV storage and peak GPU memory, with higher reference PSNR than the evaluated compression baselines.

Critical strengths

(1) The memory savings are convincing. On HY-WorldPlay, NanoKV reduces KV storage from 20.96 GiB to 0.80 GiB and peak allocated GPU memory by 35.9%. It also improves PSNR over KVTC by 1.0 dB while using roughly half its KV storage. Similar storage improvements are reported on the other two backbones. Including metadata and resident operators in the storage accounting makes these results useful for assessing deployment costs.

(2) The core compression scheme is straightforward. Shared centers capture common features, leaving the low-dimensional coefficients to encode token-specific residuals. The separate treatment of Keys and Values is sensible given their different roles in attention, and the ablation results show that the main components contribute to the final performance.

Critical weaknesses

(1) The evaluation leaves the behavior over longer generation unclear. The three backbones are evaluated on videos of 253, 169, and 125 frames. The quality-over-time plots are useful, but do not establish how the method behaves over much longer rollouts or repeated scene revisits. This matters particularly for NanoKV-Fast, whose gap from NanoKV grows later in generation. Longer tests are needed to assess how much quality is lost as compression errors propagate.

(2) The practical benefit depends on cache length, but this dependence is not measured. The main results use fixed history settings, including 32 historical latent frames for HY-WorldPlay. With a smaller sliding window, resident codec storage and encoding/decoding costs may account for a larger share of the total cost. The latency results already show limited gains: NanoKV is slower than the uncompressed reference, while NanoKV-Fast reaches only 1.06×, 1.04×, and 0.98× its speed across the three backbones. A cache-length sweep would clarify when the memory savings remain worthwhile and whether the additional computation pays off.

Questions for authors

(1) The evaluation covers three action-conditioned video models. Could you also evaluate NanoKV on a text-conditioned autoregressive model without explicit action inputs, such as LongLive, including minute-long rollouts? This would help assess whether the benefits extend beyond action-conditioned models and persist over longer generation.

(2) Table 2 adds Key block-output alignment after shared centers and Value quantization-aware fitting. Could you also apply Key alignment directly to the Joint PCA + INT4 baseline, keeping the coefficient dimensions, quantization, calibration data, and Key-training budget unchanged? Comparing this configuration with the full method would show how much the other two components contribute once Key alignment is present. Ablations removing each component from the full method would help separate their individual contributions.

(3) Could you evaluate smaller history windows, using the same window length and history-selection policy across methods? Please report quality, effective KV compression, peak GPU memory, and end-to-end latency. At what window sizes do NanoKV and NanoKV-Fast retain a useful advantage over uncompressed caching and the compression baselines?

4 comments
AuthorsOpenReview ↗

Response to Reviewer vjCw (Part 1)

We thank the reviewer for recognizing the storage savings and the rationale of our compression design. PDF revisions are marked in red.

W1: 509-frame generation and repeated scene revisits

Response. To examine error accumulation over longer rollouts and the growing gap between NanoKV-Fast and NanoKV, we extended Table 2's 20 samples from 253 to 509 frames with two out-and-back camera cycles. We kept a 32-latent-frame history window and used a same-length Original reference. PSNR, SSIM, and LPIPS measure fidelity to Original; VBench follows the paper's quality evaluation. Full-sequence results are below:

MethodPSNR ↑SSIM ↑LPIPS ↓VBench ↑
Original———0.8166
NanoKV17.320.54550.31760.8064
NanoKV-Fast16.450.51670.39640.7878

As shown above, NanoKV's full-sequence VBench remains close to the same-length Original, while NanoKV-Fast has lower reference fidelity and VBench. To determine whether these quality losses worsen later in generation, we compared early and late segments (frames 1–252 and 253–508), including Original's own quality change. Results are below:

MethodEarly PSNR ↑Early LPIPS ↓Early VBench ↑Late PSNR ↑Late LPIPS ↓Late VBench ↑
Original——0.8259——0.8175
NanoKV19.210.25560.819715.460.37870.8092
NanoKV-Fast18.650.29290.812914.290.49820.7843

As shown above, reference fidelity declines later for both compressed methods, with a larger decline for NanoKV-Fast. NanoKV's VBench decline is close to Original's, while NanoKV-Fast's is larger. This indicates accumulating compression deviations and a greater late-stage quality cost from NanoKV-Fast's additional approximation. Visual inspection of revisits shows that both retain the main subjects and scene content, with local deformation more pronounced in NanoKV-Fast. We have added these results and examples in revised Appendix E.6, Tables 24–25 (p. 32) and Figure 14 (p. 33).

W2 / Q3: Quality, storage, and latency across history windows

Response. To test whether compression remains worthwhile when resident codec costs become more prominent at smaller windows, we swept HY history windows of 8/16/32/48 latent frames on Table 2's 20 samples. Each rollout has 253 frames; methods use the same history-selection policy within each window and its corresponding Original reference.

Storage includes cached KV and resident representations; effective compression divides Original's KV storage by this total. Peak memory and total time are measured end to end. Results are below:

WindowMethodPSNR ↑SSIM ↑LPIPS ↓VBench ↑KV + resident representation (GiB) ↓Effective compression ↑Peak memory (GiB) ↓Total time (s) ↓
8Original———0.82165.241.00×39.76173.55
8KVTC17.560.54490.33900.78721.064.94×35.83187.12
8NanoKV18.500.58430.28800.80730.3017.43×35.83180.75
8NanoKV-Fast18.240.57710.30170.80470.5210.12×35.83177.07
16Original———0.822410.481.00×45.17265.77
16KVTC17.780.55650.32310.79841.228.56×35.91287.70
16NanoKV18.710.60060.26730.81660.4722.38×35.83274.95
16NanoKV-Fast18.450.58670.28890.81230.6915.29×35.83257.83
32Original———0.820020.961.00×55.92449.01
32KVTC17.890.56550.31380.79721.5513.50×36.51479.08
32NanoKV18.960.60960.25230.81420.8026.09×35.83451.82
32NanoKV-Fast18.590.59200.28370.80761.0220.53×35.99410.96
48Original———0.818231.441.00×66.58631.93
48KVTC17.960.57080.31360.79331.8816.73×39.82638.56
48NanoKV18.990.61550.25510.81211.1427.61×36.28648.66
48NanoKV-Fast18.200.58390.31730.79701.3623.19×36.42595.03

As shown above, NanoKV retains better quality and effective compression than KVTC at all tested windows, including window 8. At window 8, all compressed methods share the same memory peak because text-prompt encoding, before history KV exists, determines that peak.

For latency, standard NanoKV remains slower than Original. NanoKV-Fast is faster than Original at windows 16/32/48 and faster than KVTC at every tested window. Thus, small-window storage savings remain useful, but speed gains depend on window size.

Shorter history is not always preferable: it limits information available for generation and may affect quality and scene memory. NanoKV primarily targets applications requiring longer history. We have added this sweep in revised Appendix E.5, Table 22 (p. 31).

AuthorsOpenReview ↗

Response to Reviewer vjCw (Part 2)

Q1: Short and minute-long text-conditioned generation on LongLive

Response. To test applicability beyond action-conditioned models, including minute-long generation, we evaluated LongLive without explicit action inputs on 40 held-out prompts. We generated 165- and 969-frame videos, the latter approximately one minute. Both lengths used the native 12-latent-frame window and the same offline-fitted codecs, with the backbone frozen.

PSNR, SSIM, and LPIPS use the corresponding Original reference; VBench follows the paper's protocol. KV + codec includes cached data and resident codec storage; effective compression is Original's KV storage divided by this total. AR peak measures allocated GPU memory during autoregressive generation, whereas time is end to end. Results on an A800 are below:

Rollout length (frames)MethodPSNR ↑SSIM ↑LPIPS ↓VBench ↑KV + codec (MiB) ↓Effective compression ↑AR peak memory (GiB) ↓End-to-end time (s) ↓
165Original———0.86653290.631.00×7.2535.23
165NanoKV23.020.70690.18500.8631185.3517.75×4.3237.40
969Original———0.85763290.631.00×7.32169.69
969NanoKV16.010.60510.30320.8524185.3517.75×4.40190.11

As shown above, the same codec provides 17.75× effective compression and lower AR peak memory at both lengths, while keeping VBench close to Original. Reference fidelity decreases with rollout length, but inspected videos retain largely intact subjects and scenes without obvious deformation. These results support applicability to text-conditioned generation through minute-long rollouts. We have added the results and visual comparisons in revised Appendix E.7, Table 26 (p. 33) and Figure 15 (p. 34).

Q2: Key alignment on Joint PCA + INT4 and component removal

Response.

1. Key alignment without shared centers or Value fitting. To separate Key alignment's contribution from the preceding components in Table 2, we applied it directly to Joint PCA + INT4. We used Table 2's 20 samples and compression configuration, keeping coefficient dimensions, quantization, calibration data, and Key-training budget unchanged. Results are below:

ConfigurationPSNR ↑SSIM ↑LPIPS ↓VBench ↑
Joint PCA + INT416.380.49530.40770.7753
Joint PCA + INT4 + Key alignment18.120.58090.26540.8212
Full NanoKV20.200.66920.19680.8171

As shown above, Key alignment improves all four metrics even without shared centers or Value fitting. The full method further improves PSNR, SSIM, and LPIPS, showing that the other components remain useful when Key alignment is present.

2. Leave-one-component-out ablation. To isolate each component's contribution within the full method, we removed shared centers, Value fitting, or Key alignment individually, retaining the other components and using the same samples and compression configuration as Table 2. Results are below:

ConfigurationPSNR ↑SSIM ↑LPIPS ↓VBench ↑
Full NanoKV20.200.66920.19680.8171
Without shared centers18.580.60190.24110.8208
Without Value fitting20.170.66530.19880.8171
Without Key alignment18.540.58390.29220.8007

As shown above, shared centers and Key alignment are the main contributors to reference fidelity in this setting: removing either causes larger losses than removing Value fitting. This agrees with the additive ablation in Table 2. We have added these ablations in revised Appendix E.1, Tables 16–17 (p. 28).

While NanoKV improves efficiency for short video generation, I remain concerned about its performance on longer videos and its generation quality under text-only conditioning. Nevertheless, I acknowledge its substantial memory savings. Taking both these benefits and the remaining limitations into account, I will maintain my current rating.

Thank you for your continued evaluation and constructive feedback. We appreciate your recognition of NanoKV's memory savings and understand your remaining concerns. Your suggestions have helped us improve the paper and clarify its scope and limitations.

Reviewer_KC5YOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes NanoKV, a generation-aware KV cache compression framework for autoregressive video generation models. To achieve high compression ratios while preserving generation fidelity, NanoKV exploits cross-head and channel redundancy through joint head–channel compression, shared-center residual coding, and INT4 quantization. It further adopts role-specific optimization strategies for Keys and Values: Key decoders are trained to align Transformer block outputs, while Value encoders and decoders are optimized through the actual quantization path. The optional NanoKV-Fast variant further reduces historical-attention computation by performing mixed-dimensional attention directly from compressed representations, without retraining the backbone. Experiments on HY-WorldPlay, Matrix-Game 3.5, and LingBot-World 2.0 demonstrate substantial KV storage reductions while maintaining generation fidelity. NanoKV achieves up to 26.09× effective KV compression and reduces peak GPU memory usage by up to 35.9%.

Critical strengths

Strengths

  • This paper analyzes the redundancy of key-value (KV) data in video LLMs based on findings on KV compression in the LLM field, a crucial aspect for researching video KV compression.

  • This paper demonstrates the effectiveness of NanoKV on three autoregressive video generation models: HY-WorldPlay, Matrix-Game 3.5, and LingBot-World 2.0. Based on HY-WorldPlay, NanoKV achieves up to 26.09 times effective KV compression and reduces peak GPU memory usage by up to 35.9%, showcasing excellent compression performance.

  • The authors provide a detailed exposition, experiments, and comparisons of similar representative KV compression methods, including component ablation, pipeline procedures, and quantitative results. These findings solidify the novelty and solidity of NanoKV.

Critical weaknesses

Weaknesses

  • Limited Model-Specific Fitting and Generalization Ability. NanoKV requires encoder-decoder fitting, and NanoKV-Fast also relies on model-specific low-dimensional attention maps and history preservation strategies. These configurations are developed individually for each backbone network, which may limit their scalability and generalization capabilities.

  • The Lack of Discussion with Related Work. The authors mention Selecting historical tokens as a representative method for KV compression in the "Related Work" section. This type of method reduces KV cost by retaining only a subset of the history, and the efficiency improvements in inference latency and memory usage are obvious. However, the paper lacks comparisons with other works on this topic.

  • The Lack of Explanation of Novelty and Uniqueness. NanoKV combines low-dimensional projection, INT4 quantization, shared-center residual coding, and role-specific K/V optimization. The entire framework is reasonable for significant KV compression. However, these methods are not specific to video LLM/KV (as mentioned in the paper), and the authors do not seem to point out the uniqueness of the proposed method in video LLM.

Questions for authors

Questions

  1. NanoKV does not retrain the video generation model, but trains a KV compressor adapted to the original model while freezing the backbone. In this case, as shown in Table 1, NanoKV outperforms the baseline settings in fidelity metrics such as PSNR. Since PSNR measures fidelity relative to the original model's rollout rather than ground-truth video quality, could the authors clarify which components contribute most to this improvement?
  2. The authors repeatedly mention "reduce attention computation" in the paper, hoping to reduce inference latency. However, as shown in Table 1, NanoKV and NanoKV-Fast only achieve a maximum speedup of 1.06x under the three model settings, and only 0.98x under the LingBot-World 2.0 setting. Could the authors clarify the main bottlenecks? Also, would methods that select historical tokens to compress KV have a better speedup effect (both memory and latency)?
  3. The choice of joint head–channel compression is mainly justified by higher variance retention under matched 8× compression, which does not necessarily imply better generation quality. The comparison is also limited to HY-WorldPlay and does not account for differences in projection overhead. Could authors compare the generation quality of different compression methods and include the storage overhead of projection matrices to determine whether joint head–channel compression still offers an advantage.
  4. I think it would be more valuable for the authors to explain the differences between NanoKV, KVTC, and other PCA-based or SVD-based (Singular Value Decomposition) methods.
  • I will definitely raise my score when my problems are solved.
6 comments
AuthorsOpenReview ↗

Response to Reviewer KC5Y (Part 1)

We thank the reviewer for recognizing the importance of KV compression for video generation and our results across three backbones. PDF revisions are marked in red.

The responses are grouped as follows: Part 1 covers W1, W2, and Q1; Part 2 covers W3 and Q4; Part 3 covers Q2 and Q3.

W1: Model-specific fitting, scalability, and generalization ability

Response. Regarding the cost and scalability of model-specific fitting and configuration, we clarify:

1. NanoKV fitting. Data collection, codec fitting, and validation are automated. Each backbone undergoes one offline adaptation, and the fitted parameters are reused across test inputs, without per-video fitting.

2. NanoKV-Fast configuration. Starting from a fitted NanoKV codec, NanoKV-Fast learns to compute attention in fewer dimensions using the same compressed cache. The configuration workflow is largely automated, determining each head's dimension and which history retains full-dimensional attention, followed by final quality-guided refinement:

  1. Automated collection: Run calibration videos with standard NanoKV and collect attention features and full-dimensional attention outputs. These provide the inputs and reference outputs for fitting the low-dimensional version.
  2. Automated fitting: Using these data, fit attention maps at several candidate dimensions for each layer, so their outputs approximate the reference. The backbone and codec remain frozen.
  3. Automated initial allocation: Compare each candidate's output with the reference. Use these errors to assign dimensions to attention heads, retaining more dimensions where reduction causes larger errors, within a compute budget.
  4. Quality-guided refinement: Automatically evaluate the resulting configuration against standard NanoKV on a fixed development set, with limited manual adjustments to the remaining sensitive layers or history-protection settings, followed by re-evaluation.

The development set is separate from evaluation, and protection changes reuse fitted maps without retraining. Different backbones reuse this workflow, with separately fitted parameters and locally refined configurations. All adaptation is offline; inference uses fixed configurations with no online training or selection. Three-backbone results support cross-model applicability. We have clarified the workflow and its scope in revised Appendices B.2 (p. 15) and C.1 (p. 17).

W2: Discussion with related work: selecting historical tokens

Response. Regarding comparison with historical-token selection, we have expanded the original related-work discussion (revised Section 2, p. 3) and added a direct experiment. We adapted Forcing-KV to HY-WorldPlay and compared quality, storage, and efficiency on Table 2's 20 samples. Results are below:

MethodPSNR ↑SSIM ↑LPIPS ↓VBench ↑KV (GiB) ↓Compression ↑Peak memory (GiB) ↓Time (s) ↓Speedup ↑
Original————20.961.00×55.92449.011.00×
KVTC17.890.56550.31380.79721.5513.50×36.51479.080.94×
Forcing-KV (HY)18.640.57480.30620.810311.231.87×45.54418.961.07×
NanoKV18.960.60960.25230.81420.8026.09×35.83451.820.99×
NanoKV-Fast18.590.59200.28370.80761.0220.53×35.99410.961.09×

As shown above, NanoKV and NanoKV-Fast both retain similar or higher reference fidelity with substantially less KV storage and lower peak memory. Compared with Forcing-KV, NanoKV improves PSNR by 0.32 dB, with better SSIM and LPIPS, while reducing KV storage from 11.23 to 0.80 GiB; NanoKV-Fast reduces KV storage to 1.02 GiB with similar PSNR and speed to Forcing-KV. We have added this comparison in revised Appendix E.2, Table 18 (p. 29).

Q1: Which components contribute most to fidelity improvement?

Response. Regarding which components improve fidelity relative to Original's rollout, Table 2 identifies shared centers and Key alignment as the main contributors in this HY setting. As a complement, we removed each component from full NanoKV using the same 20 samples and compression configuration as Table 2, examining its contribution with the other components retained. Results are below:

ConfigurationPSNR ↑SSIM ↑LPIPS ↓VBench ↑
Full NanoKV20.200.66920.19680.8171
Without centers18.580.60190.24110.8208
Without Value fitting20.170.66530.19880.8171
Without Key alignment18.540.58390.29220.8007

As shown above, removing centers or Key alignment causes the largest reference-fidelity losses, while removing Value fitting has a smaller effect. Both additive and removal ablations identify centers and Key alignment as the main contributors in this setting. We have added these results in revised Appendix E.1, Table 17 (p. 28).

AuthorsOpenReview ↗

Response to Reviewer KC5Y (Part 2)

W3: Explanation of novelty and uniqueness

Response. NanoKV targets autoregressive video-generation DiTs, rather than video-understanding LLMs. Regarding novelty beyond existing compression techniques, our contribution is generation-aware, role-specific KV compression: we optimize Keys and Values according to their different uses by the frozen backbone. This matters for video generation because compressed history affects later blocks and frames, allowing errors to accumulate over a rollout.

  1. Keys: preserving history retrieval. Keys determine which history is attended to; feature reconstruction error alone does not capture changes in retrieval. We therefore train the Key decoder with block-output alignment. On HY 32×, this adds 1.75 dB PSNR over no Key post-training, versus 0.04 dB for reconstruction-only fitting.
  2. Values: preserving retrieved content. Values carry the retrieved content passed to later layers. We therefore fit Value codecs through the deployed INT4 quantization path, accounting for quantization in content recovery.
  3. Evidence for the role-specific design. Swapping the Key and Value optimization strategies lowers PSNR by 1.45 dB on HY 32×, supporting their different objectives.

These objectives operate on a compact representation built from established projection, quantization, and residual coding. We subtract shared centers and jointly compress head–channel residuals; the joint axis improves PSNR by 3.52–4.69 dB over the other tested PCA axes (Q3). We have added objective and role-swap studies in revised Appendix E.1, Tables 12 and 14 (p. 27), and expanded related work in Section 2 (p. 3) and Appendix E.2 (p. 29).

Q4: Differences from KVTC and other PCA/SVD methods

Response. Regarding the differences between NanoKV and existing PCA/SVD methods, our core contribution is generation-aware optimization tailored to the different roles of Keys and Values, combined with a compact historical KV representation—not a new low-rank decomposition. We compare KVTC, Palu, Eigen Attention, and xKV in terms of mechanisms, costs, and results.

1. Compression mechanisms and objectives. KVTC combines PCA, reconstruction-error-guided bit allocation, and entropy coding. Palu factorizes K/V projection weights and incorporates low-rank-aware quantization. Eigen Attention computes attention in low-rank activation subspaces and selects dimensions using layer-output error. xKV constructs shared representations through cross-layer SVD.

In contrast, NanoKV directly optimizes codecs with distinct objectives for the different roles of K and V: Key block-output alignment reduces historical compression’s impact on subsequent computation, while Value reconstruction through the deployed quantization path mitigates information loss. Unlike Eigen Attention’s use of output error to select dimensions, NanoKV uses this error to train the Key decoder at a fixed dimension.

2. Model adaptation and inference costs. In their basic configurations, KVTC, Palu, Eigen Attention, and NanoKV all support one-time adaptation before deployment without retraining the backbone, with the resulting parameters reused across inputs. Unlike xKV’s input-specific SVD during prefill, NanoKV directly uses fixed codecs during inference, without per-input refitting or decomposition.

3. Compression results. In Table 1, NanoKV achieves higher PSNR and SSIM with less KV storage on all three video backbones. On HY, compared with KVTC, KV storage decreases from 1.55 to 0.80 GiB while PSNR improves by 1.00 dB. Palu, Eigen Attention, and xKV also report benefits from quantized low-rank storage, low-dimensional attention, and cross-layer compression, respectively. However, these results concern different language-model tasks and should not be directly ranked numerically. Our matched comparison with KVTC directly demonstrates NanoKV’s quality–storage advantage in video generation.

We have expanded the related-work discussion in revised Section 2 (p. 3) and added a method comparison in Appendix E.2, Table 19 (p. 29).

AuthorsOpenReview ↗

Response to Reviewer KC5Y (Part 3)

Q2: Main bottlenecks in reducing inference latency

Response.

1. Bottlenecks. NanoKV-Fast aims to retain high KV compression while reducing its runtime cost through lower-dimensional attention; compression is the primary goal. To explain the reported 1.06×/0.98× speedups, we profiled HY and LingBot (end-to-end speedups are reported in Table 1), separating each generation into:

  • Denoising attention core: attention-weight computation and aggregation of Values during denoising.
  • Other denoising: the rest of each denoising step, including projections, positional transforms, feed-forward layers, cache decoding, and low-dimensional mappings.
  • History recomputation/write: preparing historical features and updating the cache for subsequent generation.
  • Remaining pipeline: input processing, VAE video decoding, and scheduling outside the above stages.

Results are below:

ModelMethodDenoising attention core (s)Other denoising (s)History recomputation/write (s)Remaining pipeline (s)Total (s)
HYOriginal91.5990.84222.0650.96455.44
HYKVTC94.69117.55244.0349.22505.49
HYNanoKV92.9497.06240.3448.26478.61
HYNanoKV-Fast67.8276.98238.5047.03430.33
LingBotOriginal20.4648.1717.0142.79128.43
LingBotKVTC20.1969.4124.1548.65162.39
LingBotNanoKV20.2354.8320.4242.67138.15
LingBotNanoKV-Fast16.6653.8219.1544.17133.79

As shown above, denoising attention accounts for only 20.10%/15.90% of Original's total runtime on HY/LingBot. NanoKV-Fast reduces this time by 27.00%/17.70% relative to NanoKV, but the limited attention share, other inference stages, and compression overhead constrain end-to-end speedup. HY spends substantial time on history recomputation; on LingBot, attention savings do not fully offset compression overhead.

2. Token selection. Regarding whether historical-token selection offers better memory and latency benefits, W2 shows that NanoKV and NanoKV-Fast substantially reduce KV storage and peak memory while retaining similar or higher reference fidelity; NanoKV-Fast also maintains end-to-end speed comparable to Forcing-KV. Thus, compressing historical representations provides greater storage and memory savings in this comparison.

We have added this runtime analysis in revised Appendix E.3, Table 20 (p. 30).

Q3: Joint head–channel compression: quality and projection overhead

Response. Regarding whether Figure 3(a)'s variance advantage translates into generation quality after accounting for projection overhead, we compared the following PCA axes on HY at matched 8× dimensional compression:

  • Spatial (separable): reduce height and width separately within each frame.
  • Spatiotemporal: reduce time, height, and width along separate axes.
  • Per-head: reduce feature channels independently within each attention head.
  • Spatial (full): jointly reduce all 1560 spatial positions to 195 dimensions; this adds a larger spatial-matrix control.
  • Joint head–channel: reduce the concatenated features of all heads together, capturing cross-head redundancy.

The full spatial control has matrix storage of the same order as joint PCA. Original KV occupies 13.10 GiB for 20 historical latent frames; total storage below includes cached data, projection matrices, and means. Results are below:

PCA axisPSNR ↑SSIM ↑LPIPS ↓Total storage (GiB) ↓Effective compression ↑
Spatial (separable)16.930.47680.48671.867.06×
Spatiotemporal17.120.48050.47321.867.06×
Per-head16.030.46210.51681.877.01×
Spatial (full)17.200.49330.48141.986.63×
Joint head–channel20.720.59610.26282.076.34×

As shown above, joint head–channel compression gives the best reference fidelity and retains 6.34× effective compression including projection overhead. Even full spatial PCA falls far short in quality. Inspected revisit examples also favor joint PCA in subject and scene preservation. We have added these results in revised Appendix E.4, Table 21 (p. 30).

Reviewer_4nVkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

NanoKV compresses the historical KV cache of autoregressive video models while keeping the backbone frozen:

  • It subtracts shared K-means centers, then jointly encodes the residuals across heads and channels into INT4 coefficients (Eq. 2).
  • For Keys, only the decoder is trained, with a block-output alignment objective Lout+LK,rec\mathcal L_{out}+\mathcal L_{K,rec} (Eq. 3). Values are fitted for quantization-aware reconstruction (Eq. 4).
  • The optional NanoKV-Fast builds low-dimensional attention directly from the coefficients and merges branches with LSE.

Experiments cover HY-WorldPlay, Matrix-Game 3.5, and LingBot-World 2.0. On HY, NanoKV reaches 26.09×26.09\times effective compression (KVTC: 13.50×13.50\times) with +1.0 dB PSNR, and Fast runs about 1.15×1.15\times faster than KVTC (Table 1).

Critical strengths
  1. Effective CR includes resident operators and metadata, and Original receives the same optimizations (zero pixel error). Everything I recomputed is self-consistent: all 15 CRs in Table 1, the peak-memory reductions (35.9% / 25.8% / 29.0%), the 1.15×1.15\times (= 1.06/0.92), and the Table 11 products.

  2. Clear frontier gain on HY. At matched 19 dB quality, NanoKV reaches about 2.84×2.84\times KVTC's compression ratio (Fig. 5a). The Key-alignment step adds +1.66 dB (Table 2).

  3. Three backbones, 400 videos per configuration, and disjoint fitting and evaluation images.

  4. The paper reports that Fast is slower on LingBot, that error accumulates over the rollout, and that MAC savings are not wall-clock savings.

Critical weaknesses
  1. Table 2 and Fig. 3(b) both compare a trained Key decoder against an untrained PCA decoder. There is no control trained with LK,rec\mathcal L_{K,rec} only, and no K/V role-swap control. The ablation also uses one backbone (HY), 20 videos, and the 16× setting. The +1.66 dB may come simply from training the decoder at all.

  2. Key alignment is not used on Matrix-Game. Table 5 marks it "Not used" for that backbone, and the main text does not say so. That is exactly where NanoKV is worse than KVTC on LPIPS (0.2425 vs 0.2380) and VBench (0.8273 vs 0.8302), with only +0.25 dB PSNR. "Consistently", and Appendix B.2's "beyond a single backbone", therefore do not hold.

  3. Limited memory gain over strong baselines. On HY, PyTorch peak memory is KVTC 36.51 vs NanoKV 35.83 GiB, a 0.68 GiB difference; on Matrix-Game the difference is 0.43 GiB. The paper does not show that the compression enables longer history or larger batches.

  4. Fast's benefit does not match its cost. End-to-end speed is 1.06× / 1.04× / 0.98×, LPIPS on HY rises from 0.2399 to 0.2787, and deployment relies on hand-specified per-layer routing for each model (Table 7).

Questions for authors
  1. Add Key-decoder controls on HY 32× and LingBot: LK,rec\mathcal L_{K,rec} only, Lout\mathcal L_{out} only, and both.

  2. Add a K/V role-swap control.

  3. Explain why Key alignment is "Not used" on Matrix-Game, report results with it enabled, and revise "consistently".

2 comments
AuthorsOpenReview ↗

Response to Reviewer 4nVk (Part 1)

We thank the reviewer for carefully verifying effective compression and memory accounting, and for recognizing the quality–compression advantage. We address related weaknesses and questions together. PDF revisions are marked in red.

W1 / Q1: Key-decoder control studies and ablations

Response.

1. Key objectives. To address whether Table 2's +1.66 dB comes merely from decoder training rather than block-output alignment, we conducted a control study comparing three Key training objectives and a no-post-training baseline on HY and LingBot at 32×, using the same 20 samples within each backbone. In these studies, we fixed shared centers and PCA Value without Value fitting, varying only the Key objective. Results on HY show an independent gain from block-output alignment; gains on LingBot are similar across objectives. Results are below:

ModelKey objectivePSNR ↑SSIM ↑LPIPS ↓VBench ↑
HY 32×No Key post-training17.240.53010.36840.7828
HY 32×Reconstruction only (LK,rec\mathcal{L}_{K,rec})17.280.53190.36650.7855
HY 32×Block-output alignment only (Lout\mathcal{L}_{out})18.990.60710.24730.8148
HY 32×Both (our objective)19.040.61110.24780.8140
LingBot 32×No Key post-training19.060.58300.23290.8282
LingBot 32×Reconstruction only (LK,rec\mathcal{L}_{K,rec})19.880.59130.21940.8273
LingBot 32×Block-output alignment only (Lout\mathcal{L}_{out})19.860.58710.22650.8278
LingBot 32×Both (our objective)19.860.58950.22530.8272

As shown above, Key post-training improves reference fidelity on both backbones, but the objectives contribute differently across backbones. On HY, block-output alignment contributes independently beyond reconstruction-only training, and the combined objective achieves the highest PSNR and SSIM; on LingBot, all three objectives yield similar quality gains.

This objective comparison also clarifies our design rationale: block-output alignment directly optimizes compression's effects on generation, while reconstruction provides auxiliary regularization to preserve the decoder's reconstruction capability.

2. Ablation coverage. Regarding your concern that ablations cover only HY, 20 videos, and 16×, we conducted LingBot 32× ablations on the same 20 samples. Results are below:

LingBot 32× configurationPSNR ↑SSIM ↑LPIPS ↓VBench ↑
Joint PCA + INT416.630.48300.36280.8287
+ Shared centers19.060.58300.23290.8282
+ Key alignment19.860.58950.22530.8272
+ Value quantization-aware fitting (full method)20.010.59590.22330.8277

As shown above, the components progressively improve reference fidelity on LingBot, while VBench remains similar. Specifically, shared centers provide the largest gain; Key alignment and Value fitting further improve fidelity. This extends component validation beyond HY and the original 16× setting. We have added these studies in revised Appendix E.1, Tables 12–13 (p. 27).

Q2: K/V role-swap control study

Response. To address the missing K/V role-swap comparison, we conducted a control study on HY 32×. Starting from full NanoKV, K codecs were fitted through deployed quantization; V encoders were fixed, and V decoders were fitted with block-output alignment and reconstruction constraints. The original strategy outperforms the swapped strategy on all quality metrics. Results are below:

HY 32× configurationPSNR ↑SSIM ↑LPIPS ↓VBench ↑
Original K/V strategy18.960.60960.25230.8142
Swapped K/V strategy17.510.54170.34210.7933

As shown above, the original strategy outperforms the swapped strategy on all four quality metrics, supporting our design of distinct optimization objectives for the different roles of K and V. We have added this study in revised Appendix E.1, Table 14 (p. 27).

AuthorsOpenReview ↗

Response to Reviewer 4nVk (Part 2)

W2 / Q3: Key alignment “Not used” on Matrix-Game and “consistently”

Response.

1. Key alignment on MG. For MG's “Not used” entry in Table 5, development tests showed small fidelity gains from Key alignment. Also, we varied only Key alignment on 20 samples under the main table's history protocol. Results are below:

MG configurationPSNR ↑SSIM ↑LPIPS ↓VBench ↑
Without Key alignment19.150.63120.24600.8370
With Key alignment19.210.63380.24170.8376

As shown above, Key alignment yields small improvements in all four MG metrics, with a smaller gain than on HY. Also, visual quality is already close to Original before alignment, suggesting limited room for improvement.

2. Cross-backbone claims. Regarding your MG–KVTC comparison—LPIPS 0.2425 vs. 0.2380, VBench 0.8273 vs. 0.8302, and +0.25 dB PSNR—we clarify: “consistently” and “beyond a single backbone” concern overall gains, not superiority of every component or metric. NanoKV improves all four quality metrics, KV storage, and speed on HY/LingBot. On MG versus KVTC, it improves PSNR/SSIM, cuts KV from 1.24 to 0.80 GiB, and runs ~1.61× as fast. Thus, Table 1 supports better overall quality–storage–efficiency trade-offs across all three backbones. We have clarified MG's omission of Key alignment in revised Section 5.2 (p. 8), added its study in Appendix E.1, Table 15 (p. 28), and clarified claim scope in the abstract, Section 1, and Appendix B.2 (p. 15).

W3: Memory gain over strong baselines and longer history

Response.

1. Longer history. To address your concern about limited peak-memory savings over KVTC (0.68 GiB on HY) and longer-history support, we tested 48/64-latent-frame windows on an A100 80 GB. At window 64, Original OOMs; NanoKV completes generation with 5.06 GiB less peak memory than KVTC. Results are below:

HY history windowMethodKV + resident representation (GiB) ↓Effective compression ↑Peak memory (GiB) ↓Total time (s) ↓
32Original20.961.00×55.92449.01
32KVTC1.5513.50×36.51479.08
32NanoKV0.8026.09×35.83451.82
48Original31.441.00×66.58631.93
48KVTC1.8816.73×39.82638.56
48NanoKV1.1427.61×36.28648.66
64Original41.921.00×OOM—
64KVTC2.2118.99×46.902510.98
64NanoKV1.4728.44×41.842301.81

Rollouts have 253 frames at windows 32/48 and 509 at window 64. As shown above, in the tested settings, NanoKV's peak-memory advantage over KVTC grows with history length, making storage savings more beneficial. KVTC's larger temporary encoding tensors also increase its peak memory.

2. Larger batches. To test larger batches, we tested 253-frame generation with 32-latent-frame history on an A100 80 GB, with identical inputs and independent noise. Original runs out of memory at batch size 2; NanoKV completes generation at batch size 4 with 71.86 GiB peak allocated memory. Thus, NanoKV's storage savings enable longer histories and larger batches. We have added long-history and batch results to revised Appendix E.5 (Table 23, p. 31; p. 32).

W4: NanoKV-Fast's benefit, cost, and hand-specified per-layer routing

Response.

1. Quality–efficiency trade-off. Regarding the 1.06×/1.04×/0.98× speedups and HY's LPIPS increase from 0.2399 to 0.2787, we clarify that NanoKV-Fast is an optional quality–efficiency configuration for different application needs, not a replacement for standard NanoKV. It trades fidelity for lower attention and compression overhead, saving storage on HY/MG without increasing Original's latency—useful for interactive games or driving simulation.

2. Limited speedup. Regarding limited speedup, NanoKV-Fast targets historical attention; Original's denoising attention takes 20.10%/15.90% of runtime on HY/LingBot. Other backbone operations, cache processing, and mappings remain, so attention savings do not yield proportional end-to-end speedups or fully offset compression overhead on LingBot.

3. Per-layer routing. Regarding hand-specified routing, we clarify that the workflow is automated, with final manual refinement. Offline steps:

  1. Automated collection: Run calibration samples and collect features for mapping fitting.
  2. Automated fitting: Fit mappings at each layer's candidate dimensions, without manual layer-by-layer training.
  3. Automated evaluation and allocation: Compute candidate attention-output errors and automatically allocate initial dimensions within a compute budget.
  4. Limited manual refinement: Inspect generations and increase dimensions, restore full-dimensional attention, or adjust history preservation only for remaining sensitive parts.

After these offline steps, configurations stay fixed without online training or selection. We have added profiling in revised Appendix E.3, Table 20 (p. 30), and clarified this workflow in Appendix C.1 (p. 17).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Author Responses and Manuscript Revisions

Summary of Author Responses and Manuscript Revisions

We thank the reviewers for their constructive feedback. We have responded to each review and incorporated additional experiments and clarifications into the revised manuscript, with changes highlighted in red.

NanoKV combines generation-aware, role-specific K/V optimization with a compact historical representation to enable high-ratio KV compression while keeping the backbone frozen. Across three video-generation backbones, NanoKV achieves higher PSNR and SSIM with less KV storage than KVTC.

Below, we summarize the additional evidence and revisions addressing the main reviewer concerns.

1. Whether Key alignment gains come merely from decoder training, and which components contribute most

We conducted Key-objective control studies on HY-WorldPlay and LingBot, a K/V optimization role-swap study, direct Key alignment on Joint PCA + INT4, and additional component ablations. These studies establish an independent contribution of block-output alignment beyond reconstruction-only training, support distinct optimization objectives for K and V, and clarify the components’ contributions to reference fidelity.

(4nVk: W1, Q1–Q2; KC5Y: W3, Q1; vjCw: Q2. Revised Appendix E.1.)

2. Whether storage savings yield practical memory benefits and support longer histories or larger batches

We evaluated quality, storage, peak memory, and latency across history windows and provided a matched batch-capacity comparison with KVTC. We also profiled runtime components to explain the main end-to-end speedup bottlenecks and added a direct comparison with Forcing-KV. The results show that NanoKV retains storage advantages at small windows and achieves a larger peak-memory advantage over KVTC at longer windows. Compared with Forcing-KV, NanoKV and NanoKV-Fast retain similar or higher reference fidelity with substantially less KV storage and lower peak memory.

(4nVk: W3–W4 and follow-up; KC5Y: W2, Q2; vjCw: W2, Q3. Revised Appendices E.2, E.3, and E.5.)

3. Behavior over longer rollouts and repeated scene revisits, and applicability to minute-long text-conditioned generation

We extended generation to 509 frames with two out-and-back camera cycles and analyzed quality using full-sequence metrics, early/late comparisons, and visual inspection. We also evaluated short and minute-long generation on LongLive using the same fitted codecs. The results show that NanoKV retains the main subjects and scene content during revisits and maintains compression and memory savings with VBench close to Original in minute-long text-conditioned generation, supporting applicability beyond action-conditioned models.

(vjCw: W1, Q1. Revised Appendices E.6–E.7.)

4. The practical advantage of joint head–channel compression and differences from existing PCA/SVD methods

We compared generation quality across PCA compression axes, including projection matrices and other resident representations in total storage. We also compared NanoKV with KVTC, Palu, Eigen Attention, and xKV in terms of optimization objectives, adaptation and inference costs, and reported results. These additions demonstrate a favorable quality–storage trade-off for joint head–channel compression after accounting for projection overhead and clarify NanoKV’s core design of optimizing K and V according to their different roles.

(KC5Y: W3, Q3–Q4. Revised Section 2 and Appendices E.2 and E.4.)

5. Model-specific adaptation costs, cross-backbone benefits, and the role of NanoKV-Fast

We clarified the automated codec-fitting and configuration steps and the limited manual refinement, added Key-alignment experiments on Matrix-Game, and refined the cross-backbone claims and quality–efficiency discussion. These clarifications explain how adaptation costs are controlled through a one-time, largely automated offline workflow with limited manual involvement and parameter reuse across inputs. They also clarify that Key-alignment gains vary across backbones and that NanoKV and NanoKV-Fast serve different quality and latency requirements.

(4nVk: W2, Q3, W4; KC5Y: W1. Revised abstract, Sections 1 and 5.2, and Appendices B.2, C.1, E.1, and E.3.)

We hope this summary helps the Area Chair and reviewers navigate the additional evidence and corresponding revisions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.