acceptodds
Withdrawn from ICLR 2027

Elastic-QVG: Efficient Long Video Generation with Elastic KV-Cache Quantization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Auto-regressive video generation uses a growing KV-cache to retain long-range context. Existing KV-cache quantization reduces the memory cost per token but still allows total memory usage to grow with context length, whereas eviction bounds memory by discarding cached context entirely. To bridge this gap, we present **Elastic-QVG, a training-free framework that progressively lowers cache precision without re-quantizing quantized caches.** A bit-level sliding window extends conventional token-level windowing to individual bit-planes, keeping recent chunks at higher precision while truncating older chunks to fit a fixed memory budget. Each chunk is quantized once, and its precision can subsequently decrease as new context arrives or the available memory budget shrinks. A truncation-exact quantizer enables hardware-friendly truncation directly on the quantized KV-cache, with no additional error relative to direct lower-bit quantization. Its nested quantization grids make dropping bit-planes exact, while separately allocated bit-plane storage and fused dequantization turn this property into memory savings without repacking the cache. Across LongCat-Video, HY-WorldPlay, and LingBot-World, Elastic-QVG improves fidelity over static QVG and retains VBench scores close to BF16 at 13–15× compression. It reduces KV memory by up to 12.7× with less than 5% latency overhead on H100, and avoids KV offloading on an RTX 5090 to achieve a 4.3× speedup over offloaded BF16-cache inference.

Primary area:
generative models
Keywords:
Elastic Quantization, Video Generation, KV Cache

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YN7XOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents Elastic-QVG, a training-free method for reducing the KV-cache memory used by autoregressive video generation. It stores new history at higher precision and gradually reduces the precision of older history when the cache reaches a fixed memory budget. The proposed encoding allows stored values to lose bits without being re-quantized, and the bit-plane storage makes it possible to release the corresponding memory. Experiments on LongCat-Video, HY-WorldPlay, and LingBot-World show substantial memory savings and generally better quality than static QVG at comparable cache sizes.

Critical strengths

(1) Training-free and easy to adopt. The method does not require fine-tuning the video model or training a separate compression model. This lowers the cost of applying it to existing generators. Reducing precision directly on the stored cache also avoids repeated decoding and re-quantization.

(2) An elegant way to control KV-cache memory, with promising results. Gradually reducing precision provides a useful choice between keeping history at a fixed precision and deleting it. The proposed storage design turns this idea into actual memory savings. The reported quality results are also encouraging: Elastic-QVG generally preserves the original model’s outputs better than static QVG at comparable cache sizes.

Critical weaknesses

(1) Insufficient validation on long videos. Long-video generation is a main motivation of the paper, but the experiments do not sufficiently show whether quality remains stable during extended generation. Figure 10 demonstrates bounded memory usage, rather than the corresponding video quality, and the total video lengths used in the main quality comparisons are not consistently specified. Small quality losses over shorter sequences may grow as generation continues and older history is compressed or removed. Quality measurements over longer rollouts, especially after the cache reaches its memory budget, are needed to support the paper’s long-video claims.

(2) Limited ablations of key design choices. The quantizer comparisons and precision-budget sweep are useful, but they do not clearly separate the contributions of companding and dynamic precision allocation. For example, comparing p=0.6p=0.6 with p=1p=1 would show the benefit of companding, while using the same quantizer under fixed and dynamic precision would help isolate the benefit of the elastic policy. A sink-plus-recent window under the same memory budget would also test whether retaining distant low-precision history is a better use of memory. Without these controlled comparisons, it is difficult to determine which design choices drive the reported improvements over QVG.

Questions for authors

(1) Ablation of companding and dynamic precision allocation. Can you compare p=0.6p=0.6 with p=1p=1, which disables companding while preserving truncation-exactness? Can you also compare fixed and dynamic precision allocation using the same quantizer under matched memory budgets? These experiments would clarify how much of the improvement comes from companding and how much comes from the elastic allocation policy.

(2) Long-video validation on text-only streaming models. Can you evaluate substantially longer rollouts, such as minute-long videos, on text-only streaming generation models such as LongLive or Rolling Forcing? Please report quality as a function of generation length, including after the cache reaches its budget and older chunks are heavily compressed or evicted. Additional action-conditioned world-model results would not resolve this concern: this evaluation should exclude action inputs so that the effect of KV compression can be assessed without action guidance. Please specify the frame rate, total generated frames, and history-window settings.

(3) Comparison with fixed-bit quantization and a sink-plus-recent sliding window. Can you compare Elastic-QVG with a baseline that directly quantizes the retained KV cache at a fixed bit-width and keeps only sink tokens and recent history, without progressively reducing the precision of older chunks? Please adjust the window size so that both methods use the same actual KV-memory budget, including metadata, and evaluate them on the same long-video workloads. This comparison would help determine whether the proposed progressive precision reduction preserves video quality better than a simpler combination of fixed-bit quantization and sliding-window eviction.

Reviewer_TPoPOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Elastic-QVG is a training-free method for reducing KV-cache memory in autoregressive video generation. The main idea is to lower the precision of older cached chunks as the cache fills, rather than keeping every chunk at a fixed bit width or immediately discarding old context. A prefix-nested quantizer makes bit truncation match direct lower-bit quantization under the same block normalization and companding transform; this avoids re-quantization, but does not remove the error from using fewer bits. The paper combines this quantizer with a bit-level sliding window and a bit-major layout with fused GPU kernels. Table 1 reports roughly 2–3 dB PSNR gains over static QVG at 2-bit budgets and about 5 dB at 1-bit budgets across LongCat-Video, HY-WorldPlay, and LingBot-World. On H100, the paper reports up to 12.7× lower KV-cache memory with less than 5% latency overhead.

Critical strengths

Fixed-bit quantization reduces the cost per token but does not stop the cache from growing. I like the idea of gradually lowering precision before resorting to eviction. The nested indices give a clean way to do this without recovering the original KV values or adding another round of quantization error. Pairing the representation with a memory-reclamation design is an important part of making the idea practical, although the implementation details need clarification.

Critical weaknesses

I have doubts about the authenticity of the experimental results in the paper. The paper reports evaluations on HY-WorldPlay, LingBot-World, and LingBot-World-v2 using MovieGen prompts. However, these models are primarily designed for image-conditioned video generation, whereas MovieGen Video Bench provides text prompts rather than the initial images, camera trajectories, and action conditions used by these models. The paper does not explain how these text prompts were adapted to the required conditioning inputs. In particular, it remains unclear how the initial images were obtained, how the camera trajectories or action conditions were specified, and whether the same conditioning inputs were used across all compared methods. These details are important because the choice of initial images and control signals can substantially affect the generated content and its temporal dynamics, potentially influencing the measured quantization fidelity. Without a clear description of the evaluation protocol, it is difficult to assess the reproducibility of the experiments and the robustness of the reported improvements across different generation scenarios.

Questions for authors
  1. The paper reports evaluations using MovieGen prompts on HY-WorldPlay, LingBot-World, and LingBot-World-v2, although these models primarily support image-conditioned generation. Could the authors clarify how the text prompts were converted into the required conditioning inputs? Specifically, how were the initial images obtained, and how were the camera trajectories or action conditions specified? Were identical conditioning inputs used for BF16 and all quantization baselines? A detailed description of the evaluation protocol would help establish the reproducibility and reliability of the reported results.

  2. Elastic-QVG introduces both a new truncation-exact quantizer with mid-rise reconstruction and companding, and a bit-level sliding window for dynamic precision allocation. However, the current comparison with static QVG does not clearly isolate the contributions of these components. How does the proposed quantizer perform under uniform INT2 or INT1 precision without the bit-level sliding window? What is the individual contribution of companding to the reported fidelity improvements? These ablations would help clarify whether the gains primarily arise from the quantizer design or the elastic precision allocation strategy.

  3. The proposed bit-level sliding window progressively reduces the precision of older chunks based primarily on their age. However, temporal distance does not necessarily correlate with quantization sensitivity, as some distant historical states may remain important for maintaining object identity, scene consistency, and long-range dependencies. Consequently, aggressively quantizing or evicting older chunks may introduce substantial degradation in scenarios requiring information from distant history. While Figure 10 demonstrates bounded KV-cache memory as the context grows, corresponding generation-quality results are not provided. Could the authors evaluate substantially longer rollouts, particularly scenarios requiring the model to recall previously observed objects or scenes? Additionally, how does Elastic-QVG compare with conventional sliding-window eviction under matched memory budgets? Such evaluations would help establish whether the proposed age-based precision reduction strategy effectively preserves long-range information.

Reviewer_eayZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

A training-free KV-cache compression method. Each chunk is written once at 4 bits. When the footprint exceeds the budget B=btarget⋅NplannedB=b_{\text{target}}\cdot N_{\text{planned}}, low bit-planes are dropped from the oldest chunks, stepping down 4→3→2→1→04\to3\to2\to1\to0 (Alg. 1). Indices are un=⌊2nv⌋u_n=\lfloor 2^n v\rfloor, which satisfy ⌊un/2k⌋=un−k\lfloor u_n/2^k\rfloor=u_{n-k} (Eq. 4), so truncation exactly matches direct lower-bit quantization. Reconstruction is mid-rise, Qn(x)=sX,n(qn(x)+12)Q_n(x)=s_{X,n}\big(q_n(x)+\tfrac12\big), combined with companding g(x)=sign⁡(x),∣x∣pg(x)=\operatorname{sign}(x),|x|^{p} with p=0.6p=0.6. Reported results: 2–5 dB PSNR over QVG at matched compression; up to 12.7×12.7\times KV memory reduction on H100 with under 5% overhead; and a 4.3×4.3\times speedup on RTX 5090.

Critical strengths
  1. Clear framing: one cache serves many memory budgets.

  2. Exactness is rigorously proven, and the numerical example in App. D.3 is checkable: truncation gives −0.69→−0.75-0.69\to-0.75, exactly the result of direct 2-bit quantization.

  3. Very low systems overhead: only 1.4–2.7% over BF16 on B300, on par with static QVG.

Critical weaknesses
  1. The method changes three things at once: the quantizer (mid-rise uses all 2n2^n levels), companding, and precision scheduling, with no ablation of any of them. The missing control is the proposed quantizer at a static uniform bit-width.

  2. There is no equal-memory sliding-window eviction and no L=4,0L={4,0} control, so the claim that intermediate precision tiers beat outright eviction is untested.

  3. Appendix Table 3 contradicts the main text. At 2 bits, Elastic's SSIM is 0.695 (QVG: 0.713) and its LPIPS is 0.238 (QVG: 0.223), so both are worse; compression also differs (5.35×5.35\times vs 5.12×5.12\times). In addition, this table lists 1003 prompts while Table 2 lists 100.

  4. The 4.3×4.3\times speedup includes FP8's contribution. FP8 alone already reaches 1.9×1.9\times (1.26 s); Elastic's own contribution is about 1.26/0.55≈2.3×1.26/0.55\approx2.3\times. FP8 + QVG INT2 is not tested, and Fig. 9 reports 26 GB while Fig. 10 reports about 31 GB, with no explanation of the difference.

Questions for authors
  1. Report the proposed quantizer at static uniform 2/1 bits.

  2. Compare against sliding-window eviction (with sinks) and L=4,0L={4,0} at equal peak memory.

  3. Explain the counterexample in Table 3 and the 100 vs 1003 prompt discrepancy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.