acceptodds
Under review as a conference paper at ICLR 2027

Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion

14 views

Abstract

Chunk-wise autoregressive video diffusion models rely on a KV cache of previously generated chunks to avoid redundant computation, and this cache quickly becomes a memory bottleneck as videos grow longer. Methods that quantize the KV cache to low bitwidths reduce memory pressure but degrade video quality. We show that a key driver of this degradation is a systematic bias in attention weights: due to the convexity of the exponential in softmax attention, quantization noise inflates the contribution of cached keys, a phenomenon we call the Jensen bias. This effect causes quantized keys to steal attention mass from the unquantized current chunk. We derive a per-attention-score correction that removes this bias in expectation, computed on the fly from the quantization step sizes of the cached keys and the query norm. With a second-order Taylor approximation, the correction adds negligible computational overhead and needs no additional memory alongside the cache. Evaluated on MAGI-1, SkyReels-V2, and HY-WorldPlay at INT2 quantization, our correction recovers most of the quality lost to aggressive quantization, reaching near-BF16 video quality, and can match or exceed INT4 quantization while using 50% less memory. Because it removes the residual Jensen bias rather than the quantization error itself, the correction complements upstream quantizers such as QuantVideoGen.

Primary area:
generative models
Keywords:
KV cache quantization, autoregressive video diffusion, efficient multimodal AI systems, memory-efficient inference, long-form video generation, attention bias correction, post-training quantization, video generation, softmax attention

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sPWdOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates quality degradation caused by low-bit KV-cache quantization in autoregressive video models. It finds that quantization errors shift attention mass toward cached keys, which the paper calls Jensen bias. Based on this observation and a mathematical derivation, the authors propose a bias correction method. Experiments on three video models demonstrate consistent quality improvements under INT2 quantization.

Critical strengths
  1. The core observation is intuitive and insightful. Starting from the mathematical formulation of attention, the paper identifies a systematic bias introduced by key quantization and derives a corresponding correction. The progression from observation to analysis and solution is concise and elegant.
  2. The method demonstrates consistent quality gains across three autoregressive video models and multiple quantization schemes. It requires no retraining, making it easy to integrate.
Critical weaknesses
  1. A key motivation for KV-cache compression is to retain more historical context under a fixed memory budget, which is particularly important for long-video generation. However, the evaluation is limited to 7–10-second videos, leaving the method’s effectiveness over substantially longer generation horizons unclear.
  2. The reported end-to-end overhead is estimated rather than directly measured in the full generation pipeline. A direct end-to-end latency comparison would strengthen the practical efficiency claims.
Questions for authors
  1. Could the authors report results on substantially longer video sequences?
  2. Could the authors report directly measured end-to-end generation latency comparison?
3 comments
AuthorsOpenReview ↗

Response to Reviewer sPWd

W1/Q1: Longer video sequences

Reply: We thank the reviewer for this point. We agree that KV-cache compression matters substantially for long-horizon generation.

"Longer videos." We are currently checking whether an evaluation on substantially longer videos is feasible with our compute resources. We will come back to the reviewer on this point within the next few days.

"More context at the same memory budget." KV-cache compression can also enable longer context windows without increasing memory usage. The cache footprint grows with the window size and the bits per element. At equal window size, INT2 with our correction matches or exceeds uncorrected INT4 in PSNR, SSIM and LPIPS (Section 5.3). INT2 can therefore use a window approximately twice as long as INT4 at the same footprint. The evaluated models limit how much past video they attend to: MAGI-1 and SkyReels-V2 use sliding windows, while HY-WorldPlay selects a fixed number of past latent frames. Although KV-cache compression could allow these models to retain more historical context at the same memory budget, evaluating whether this improves generation quality would require repeating the experiments with larger context windows, including the BF16 references. The computational cost makes such an evaluation impractical within the rebuttal period, so we will discuss it as future work in the revised paper.

W2/Q2: Measured end-to-end latency

Reply: Since our correction only modifies the attention computation, we deliberately measured the correction in isolation, at the attention kernel, to give a detailed picture of its cost across all attention shapes. The 1.8% stated in Section 4.3 is an estimate derived from this kernel measurement and the attention share of the baseline pipeline.

In response to the reviewer's request, we also measured the end-to-end generation latency directly. With our fused Triton kernels (Appendix C.4) integrated into the complete MAGI-1 pipeline, generating a 10 s video takes on average 702.9 s without and 708.8 s with the correction on an NVIDIA A100, an increase of 0.84%. We will add these measured numbers to Section 4.3.

AuthorsOpenReview ↗

Response to Reviewer sPWd (Additional results)

As promised in our initial response, we have evaluated our correction on substantially longer videos.

Setup. We generated 30 s videos (720 frames, three times the longest videos in the paper) with MAGI-1 for all 40 long-form prompts of VBench’s long-prompt list (competitions/long_prompt_list.txt), none of which were part of our original evaluation. All variants use identical prompts and seeds, so comparisons are paired per prompt. Quantization is INT2 RTN with group size 32, with and without our correction. We also report a BF16 reference and an uncorrected INT4 baseline with the same RTN quantizer and group size. We use MAGI-1's default KV-range schedule.

Over long horizons, quantization slightly perturbs the autoregressive trajectory so frames drift out of alignment with the BF16 reference. This drift compounds over the video length, so frame-aligned PSNR, SSIM and LPIPS become increasingly unreliable, and can even favor blurry frames over sharp ones that are slightly displaced, a known limitation of pixel-aligned metrics for generated video (Mathieu et al., 2016; Denton & Fergus, 2018; Unterthiner et al., 2018). We therefore use VBench-Long’s reference-free evaluation protocol. Over the full 30 s videos, we evaluate frame-wise imaging and aesthetic quality, motion smoothness, dynamic degree, and subject- and background-consistency, which VBench-Long scores as a blend of within-clip and across-clip (long-range) consistency.

These are the dimensions VBench-Long supports for prompts outside its standard prompt suite: its semantic dimensions (e.g. object class, color, spatial relationship) rely on per-prompt annotations that exist only for the standard suite, not for the long-prompt list.

Mean ± standard error over the 40 prompts (VBench scale, 0–100):

VariantImagingAestheticMotion smoothnessDynamic degreeSubject cons.Background cons.
BF1662.32 ± 1.9361.97 ± 1.3999.33 ± 0.0820.68 ± 5.2298.32 ± 0.3097.59 ± 0.23
INT252.97 ± 1.8153.23 ± 1.3498.91 ± 0.205.81 ± 2.1798.55 ± 0.2197.36 ± 0.15
INT2 + corr.60.82 ± 2.0960.63 ± 1.2199.37 ± 0.0716.41 ± 4.4798.33 ± 0.2797.49 ± 0.21
INT457.00 ± 2.0653.54 ± 1.3898.79 ± 0.175.30 ± 2.4098.38 ± 0.2698.14 ± 0.14

At 30 s, our correction recovers most of the quality lost to INT2 quantization: about 84% of the imaging-quality gap and 85% of the aesthetic-quality gap to BF16. It also restores motion: uncorrected INT2 and INT4 videos frequently degrade into heavily distorted frames that often remain largely static, which lowers dynamic degree (5.8 and 5.3 vs. 20.7 for BF16). The correction largely recovers it (16.4), and motion smoothness returns to the BF16 level. Corrected INT2 also outperforms uncorrected INT4 on imaging and aesthetic quality while using about 42% less KV-cache memory (2.75 vs 4.75 effective bits per element). The table below gives per-prompt paired tests of corrected against uncorrected INT2 (exact two-sided Wilcoxon signed-rank test, Holm-corrected across the five dimensions).

DimensionPrompts improvedpHolm-corrected p
Imaging32/401.7×10⁻⁵6.7×10⁻⁵
Aesthetic36/402.0×10⁻⁷1.0×10⁻⁶
Motion smoothness31/402.4×10⁻⁴7.1×10⁻⁴
Subject cons.24/400.880.88
Background cons.23/400.350.70

Imaging quality, aesthetic quality and motion smoothness improve significantly. Subject and background consistency are saturated for all variants and show no significant difference. VBench reports a known trade-off between these consistency dimensions and dynamic degree, as less dynamic videos score higher on consistency (Huang et al., 2024). The slightly higher consistency of uncorrected INT2 and INT4 is in line with their low dynamic degree (5.8 and 5.3 vs. 20.7 for BF16).

We conclude that our approach remains effective for longer videos. We will add these results to the main manuscript in Section 5, together with qualitative frames from the 30 s videos (BF16, INT2, INT2 with correction, INT4) in the appendix.

I thank the authors for their detailed response and additional experiments.

The results on longer video sequences and the end-to-end latency measurements provide further support for the paper’s conclusions. I encourage the authors to include these results in the revised manuscript.

Overall, the authors have adequately addressed my main concerns. I maintain my rating of 3 (Weak Acceptance) and raise my confidence score.

Reviewer_pZ43OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the degradation caused by low-bit KV-cache quantization in autoregressive video diffusion models. The authors identify a systematic bias, termed Jensen bias, whereby approximately zero-mean quantization noise in cached keys inflates their expected unnormalized contributions to softmax attention. This provides an explanation for the observed shift in attention mass toward cached tokens and away from the unquantized current chunk. To address this issue, the authors derive an exact correction for the expected exponential contribution and propose a practical approximation based on a second-order expansion of the score-noise cumulant generating function. Under the assumed quantization-noise model, the correction can be computed from per-group squared query norms and quantization step sizes without requiring additional KV-cache metadata. Experiments on MAGI-1, SkyReels-V2, and HY-WorldPlay demonstrate improved fidelity relative to BF16 outputs under INT2 quantization, along with substantial VBench improvements on the first two models.

Critical strengths
  1. Insightful analysis of quantization-induced attention bias. The paper identifies an interesting failure mechanism beyond conventional KV reconstruction errors. The observation that approximately zero-mean key quantization noise can systematically inflate expected unnormalized attention contributions is well motivated by the convexity of the exponential function. The theoretical analysis provides an interpretable correction based on query magnitudes and quantization step sizes.

  2. Simple and effective correction with minimal overhead. The proposed correction is training-free, reuses existing quantization scales, and requires no additional persistent KV-cache storage. The experimental results demonstrate substantial improvements across several INT2 quantization settings.

Critical weaknesses

The theoretical analysis in Section 4.2 establishes that the proposed correction removes the Jensen bias in the expected exponential contributions of quantized cached keys. Specifically, Eq. (7) shows that the corrected cached partition sum is unbiased in expectation. However, this does not necessarily guarantee unbiased normalized attention weights or total attention mass assigned to cached tokens, since the softmax denominator is also affected by quantization noise. In general, the expectation of a ratio differs from the ratio of expectations. Consequently, even an exact correction of the expected exponential contributions may leave residual bias after softmax normalization. Although the experiments demonstrate reduced attention-mass shifts, the theoretical analysis does not fully characterize this residual bias or establish the conditions under which the corrected attention distribution approaches the BF16 reference. Further analysis of the normalized attention weights would strengthen the theoretical foundation of the method.

Questions for authors
  1. Theoretical guarantee for normalized attention weights. Eq. (7) establishes that the corrected cached partition sum is unbiased in expectation, but this does not directly guarantee unbiased normalized attention weights. Could the authors clarify how this theoretical guarantee relates to the resulting attention distribution? In particular, could the authors characterize the residual bias in the normalized attention mass after correction, either theoretically or empirically?

  2. Potential overcorrection and calibration. Appendix B shows that the estimated correction can overestimate the measured Jensen bias, particularly under RTN. Have the authors considered introducing a simple calibration factor to mitigate this overcorrection? For example, could scaling the correction term further improve attention alignment or downstream generation quality? It would be helpful to understand whether the theoretically derived correction is close to optimal in practice.

  3. Limited evaluation scale and representativeness. The experimental evaluation is relatively limited, particularly for HY-WorldPlay, which is evaluated on only 10 image-prompt pairs. For MAGI-1 and SkyReels-V2, the authors use only the first 30% of prompts from each VBench-Long dimension, raising concerns about the representativeness of the selected subsets. These limitations make it difficult to assess whether the reported improvements generalize across diverse generation scenarios. Could the authors expand the evaluation to a larger and more representative set of prompts, particularly for HY-WorldPlay, and verify whether the improvements remain consistent across different evaluation subsets?

3 comments
AuthorsOpenReview ↗

Response to Reviewer pZ43

W1/Q1: Normalized attention bias

Reply: We agree with the reviewer. The correction removes the bias of the cached partition sum only on average, which does not guarantee the same for the normalized attention weights. We characterize the remaining bias below.

With the correction in Eq. (6), we have E[Z~S]=ZS\mathbb{E}[\widetilde{Z}_S] = Z_S. Since f(x)=x/(x+ZR)f(x) = x/(x+Z_R) is concave, Jensen's inequality gives E[P~S]≤PS\mathbb{E}[\widetilde{P}_S] \leq P_S. More precisely, the residual normalized attention bias is

E[P~S]−PS=−ZR(ZS+ZR)2E[(Z~S−ZS)2Z~S+ZR].\mathbb{E}[\widetilde{P}_S] - P_S = -\frac{Z_R}{(Z_S+Z_R)^2} \mathbb{E}\left[ \frac{(\widetilde{Z}_S-Z_S)^2}{\widetilde{Z}_S+Z_R} \right].

Thus, even if Eq. (6) is evaluated without the approximations leading to Eq. (10), the correction leaves a non-positive residual normalized bias due to fluctuations in the corrected cached partition sum. For sufficiently small fluctuations,

E[P~S]−PS≈−ZRVar⁡(Z~S)(ZS+ZR)3.\mathbb{E}[\widetilde{P}_S] - P_S \approx -\frac{Z_R \operatorname{Var}(\widetilde{Z}_S)} {(Z_S+Z_R)^3}.

Under small, zero-mean score noise independent across cached tokens, this further simplifies to

E[P~S]−PS≈−(1−PS)∑i∈Spi2σi2\mathbb{E}[\widetilde{P}_S] - P_S \approx -(1-P_S)\sum_{i \in S}p_i^2\sigma_i^2

where pip_i denotes the original attention probability and σi2\sigma_i^2 the score-noise variance of cached token ii. This predicts a smaller residual when attention is distributed across many cached tokens rather than concentrated on a few. Importantly, this normalization-induced residual is one possible contributor to the slight overcorrection observed in Fig. 4, alongside the approximation error characterized in Appendix B. We will clarify this distinction and include this analysis in the appendix.

Alternatively, directly targeting E[P~S]=PS\mathbb{E}[\widetilde{P}_S] = P_S yields a normalization-aware correction

bi≈σi22(1−2pi)b_i \approx \frac{\sigma_i^2}{2}(1-2p_i)

through a second-order Taylor expansion of the normalized softmax, assuming small, zero-mean score noise independent across cached tokens. This approaches our original correction when individual attention weights pip_i are small. However, it requires the original BF16 attention probabilities, which are unavailable during quantized inference. Our proposed correction avoids this requirement, relying only on the query and quantization step sizes.

Q2: Overcorrection and calibration

Reply: As the reviewer notes, the correction overestimates the measured Jensen bias (Appendix B), by about 30% under RTN and about 8% under QuaRot+RTN. A scaling factor slightly below one should therefore match the bias better, mainly under RTN.

Note, however, that a fitted factor requires calibration data, which the correction in Eq. (10) does not. We are currently running a grid search over a single factor applied to all correction terms bib_i (Eq. 10), and will post the results during the discussion period.

Q3: Evaluation scale and representativeness

Reply: We thank the reviewer for this point and agree that broader evaluation coverage would strengthen the paper.

Current evaluation (MAGI-1, SkyReels-V2). We use the first 30% of the prompts in each VBench-Long dimension, which gives 286 prompts per model. This selection rule was fixed before we looked at any results, and all comparisons are paired on identical prompts and seeds. On this subset, 14 of the 15 fidelity improvements are statistically significant after Holm correction (Appendix D.2), and the correction improves 68% to 100% of the individual prompts.

Additional evaluation. Our subset is not randomly sampled. We estimate that evaluating the remaining 70% in full would take roughly 2,000 NVIDIA L4 GPU-hours, which exceeds our available compute budget. We are currently exploring the feasibility of evaluating an additional random subset of previously unused prompts.

HY-WorldPlay. The model needs an image, a text prompt and per-frame keyboard actions as input, and no VBench suite provides these. We therefore use the 10 image–prompt pairs released with the official repository, treating HY-WorldPlay as supporting evidence on a third, action-conditioned architecture rather than our main benchmark. Despite the small sample size, LPIPS improves on all 10 videos under both quantization schemes, as does SSIM under QuaRot+RTN. These improvements are statistically significant after Holm correction (exact Wilcoxon signed-rank test, p = 0.002). If the reviewer has a specific evaluation set for action-conditioned world models in mind, we would be glad to consider it for additional evaluation of our method on HY-WorldPlay.

AuthorsOpenReview ↗

Response to Reviewer pZ43 (Additional results)

Q2: Overcorrection and calibration

Reply: As promised, we evaluated whether scaling the correction terms bib_i in Eq. (10) by a single factor α\alpha can further improve generation fidelity.

Setup. We evaluated MAGI-1 with INT2 KV-cache quantization (group size 32), under RTN and QuaRot+RTN. We sweep α∈0,0.1,…,0.9,1\alpha \in{ 0, 0.1, \ldots, 0.9, 1} and additionally test the measured Jensen-bias ratios y/by/b from Appendix B (0.7720.772 for RTN and 0.9280.928 for QuaRot+RTN). We report selected scaling factors in the table below. As in the memory–quality trade-off ablation (Fig. 5), we use a reduced set of 11 VBench prompts with the same seeds as the main evaluation, reporting mean ± standard error relative to BF16.

α\alphaRTN PSNR↑RTN SSIM↑RTN LPIPS↓QuaRot+RTN PSNR↑QuaRot+RTN SSIM↑QuaRot+RTN LPIPS↓
024.01 ± 1.110.836 ± 0.0280.185 ± 0.02317.51 ± 0.700.683 ± 0.0520.437 ± 0.046
0.225.33 ± 1.490.859 ± 0.0270.131 ± 0.02623.26 ± 1.390.821 ± 0.0320.219 ± 0.027
0.425.49 ± 1.520.860 ± 0.0280.122 ± 0.02723.51 ± 1.430.834 ± 0.0330.165 ± 0.023
0.525.58 ± 1.410.865 ± 0.0240.111 ± 0.01823.98 ± 1.580.834 ± 0.0340.155 ± 0.025
0.625.32 ± 1.580.858 ± 0.0260.116 ± 0.02323.26 ± 1.500.825 ± 0.0340.152 ± 0.023
y/b=0.772y/b=0.77224.97 ± 1.620.851 ± 0.0260.122 ± 0.023———
0.824.89 ± 1.650.850 ± 0.0260.122 ± 0.02223.55 ± 1.530.826 ± 0.0340.142 ± 0.023
y/b=0.928y/b=0.928———23.87 ± 1.550.830 ± 0.0340.141 ± 0.023
124.95 ± 1.560.852 ± 0.0250.122 ± 0.01923.62 ± 1.530.826 ± 0.0350.143 ± 0.024

Results. Under RTN, α=0.5\alpha=0.5 gives the best mean scores among the reported factors, improving over our original correction (α=1\alpha=1) by +0.63+0.63 dB PSNR, +0.013+0.013 SSIM, and −0.011-0.011 LPIPS. Interestingly, this factor is lower than the measured Jensen-bias ratio of 0.772, which performs nearly identically to α=1\alpha=1. Under QuaRot+RTN, the measured ratio of 0.928 gives the lowest reported LPIPS (0.141 versus 0.143 for α=1\alpha=1), while α=0.5\alpha=0.5 gives the highest PSNR and SSIM. Thus, different metrics favor different scaling factors. Given the small sample of 11 prompts, these differences should be interpreted cautiously.

Discussion. These experiments suggest that calibration can provide modest additional improvements, but our original correction remains competitive without requiring any calibration data. Moreover, it appears difficult to identify a single scaling factor that works well across quantization schemes. We therefore retain the calibration-free correction as our proposed method and will include this calibration study as an additional ablation in the revised manuscript.

Q3: Evaluation scale and representativeness

Reply: To address the concern about evaluation scale and representativeness, we conducted an additional evaluation on 95 held-out prompts (10% of VBench), uniformly sampled at random from the 658 prompts not used in our original evaluation. The new set has no overlap with the paper's 286 prompts and covers all 16 VBench dimensions.

We evaluated MAGI-1 and SkyReels-V2 using the same generation settings, quantization configurations, and evaluation pipeline as in the paper, including RTN and QuaRot+RTN for both models and QVG for MAGI-1.

Results on held-out prompts (INT2 → INT2 + correction):

ModelSchemePSNR ↑SSIM ↑LPIPS ↓VBench Total ↑
MAGI-1QuaRot+RTN16.79 → 23.690.590 → 0.8150.474 → 0.13671.00 → 80.97
MAGI-1RTN23.72 → 24.340.796 → 0.8320.177 → 0.11777.65 → 80.68
MAGI-1QVG23.33 → 25.290.827 → 0.8590.122 → 0.09680.15 → 80.46
SkyReels-V2QuaRot+RTN19.86 → 21.740.736 → 0.8280.288 → 0.14571.16 → 77.41
SkyReels-V2RTN18.58 → 18.860.678 → 0.7700.368 → 0.23568.17 → 78.55

As reference, the BF16 VBench Total scores on this held-out subset are 80.45 for MAGI-1 and 77.58 for SkyReels-V2.

The held-out results are consistent with our original findings, showing improvements across all five configurations in PSNR, SSIM, LPIPS, and VBench Total score. The corrected VBench scores also approach the BF16 reference, providing further evidence that the improvements generalize beyond the original evaluation subset.

In the revised paper, we will combine the original 286 prompts with the 95 held-out prompts, expanding our main evaluation to 381 prompts (40% of VBench), and update Table 1 and the main text accordingly. We will also clarify the prompt selection procedure in the appendix and report the isolated results for the original fixed subset and the additional randomly sampled prompts.

Thank you for the detailed rebuttal and additional experiments, which have addressed most of my concerns.

I encourage the authors to further evaluate HY-WorldPlay on WorldMark to strengthen the experimental validation.

Overall, I have increased my rating from 2 to 3.

Reviewer_kpdBOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses an interesting problem. when autoregresssive video dit is using low bit like 2 bit KV cache, the attention activation is biased and the author want to resolve this bias.

Critical strengths
  1. super intersting problem. I really like the motivation
  2. the method is sound. lognormal correction is solid. I think it will work.
  3. simple and effective
Critical weaknesses
  1. is this problem really important? How many people really use 2 bits in autoregressive video KV cache? Even if they use, how on earth people will care about this?
  2. There are many stocastic methods like ComfyUI's [1] stocastic post processing for low bit weights. I want to say that maybe those existing methods are already overkill for this small bias, but I think this might be too harse and I will not penalize this
  3. There are many over claims like "exact for any noise distribution"

[1] ComfyUI, comfy/float.py, stochastic_rounding() (commit 6842961). https://github.com/Comfy-Org/ComfyUI/blob/684296148ed30bc39e34c7c069905e166153c11e/comfy/float.py#L65-L86

Questions for authors

how the KV cache is built for Wan familily and SkyReel? There is no consensus methods for 14B models I think?

2 comments
AuthorsOpenReview ↗

Response to Reviewer kpdB (Part: 1/2)

W1: Is this problem really important? How many people really use 2 bits in autoregressive video KV cache?

Reply: We thank the reviewer for raising this point.

In autoregressive video models, the KV cache acts as the model's memory of everything generated so far, and it grows quickly. Quant VideoGen (Xi et al., 2026, ICML 2026) reports that generating a 5-second, 480p video with LongCat-Video already requires approximately 34 GB of KV cache. That paper builds a method around 2-bit KV caches for exactly this reason. For models like MAGI-1, a smaller cache can enable longer context windows under the same memory budget, potentially improving long-range temporal consistency (see also Samuel et al., 2026).

Importantly, our method is not limited to video models. We also present preliminary results on three LLMs in Section 5.4. Beyond our work, 2-bit KV-cache quantization has been extensively studied for LLM inference, including KIVI (Liu et al., 2024), KVQuant (Hooper et al., 2024), GEAR (Kang et al., 2024), AsymKV (Tao et al., 2024), and RotateKV (Su et al., 2025).

Our contribution is not a new quantizer but a correction for anyone already working in this regime. With it, INT2 matches or exceeds uncorrected INT4 at roughly half the memory (Fig. 5). It also composes with Quant VideoGen, achieving our best quantized MAGI-1 results (Table 1).

W2: Stochastic rounding (ComfyUI) as an alternative

Reply: We thank the reviewer for highlighting stochastic rounding as a related approach. Stochastic rounding and our correction address different sources of bias. Stochastic rounding aims to make quantization error zero-mean, whereas the Jensen bias arises even when the noise is perfectly zero-mean. Specifically, by Jensen's inequality, E[eδi]≥eE[δi]=1\mathbb{E}[e^{\delta_i}] \geq e^{\mathbb{E}[\delta_i]} = 1 for zero-mean score noise. In Appendix B, we empirically verify that our quantization noise is already approximately zero-mean under round-to-nearest quantization, so making the error zero-mean, which is what stochastic rounding provides, would not remove the bias we observe.

Stochastic rounding can actually increase the Jensen bias rather than reducing it. It achieves a zero conditional mean at the cost of a larger noise variance: for step size Δ\Delta, the per-element variance is Δ2p(1−p)\Delta^2 p(1-p) with pp denoting the fractional position. Under uniformly distributed fractional positions, this averages to Δ2/6\Delta^2/6, compared with Δ2/12\Delta^2/12 for round-to-nearest under the standard uniform-error model. Since the Jensen bias scales to leading order with the score noise variance, bi≈σi2/2b_i \approx \sigma_i^2/2, stochastic rounding can roughly double the inflation of cached-key attention contributions under these assumptions. Our correction remains applicable in this setting: Eq. (6) holds for any noise distribution, and the practical approximation in Eq. (10) only requires replacing the round-to-nearest noise variance by that of stochastic rounding, so the two approaches could potentially be combined.

Importantly, the effect is not negligible in practice: on MAGI-1 with INT2 QuaRot+RTN, our correction improves VBench from 70.24 to 78.02, nearly recovering the BF16 baseline of 78.27 (Table 1).

AuthorsOpenReview ↗

Response to Reviewer kpdB (Part: 2/2)

W3: There are many over claims like "exact for any noise distribution"

Reply: We thank the reviewer and agree that this wording can be misread. "Exact" is correct only in the narrow context of Eq. (6): bi=log⁡E[eδi]b_i = \log \mathbb{E}[e^{\delta_i}] makes the expected cached partition sum in Eq. (7) unbiased for any noise distribution with finite E[eδi]\mathbb{E}[e^{\delta_i}]. As a description of the method as a whole, the term is misleading. The correction compensates for the bias only on average: it subtracts the expected inflation bib_i, while the score noise δi\delta_i that key quantization adds to each individual attention score remains. We will therefore remove "exact" and revise the following statements. We will also check the remaining claims in the paper for similar wording.

  • Contributions: "a per-attention-score correction that is exact for any noise distribution" becomes "a per-attention-score correction that removes the bias of each cached key's softmax contribution in expectation, without assumptions on the noise distribution".
  • Section 4.2: "The correction in Eq. (6) is exact for any noise distribution" becomes "The correction in Eq. (6) makes the cached partition sum unbiased in expectation for any noise distribution. It does not remove the quantization-induced score noise δi\delta_i itself, and the normalized attention weights are unbiased only approximately."

Q: KV cache construction for Wan family / SkyReels

Reply: The original Wan models, including the 14B variants, have no KV cache. They denoise all frames of a fixed-length clip jointly with bidirectional attention, so keys and values are recomputed at every step and nothing is stored that could be quantized. They are therefore outside the scope of our study.

We evaluate only autoregressive models, which generate chunk by chunk and are therefore not limited to a fixed clip length. They reuse cached keys and values of earlier chunks:

  • MAGI-1: Keys and values of a chunk are written to the cache once the chunk is fully denoised. We use the released inference code with its default sliding window.
  • SkyReels-V2: Although built on Wan2.1, it is trained with diffusion forcing, which permits chunk-wise autoregressive generation with a KV cache. We use the FlowCache implementation (Ma et al., 2026, ICLR 2026) of SkyReels-V2 DF 1.3B with a five-chunk window. Unlike MAGI-1, the FlowCache implementation also writes keys and values of chunks that are still being denoised to the KV cache. Our correction applies unchanged in this case, since it only depends on the quantization step sizes and query norms.

Our correction would also apply to causal Wan variants with a KV cache, such as Self Forcing (Huang et al., 2025, NeurIPS 2025). This includes the 14B scale: Krea Realtime 14B (Krea AI, 2025) converts Wan2.1-14B into an autoregressive model via Self Forcing and generates with a persistent KV cache. We will add these implementation details to the main text of the manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised manuscript uploaded

We thank all reviewers for the constructive discussion.

We have uploaded a revised version of the paper that contains the changes described in our responses to each reviewer. Added text is marked in blue. Removed text is not shown. The main text is within the 10-page limit for the rebuttal version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.