- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces KAVA, a video codec that specializes a frozen pretrained video-diffusion model (Wan2.1) to each target video. The core idea is to transmit only two things: (i) sparse, heavily compressed keyframes (via x265), and (ii) a tiny rank-one adapter (81,360 parameters) fitted to each video. The decoder applies the adapter to the frozen model and generates every frame—including the keyframe positions—by interpolating between decoded keyframes. The adapter is placed only on the model's interface layers (patchify layer and output head) and conditioning layers (text and time embeddings), leaving all 40 transformer blocks frozen. This yields state-of-the-art perceptual performance at extremely low rates (0.0015–0.0062 bpp) on UVG, with 23–92% bit savings over traditional and learned codecs at equal perceptual quality.
Critical strengths
1、The paper reframes extreme video compression as specializing a shared generative prior to a single video. The bitstream carries only what the frozen model cannot predict—keyframes and a small adapter—rather than coded latents for every frame. This is a clean and well-motivated departure from both learned codecs (which transmit latents) and neural representations (which transmit network weights).
2、The choice to adapt only the interface and conditioning layers (81,360 parameters, rank one) rather than attention projections (1.6–4.1M parameters) is well-justified and empirically validated (Table 4). The ablation showing that interface-only adaptation (10,448 params) is insufficient, but adding conditioning layers brings large gains, is informative.
3、KAVA achieves the lowest LPIPS and DISTS on UVG at both resolutions, and dominates all baselines in the rate-perception plane (Fig. 5). The BD-rate savings (Table 2) are substantial: 49–94% against traditional and learned codecs on perceptual metrics. Temporal consistency (Table 3) is also best or near-best.
4、The paper carefully documents rate denominators, BD-rate computation, and the structural asymmetry that KAVA's fixed adapter cost does not amortize over short prefixes. The separate evaluation on the frames GNVC-VD and VoV score (Appendix F) is a fair accommodation.
5、Table 4 systematically tests: (a) adapter placement (interface vs. conditioning vs. all), (b) alternative parameterizations (VeRA, LoRA-XS), and (c) the contribution of the interface layers. The results support the design choices.
6、Appendix B provides detailed training settings, baseline invocations, and hyperparameters. The frozen model is publicly released.
Critical weaknesses
1、The adapter is fitted per video, per CRF, and per resolution, which is a significant encoding cost. Appendix G reports a median encoding cost of 3.9 GPU-minutes per frame at 832×480 and 9.9 GPU-minutes per frame at 1920×1088. For a 600-frame sequence, this is ~39–99 GPU-hours per operating point. The paper reports five CRFs at 832×480 and three at 1920×1088, so encoding a single sequence across all rates costs hundreds of GPU-hours. This is orders of magnitude more expensive than conventional codecs and is not discussed as a limitation. While the paper focuses on perceptual quality, the practical viability of such an encoder is questionable.
2、The adapter's rate is charged as a fixed cost, but the paper does not explore how it scales with video length or content. On UVG, the adapter costs 68,600 bytes (0.00052–0.00272 bpp depending on resolution and CRF). For short videos, this dominates; for long videos, it amortizes. The paper acknowledges this but does not provide guidance on the minimum video length for which KAVA is competitive. A plot of rate vs. video length would be useful.
3、The comparison to VoV and GNVC-VD is somewhat uneven. Both are evaluated on short prefixes (81 and 96 frames), while KAVA is evaluated on full sequences. The paper addresses this in Appendix F by scoring KAVA on the same prefixes, but with the prefix-only rate (which charges the full adapter), KAVA's LPIPS is 0.011–0.024 higher than GNVC-VD's at equal rate at 1920×1088. This is a meaningful caveat that the main text does not emphasize. The claim of "state-of-the-art perceptual performance" should be qualified: it holds on full sequences where the adapter amortizes, but not necessarily on short clips.
4、The generative model can hallucinate content that looks plausible but is wrong. Figure 11 shows text becoming "text-like marks," and Figure 13 shows the bee in HoneyBee being smeared and displaced. The paper acknowledges this in the ethics statement, but does not quantify hallucination rates or propose mitigation. For a compression method, this is a fundamental limitation: the reconstruction is not a faithful representation of the source. The paper should discuss whether there is a rate regime where KAVA's output is trustworthy, or whether it is inherently unsuitable for applications requiring fidelity (e.g., surveillance, medical, evidence).
5、The paper does not compare against a simple baseline: transmitting the adapter alone (without keyframes) or keyframes alone (without adapter). Tables 9–11 report "adapter removed" rows, which show that the adapter provides large gains (e.g., LPIPS 0.3943 → 0.3128 at CRF 38, 1920×1088). However, the reverse ablation—adapter without keyframes—is not shown. Since the adapter alone costs 68,600 bytes and the keyframes are heavily compressed, it would be informative to know whether the keyframes or the adapter contribute more to the final quality.
6、The paper claims "the output at a keyframe position can be closer to the source than the decoded keyframe" (Fig. 7), but this is only shown qualitatively. A quantitative analysis (e.g., average LPIPS at keyframe positions vs. decoded keyframes) would strengthen this claim. Figure 7 shows three examples with LPIPS improvements, but the aggregate effect is unclear.
7、The prompt construction is complex and its necessity is not fully ablated. The prompt combines a keyword (clip index), a generic caption, and 243 neutral words rotated per clip. Appendix A.1 explains why neutral words help (they fill padding positions and give each clip different text embeddings). However, the paper does not ablate the prompt design—e.g., what happens with only the keyword, or with a fixed caption, or with no rotation? This is a key component of the method and deserves more analysis.
8、The paper uses LPIPS with VGG features, but DISTS and PSNR are also reported. The choice of VGG-LPIPS is justified by comparability with GNVC-VD and VoV, but it would be useful to report LPIPS with AlexNet features as well, since some prior work uses that.
9、The temporal consistency metrics (warp error, CLIP-F) are reported in Table 3, but the paper does not discuss the clip-boundary artifact in detail. The warp error at clip boundaries is 49.13 vs. 11.87 for other pairs—a 4× increase. This is a visible artifact (Fig. 10) and should be discussed as a limitation.
10、The paper does not report encoding/decoding latency for the baselines. Table 19 and Appendix G report KAVA's costs, and VoV's encoding cost is mentioned, but the traditional and learned codecs' latencies are not given. Since KAVA's decoding is 4–30 seconds per frame, a comparison would be useful.
11、The rate-distortion (PSNR) results are reported in Appendix D, but the main text focuses on perceptual metrics. The paper correctly notes the perception-distortion tradeoff, but it would be useful to discuss whether KAVA can be tuned for PSNR (e.g., by adjusting the loss or training) and how it compares in that regime.
12、The paper uses x265 for keyframes, but does not compare against using a learned codec (e.g., DCVC-RT) for keyframes. Since keyframes are a small fraction of the rate, this may not matter, but it would be informative.
13、The adapter is quantized to 8 bits with per-tensor scales, but the paper does not explore lower precision (e.g., 4-bit) or other compression methods. Given the adapter is only 81,360 parameters, further compression could reduce its rate.
14、The paper claims "the decoder derives each clip's prompt from its index at no rate cost," but the prompt construction requires the decoder to hold the vocabulary V and the generic caption p. This is fine, but the paper should clarify that these are shared side information, not transmitted.
15、The paper does not discuss failure cases beyond the bee and text examples. Are there video types (e.g., high motion, occlusion, scene cuts) where KAVA performs poorly? A brief discussion would be valuable.
Questions for authors
1、What is the total encoding cost for a single video across all operating points? For a 600-frame UVG sequence, five CRFs at 832×480 would cost ~5 × 600 × 3.9 GPU-min = 195 GPU-hours. Is this practical?
2、How does KAVA perform on short clips (e.g., 81 frames) when the adapter cost is fully charged? Appendix F suggests it is not competitive with GNVC-VD on LPIPS at equal rate. Should the "state-of-the-art" claim be restricted to long videos?
3、What is the minimum video length for KAVA to be competitive? A plot of rate vs. video length (or adapter cost as a fraction of total rate) would be useful.
4、How much do the keyframes contribute vs. the adapter? An ablation transmitting only the adapter (no keyframes) or only keyframes (no adapter, already in Tables 9–11) would isolate their roles.
5、Can the adapter be compressed further (e.g., 4-bit quantization)? What is the rate-quality tradeoff?
6、How does the prompt design affect performance? An ablation on the neutral-word rotation, the generic caption, and the keyword would clarify their contributions.
7、What is the hallucination rate? Can the authors quantify how often the generated content differs semantically from the source (e.g., using a semantic segmentation model)?
8、How does KAVA handle scene cuts or large motion? The keyframe stride is fixed at 24 frames; would adaptive keyframe placement help?
9、What is the decoding latency compared to baselines? Table 19 reports KAVA's decoding cost, but not the baselines'. A comparison would be useful.
10、Can KAVA be extended to multiple videos with a shared adapter? The current design fits one adapter per video; a meta-learned adapter or a small set of adapters could reduce encoding cost.
6 comments
Response to Reviewer UCUi (2/5)
Is the state-of-the-art claim limited to full-length videos? (W3, Q2)
Q (W3, Q2): "How does KAVA perform on short clips (e.g., 81 frames) when the adapter cost is fully charged? [...] Should the "state-of-the-art" claim be restricted to long videos?"
A: KAVA's state-of-the-art claim is for full-length UVG sequences, and the revision states this scope wherever the claim appears. The submitted Abstract already qualified it for "full-length UVG sequences" (l.021-022), and the revision repeats the scope in Section 1, Section 3.1, Section 4 (Protocol) and Limitations (Short videos). Even on short clips, with the full adapter counted and at VoV's model size, KAVA on the 1.3B model outperforms all three runs of VoV's released code on LPIPS, DISTS and PSNR and uses 21%–31% fewer bits on ShakeNDry's first 81 frames at 832 × 480 (Section 4.1). It also outperforms the middle point of VoV+Scaling on all three measures, with 33% fewer bits (Table 27). On GNVC-VD's (CVPR'26) own 96 frames at 1920 × 1088, KAVA matches GNVC-VD's published LPIPS at equal rate and has 32%–37% lower DISTS, at keyframe CRF 28 and 33 with the full adapter counted (Section 4.1, Appendix F.1). Section 4.1 and Appendix F.1 (Table 20) give these short-clip results, with the adapter at four and at eight bits, and general comment, points 1 and 3 give the detail.
How much of the decoded video is content that is not in the source? (W4, Q7)
Q (W4, Q7): "The paper should discuss whether there is a rate regime where KAVA's output is trustworthy, or whether it is inherently unsuitable for applications requiring fidelity (e.g., surveillance, medical, evidence). [...] What is the hallucination rate?"
A: At 832 × 480, a segmentation model labels 4.1%, 2.6% and 4.2% of KAVA's pixels with a class absent from the source frame, at CRF 32, 38 and 44, against 15.6% to 38.6% for the other codecs; the values are means over the seven UVG sequences. Pixel agreement, the share of pixels whose label matches the source's, is highest for KAVA at four of five operating points: 0.838 against 0.592 to 0.706 at CRF 32 (Table 25). High agreement means the right objects sit in the right places in the sampled frames; few absent classes mean it rarely renders unrelated objects. Both numbers are proxies and depend on the segmentation model: VTM, a traditional codec that generates no content, has 25.2% of its pixels labelled with an absent class on frames 4 to 76 (Table 26), because the segmentation model mislabels its degraded frames. In our reconstructions, most of the other codecs at these extremely low rates (below 0.01 bits per pixel) blur the frame and lose texture, small details and at times whole small objects, while a generative decoder keeps detail sharp but can render it wrong. The proxies miss changes within a class that is present, such as text becoming text-like marks.
KAVA compresses very strongly with a generative prior, and hallucination risk from the underlying model always remains. The Ethics Statement says its output "should not be used as evidence of what a scene contained". We argue the keyframes reduce this risk. Every generated clip is conditioned on real keyframes, which give the model spatial information. VoV also uses a generative prior but has no keyframes: 19.6% at 0.00587 bpp on frames 4 to 76, against 4.1% for KAVA on the same frames at 0.00616 bpp (Table 26). Replacing the keyframes with grey frames raises LPIPS on ShakeNDry from 0.300 to 0.688 (Section 4.1). The encoder holds the source, so untested future work could find clips that depart from it, add keyframes, lower the keyframe CRF, or flag them. Details: Appendix I (Tables 25, 26), Limitations ("Content between keyframes"), general comment, point 5.
Which part carries more of the quality, keyframes or adapter? (W5, Q4)
Q (W5, Q4): "The paper does not compare against a simple baseline: transmitting the adapter alone (without keyframes) or keyframes alone (without adapter). [...] How much do the keyframes contribute vs. the adapter?"
A: Keyframes and the adapter are both needed: on ShakeNDry at 832 × 480, CRF 38, keyframes alone give LPIPS 0.449, the adapter alone 0.688, and both together 0.300. Keyframes carry more of the basic reconstruction, but neither part alone comes near the pair:
| Decode | LPIPS | DISTS | PSNR (dB) |
|---|---|---|---|
| Adapter only (keyframes replaced by grey frames) | 0.688 | 0.687 | 14.68 |
| Keyframes only (adapter removed) | 0.449 | 0.189 | 20.53 |
| Both | 0.300 | 0.111 | 25.66 |
In the table above, the DISTS and PSNR of the adapter-only decode, and the PSNR of the decode with both, are new, reported here. Section 4.1 ("What the adapter adds") gives the three LPIPS values, Table 8 the keyframes-only row, and Tables 11 to 13 (Appendix C) the adapter-removed rows at every operating point.
(Continued.)
Response to Reviewer UCUi (3/5)
Is the output at keyframe positions really closer to the source? (W6)
Q (W6): "The paper claims "the output at a keyframe position can be closer to the source than the decoded keyframe" (Fig. 7), but this is only shown qualitatively. A quantitative analysis [...] would strengthen this claim."
A: The KAVA output at keyframe positions is closer to the source than the decoded keyframes, in aggregate and at most individual positions. Over all 1,565 keyframe positions of UVG at both resolutions and HEVC Class B, mean LPIPS is 0.292 against 0.351 and mean DISTS 0.095 against 0.134, and the output is closer at 1,281 positions (82%). The decoder therefore does not copy the decoded keyframes: it uses them as conditioning and generates frames at their positions that are closer to the source. Figure 7, top (Appendix A.3), gives the per-setting table, and Section 3.4 points to it.
Do the rotating neutral words in the prompt matter? (W7, Q6)
Q (W7, Q6): "The prompt construction is complex and its necessity is not fully ablated. [...] How does the prompt design affect performance?"
A: Adding the neutral words, which change from clip to clip, improves LPIPS on all seven UVG sequences, by 2.8% on average. DISTS falls as well, by 2.2% on average. We compared two prompts at 832 × 480, CRF 44. The clip keyword and the generic caption stay in both prompts, and one prompt adds the neutral words; Table 6 gives the per-sequence values (Appendix A.1, Section 3.3, and general comment, point 6).
Do the savings hold with AlexNet-LPIPS? (W8)
Q (W8): "The choice of VGG-LPIPS is justified by comparability with GNVC-VD and VoV, but it would be useful to report LPIPS with AlexNet features as well, since some prior work uses that."
A: Yes, and against HM, VTM and x265 the savings are larger: with AlexNet features and the adapter at four bits, KAVA saves 55.6%–74.0% of the bits, more than with VGG in each of those comparisons (35.4%–70.6% at eight bits). Against DCVC-RT and DCVC-FM at 832 × 480 the AlexNet saving is also larger than with VGG. The table gives per-codec AlexNet BD-rates in percent, with the adapter at four bits and the VGG values from the 4-bit rows of Table 2 in parentheses:
| Codec | UVG 832 × 480 | UVG 1920 × 1088 | HEVC-B 1920 × 1088 |
|---|---|---|---|
| HM | −64.4 (−55.5) | −62.5 (−54.8) | −74.0 (−65.9) |
| VTM | −55.6 (−46.3) | −61.8 (−48.0) | −68.3 (−62.6) |
| x265 | −72.0 (−69.0) | −66.6 (−66.0) | −70.8 (−68.4) |
| DCVC-RT | −67.2 (−62.3) | −61.4 (−64.4) | −63.6 (−61.8) |
| DCVC-FM | −65.2 (−60.4) | −60.7 (−65.7) | −67.2 (−65.9) |
The DCVC entries at 1920 × 1088 and on HEVC-B are new, reported here; at UVG 1920 × 1088 they leave out each DCVC codec's highest-rate point, which we did not rescore and which does not change the VGG values. GLC-Video has an AlexNet BD-rate only on ShakeNDry, VoV and VoV+Scaling have no AlexNet curves, and GNVC-VD cannot be rescored because we read its curves from published figures; the complete AlexNet curves are Figure 10 (Appendix D).
Is the clip-boundary artifact discussed as a limitation? (W9)
Q (W9): "The warp error at clip boundaries is 49.13 vs. 11.87 for other pairs [...]. This is a visible artifact (Fig. 10) and should be discussed as a limitation."
A: The revision's Limitations has a "Clip boundaries" paragraph with these numbers: warp error 49.13 at clip boundaries against 11.87 for interior pairs. Section 4.1 ("Temporal consistency") reports the values, Section 5 ("Clip boundaries") gives the discussion, and general comment, point 5 adds detail.
How does KAVA's speed compare with the other codecs? (W10, Q9)
Q (W10, Q9): "The paper does not report encoding/decoding latency for the baselines. [...] What is the decoding latency compared to baselines?"
A: Among codecs that run a video-generation model, KAVA decodes as fast as VoV (2.5 versus 2.48 seconds per frame at 832 × 480), and it encodes about 31 times faster than our runs of VoV's released code. The 4.0 and 30.4 seconds per frame in the submission included quality scoring during validation. Timed alone, decoding takes 2.5 seconds at 832 × 480 and 23 seconds at 1920 × 1088 (Table 23). VoV+Scaling, which is VoV with candidate search, decodes in 25 seconds per frame, 10 times our time at 832 × 480, as reported in VoV's paper. The traditional and learned codecs, which run no video-generation model, are much faster: they decode in 0.007 to 0.26 seconds per frame at both resolutions (0.007 to 0.068 at 832 × 480). Table 23 and Appendix G give the full times, and Limitations (Compute) gives the total encoding cost per video.
(Continued.)
Response to Reviewer UCUi (1/5)
We thank the reviewer for recognizing the reframing of video compression as specializing a shared generative prior to one video, the choice to adapt only the interface and conditioning layers, validated in Table 4, the lowest LPIPS and DISTS on UVG at both resolutions, the careful rate accounting, and the systematic ablations. As the reviewer suggested, the revision now sends the adapter at four bits, because quality is practically unchanged (Figure 5, Section 4.1, Appendix A.2, Table 27). The test at 8, 6 and 4 bits appears in Appendix A.2. It further adds the requested keyframes-versus-adapter test (Section 4.1), the requested quantitative keyframe-position analysis (Appendix A.3), the prompt ablation (Appendix A.1), the AlexNet-LPIPS curves, as suggested (Appendix D), break-even lengths per codec (Section 4.1, Table 21), and a measurement of generated content that is absent from the source (Appendix I).
Table, figure and section numbers refer to the revised PDF. For the reviewer's convenience, new and changed text there is blue and the additions for the four-bit adapter are red; both will become regular text before the final deadline. Quotes from the review keep the submitted numbers. Results that are not in the revised paper are marked "(new, reported here)". In this reply, "KAVA" alone means the adapter sent at four bits, as in the revised paper. Figure 1, Table 3, the ablations on ShakeNDry, and appendix results not marked four-bit use the same checkpoints at eight bits.
What does it cost to encode one video, and is that realistic? (W1, Q1)
Q (W1, Q1): "The adapter is fitted per video, per CRF, and per resolution, which is a significant encoding cost. [...] What is the total encoding cost for a single video across all operating points?"
A: KAVA encodes a video once, at a single rate: a 600-frame UVG sequence costs about 39 GPU-hours at 832 × 480 and 99 at 1920 × 1088. The median encoding cost is 3.9 GPU-minutes per frame at 832 × 480 and 9.9 at 1920 × 1088. The 195 GPU-hours in the question is the total for the five CRFs we trained at 832 × 480 to draw the rate-distortion curve for evaluation; Section 3.1 now states that one encoding produces a single operating point. Encoding is also about 31 times cheaper per frame than our runs of the released VoV (ICML'26) code (W10, Q9). KAVA's cost is Wan2.1's cost. Wan2.1's sampling steps take 95% of decoding time (Appendix G), and each encoding step is one forward and one backward pass of the same frozen model, so a faster video model speeds up both encoding and decoding. Limitations (Compute) and Appendix G give the details.
At what video length does KAVA start to pay off? (W2, Q3)
Q (W2, Q3): "The adapter's rate is charged as a fixed cost, but the paper does not explore how it scales with video length or content. [...] What is the minimum video length for KAVA to be competitive?"
A: On LPIPS, KAVA needs fewer bits than HM and VTM (LD and RA), x265, DCVC-RT and DCVC-FM from 121 frames on at 1920 × 1088 and from 193 frames on at 832 × 480, with the whole adapter and the keyframes counted. At 30 fps, 121 and 193 frames are 4.0 and 6.4 seconds. Table 21 (Appendix F.2) gives the lengths per codec at four and at eight bits, and Figure 12 shows our bit saving against video length with the adapter at four bits (Appendix F.2). Every UVG point is counted at the largest adapter in that dataset, 37,548 bytes, and the coded adapter changes little across sequences (Table 9 for UVG at 832 × 480 and CRF 38, Table 14 for HEVC Class B). GLC-Video is the one exception among the codecs we ran, and VoV and VoV+Scaling are compared in general comment, point 3.
(Continued.)