acceptodds
Under review as a conference paper at ICLR 2027

KAVA: Keyframe-Anchored Video Adapters for Extreme Video Compression

Abstract

Video codecs, classical and learned, transmit whatever their decoder cannot predict from the frames it has already decoded, and at low rates they drop fine details. A pretrained video-diffusion model has learned much of how natural video looks and moves, and if the same copy sits at both ends of the channel, it can predict much of what a codec would otherwise transmit. We present KAVA, a codec that specializes one such frozen model, Wan2.1, to each target video. A standard codec sends sparse, heavily compressed keyframes, and the model generates every frame from them, the keyframe positions included. Because the model alone would produce plausible motion rather than the target's, we fit a rank-one adapter to each video on the layers where tokens enter and leave the transformer and on its text and time conditioning, leaving every transformer block frozen. The bitstream is these keyframes and this adapter alone. On full-length UVG sequences, KAVA delivers state-of-the-art perceptual performance from to bits per pixel against traditional, learned and prior generative codecs. We see specializing such a shared model to each video, so that the bitstream carries only what the model cannot predict, as a key step toward extreme video compression.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video compression, learned video coding, video diffusion models, low-rank adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UCUiOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces KAVA, a video codec that specializes a frozen pretrained video-diffusion model (Wan2.1) to each target video. The core idea is to transmit only two things: (i) sparse, heavily compressed keyframes (via x265), and (ii) a tiny rank-one adapter (81,360 parameters) fitted to each video. The decoder applies the adapter to the frozen model and generates every frame—including the keyframe positions—by interpolating between decoded keyframes. The adapter is placed only on the model's interface layers (patchify layer and output head) and conditioning layers (text and time embeddings), leaving all 40 transformer blocks frozen. This yields state-of-the-art perceptual performance at extremely low rates (0.0015–0.0062 bpp) on UVG, with 23–92% bit savings over traditional and learned codecs at equal perceptual quality.

Critical strengths

1、The paper reframes extreme video compression as specializing a shared generative prior to a single video. The bitstream carries only what the frozen model cannot predict—keyframes and a small adapter—rather than coded latents for every frame. This is a clean and well-motivated departure from both learned codecs (which transmit latents) and neural representations (which transmit network weights).

2、The choice to adapt only the interface and conditioning layers (81,360 parameters, rank one) rather than attention projections (1.6–4.1M parameters) is well-justified and empirically validated (Table 4). The ablation showing that interface-only adaptation (10,448 params) is insufficient, but adding conditioning layers brings large gains, is informative.

3、KAVA achieves the lowest LPIPS and DISTS on UVG at both resolutions, and dominates all baselines in the rate-perception plane (Fig. 5). The BD-rate savings (Table 2) are substantial: 49–94% against traditional and learned codecs on perceptual metrics. Temporal consistency (Table 3) is also best or near-best.

4、The paper carefully documents rate denominators, BD-rate computation, and the structural asymmetry that KAVA's fixed adapter cost does not amortize over short prefixes. The separate evaluation on the frames GNVC-VD and VoV score (Appendix F) is a fair accommodation.

5、Table 4 systematically tests: (a) adapter placement (interface vs. conditioning vs. all), (b) alternative parameterizations (VeRA, LoRA-XS), and (c) the contribution of the interface layers. The results support the design choices.

6、Appendix B provides detailed training settings, baseline invocations, and hyperparameters. The frozen model is publicly released.

Critical weaknesses

1、The adapter is fitted per video, per CRF, and per resolution, which is a significant encoding cost. Appendix G reports a median encoding cost of 3.9 GPU-minutes per frame at 832×480 and 9.9 GPU-minutes per frame at 1920×1088. For a 600-frame sequence, this is ~39–99 GPU-hours per operating point. The paper reports five CRFs at 832×480 and three at 1920×1088, so encoding a single sequence across all rates costs hundreds of GPU-hours. This is orders of magnitude more expensive than conventional codecs and is not discussed as a limitation. While the paper focuses on perceptual quality, the practical viability of such an encoder is questionable.

2、The adapter's rate is charged as a fixed cost, but the paper does not explore how it scales with video length or content. On UVG, the adapter costs 68,600 bytes (0.00052–0.00272 bpp depending on resolution and CRF). For short videos, this dominates; for long videos, it amortizes. The paper acknowledges this but does not provide guidance on the minimum video length for which KAVA is competitive. A plot of rate vs. video length would be useful.

3、The comparison to VoV and GNVC-VD is somewhat uneven. Both are evaluated on short prefixes (81 and 96 frames), while KAVA is evaluated on full sequences. The paper addresses this in Appendix F by scoring KAVA on the same prefixes, but with the prefix-only rate (which charges the full adapter), KAVA's LPIPS is 0.011–0.024 higher than GNVC-VD's at equal rate at 1920×1088. This is a meaningful caveat that the main text does not emphasize. The claim of "state-of-the-art perceptual performance" should be qualified: it holds on full sequences where the adapter amortizes, but not necessarily on short clips.

4、The generative model can hallucinate content that looks plausible but is wrong. Figure 11 shows text becoming "text-like marks," and Figure 13 shows the bee in HoneyBee being smeared and displaced. The paper acknowledges this in the ethics statement, but does not quantify hallucination rates or propose mitigation. For a compression method, this is a fundamental limitation: the reconstruction is not a faithful representation of the source. The paper should discuss whether there is a rate regime where KAVA's output is trustworthy, or whether it is inherently unsuitable for applications requiring fidelity (e.g., surveillance, medical, evidence).

5、The paper does not compare against a simple baseline: transmitting the adapter alone (without keyframes) or keyframes alone (without adapter). Tables 9–11 report "adapter removed" rows, which show that the adapter provides large gains (e.g., LPIPS 0.3943 → 0.3128 at CRF 38, 1920×1088). However, the reverse ablation—adapter without keyframes—is not shown. Since the adapter alone costs 68,600 bytes and the keyframes are heavily compressed, it would be informative to know whether the keyframes or the adapter contribute more to the final quality.

6、The paper claims "the output at a keyframe position can be closer to the source than the decoded keyframe" (Fig. 7), but this is only shown qualitatively. A quantitative analysis (e.g., average LPIPS at keyframe positions vs. decoded keyframes) would strengthen this claim. Figure 7 shows three examples with LPIPS improvements, but the aggregate effect is unclear.

7、The prompt construction is complex and its necessity is not fully ablated. The prompt combines a keyword (clip index), a generic caption, and 243 neutral words rotated per clip. Appendix A.1 explains why neutral words help (they fill padding positions and give each clip different text embeddings). However, the paper does not ablate the prompt design—e.g., what happens with only the keyword, or with a fixed caption, or with no rotation? This is a key component of the method and deserves more analysis.

8、The paper uses LPIPS with VGG features, but DISTS and PSNR are also reported. The choice of VGG-LPIPS is justified by comparability with GNVC-VD and VoV, but it would be useful to report LPIPS with AlexNet features as well, since some prior work uses that.

9、The temporal consistency metrics (warp error, CLIP-F) are reported in Table 3, but the paper does not discuss the clip-boundary artifact in detail. The warp error at clip boundaries is 49.13 vs. 11.87 for other pairs—a 4× increase. This is a visible artifact (Fig. 10) and should be discussed as a limitation.

10、The paper does not report encoding/decoding latency for the baselines. Table 19 and Appendix G report KAVA's costs, and VoV's encoding cost is mentioned, but the traditional and learned codecs' latencies are not given. Since KAVA's decoding is 4–30 seconds per frame, a comparison would be useful.

11、The rate-distortion (PSNR) results are reported in Appendix D, but the main text focuses on perceptual metrics. The paper correctly notes the perception-distortion tradeoff, but it would be useful to discuss whether KAVA can be tuned for PSNR (e.g., by adjusting the loss or training) and how it compares in that regime.

12、The paper uses x265 for keyframes, but does not compare against using a learned codec (e.g., DCVC-RT) for keyframes. Since keyframes are a small fraction of the rate, this may not matter, but it would be informative.

13、The adapter is quantized to 8 bits with per-tensor scales, but the paper does not explore lower precision (e.g., 4-bit) or other compression methods. Given the adapter is only 81,360 parameters, further compression could reduce its rate.

14、The paper claims "the decoder derives each clip's prompt from its index at no rate cost," but the prompt construction requires the decoder to hold the vocabulary V and the generic caption p. This is fine, but the paper should clarify that these are shared side information, not transmitted.

15、The paper does not discuss failure cases beyond the bee and text examples. Are there video types (e.g., high motion, occlusion, scene cuts) where KAVA performs poorly? A brief discussion would be valuable.

Questions for authors

1、What is the total encoding cost for a single video across all operating points? For a 600-frame UVG sequence, five CRFs at 832×480 would cost ~5 × 600 × 3.9 GPU-min = 195 GPU-hours. Is this practical?

2、How does KAVA perform on short clips (e.g., 81 frames) when the adapter cost is fully charged? Appendix F suggests it is not competitive with GNVC-VD on LPIPS at equal rate. Should the "state-of-the-art" claim be restricted to long videos?

3、What is the minimum video length for KAVA to be competitive? A plot of rate vs. video length (or adapter cost as a fraction of total rate) would be useful.

4、How much do the keyframes contribute vs. the adapter? An ablation transmitting only the adapter (no keyframes) or only keyframes (no adapter, already in Tables 9–11) would isolate their roles.

5、Can the adapter be compressed further (e.g., 4-bit quantization)? What is the rate-quality tradeoff?

6、How does the prompt design affect performance? An ablation on the neutral-word rotation, the generic caption, and the keyword would clarify their contributions.

7、What is the hallucination rate? Can the authors quantify how often the generated content differs semantically from the source (e.g., using a semantic segmentation model)?

8、How does KAVA handle scene cuts or large motion? The keyframe stride is fixed at 24 frames; would adaptive keyframe placement help?

9、What is the decoding latency compared to baselines? Table 19 reports KAVA's decoding cost, but not the baselines'. A comparison would be useful.

10、Can KAVA be extended to multiple videos with a shared adapter? The current design fits one adapter per video; a meta-learned adapter or a small set of adapters could reduce encoding cost.

6 comments
AuthorsOpenReview ↗

Response to Reviewer UCUi (1/5)

We thank the reviewer for recognizing the reframing of video compression as specializing a shared generative prior to one video, the choice to adapt only the interface and conditioning layers, validated in Table 4, the lowest LPIPS and DISTS on UVG at both resolutions, the careful rate accounting, and the systematic ablations. As the reviewer suggested, the revision now sends the adapter at four bits, because quality is practically unchanged (Figure 5, Section 4.1, Appendix A.2, Table 27). The test at 8, 6 and 4 bits appears in Appendix A.2. It further adds the requested keyframes-versus-adapter test (Section 4.1), the requested quantitative keyframe-position analysis (Appendix A.3), the prompt ablation (Appendix A.1), the AlexNet-LPIPS curves, as suggested (Appendix D), break-even lengths per codec (Section 4.1, Table 21), and a measurement of generated content that is absent from the source (Appendix I).

Table, figure and section numbers refer to the revised PDF. For the reviewer's convenience, new and changed text there is blue and the additions for the four-bit adapter are red; both will become regular text before the final deadline. Quotes from the review keep the submitted numbers. Results that are not in the revised paper are marked "(new, reported here)". In this reply, "KAVA" alone means the adapter sent at four bits, as in the revised paper. Figure 1, Table 3, the ablations on ShakeNDry, and appendix results not marked four-bit use the same checkpoints at eight bits.

What does it cost to encode one video, and is that realistic? (W1, Q1)

Q (W1, Q1): "The adapter is fitted per video, per CRF, and per resolution, which is a significant encoding cost. [...] What is the total encoding cost for a single video across all operating points?"

A: KAVA encodes a video once, at a single rate: a 600-frame UVG sequence costs about 39 GPU-hours at 832 × 480 and 99 at 1920 × 1088. The median encoding cost is 3.9 GPU-minutes per frame at 832 × 480 and 9.9 at 1920 × 1088. The 195 GPU-hours in the question is the total for the five CRFs we trained at 832 × 480 to draw the rate-distortion curve for evaluation; Section 3.1 now states that one encoding produces a single operating point. Encoding is also about 31 times cheaper per frame than our runs of the released VoV (ICML'26) code (W10, Q9). KAVA's cost is Wan2.1's cost. Wan2.1's sampling steps take 95% of decoding time (Appendix G), and each encoding step is one forward and one backward pass of the same frozen model, so a faster video model speeds up both encoding and decoding. Limitations (Compute) and Appendix G give the details.

At what video length does KAVA start to pay off? (W2, Q3)

Q (W2, Q3): "The adapter's rate is charged as a fixed cost, but the paper does not explore how it scales with video length or content. [...] What is the minimum video length for KAVA to be competitive?"

A: On LPIPS, KAVA needs fewer bits than HM and VTM (LD and RA), x265, DCVC-RT and DCVC-FM from 121 frames on at 1920 × 1088 and from 193 frames on at 832 × 480, with the whole adapter and the keyframes counted. At 30 fps, 121 and 193 frames are 4.0 and 6.4 seconds. Table 21 (Appendix F.2) gives the lengths per codec at four and at eight bits, and Figure 12 shows our bit saving against video length with the adapter at four bits (Appendix F.2). Every UVG point is counted at the largest adapter in that dataset, 37,548 bytes, and the coded adapter changes little across sequences (Table 9 for UVG at 832 × 480 and CRF 38, Table 14 for HEVC Class B). GLC-Video is the one exception among the codecs we ran, and VoV and VoV+Scaling are compared in general comment, point 3.

(Continued.)

AuthorsOpenReview ↗

Response to Reviewer UCUi (2/5)

Is the state-of-the-art claim limited to full-length videos? (W3, Q2)

Q (W3, Q2): "How does KAVA perform on short clips (e.g., 81 frames) when the adapter cost is fully charged? [...] Should the "state-of-the-art" claim be restricted to long videos?"

A: KAVA's state-of-the-art claim is for full-length UVG sequences, and the revision states this scope wherever the claim appears. The submitted Abstract already qualified it for "full-length UVG sequences" (l.021-022), and the revision repeats the scope in Section 1, Section 3.1, Section 4 (Protocol) and Limitations (Short videos). Even on short clips, with the full adapter counted and at VoV's model size, KAVA on the 1.3B model outperforms all three runs of VoV's released code on LPIPS, DISTS and PSNR and uses 21%–31% fewer bits on ShakeNDry's first 81 frames at 832 × 480 (Section 4.1). It also outperforms the middle point of VoV+Scaling on all three measures, with 33% fewer bits (Table 27). On GNVC-VD's (CVPR'26) own 96 frames at 1920 × 1088, KAVA matches GNVC-VD's published LPIPS at equal rate and has 32%–37% lower DISTS, at keyframe CRF 28 and 33 with the full adapter counted (Section 4.1, Appendix F.1). Section 4.1 and Appendix F.1 (Table 20) give these short-clip results, with the adapter at four and at eight bits, and general comment, points 1 and 3 give the detail.

How much of the decoded video is content that is not in the source? (W4, Q7)

Q (W4, Q7): "The paper should discuss whether there is a rate regime where KAVA's output is trustworthy, or whether it is inherently unsuitable for applications requiring fidelity (e.g., surveillance, medical, evidence). [...] What is the hallucination rate?"

A: At 832 × 480, a segmentation model labels 4.1%, 2.6% and 4.2% of KAVA's pixels with a class absent from the source frame, at CRF 32, 38 and 44, against 15.6% to 38.6% for the other codecs; the values are means over the seven UVG sequences. Pixel agreement, the share of pixels whose label matches the source's, is highest for KAVA at four of five operating points: 0.838 against 0.592 to 0.706 at CRF 32 (Table 25). High agreement means the right objects sit in the right places in the sampled frames; few absent classes mean it rarely renders unrelated objects. Both numbers are proxies and depend on the segmentation model: VTM, a traditional codec that generates no content, has 25.2% of its pixels labelled with an absent class on frames 4 to 76 (Table 26), because the segmentation model mislabels its degraded frames. In our reconstructions, most of the other codecs at these extremely low rates (below 0.01 bits per pixel) blur the frame and lose texture, small details and at times whole small objects, while a generative decoder keeps detail sharp but can render it wrong. The proxies miss changes within a class that is present, such as text becoming text-like marks.

KAVA compresses very strongly with a generative prior, and hallucination risk from the underlying model always remains. The Ethics Statement says its output "should not be used as evidence of what a scene contained". We argue the keyframes reduce this risk. Every generated clip is conditioned on real keyframes, which give the model spatial information. VoV also uses a generative prior but has no keyframes: 19.6% at 0.00587 bpp on frames 4 to 76, against 4.1% for KAVA on the same frames at 0.00616 bpp (Table 26). Replacing the keyframes with grey frames raises LPIPS on ShakeNDry from 0.300 to 0.688 (Section 4.1). The encoder holds the source, so untested future work could find clips that depart from it, add keyframes, lower the keyframe CRF, or flag them. Details: Appendix I (Tables 25, 26), Limitations ("Content between keyframes"), general comment, point 5.

Which part carries more of the quality, keyframes or adapter? (W5, Q4)

Q (W5, Q4): "The paper does not compare against a simple baseline: transmitting the adapter alone (without keyframes) or keyframes alone (without adapter). [...] How much do the keyframes contribute vs. the adapter?"

A: Keyframes and the adapter are both needed: on ShakeNDry at 832 × 480, CRF 38, keyframes alone give LPIPS 0.449, the adapter alone 0.688, and both together 0.300. Keyframes carry more of the basic reconstruction, but neither part alone comes near the pair:

DecodeLPIPSDISTSPSNR (dB)
Adapter only (keyframes replaced by grey frames)0.6880.68714.68
Keyframes only (adapter removed)0.4490.18920.53
Both0.3000.11125.66

In the table above, the DISTS and PSNR of the adapter-only decode, and the PSNR of the decode with both, are new, reported here. Section 4.1 ("What the adapter adds") gives the three LPIPS values, Table 8 the keyframes-only row, and Tables 11 to 13 (Appendix C) the adapter-removed rows at every operating point.

(Continued.)

AuthorsOpenReview ↗

Response to Reviewer UCUi (3/5)

Is the output at keyframe positions really closer to the source? (W6)

Q (W6): "The paper claims "the output at a keyframe position can be closer to the source than the decoded keyframe" (Fig. 7), but this is only shown qualitatively. A quantitative analysis [...] would strengthen this claim."

A: The KAVA output at keyframe positions is closer to the source than the decoded keyframes, in aggregate and at most individual positions. Over all 1,565 keyframe positions of UVG at both resolutions and HEVC Class B, mean LPIPS is 0.292 against 0.351 and mean DISTS 0.095 against 0.134, and the output is closer at 1,281 positions (82%). The decoder therefore does not copy the decoded keyframes: it uses them as conditioning and generates frames at their positions that are closer to the source. Figure 7, top (Appendix A.3), gives the per-setting table, and Section 3.4 points to it.

Do the rotating neutral words in the prompt matter? (W7, Q6)

Q (W7, Q6): "The prompt construction is complex and its necessity is not fully ablated. [...] How does the prompt design affect performance?"

A: Adding the neutral words, which change from clip to clip, improves LPIPS on all seven UVG sequences, by 2.8% on average. DISTS falls as well, by 2.2% on average. We compared two prompts at 832 × 480, CRF 44. The clip keyword and the generic caption stay in both prompts, and one prompt adds the neutral words; Table 6 gives the per-sequence values (Appendix A.1, Section 3.3, and general comment, point 6).

Do the savings hold with AlexNet-LPIPS? (W8)

Q (W8): "The choice of VGG-LPIPS is justified by comparability with GNVC-VD and VoV, but it would be useful to report LPIPS with AlexNet features as well, since some prior work uses that."

A: Yes, and against HM, VTM and x265 the savings are larger: with AlexNet features and the adapter at four bits, KAVA saves 55.6%–74.0% of the bits, more than with VGG in each of those comparisons (35.4%–70.6% at eight bits). Against DCVC-RT and DCVC-FM at 832 × 480 the AlexNet saving is also larger than with VGG. The table gives per-codec AlexNet BD-rates in percent, with the adapter at four bits and the VGG values from the 4-bit rows of Table 2 in parentheses:

CodecUVG 832 × 480UVG 1920 × 1088HEVC-B 1920 × 1088
HM−64.4 (−55.5)−62.5 (−54.8)−74.0 (−65.9)
VTM−55.6 (−46.3)−61.8 (−48.0)−68.3 (−62.6)
x265−72.0 (−69.0)−66.6 (−66.0)−70.8 (−68.4)
DCVC-RT−67.2 (−62.3)−61.4 (−64.4)−63.6 (−61.8)
DCVC-FM−65.2 (−60.4)−60.7 (−65.7)−67.2 (−65.9)

The DCVC entries at 1920 × 1088 and on HEVC-B are new, reported here; at UVG 1920 × 1088 they leave out each DCVC codec's highest-rate point, which we did not rescore and which does not change the VGG values. GLC-Video has an AlexNet BD-rate only on ShakeNDry, VoV and VoV+Scaling have no AlexNet curves, and GNVC-VD cannot be rescored because we read its curves from published figures; the complete AlexNet curves are Figure 10 (Appendix D).

Is the clip-boundary artifact discussed as a limitation? (W9)

Q (W9): "The warp error at clip boundaries is 49.13 vs. 11.87 for other pairs [...]. This is a visible artifact (Fig. 10) and should be discussed as a limitation."

A: The revision's Limitations has a "Clip boundaries" paragraph with these numbers: warp error 49.13 at clip boundaries against 11.87 for interior pairs. Section 4.1 ("Temporal consistency") reports the values, Section 5 ("Clip boundaries") gives the discussion, and general comment, point 5 adds detail.

How does KAVA's speed compare with the other codecs? (W10, Q9)

Q (W10, Q9): "The paper does not report encoding/decoding latency for the baselines. [...] What is the decoding latency compared to baselines?"

A: Among codecs that run a video-generation model, KAVA decodes as fast as VoV (2.5 versus 2.48 seconds per frame at 832 × 480), and it encodes about 31 times faster than our runs of VoV's released code. The 4.0 and 30.4 seconds per frame in the submission included quality scoring during validation. Timed alone, decoding takes 2.5 seconds at 832 × 480 and 23 seconds at 1920 × 1088 (Table 23). VoV+Scaling, which is VoV with candidate search, decodes in 25 seconds per frame, 10 times our time at 832 × 480, as reported in VoV's paper. The traditional and learned codecs, which run no video-generation model, are much faster: they decode in 0.007 to 0.26 seconds per frame at both resolutions (0.007 to 0.068 at 832 × 480). Table 23 and Appendix G give the full times, and Limitations (Compute) gives the total encoding cost per video.

(Continued.)

Reviewer_xfZFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

KAVA fits a rank-one LoRA at the encoder for each video and keyframe CRF configuration to specialize a frozen Wan2.1-FLF2V-14B model shared by the encoder and decoder. The bitstream consists of sparse keyframes coded with x265 and a quantized, compressed adapter. The adapter acts only on seven interface and text/time conditioning layers, totaling 81,360 parameters. At the decoder, consecutive keyframes and a prompt constructed by fixed rules condition video-clip reconstruction through 50 Euler steps; the output at keyframe positions is also generated. The method is per-video optimized, offline, bidirectional generative compression at extremely low rates. On UVG and HEVC Class B, the paper reports substantial LPIPS/DISTS improvements over the traditional and learned codecs evaluated.

Critical strengths
  1. The coding representation is compact and has research value. A standard keyframe stream anchors the content, while a small video-level adapter specializes the shared generative prior, without transmitting per-frame residuals or a complete per-video network.
  2. Limitations and experimental details are disclosed relatively thoroughly. The authors provide per-sequence results and explicitly report insufficient amortization on short videos, lower PSNR, content errors, high warp error at clip boundaries, and substantial computational cost, which help establish the method's scope of applicability.
Critical weaknesses
  1. Backbone scale is not controlled. KAVA uses a large Wan 14B model, so gains over VoV and GNVC-VD may partly reflect backbone differences rather than the proposed design. Please test a compatible smaller backbone or provide a same-backbone control, with matched frame counts and complete rate accounting.

  2. A compute-matched conventional codec baseline is missing. Could a conventional learned codec achieve similar gains with more model capacity, encoder-side search, or video-level perceptual optimization? Please evaluate one feasible baseline under a comparable computational budget, or explain why this is not possible.

  3. Internal adaptation layers are not evaluated. Beyond the existing conditioning and interface ablations, please test adding a few internal attention adaptation layers or reallocating the adapter budget to them. Compare rate-distortion performance across several videos with consistent initialization and training budgets, reporting actual coded bytes, PSNR, and LPIPS/DISTS.

  4. Figure 1 reports rate but not quality metrics. Please add PSNR and LPIPS/DISTS for the displayed coding configurations and specify whether they measure the crop, full frame, or full video. Table 17 uses different operating points and does not supply these measurements.

Questions for authors

See Weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer xfZF (1/3)

We thank the reviewer for recognizing a compact coding representation with research value, where a standard keyframe stream anchors the content while a small video-level adapter specializes the shared generative prior, and the per-sequence results. The revision adds a comparison on a smaller backbone, the 1.3B model, the size of VoV's model, as the reviewer requested (Appendix J, Table 27, Figure 17). It further adds per-frame encoding and decoding times for KAVA, VTM and VoV (Table 23), with a compute paragraph in the limitations (Section 5), random-access HM and VTM (Table 2), the requested quality scores for Figure 1, on the crop and the full frame (Table 24, Appendix H), and a switch to sending the adapter at four bits, which leaves quality practically unchanged (Figure 5, Appendix A.2).

Table, figure and section numbers refer to the revised PDF. For the reviewer's convenience, new and changed text there is blue and the additions for the four-bit adapter are red; both will become regular text before the final deadline. Quotes from the review keep the submitted numbers. Results that are not in the revised paper are marked "(new, reported here)". In this reply, "KAVA" alone means the adapter sent at four bits, as in the revised paper. Figure 1, Table 3, the ablations on ShakeNDry, and appendix results not marked four-bit use the same checkpoints at eight bits.

Q (W1): "Backbone scale is not controlled. [...] Please test a compatible smaller backbone or provide a same-backbone control, with matched frame counts and complete rate accounting."

A: KAVA outperforms VoV (ICML'26) under VoV's own evaluation protocol. Its lead grows under our whole-sequence protocol, and the improvements over VoV also appear on a 1.3B model about the size of VoV's.

Under VoV's protocol (Table 20, Appendix F: seven UVG sequences, 832 × 480, frames 0 to 80, rate counted over those 81 frames), KAVA on the 14B model has lower LPIPS and lower DISTS than VoV at about the same rate. Each KAVA adapter was trained on its whole sequence. The rate charges the whole adapter, and every keyframe needed to decode frames 0 to 80 (those at frames 0 to 96), to those 81 frames. VoV at its highest published rate: 0.0118 bpp, LPIPS 0.350, DISTS 0.106. KAVA at CRF 41: 0.01192 bpp, LPIPS 0.289, DISTS 0.100. That is 1% more bits, LPIPS 17% lower, DISTS 6% lower.

Under our whole-sequence protocol (Table 1, Section 4.1: seven UVG sequences, 832 × 480, whole sequences), KAVA's lead grows. KAVA: 0.00493 bpp, LPIPS 0.2562, DISTS 0.0908. VoV, its published 81-frame results: 0.0059 bpp, LPIPS 0.428, DISTS 0.131. KAVA uses 16% fewer bits, LPIPS 40% lower, DISTS 31% lower. The adapter is sent once per video, so its cost per frame falls as the sequence grows (Section 4, "Protocol"). VoV appears with its published 81-frame results because running it on whole sequences was not practical. In our runs of its released code, VoV takes 7,286 H100 GPU-seconds per frame to encode, 31 times KAVA's 235 (Table 23, Appendix G). At that cost per frame, one 600-frame UVG sequence would take about 1,200 H100 GPU-hours for each rate point. VoV's own paper lists its long encoding time as a limitation.

The improvements over VoV also appear with a smaller model. On the 1.3B model, about 10 times smaller than the 14B one and about the size of VoV's model, KAVA outperforms our runs of VoV's released code on LPIPS, DISTS and PSNR at all three of VoV's rates, using 21% to 31% fewer bits. This is reported in Section 4.1, Appendix J and Table 27; the rate includes the adapter. The VoV rows are our runs of VoV's released code:

ShakeNDry 832 × 480, frames 0 to 80bppLPIPS
KAVA on the 1.3B model, CRF 440.004690.4030
VoV, lowest rate0.005920.4467
KAVA on the 1.3B model, CRF 380.006710.3640
VoV, middle rate0.009660.4286
KAVA on the 1.3B model, CRF 350.008580.3456
VoV, highest rate0.012370.4262

At CRF 38, KAVA also outperforms the middle point of VoV with candidate search (VoV+Scaling) on all three measures while using 33% fewer bits. The 1.3B and 14B models give the same LPIPS at about the same rate: 1.3B at CRF 32 gives 0.01147 bpp, LPIPS 0.3313; 14B at CRF 41 on the same frames gives 0.01144 bpp, LPIPS 0.3312 (new, reported here).

Table 27 and Figure 17 show KAVA on the 1.3B model (Wan2.1-Fun 1.3B InP, same adapter design), adapter at eight and four bits, next to VoV and VoV+Scaling. Appendix J states that rates include the whole adapter and the keyframes. General comment, point 3 adds KAVA on the 1.3B model at 1920 × 1088 on ShakeNDry; general comment, point 1 explains why we evaluate whole sequences and compares KAVA with GNVC-VD.

(Continued.)

AuthorsOpenReview ↗

Response to Reviewer xfZF (2/3)

Q (W2): "A compute-matched conventional codec baseline is missing. Could a conventional learned codec achieve similar gains with more model capacity, encoder-side search, or video-level perceptual optimization? Please evaluate one feasible baseline under a comparable computational budget, or explain why this is not possible."

A: We agree that a compute-matched comparison would help isolate the benefits of our generative prior. However, matching KAVA's computational budget would require substantial changes to the competing codecs, such as video-specific optimization or more extensive encoder-side search. We therefore evaluate released codec implementations under their supported configurations, including the stronger random-access settings of HM and VTM. We also compare against VoV, which uses a generative backbone and substantially more encoding compute.

Our comparison includes VoV (ICML'26) and GNVC-VD (CVPR'26), the latest state-of-the-art methods for extreme compression with generative models. More model capacity would mean a new codec trained on a large video dataset. GNVC-VD takes that route. It follows DCVC-RT and adds a 2.2-billion-parameter video diffusion transformer at the decoder.

HM and VTM already search coding modes by rate and distortion, in 1.9 to 4.1 seconds per frame on one CPU thread at 832 × 480, and no setting of theirs spends compute near KAVA's 235 GPU-seconds per frame (Table 23). We have now added their random-access configuration (Table 2), where KAVA spends 43%–55% fewer bits than random-access VTM and 61%–69% fewer than random-access HM at equal LPIPS.

DCVC-RT and DCVC-FM share one decoder and encode each frame in a single pass, so spending more compute means turning them into per-video codecs. Per-video fine-tuning (van Rozendaal et al., Instance-Adaptive Video Compression, TMLR 2023) saves 17%–27% of the bits at equal PSNR, and its output still looks blurred at low rates. A perceptual version would be a new method that needs training runs. VoV takes this route with a generative model. It fits a small vector on each video that adapts a frozen text-to-video model. On full-length UVG sequences, VoV's points and GNVC-VD's published points lie above our rate-distortion curves, so at equal rate KAVA outperforms VoV and GNVC-VD on LPIPS and DISTS (Section 4.1, Figure 5). It does so with 31 times less encoder compute than VoV on the same GPU type, and it decodes a frame at 832 × 480 in 2.5 seconds, about as fast as VoV and 10 times faster than VoV+Scaling (Table 23).

The encoder cost is paid once per video: KAVA is built for full-length videos coded once offline and decoded many times. General comment, point 1 discusses amortization over full-length videos, point 2 the encoding cost, and point 4 the random-access HM and VTM baselines.

Q (W3): "Internal adaptation layers are not evaluated. Beyond the existing conditioning and interface ablations, please test adding a few internal attention adaptation layers or reallocating the adapter budget to them."

A: We did not train adapters on internal attention layers in the discussion period. However, placement decides the rate, because every adapted parameter is transmitted. Our conditioning layers feed all 40 blocks: the text pathway gives the context tokens that every block's cross-attention reads, and the time projection sets every block's modulation, so one rank-one change there acts on all 40 blocks. Inside a block, one rank-one LoRA on one attention projection (5,120 × 5,120) has 10,240 parameters, which is 13% of our 81,360-parameter adapter. The query, key, value and output projections of one block have 40,960, about half our adapter, and across all 40 blocks these projections would make the adapter 20 to 50 times larger (Section 3.2, Table 5). Reallocating the budget means removing conditioning layers, and LPIPS rises with every conditioning layer removed (Table 4).

The paragraph of Section 1 that introduces the adapter states the placement, and Table 5 lists the seven adapted layers with their parameter counts.

(Continued.)

AuthorsOpenReview ↗

Response to Reviewer xfZF (3/3)

Q (W3): "Compare rate-distortion performance across several videos with consistent initialization and training budgets, reporting actual coded bytes, PSNR, and LPIPS/DISTS."

A: LPIPS falls as more conditioning layers are adapted, and the best result comes from adapting all five conditioning layers and training the interface layers. Tables 4 and 8 give LPIPS, DISTS, PSNR and the parameter count for every adapter configuration.

AdapterCoded bytes, adapter at 8 bits (new, reported here)LPIPSDISTSPSNR (dB)
Time projection32,0600.31240.114425.48
All five conditioning layers63,8920.29510.108926.06
All five + interface layers trained (Table 8)70,3920.29280.106526.16

The rows cover a single video, ShakeNDry (832 × 480, CRF 38). Table 4 lists all adapter configurations with their parameter counts, and Table 8 lists the interface ablation. The layer ablations use only this video (Section 4.2), since more videos were not possible within the discussion period. Two ablations cover all seven UVG sequences (general comment, point 6). Adding the neutral words to the prompt (Table 6, Appendix A.1) lowers LPIPS and DISTS on all seven; LPIPS is 3.3% lower on ShakeNDry and 2.8% lower on average. Storing the adapter at 6 instead of 8 bits (Table 9, Appendix A.2) raises LPIPS by at most 0.0006 on any sequence. In both cases, ShakeNDry's conclusion held on all seven. This suggests that ShakeNDry's layer ranking is a good guide for the other sequences.

Q (W4): "Figure 1 reports rate but not quality metrics. Please add PSNR and LPIPS/DISTS for the displayed coding configurations and specify whether they measure the crop, full frame, or full video. [...]"

A: On both sequences in Figure 1, KAVA has the lowest LPIPS and DISTS of the five methods on the shown crop and on the full frame.

The crop and the full frame are scored against the source image shown in the figure; the full video is scored the same way as everywhere else in the paper. Table 24 (Appendix H) gives LPIPS, DISTS and PSNR on the crop and the full frame for all five methods, and the Figure 1 caption now states that each crop is one frame and points to Appendix H. On the full video, KAVA has lower DISTS than random-access VTM on both sequences (Table 2); Table 16 gives KAVA's full-video LPIPS and DISTS.

Presentation.

A: Section 1 now states the setting KAVA is built for, full-length videos coded once offline and decoded many times, and where the adapter is placed. A new "Protocol" paragraph in Section 4, ahead of the Section 4.1 results, gives the evaluation setting, and Section 4.1 gives the break-even sequence lengths.

Reviewer_niQEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

KAVA is a compression method which propose to use frozen video generative model Wan2.1 and a small video-specific adapter for a shared, transmitting sparsely sampled, x265-compressed keyframes. The decoder reconstructs 25 frame overlapping clips from their boundary keyframes, with a deterministic clip index related prompt and noise seed. The adapter is quantized and losslessly compressed. Experiments on seven UVG and five HEVC Class B sequences show substantial LPIPS/DISTS improvements over the conventional and learned codecs evaluated in the paper. The main advantages occur when the adapter works over long sequences, at the expense of high computation, reduced pixel fidelity, and errors at clip boundaries.

Critical strengths
  1. A compact and useful adaptation mechanism. KAVA obtains large reported improvements over the same frozen generator using identical keyframe streams, while sending a small adapter rather than per-frame generative latents or an entire fitted network. Tables 9–11 show that this benefit holds across all reported dataset/resolution operating points. This is consequential evidence for the effectiveness of the transmitted adaptation.
  2. The paper includes per-sequence results, both perceptual and pixel-fidelity metrics, a shorter-prefix evaluation that weakens its own headline comparison, and explicit examples of reconstruction failures. It also identifies a roughly fourfold increase in warp error across independently decoded clip boundaries. This makes the intended use case and important limitations assessable rather than hiding them behind aggregate perceptual scores.
Critical weaknesses
  1. The headline comparisons use different coding settings. KAVA amortizes its adapter over long sequences, while published VoV and GNVC-VD results cover short prefixes. With matched prefix accounting, KAVA no longer dominates VoV, and its comparison with GNVC-VD becomes a trade-off between LPIPS and DISTS. HM/VTM also use low-delay settings despite KAVA being offline and bidirectional, and the closely related DiV-INR is missing from quantitative evaluation. These gaps make the claimed advantage difficult to assess fairly.
  2. The design ablations are limited to one sequence and one run per setting. LoRA continues from a trained adapter, while VeRA and LoRA-XS initialize their replacement terms from scratch (§4.2, Appendix A.2). The top half of Table 7 compares further training with freezing the existing interface adapters; it does not test removing them from the full model. The rotated prompt also lacks an ablation. These experiments leave the contribution of individual design choices uncertain.
Questions for authors
  1. Would KAVA retain its advantage if HM/VTM used an offline/random-access configuration and the generative codecs were evaluated on the same frames with all required bits counted?
  2. Have you compared KAVA with quantizing and entropy-coding video-VAE latents, such as those from LTX-Video, followed by VAE-only decoding? At matched coded bitrate, how do reconstruction quality and encoding/decoding time compare? If useful reconstruction is not possible at KAVA's rates, evidence of that limitation would help establish the benefit of generative decoding.
  3. Does LoRA still outperform VeRA and LoRA-XS with comparable initialization and total training budgets, including training before the ablations, and on sequences beyond ShakeNDry? Does the rotated neutral-word prompt improve reconstruction over a fixed prompt with the same words across clips?
  4. Appendix B specifies Wan2.1-FLF2V-14B. Have you tested a smaller backbone with comparable first-and-last-frame conditioning? A comparison of reconstruction quality at matched coded bitrate, runtime, and peak memory on the same hardware would clarify how much of KAVA's performance depends on this model scale.
5 comments
AuthorsOpenReview ↗

Response to Reviewer niQE (1/3)

We thank the reviewer for recognizing a compact and useful adaptation mechanism, the large gains over the same frozen generator with identical keyframe streams, which hold at every dataset and resolution, and the per-sequence results with perceptual and pixel measures. As the reviewer requested, the revision adds break-even lengths (Section 4.1), random-access rows for HM and VTM (Table 2) and a comparison on a smaller backbone (Appendix J). It also adds new ablation results (Tables 6 and 9) and sends the adapter at four bits, which leaves quality practically unchanged (Figure 5, Appendix A.2).

Table, figure and section numbers refer to the revised PDF. For the reviewer's convenience, new and changed text there is blue and the additions for the four-bit adapter are red; both will become regular text before the final deadline. Quotes from the review keep the submitted numbers. Results that are not in the revised paper are marked "(new, reported here)". In this reply, "KAVA" alone means the adapter sent at four bits, as in the revised paper. Figure 1, Table 3, the ablations on ShakeNDry, and appendix results not marked four-bit use the same checkpoints at eight bits.

1. At what video length does KAVA pay off, and does KAVA outperform VoV (ICML'26) at VoV's own model size? (W1 (a))

Q (W1 (a)): "KAVA amortizes its adapter over long sequences, while published VoV and GNVC-VD results cover short prefixes. With matched prefix accounting, KAVA no longer dominates VoV, and its comparison with GNVC-VD becomes a trade-off between LPIPS and DISTS."

A: KAVA needs fewer bits than HM, VTM, x265, DCVC-RT and DCVC-FM from 193 frames (LPIPS) and 73 (DISTS) at 832 × 480, and fewer bits than the published points of GNVC-VD (CVPR'26) from 97 frames (LPIPS) and 25 (DISTS) at 1920 × 1088. KAVA sends its adapter once per video, so its cost per frame drops as the video gets longer; every bit is counted, keyframes and the coded adapter included. GNVC-VD has no released code, so we use its published points and hold its rate per frame fixed. On that basis, the trade-off you describe appears only on clips shorter than these lengths. On whole videos at 1920 × 1088, KAVA uses 40% fewer bits than GNVC-VD, with LPIPS 0.2895 against 0.291 and DISTS 0.0750 against 0.115 (Table 1). Even on GNVC-VD's own 96 frames, with the full adapter counted, KAVA matches GNVC-VD's published LPIPS at equal rate and has 32%–37% lower DISTS at keyframe CRF 28 and 33 (Section 4.1, Appendix F.1).

On the short clips, at VoV's model size and with the adapter included in the rate, KAVA on the 1.3B model outperforms all three runs of VoV's released code (our runs, without candidate search) on the same 81 frames of ShakeNDry at 832 × 480, across LPIPS, DISTS and PSNR, with 21%–31% fewer bits. It also outperforms the middle point of VoV+Scaling on all three measures, with 33% fewer bits (Section 4.1, Table 27).

Details: Section 4.1; Appendix F (Table 20 on the bit accounting on short clips; Table 21, the break-even lengths at both resolutions at four and at eight bits; Figure 12 with the adapter at four bits); Appendix J (Table 27).

2. Does KAVA still use fewer bits than random-access HM and VTM, and than the generative codecs with all bits counted? (W1 (b), Q1)

Q (W1 (b), Q1): "HM/VTM also use low-delay settings despite KAVA being offline and bidirectional[...]." "Would KAVA retain its advantage if HM/VTM used an offline/random-access configuration and the generative codecs were evaluated on the same frames with all required bits counted?"

A: KAVA uses 43%–55% fewer bits than random-access VTM and 61%–69% fewer than random-access HM at equal LPIPS, and 70%–82% fewer at equal DISTS where a BD-rate exists (Table 2). Random access suits an offline codec: its encoder predicts from later frames, as KAVA does.

Counting all required bits, KAVA also needs fewer bits than GNVC-VD's published points at 1920 × 1088, from 97 frames on LPIPS and 25 on DISTS. KAVA on the 1.3B model is better than every run of VoV's released code on VoV's 81 frames (answer 1 above). Details: Table 2 (Section 4.1, rows "HM-RA" and "VTM-RA"); general comment, points 1 and 4.

(Continued.)

AuthorsOpenReview ↗

Response to Reviewer niQE (2/3)

3. Do the gains depend on the 14B base model? (Q4)

Q (Q4): "Appendix B specifies Wan2.1-FLF2V-14B. Have you tested a smaller backbone with comparable first-and-last-frame conditioning? A comparison of reconstruction quality at matched coded bitrate, runtime, and peak memory [...] would clarify how much of KAVA's performance depends on this model scale."

A: KAVA's advantage holds with a backbone about 10 times smaller. Wan2.1 only has a first-and-last-frame model at 14B, so we used the closest 1.3B model that accepts boundary frames, Wan2.1-Fun 1.3B InP. At matched rate on the first 81 frames of ShakeNDry (UVG, 832 × 480), both backbones reach LPIPS 0.331: 0.01147 bpp for 1.3B (Appendix J, Table 27) and 0.01144 bpp for 14B (new, reported here). This supports the view that the gains do not require a 14B backbone. The 1.3B model is a different model, not a smaller copy, so this comparison does not fully separate model size from architecture. On one H100 GPU, the 1.3B model takes 0.56 s per generated frame and the 14B model 2.53 s (Table 23); peak memory is 15.9 and 29.0 GiB (new, reported here). General comment, point 3 adds 1920 × 1088 results for the 1.3B model.

4. Is coding the VAE latents enough at KAVA's rates? (Q2)

Q (Q2): "Have you compared KAVA with quantizing and entropy-coding video-VAE latents, such as those from LTX-Video, followed by VAE-only decoding? At matched coded bitrate, how do reconstruction quality and encoding/decoding time compare? [...]"

A: At 1.1 and 2.2 times KAVA's rate, decoding quantized Wan2.1 VAE latents with the VAE alone gives LPIPS 0.721 and 0.685, against 0.304 for KAVA, and the VAE alone needs about 0.0580 bpp, 14.8 times KAVA's rate, to reach KAVA's LPIPS (new, reported here). We tested the Wan2.1 VAE, the VAE KAVA decodes with, on ShakeNDry at 832 × 480:

ShakeNDry, 832 × 480bppLPIPSDISTSPSNR
KAVA, CRF 380.003930.3040.11225.6
VAE only0.004170.7210.42715.3
VAE only0.008600.6850.39916.0

Without quantization, the VAE reaches LPIPS 0.088, so the poor quality at KAVA's rates comes from coding the latents with so few bits, not from the VAE itself. For decoding time, KAVA takes 2.5 s per frame at this setting; the VAE alone is faster, but at KAVA's rate its output is not usable. KAVA's encode and decode times are in Appendix G (Table 23).

5. Are the design ablations convincing? (W2 (a) to W2 (d), Q3)

Q (W2 (a)): "The design ablations are limited to one sequence and one run per setting."

A: Two ablations cover all seven UVG sequences, and each gives the same result on every sequence. The layer and adapter-type ablations use one video, ShakeNDry at 832 × 480 and CRF 38, with one run per row (Section 4.2). On all seven sequences, adding the neutral words to the prompt lowers LPIPS, by 2.8% on average (Table 6, Appendix A.1), and storing the adapter at 6 bits instead of 8 raises LPIPS by at most 0.0006 (Table 9, Appendix A.2).

Q (W2 (b), Q3): "LoRA continues from a trained adapter, while VeRA and LoRA-XS initialize their replacement terms from scratch (§4.2, Appendix A.2)." "Does LoRA still outperform VeRA and LoRA-XS with comparable initialization and total training budgets, including training before the ablations, and on sequences beyond ShakeNDry? Does the rotated neutral-word prompt improve reconstruction over a fixed prompt with the same words across clips?"

A: Yes, with the same starting point and the same training budget, LoRA still outperforms VeRA and LoRA-XS on all three measures. In the revision, the right half of Table 4 now compares adapter types that all start from scratch with the same budget: one run per row, 2,000 epochs, on ShakeNDry at 832 × 480 and CRF 38 (Section 4.2, Appendix A.2). LoRA (r=1), with the five conditioning layers trained from default initialization, reaches LPIPS 0.3003, DISTS 0.1094, and PSNR 26.13 dB. VeRA (q=256), which even keeps the fitted interface layers of the reported CRF-38 adapter, reaches LPIPS 0.3087, DISTS 0.1117, and PSNR 25.65 dB. LoRA-XS (q=32) reaches LPIPS 0.4482, close to the unadapted model's 0.4490. In both halves of Table 4, quality improves as the adapter gets more parameters; in the left half, LPIPS falls from 0.3124 to 0.2951 as the count grows. Fitting one video is thus limited more by the number of adapted parameters than by the adapter type. Rank-one LoRA is the best trade-off: it is the smallest LoRA and it can adapt every layer of our adapter, including the 3D convolution of the patchify layer, which VeRA and LoRA-XS cannot. At 1920 × 1088, the adapter is also a small part of the total rate, and the keyframes take most of the bits (Table 11, Appendix C). This adapter-type comparison covers one video. We answer the prompt question under W2 (d) below.

(Continued.)

AuthorsOpenReview ↗

Response to Reviewer niQE (3/3)

Q (W2 (c)): "The top half of Table 7 compares further training with freezing the existing interface adapters; it does not test removing them from the full model."

A: Most of the gain comes from the five conditioning layers; the interface layers add a small, consistent improvement. Training the interface layers further, rather than keeping the CRF-38 adapter's values for them, improves all three measures: LPIPS 0.2951 to 0.2928, DISTS 0.1089 to 0.1065 and PSNR 26.06 to 26.16 dB, with lower LPIPS at 35 of 40 validation checkpoints during training. An adapter on the interface layers alone lowers DISTS by 9%, from 0.1886 (unadapted) to 0.1714, while LPIPS moves only from 0.4490 to 0.4475. Details: Table 8 (Appendix A.2).

Q (W2 (d)): "The rotated prompt also lacks an ablation."

A: Adding the neutral words improves reconstruction on all seven UVG sequences and adds no bits. LPIPS is lower on every sequence (832 × 480, CRF 44), by 2.8% on average. Both prompts contain the clip keyword and the generic caption; the comparison is the full rotated prompt against the same prompt without the neutral words. Details: Table 6 (Appendix A.1), cited in Section 3.3.

6. Why is DiV-INR missing from the tables? (W1 (c))

Q (W1 (c)): "[...] the closely related DiV-INR is missing from quantitative evaluation."

A: DiV-INR's lowest UVG rate, about 0.0067 bpp as read from Figure 4 of the DiV-INR paper (not from ours), lies above KAVA's highest rate (0.0049 bpp at 832 × 480, 0.0035 bpp at 1920 × 1088), so the two rate ranges do not overlap and no comparison at matched rate exists. The DiV-INR paper also evaluates UVG at 1024 × 576, a size none of the other compared methods uses. Section 2 discusses the method.

Encoding and decoding cost is covered in general comment, point 2. The review's submitted Tables 7 and 9 to 11 appear as Tables 8 and 11 to 13 in the revised PDF.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General response (1/2)

We thank the chairs and the reviewers for their feedback and time. The reviewers single out specializing a shared frozen video model to each video with a small adapter, while a standard keyframe stream anchors the content and the departure from learned codecs that send latents per frame and from neural representations that send a whole network. One review also points to the placement of the adapter at the interface and conditioning layers, justified and tested by ablation, and two point to perceptual gains over traditional and learned codecs, with the lowest LPIPS and DISTS on UVG at both resolutions. The reviews also value thorough per-sequence evaluation with perceptual and pixel measures and careful rate accounting.

The reviewers ask for more evidence on the video length at which KAVA pays off when every bit is counted, encoding and decoding compute, whether the gains come from the size of the base model, stronger configurations of the conventional codecs, and whether the generated content stays faithful to the source. In the rebuttal, we add break-even video lengths against the traditional and learned codecs and GNVC-VD, and random-access configurations of HM and VTM, as suggested by the reviewers. We further add a comparison on a 1.3B backbone, the size of VoV's model, encoding and decoding times next to VoV and VTM, a measurement of generated content that is absent from the source, and we now send the adapter at four bits. We answer the common concerns below.

Table, figure and section numbers refer to the revised PDF. For the reviewers' convenience, new and changed text there is blue and the additions for the adapter stored at four bits are red; both will become regular text before the final deadline. Quotes from the reviews keep the submitted numbers. Results that are not in the revised paper are marked "(new, reported here)". In our replies, "KAVA" alone means the adapter sent at four bits, as in the revised paper. Figure 1, Table 3, the ablations on ShakeNDry, and appendix results not marked four-bit use the same checkpoints at eight bits.

1. At what video length does KAVA pay off? (niQE W1, Q1; UCUi W2, Q3, W3, Q2; xfZF W1)

Counting the whole adapter and the keyframes, KAVA needs fewer bits than GNVC-VD (CVPR'26), the closest generative competitor, from 97 frames on LPIPS and 25 on DISTS, and fewer than HM, VTM, x265, DCVC-RT and DCVC-FM from at most 193 frames.

The exact lengths, in frames and in seconds at 30 fps (common today), 60 fps (increasingly common) and 120 fps (the native frame rate of UVG):

AgainstResolution, measureFrames30 fps60 fps120 fps
GNVC-VD1920 × 1088, LPIPS973.2 s1.6 s0.8 s
GNVC-VD1920 × 1088, DISTS250.8 s0.4 s0.2 s
HM, VTM, x265, DCVC-RT, DCVC-FM1920 × 1088, LPIPS1214.0 s2.0 s1.0 s
HM, VTM, x265, DCVC-RT, DCVC-FM1920 × 1088, DISTS732.4 s1.2 s0.6 s
HM, VTM, x265, DCVC-RT, DCVC-FM832 × 480, LPIPS1936.4 s3.2 s1.6 s
HM, VTM, x265, DCVC-RT, DCVC-FM832 × 480, DISTS732.4 s1.2 s0.6 s

The adapter (37,548 bytes) is sent once per video, so its cost per frame falls with length, from 0.0093 bpp over 81 frames (Appendix F.1) to 0.0013 bpp over a full UVG sequence at 832 × 480. HM, VTM and x265 are measured on the same short frame ranges. For DCVC-RT, DCVC-FM and GNVC-VD we hold the rate per frame fixed; GNVC-VD has no released code, so we use its published points.

No length applies to GLC-Video, whose curve does not overlap ours on several sequences, or to VoV and VoV+Scaling, whose adapter cost per frame also falls with length.

On whole videos at 1920 × 1088, KAVA uses 40% fewer bits than GNVC-VD, with LPIPS 0.2895 against 0.291 and DISTS 0.0750 against 0.115 (Table 1). Even on GNVC-VD's own 96 frames, KAVA matches GNVC-VD's published LPIPS at equal rate and has 32%–37% lower DISTS at keyframe CRF 28 and 33 (Appendix F.1). Details: Table 21 and Figure 12 (Appendix F.2).

2. Is KAVA's compute practical? (UCUi W1, Q1; niQE summary; xfZF W2)

KAVA remains computationally expensive relative to conventional codecs, reflecting the cost of its diffusion backbone. However, among generative approaches, it requires substantially less encoding compute than VoV (ICML'26): 235 versus 7,286 GPU-seconds per frame in our experiments, a 31-fold reduction. Clips are generated separately, so cost is linear in length, not quadratic as with whole-video attention. The backbone is 95% of decoding time, so the cost is that of today's diffusion models; generative-prior compression is just emerging, so it would fall as they speed up. A video is encoded once, at one rate, and decoded many times. Details: Section 5 "Compute" and Table 23 (Appendix G).

(Continued.)

AuthorsOpenReview ↗

General response (2/2)

3. Are the gains purely due to a larger base model? (xfZF W1; niQE Q4)

KAVA's advantage holds on the 1.3B model, about 10 times smaller than the 14B one and the size of VoV's model. On VoV's 81 frames of ShakeNDry at 832 × 480, with the adapter included in the rate, KAVA on the 1.3B model outperforms all three runs of VoV's released code on LPIPS, DISTS and PSNR with 21%–31% fewer bits. It also outperforms the middle point of VoV+Scaling on all three measures with 33% fewer bits (Section 4.1; Table 27, Figure 17).

On all 289 coded frames of ShakeNDry at 1920 × 1088, KAVA on the 1.3B model (8-bit adapter in the rate) has lower DISTS than every codec at all three rates, and lower LPIPS at two of them (new, reported here). Each codec is at its lowest-rate setting that uses 1 to 3 times KAVA's rate (shown as a multiple, as in Table 1); GLC-Video has none at the lowest rate.

bppLPIPSBest codec, LPIPSDISTSBest codec, DISTS
0.003090.2990.304 (VTM, 1.3×)0.08760.1062 (x265, 1.2×)
0.001710.3310.304 (VTM, 2.3×)0.09800.1234 (GLC-Video, 2.9×)
0.001000.3660.409 (VTM, 1.1×)0.10340.1768 (HM, 1.2×)

4. Does the advantage hold against random-access HM and VTM? (niQE W1, Q1)

KAVA spends 43%–55% fewer bits than random-access VTM and 61%–69% fewer than random-access HM at equal LPIPS, across UVG at 832 × 480, UVG at 1920 × 1088 and HEVC Class B (Table 2). At equal DISTS the saving is 70%–82% where a BD-rate exists. Details: Table 2, where BD-rate is the average bit saving at equal quality (the random-access rows are new in the revision).

5. Does the generated video stay faithful and consistent? (niQE summary; UCUi W4, W9, W11)

KAVA has the highest pixel agreement with the source (share of pixels with the same segmentation label) at four of the five settings tested, for example 0.838 against 0.592 to 0.706 for the codecs on UVG at 832 × 480. PSNR is lower than the codecs', as expected for a method that optimizes perception (Appendix D). On temporal consistency, KAVA has the lowest mean warp error of the codecs in Table 3 on UVG at 1920 × 1088, 13.35 against 13.63 to 32.86. Clip-boundary artifacts are reported in Section 5 "Limitations". Appendix I (Tables 25, 26) also reports how often the segmentation model labels pixels with a class absent from the source (our reply to UCUi, W4).

6. Are the ablation results reliable? (niQE W2; xfZF W3)

Two ablations cover all seven UVG sequences, and each gives the same result on every sequence. The layer and adapter-type ablations use one video, ShakeNDry at 832 × 480 and CRF 38, with one run per row (Section 4.2). On all seven sequences, adding the neutral words to the prompt lowers LPIPS, by 2.8% on average (Table 6, Appendix A.1), and storing the adapter at 6 bits instead of 8 raises LPIPS by at most 0.0006 (Table 9, Appendix A.2). On ShakeNDry, the run that trains the interface layers further has lower LPIPS than the run that keeps the CRF-38 adapter's values for them, at 35 of the 40 validation points both runs share, not only at the end of training (Table 8).

Details: Table 4 (Section 4.2) for the layer ablations, Table 8 and Table 9 (Appendix A.2), and Section 3.3 and Table 6 (Appendix A.1) for the prompt ablation.

7. Could the VAE latents alone carry the video? (niQE Q2)

With the Wan2.1 VAE on ShakeNDry at 832 × 480, VAE-only decoding reaches KAVA's LPIPS only at 14.8 times KAVA's rate. Near KAVA's rates, the decode is a grey field with coloured blocks. Matching KAVA's LPIPS of 0.304 at 0.00393 bpp (CRF 38) takes about 0.0580 bpp (new, reported here).

What changed in the revision

  • The abstract now states the comparison against traditional and learned codecs, and Section 4.1 notes that VoV and GNVC-VD evaluate only short frame ranges.
  • We add the break-even lengths to Section 4.1, with the lengths at both widths in Table 21 and at four bits in Figure 12 (Appendix F.2).
  • We add random-access HM and VTM results to Table 2.
  • Section 5 "Limitations" gains a "Compute" paragraph, with per-frame medians in Appendix G.
  • The revision now sends the adapter at four bits, 44%–47% smaller, with quality practically unchanged: mean LPIPS 0.0005 to 0.0017 higher per test set, mean DISTS changing by at most 0.0005, and mean PSNR 0.04 to 0.06 dB lower. The abstract, introduction, Tables 1 and 2, Figure 5 and the conclusion report four bits; the 8-bit results remain in Table 2's second rows, Figure 5's hollow squares, Tables 11 to 18 (Appendix C), Tables 20 and 21, Table 27 and Figure 17.
  • Appendix J adds KAVA on the 1.3B backbone, and Appendix I adds the faithfulness evidence.

We believe that the revised paper, together with our replies, addresses the main points raised in the reviews.

Area_Chair_UJxCOpenReview ↗

Please join the discussion before it closes

Thank you to the authors for the rebuttal and to the reviewers for engaging in the discussion. As the discussion period draws to a close, I encourage everyone to keep the conversation going. I’d appreciate a brief final assessment from each reviewer. After reading the rebuttal and the discussion so far, what is the main reason you lean toward acceptance or rejection, and what evidence supports that view? Please note which concerns the rebuttal resolved and which remaining concerns, if any, are decisive for your recommendation.

If your assessment differs from another reviewer’s, please point to the specific claim or evidence you interpret differently. Authors are welcome to clarify any factual points that remain open. This will help us identify where we agree and understand the reasons for any remaining disagreement.

AuthorsOpenReview ↗

Summary of the rebuttal and revision

Dear Area Chairs and Reviewers,

Thank you very much for your time and for the effort you put into reviewing our paper. Below, we provide a brief summary of the rebuttal and the revision. This summary lists main points; all details are in our 17 replies and the revision, which adds five appendix sections, nine tables, and four figures.

Highlights

  • Clearer focus and claims. The rewritten abstract and introduction say what the paper does: we compress full-length video at low bitrate by conditioning a frozen video foundation model on heavily compressed keyframes and adapting it with a small adapter, sent once per video. The claim is scoped to full-length UVG sequences, 0.0013 to 0.0049 bpp, against traditional and learned codecs (Abstract, Section 1).
  • Four-bit adapter. The adapter is sent at four bits, which makes it 44% to 47% smaller than at eight bits, and mean LPIPS per test set changes by at most 0.0017. The main results use it (Section 3.2; Appendix A.2, Table 9).
  • Main result. On full-length UVG sequences, KAVA has the lowest LPIPS and DISTS at 832 × 480. At 1920 × 1088 it spends 40% fewer bits than GNVC-VD (CVPR'26) with matching LPIPS and lower DISTS (Table 1).
  • The adapter is needed. With the same frozen 14B model and keyframes, removing the adapter hurts at all 11 operating points, by 0.125 LPIPS on average (Section 4.1).
  • Smaller model. At VoV's model size (1.3B, about 10 times smaller), KAVA outperforms all three runs of VoV's released code on LPIPS, DISTS and PSNR with 21% to 31% fewer bits. This is on VoV's 81 frames of ShakeNDry, the only sequence run at that size, where it also needs about 106 times less encoding compute per frame than VoV (Appendix J, Tables 23 and 27).
  • Short videos. Counting the whole adapter and the keyframes, KAVA needs fewer bits than GNVC-VD from 97 frames (LPIPS) and 25 (DISTS) at 1920 × 1088, and fewer bits than HM, VTM, x265, DCVC-RT and DCVC-FM from at most 193 frames (Appendix F.2, Table 21, Figure 12).
  • Random-access codecs. At equal LPIPS, KAVA spends 43% to 55% fewer bits than random-access VTM and 61% to 69% fewer than random-access HM. On every UVG sequence at 1920 × 1088, every KAVA point has lower DISTS than every random-access VTM point, so no DISTS BD-rate exists there (Table 2; Section 5).
  • Compute. At 832 × 480, KAVA encodes with 31 times less GPU compute per frame than VoV (ICML'26) and decodes in about the time VoV reports (Section 5, Table 23).
  • Faithfulness. KAVA has the lowest hallucination rate at every operating point: 2.6% to 4.2% of pixels at 832 × 480, against 15.6% to 38.6% for the other codecs (Appendix I).

What the revision adds

New experiments

  • On VoV's 81 frames, KAVA reaches the same LPIPS on the 1.3B and the 14B model at about the same rate (Appendix J).
  • A reverse test puts grey frames in place of the keyframes. Both the adapter and the keyframes are needed (Section 4.1).
  • The Wan2.1 VAE alone needs about 25 times KAVA's rate to match KAVA's LPIPS on seven UVG sequences (Appendix K).
  • We score our reconstructions on the frames GNVC-VD and VoV evaluate (Appendix F.1, Table 20).
  • We train adapter types (LoRA, VeRA, LoRA-XS) from scratch; LoRA is best on all three measures (Table 4).
  • We ablate the prompt on all seven UVG sequences, and the neutral words lower LPIPS on every one. A new figure shows the per-clip prompt, built from a keyword, a generic caption and the rotated neutral words (Appendix A.1, Table 6, Figure 6).

New measures and analyses

  • Pixel agreement and hallucination rate (Appendix I, Tables 25-26).
  • PSNR curves and LPIPS with AlexNet features (Appendix D, Figures 9 and 10).
  • Encoding and decoding times per frame for every codec we ran, VoV and KAVA (Appendix G, Table 23), and model sizes (Table 22).
  • Output at keyframe positions compared with the decoded keyframe (Appendix A.3, Figure 7).
  • An analysis of the operating points where a codec is better, and why (Appendix C).
  • Quality values for the Figure 1 cells (Appendix H, Table 24).

Accounting and clarity

  • A new "Protocol" paragraph explains that the adapter is sent once per video, so we evaluate on full-length video (Section 4).
  • Figure 2, on four ways to code a video, now appears in Related Work.
  • The method states that one encoded video gives one operating point, that the CRF sweep is only for evaluation, that the adapter is entropy coded with LZMA, and that its cost per pixel falls as 1/N over N frames (Section 3).
  • Qualitative figure captions state the rate multiple wherever a competitor at a higher rate is better.
  • We state the limitations: compute, short videos, clip boundaries, one-sequence ablations (Section 5).

In the revised PDF, new and changed text is blue and additions for the four-bit adapter are red. We keep these colours for the discussion and assessment period; if the paper is accepted, the camera-ready version will print all text in the normal black.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.