- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
C2V is a training-free video token compressor applied before the LLM. It selects current-frame anchors by minimizing MMD between an attention-weighted scene distribution and an optimal mixture of the preceding compressed state and current anchors. Feature aggregation produces the LLM inputs and next-frame state. Experiments cover three backbones, five QA benchmarks, and four retention ratios.
Critical strengths
The formulation is well specified, with a closed-form mixture coefficient and detailed selection, aggregation, and position-handling rules. Component ablations examine the state, mixture weights, and feature fusion.
Critical weaknesses
-
The coverage analysis does not isolate the contribution of recurrence. Figure 1 evaluates the full pipeline; Table 5 ablates the previous-frame state only through QA accuracy (60.3% versus 59.2%). Report final-output coverage for the existing no-state MMD and fixed-mixture variants, keeping aggregation and token budgets identical.
-
Long-range redundancy and scalability remain insufficiently tested. The state stores only the preceding frame's compressed features. Test repeated or interleaved scenes and extend the current 128-frame evaluation up to 768 frames, with intermediate settings. Include Qwen3.5-27B and Qwen3.5-35B-A3B to extend beyond 7–8B dense models. Report accuracy, compression overhead, end-to-end latency, and peak memory against no-state selection.
-
Task coverage is limited to QA. Add temporal grounding and video captioning on suitable benchmarks, reporting timestamp accuracy and caption quality.
-
The baseline comparison is insufficiently comprehensive. Add a few representative baselines, such as SparseVLM [1], DyCoke [2], VidCom2 [3], HoliTom [4], and the now-public MMG-Vid. Compare accuracy and efficiency under matched inputs and comparable compression budgets, distinguishing before-LLM and hybrid configurations and including compression overhead.
[1] Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis A. Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, and Shanghang Zhang. “SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference.” ICML, 2025.
[2] Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. “DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models.” CVPR, 2025.
[3] Xuyang Liu, Yiyu Wang, Junpeng Ma, and Linfeng Zhang. “Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Models.” EMNLP, 2025.
[4] Kele Shao, Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. “HoliTom: Holistic Token Merging for Fast Video Large Language Models.” NeurIPS, 2025.
Questions for authors
See weekness.
3 comments
Response to W2: Long-Range Redundancy, Longer Inputs, and Larger Backbones
Q2. Long-range redundancy, longer inputs, and larger backbones. Following your suggestions, we added interleaved-scene tests, extended evaluation to 768 sampled frames with intermediate settings, and evaluated both Qwen3.5-27B and Qwen3.5-35B-A3B. Comparisons use no-state MMD with the same aggregation, inputs, and output budgets.
1. Interleaved scenes. Using Qwen3-VL-8B features from 900 VideoMME videos (32 sampled frames per source video), we split each video into consecutive segments and interleaved them with other videos: , with and unchanged per-frame budgets. Cross-visit duplication is the percentage of later-visit output tokens matching an earlier-visit output at cosine similarity .
| Interruptions | Duplication reduction vs. no-state, 10% (pp) | 20% (pp) |
|---|---|---|
| 1 | 1.83 | 1.17 |
| 3 | 1.77 | 1.07 |
| 7 | 1.33 | 0.84 |
| 15 | 0.48 | 0.29 |
All reductions have paired video-level bootstrap 95% CIs excluding zero. The reduction persists under interleaving, although it narrows with frequent interruptions. This indicates reduced cross-visit redundancy in the final outputs, not explicit long-term memory of earlier visits.
2. Scaling to 768 frames. We evaluated Qwen3-VL-8B at 128/256/320/512/768 sampled frames, keeping spatial preprocessing unchanged and the LLM visual-token budget fixed to the uncompressed 32-frame setting (). Entries below are C²V / no-state accuracy (%); Avg. covers the three benchmarks.
| Frames | Retention | VideoMME | LongVideoBench | LVBench | Avg. |
|---|---|---|---|---|---|
| 128 | 25% | 67.9/67.8 | 62.1/60.8 | 43.1/41.9 | 57.7/56.8 |
| 256 | 12.5% | 68.0/68.0 | 62.9/61.4 | 44.9/44.3 | 58.6/57.9 |
| 320 | 10% | 69.0/67.4 | 63.1/61.8 | 47.3/46.3 | 59.8/58.5 |
| 512 | 6.25% | 67.6/66.4 | 62.2/61.1 | 46.7/45.6 | 58.8/57.7 |
| 768 | 4.17% | 66.9/66.1 | 62.5/61.2 | 46.5/45.6 | 58.6/57.6 |
C²V achieves a higher mean in all five settings, with gains of 0.7–1.3 pp. At 768 frames, its mean is 58.6, vs. 57.6 for no-state and 54.5 for the uncompressed 32-frame baseline.
3. Larger dense and MoE backbones. We evaluated both requested Qwen3.5 models with 32 sampled frames, 10% retention, and non-thinking inference, following the same three-benchmark protocol. Entries are again C²V / no-state.
| Model | VideoMME | LongVideoBench | LVBench | Avg. |
|---|---|---|---|---|
| Qwen3.5-27B | 66.4/66.1 | 62.8/61.5 | 39.9/38.9 | 56.4/55.5 |
| Qwen3.5-35B-A3B | 63.7/63.1 | 59.0/58.1 | 40.6/39.5 | 54.4/53.6 |
C²V improves all three benchmarks on both backbones, with mean gains of approximately 0.8–0.9 pp.
4. Compression overhead, latency, and memory. Measurements use RTX A5000 GPUs: two for Qwen3-VL-8B (vision/compression and LLM on separate GPUs), and four for each Qwen3.5 model. All timings are medians over videos, measured within end-to-end runs that include vision encoding, compression, and LLM inference, with generation limited to one token (max_new_tokens=1) and host synchronization. Memory is the sum of per-GPU peak allocations. All pairs are C²V / no-state.
| Model / frames | Compression overhead (ms) | E2E (1 token, s) | Peak memory (GiB) |
|---|---|---|---|
| Qwen3-VL-8B / 128 | 167.3/148.9 | 4.89/4.88 | 25.54/25.54 |
| Qwen3-VL-8B / 256 | 209.6/181.0 | 7.99/7.95 | 27.63/27.63 |
| Qwen3-VL-8B / 320 | 233.0/200.1 | 9.56/9.49 | 28.06/28.06 |
| Qwen3-VL-8B / 512 | 294.4/255.7 | 14.24/14.27 | 31.23/31.23 |
| Qwen3-VL-8B / 768 | 448.3/354.1 | 20.82/20.72 | 35.87/35.87 |
| Qwen3.5-27B / 32 | 27.6/23.8 | 1.41/1.40 | 53.34/53.34 |
| Qwen3.5-35B-A3B / 32 | 20.9/17.6 | 1.39/1.37 | 67.50/67.50 |
In a separate measurement with a 16-token output, E2E latency is 3.40/3.39 s for Qwen3.5-27B and 3.06/3.03 s for Qwen3.5-35B-A3B. State conditioning increases compression overhead, but its effect on E2E latency is small in these measurements; peak allocations are unchanged at the reported precision.
These results extend the evidence for state conditioning to interleaved inputs, substantially longer sequences, and both larger backbone types.
Response to W3–W4: Temporal Grounding, Captioning, and Additional Baselines
Q3. Beyond QA: temporal grounding and video captioning. We evaluated Qwen3-VL-8B with 32 sampled frames at nominal 10%/20% retention, using common inputs and prompts. Baselines are FlashVID† and FastVID from our original evaluation, plus MMG-Vid and EchoPrune, related methods using past information whose code is now available.
Temporal grounding. On Charades-STA test (3,720 queries, 1,334 videos), R@θ is the percentage with temporal IoU ≥ θ; mIoU is mean IoU. Unparseable start–end intervals receive IoU 0. Scores are percentages.
| R | Method | [email protected] | [email protected] | [email protected] | mIoU |
|---|---|---|---|---|---|
| 100% | Uncompressed | 78.36 | 62.55 | 36.56 | 54.07 |
| 10% | FlashVID† | 66.24 | 49.27 | 26.29 | 44.79 |
| FastVID | 56.02 | 38.17 | 17.93 | 36.70 | |
| MMG-Vid | 68.12 | 49.41 | 26.59 | 45.54 | |
| EchoPrune | – | – | – | – | |
| C²V | 71.56 | 52.39 | 27.31 | 47.64 | |
| 20% | FlashVID† | 71.94 | 55.75 | 31.67 | 49.59 |
| FastVID | 64.14 | 47.42 | 23.90 | 42.95 | |
| MMG-Vid | 74.06 | 56.21 | 31.59 | 50.31 | |
| EchoPrune | – | – | – | – | |
| C²V | 74.70 | 56.94 | 32.96 | 50.97 |
C²V leads the reported compressors on all four metrics at both ratios. At 10%, gains over MMG-Vid are +2.10 pp mIoU (95% CI [1.15,3.03]) and +2.98 pp [email protected] ([1.28,4.67]); CIs use paired video-level bootstrap (2,000 resamples).
“–”: EchoPrune's outputs could rarely be parsed as a start–end interval (8.3%/19.4% at 10%/20%; mostly a single timestamp), so we did not run its full grounding evaluation. Its captioning results are reported below.
Video captioning. On YouCook2 validation (3,178 clips, 414 videos), CIDEr is the primary metric; BERTScore is F1 × 100.
| R | Method | CIDEr | METEOR | BERTScore |
|---|---|---|---|---|
| 100% | Uncompressed | 53.51 | 14.06 | 29.6 |
| 10% | FlashVID† | 38.95 | 11.07 | 25.2 |
| FastVID | 38.30 | 11.23 | 24.9 | |
| MMG-Vid | 36.34 | 10.72 | 24.9 | |
| EchoPrune | 32.58 | 11.30 | 24.0 | |
| C²V | 45.47 | 12.00 | 27.0 | |
| 20% | FlashVID† | 45.06 | 12.80 | 27.3 |
| FastVID | 41.49 | 12.44 | 26.5 | |
| MMG-Vid | 42.41 | 11.94 | 26.8 | |
| EchoPrune | 38.40 | 12.69 | 26.1 | |
| C²V | 50.21 | 13.01 | 28.4 |
C²V exceeds all four baselines on the three reported metrics at both ratios. Against FlashVID†, the highest-CIDEr baseline here, paired 95% CIs for CIDEr gains are [4.28,8.78] at 10% and [3.28,7.03] at 20%. These results extend the evidence to temporal localization and free-form video description.
Q4. Additional baselines and compression overhead. We added VidCom2, HoliTom, MMG-Vid, and EchoPrune on Qwen3-VL-8B, using the same inputs and comparable LLM-input token budgets. All are evaluated before the LLM. For HoliTom, we use the authors' w/o M configuration, disabling inner-LLM merging; FlashVID† likewise denotes its before-LLM component.
SparseVLM requires a backbone-specific pruning-layer and retention schedule, which we did not assign without validation. DyCoke's original four-frame grouping preserves a full reference frame and cannot match 10%/20% LLM-input retention without modification.
QA accuracy (%); Avg. is the five-benchmark mean (32 frames).
| R | Method | MVBench | TempCompass | VideoMME | LongVideoBench | LVBench | Avg. |
|---|---|---|---|---|---|---|---|
| 10% | VidCom2 | 59.6 | 63.9 | 56.7 | 54.0 | 36.8 | 54.2 |
| HoliTom (w/o M) | 65.6 | 68.4 | 58.5 | 55.0 | 35.5 | 56.6 | |
| MMG-Vid | 64.2 | 69.7 | 56.8 | 56.4 | 36.0 | 56.6 | |
| EchoPrune | 60.2 | 65.9 | 58.6 | 54.3 | 35.4 | 54.9 | |
| C²V | 66.2 | 70.1 | 59.3 | 55.5 | 36.7 | 57.6 | |
| 20% | VidCom2 | 65.3 | 68.5 | 60.6 | 55.6 | 37.1 | 57.4 |
| HoliTom (w/o M) | 67.2 | 71.0 | 60.7 | 56.4 | 37.3 | 58.5 | |
| MMG-Vid | 67.2 | 72.0 | 59.8 | 57.3 | 36.9 | 58.6 | |
| EchoPrune | 63.1 | 69.5 | 61.4 | 56.1 | 37.1 | 57.5 | |
| C²V | 68.3 | 71.5 | 61.0 | 57.8 | 37.8 | 59.3 |
C²V retains the highest five-benchmark average at both ratios, exceeding the strongest added baseline by 1.0/0.7 pp.
Compression overhead at 10% (ms): VidCom2 12.01, HoliTom (w/o M) 39.72, MMG-Vid 26.11, EchoPrune 13.29, C²V 26.58. These use the QA configurations, averaged over 300 VideoMME videos on an RTX A5000, from vision-feature output to LLM input, excluding encoding and LLM computation.
Planned revisions. In the final version, we will add the coverage, long-input, and larger-model comparisons with the no-state variant (Q1, Q2) to a new appendix section, referenced from Secs. 4.3–4.4; the cost measurements to Appendix E; the limitation on re-selection after long intervals to a Limitations section; the captioning results to Sec. 4.4 alongside the coverage analysis and the grounding results to the appendix (Q3); and the additional baselines to the main results (Q4).
Thank you again for your careful review and concrete suggestions. We hope these additional results address your concerns and would be grateful if you would consider them in your final assessment.
Response to W1: State Conditioning and Final-Output Coverage
Dear Reviewer bGcj,
Thank you for your careful and detailed review. We appreciate your recognition of our well-specified formulation and component ablations, as well as your constructive suggestions on recurrence, scalability, and broader evaluations. Your comments have provided a valuable opportunity to clarify our contributions and strengthen the empirical evidence. We address your concerns with additional experiments and analyses below.
Summary. (1) With identical aggregation and token budgets, state conditioning raises final-output coverage over no-state MMD by 0.9–2.5 pp and over a fixed mixture by 0.5–1.9 pp in all 12 conditions. (2) It reduces cross-visit duplication under interleaving, without constituting long-term memory, and improves accuracy over no-state at every tested input length from 128 to 768 frames (+0.7–1.3 pp) and on Qwen3.5-27B and Qwen3.5-35B-A3B (+0.8–0.9 pp), changing E2E latency by at most 0.1 s and leaving peak memory unchanged. (3) On Charades-STA grounding and YouCook2 captioning, C²V leads the reported compressors on every metric at 10% and 20%, e.g., +2.10 mIoU over MMG-Vid and +6.52 CIDEr over FlashVID† at 10%. (4) Against VidCom2, HoliTom (w/o M), MMG-Vid, and EchoPrune, C²V keeps the highest five-benchmark average (57.6/59.3 vs. at most 56.6/58.6 at 10%/20%), with 26.6 ms of compression overhead, within the 12.0–39.7 ms range of these baselines.
Q1. State conditioning and final-output coverage. You are right that Fig. 1 evaluates the full pipeline rather than isolating recurrence. Following your suggestion, we compared the existing no-state MMD (λ=0), fixed-mixture (λ=0.5), and full C²V variants using the same inputs, aggregation rule, and token budgets. We followed the Fig. 1 protocol (Qwen3-VL-8B, VideoMME, 32 sampled frames, 10% retention), measuring coverage from the final aggregated outputs, not the selected anchors alone.
MMD selection with aggregation already yields high coverage without a recurrent state. The full pipeline's coverage advantage should therefore not be attributed entirely to recurrence. Nevertheless, fixed mixing improves over no-state in all 12 rank–weighting conditions, and C²V improves over no-state by 0.9–2.5 percentage points and over fixed mixing by 0.5–1.9 points, again in all 12 conditions. With the aggregation rule and budgets unchanged, these results show the additional benefits of using the preceding compressed state and adapting its mixture weight to each candidate set.