- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents CoGCo, a training-free video token compression method guided by codec reference-update framework. It selects semantic anchors from I-frames and complementary P-frame updates using attention, feature diversity, motion vectors, and residuals. Packet sizes guide budget allocation, while update-guided reference recovery and global completion improve coverage. Codec-guided aggregation incorporates compatible unselected features into retained representatives without updating model weights. Experiments on four backbones and four video-QA benchmarks report improved average accuracy over existing methods across multiple retention ratios settings.
Critical strengths
-
The paper is well written and clearly presented. The overall pipeline, individual components, and their connections to video coding are easy to follow, while the overview figure provides useful explanations of the method.
-
The motivation is clear and intuitive. Persistent appearance and temporal updates serve different roles in video understanding, so using codec information to coordinate their representation provides a reasonable approach to token compression.
-
The experimental evaluation is reasonably extensive. Four backbones, four benchmarks, multiple compression budgets, component ablations, and hyperparameter sensitivity analyses provide a useful assessment of the method.
Critical weaknesses
-
The distinction from related temporal-reuse methods needs stronger justification. The reference-update organization is closely related to codec-driven sparsity in OneVision-Encoder [1] and reference-feature reuse in STC [2], although their implementations and operating stages differ. OneVision-Encoder is discussed, but neither approach is experimentally compared. Both address visual encoding costs, which can dominate video inference, whereas CoGCo compresses already encoded features. The authors should clarify the methodological differences and compare their accuracy-efficiency trade-offs under compatible settings.
-
The improvements over FlashVID [3] are modest. At 15% nominal retention, Table 1 reports average accuracy gains of 0.5, 0.1, and 0.9 percentage points on LLaVA-OneVision, LLaVA-Video, and InternVL3.5, respectively, averaging approximately 0.5 points. Figure 4 also shows only a 2.6% TTFT reduction, from 1320 ms to 1286 ms, at comparable accuracy. These margins require stronger evidence of practical significance and statistical reliability.
-
The efficiency comparison is insufficiently comprehensive. Figure 4 only compares CoGCo with FlashVID and UniComp [4], leaving its advantage over simpler methods such as VisionZip [5] unclear. The authors should include VisionZip under both matched token budgets and comparable accuracy, with a breakdown of visual encoding, compression, and LLM prefill costs. The analysis also omits peak GPU memory consumption, an important metric for assessing deployment feasibility and scalability. Since codec-sidecar preparation is excluded from the reported TTFT, its preprocessing cost and amortization should also be reported.
-
The sparse frame settings insufficiently validate codec-guided temporal aggregation. General benchmarks use only 32 or 64 frames, which can leave large temporal gaps in long videos and reduce the predictability between adjacent sampled frames. Since codec metadata is extracted from the prepared sequence, the resulting correspondence and redundancy may differ substantially from densely sampled video. Experiments with 128, 256, and 512 frames on VideoMME-Long and LongVideoBench are needed to establish the design’s effectiveness and scalability.
-
Task diversity remains limited. All evaluated benchmarks concern video question answering (VideoQA), leaving applicability to temporal grounding and video captioning unverified. These tasks require precise temporal localization and broader event coverage, respectively, and would provide stronger evidence that compression preserves information beyond QA-specific cues.
-
Model-scale generalization is insufficiently evaluated. All tested backbones contain fewer than 10B parameters. Experiments on the MoE model Qwen3.5-35B-A3B and the larger dense model Qwen3.5-27B would help establish whether the method remains effective on larger, practically relevant models that are still accessible to research laboratories.
[1] Feilong Tang et al. “OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence.” arXiv preprint arXiv:2602.08683, 2026.
[2] Yiyu Wang, Xuyang Liu, Xiyan Gui, Xinying Lin, Boxue Yang, Chenfei Liao, Tailai Chen, and Linfeng Zhang. “Accelerating Streaming Video Large Language Models via Hierarchical Token Compression.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.
[3] Ziyang Fan, Keyu Chen, Ruilong Xing, Yulin Li, Li Jiang, and Zhuotao Tian. “FlashVID: Efficient Video Large Language Models via Training-Free Tree-Based Spatiotemporal Token Merging.” In Proceedings of the International Conference on Learning Representations (ICLR), 2026.
[4] Chao Yuan, Shimin Chen, Minliang Lin, Limeng Qiao, Guanglu Wan, and Lin Ma. “UniComp: Rethinking Video Compression Through Informational Uniqueness.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.
[5] Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. “VisionZip: Longer Is Better but Not Necessary in Vision Language Models.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
Questions for authors
-
Can the authors provide comparisons with OneVision-Encoder and STC-Cacher under compatible settings, including separate measurements of visual encoding and LLM prefill costs?
-
Can the authors add efficiency comparisons with VisionZip at matched token budgets and comparable accuracy, and separately report codec preprocessing costs?
-
Can the authors evaluate more input frames on long-video benchmarks, reporting accuracy, TTFT, and peak memory?
-
Can the authors provide additional experiments on more video tasks beyond VideoQA, such as temporal grounding and video captioning?
-
Can the authors evaluate larger models, such as Qwen3.5-35B-A3B and Qwen3.5-27B, to demonstrate generalization across model scales and architectures?
6 comments
Response to Reviewer X1D3 (2)
R4 Sparse sampling and larger frame counts
CoGCo does not assume strong redundancy between frames. Instead, codec motion vectors and residuals propose candidate updates, while feature compatibility filters out unreliable associations. Table R8 reports our extended evaluation at 128 and 256 frames across LLaVA-OneVision and InternVL3.5.
Table R8. Accuracy comparison at 128 and 256 input frames on VideoMME-Long and LongVideoBench.
| Frames | Ratio | Method | VideoMME-Long OV | VideoMME-Long InternVL3.5 | LongVideoBench OV | LongVideoBench InternVL3.5 | Average |
|---|---|---|---|---|---|---|---|
| 128 | 25% | FlashVID | 49.4 | 52.8 | 58.4 | 61.1 | 55.4 |
| 128 | 25% | CoGCo | 51.0 | 55.9 | 57.9 | 60.9 | 56.4 |
| 128 | 20% | FlashVID | 51.0 | 52.4 | 57.5 | 60.8 | 55.4 |
| 128 | 20% | CoGCo | 49.8 | 56.3 | 58.8 | 60.1 | 56.3 |
| 128 | 15% | FlashVID | 49.5 | 51.7 | 58.0 | 59.7 | 54.7 |
| 128 | 15% | CoGCo | 50.9 | 54.9 | 57.8 | 59.9 | 55.9 |
| 256 | 25% | FlashVID | 50.6 | 53.2 | 57.7 | 62.0 | 55.9 |
| 256 | 25% | CoGCo | 50.3 | 56.4 | 57.7 | 61.6 | 56.5 |
| 256 | 20% | FlashVID | 50.7 | 53.8 | 57.1 | 61.6 | 55.8 |
| 256 | 20% | CoGCo | 50.9 | 55.9 | 58.5 | 61.7 | 56.8 |
| 256 | 15% | FlashVID | 50.3 | 51.6 | 57.8 | 62.0 | 55.4 |
| 256 | 15% | CoGCo | 50.3 | 56.2 | 58.4 | 60.9 | 56.5 |
CoGCo outperforms FlashVID across all six frame-count and retention averages. Notably, the largest average gains occur at the tightest 15% budget, and on VideoMME-Long where InternVL3.5 improves by +3.1 to +4.6 points. Increasing density to 128/256 frames reduces sampling stride by 4×–8×, directly shifting motion vector and residual distributions to stress-test temporal generalization. Scaling further to 512 frames introduces quadratic pre-compression encoding and LLM prefill memory bottlenecks that exceed our GPU envelope for unpruned baselines and fall outside standard training-free evaluation protocols; we leave 512-frame streaming scaling as future work.
R5 Temporal grounding beyond VideoQA
We agree that evaluation beyond VideoQA should test whether compression preserves temporal evidence for a different output task. We add a temporal-grounding comparison on Charades-TimeLens with Qwen3-VL-8B because LLaVA-OneVision and LLaVA-Video do not support grounding task [13].
Table R9a Temporal grounding on Charades TimeLens with Qwen3-VL
| Setting | mIoU | [email protected] | [email protected] | [email protected] |
|---|---|---|---|---|
| Full tokens | 47.73 | 68.60 | 52.42 | 26.85 |
| FlashVID 25% | 46.14 | 65.69 | 49.99 | 26.32 |
| CoGCo 25% | 46.78 | 66.96 | 51.50 | 26.73 |
| FlashVID 20% | 45.46 | 65.69 | 48.71 | 24.89 |
| CoGCo 20% | 46.05 | 66.07 | 50.10 | 26.32 |
| FlashVID 15% | 44.61 | 64.59 | 47.75 | 24.32 |
| CoGCo 15% | 45.44 | 65.45 | 49.84 | 25.99 |
The improvement is consistent across localization overlap and all three recall thresholds. CoGCo raises mIoU from 46.14 to 46.78 and [email protected] from 49.99 to 51.50.
Second, we evaluate detailed video captioning on VideoDetailCaption across LLaVA-OneVision and InternVL3.5:
Table R9b. Video captioning scores on VideoDetailCaption
| Method | Ratio | LLaVA-OneVision-7B | InternVL3.5-8B | Average |
|---|---|---|---|---|
| Vanilla | 100% | 3.36 | 3.29 | 3.33 |
| FlashVID | 25% | 3.08 | 2.96 | 3.02 |
| CoGCo | 25% | 3.02 | 2.99 | 3.01 |
| FlashVID | 20% | 3.05 | 2.93 | 2.99 |
| CoGCo | 20% | 3.06 | 3.00 | 3.03 |
| FlashVID | 15% | 2.97 | 2.90 | 2.94 |
| CoGCo | 15% | 2.98 | 2.92 | 2.95 |
CoGCo outperforms FlashVID across all three retention ratios on InternVL3.5, confirming that it preserves fine-grained visual details.
R6 Larger dense and MoE models
We thank the reviewer for suggesting both larger architectures. To evaluate scalability, we prioritized Qwen3.5-35B-A3B (Table R10). Although Qwen3.5-35B-A3B activates 3B parameters, its 35B capacity and fine-grained expert routing make it sensitive to token distortion. Under same protocol, CoGCo achieves consistent gains across all budgets. By contrast, evaluating the 27B dense model requires substantially higher compute and memory bandwidth than was feasible to complete across all benchmarks during the rebuttal window; we are actively securing additional compute resources and will gladly include the full Qwen3.5-27B dense results in the camera-ready version.
Table R10: Evaluation on larger MoE model Qwen3.5-35B-A3B across four benchmarks
| Ratio | Method | VideoMME | MVBench | MotionBench | LongVideoBench | Average |
|---|---|---|---|---|---|---|
| 25% | CoGCo | 64.6 | 69.3 | 59.8 | 61.0 | 63.7 |
| 25% | FlashVID | 64.5 | 69.2 | 59.5 | 59.6 | 63.2 |
| 20% | CoGCo | 64.5 | 68.2 | 59.5 | 60.2 | 63.1 |
| 20% | FlashVID | 63.5 | 69.0 | 59.5 | 58.0 | 62.5 |
| 15% | CoGCo | 63.5 | 67.8 | 59.3 | 59.6 | 62.6 |
| 15% | FlashVID | 63.1 | 66.9 | 58.8 | 57.3 | 61.5 |
References
[3] Feilong Tang et al. OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence. arXiv:2602.08683v3, 2026. https://arxiv.org/abs/2602.08683v3
[4] Yiyu Wang et al. Accelerating Streaming Video Large Language Models via Hierarchical Token Compression. arXiv:2512.00891, 2025. https://arxiv.org/abs/2512.00891
[13] Minseok Kang et al. OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models. arXiv:2605.11803v1, 2026. https://arxiv.org/abs/2605.11803v1
[15] Xiang An et al. LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence. arXiv:2605.25979, 2026. https://arxiv.org/abs/2605.25979
Response to Reviewer X1D3 (1)
We thank the reviewer for recognizing the clarity, motivation, and breadth of our evaluation. To address your questions, we clarify our methodological distinctions from pre-encoder/streaming approaches alongside empirical controls, demonstrate upper-bound gap recovery and TTFT/memory savings, and provide new experiments extending CoGCo to denser frame inputs, the larger Qwen3.5-35B-A3B model, and non-QA tasks.
R1 Distinction from OneVision2 Encoder and STC
We thank the reviewer for highlighting these works and noting that their implementations and operating stages differ from CoGCo. Specifically, that OneVision-Encoder [3] [15] relies on codec-aligned pre-training to drop patches before/during visual encoding and STC [4] performs feature-level streaming cache reuse, whereas CoGCo is a strictly training-free, post-encoder framework. Beyond operating stage and training-free compatibility, CoGCo differs methodologically in three concrete ways: (1)Backward Reference Recovery: Unlike forward sparsification or forward feature caching, CoGCo traces P-frame prediction chains backward to recover overlooked I-frame anchor regions that carry structural context. (2)Motion-Compensated Residual Filtering: Rather than relying solely on raw block sparsity or embedding similarity, CoGCo separates global camera motion from local object dynamics, suppressing camera-induced false updates. (3)Packet-Guided Allocation and Cross-Frame Aggregation: Instead of discarding unselected P-frame tokens, CoGCo uses bitstream packet sizes to allocate budgets and aggregates compatible P-frame update features into retained anchor tokens along motion-verified correspondences. To provide an empirical reference under comparable token budgets (Table R5), we compare OneVision2 and STC Pruner results against CoGCo on VideoMME and MVBench. While OneVision2 requires 65% retention on VideoMME to reach 58.3%, CoGCo achieves 59.5% at 25% retention without modifying or retraining the vision encoder.
Table R5 Codec-gating controls and STC results
R2 Practical value and reliability of the gains
We thank the reviewer for asking to clarify the practical significance and reliability of these gains. In training-free video token compression, strong baselines already operate within 1.3–4.2 points of the full-token ceiling, so sub-point improvements recover a substantial share of the remaining accuracy gap. Specifically, CoGCo recovers up to 42% of the remaining gap between FlashVID and the upper bound on InternVL3.5-8B, 30% on Qwen3-VL-8B, and 31% on LLaVA-Video, while achieving >100% recovery across all budgets on LLaVA-OV. Translated into compute savings at matched accuracy, CoGCo at 15% retention reaches 57.0% on LLaVA-OV v.s. FlashVID's 56.7% at 25%—eliminating 40% of visual tokens, accelerating LLM prefill by 46.9%, reducing total model TTFT by 17.0% (1,321 to 1,096 ms). Finally, these gains hold consistently across all 12 backbone–budget, and widen in more demanding regimes: reaching +0.5 to +1.1 points on the larger Qwen3.5-35B-A3B model (Table R10), alongside non-QA tasks (Tables R9a–R9b).
R3 VisionZip efficiency and GPU memory
We evaluate VisionZip on LLaVA-OV, Table R6 compares accuracy and latency at matched token budgets and comparable accuracy.
Table R6. Accuracy and latency breakdown on LLaVA-OV (ms).
At the matched 15% budget, CoGCo improves average accuracy by 3.7 pp over VisionZip, with 12.9% higher TTFT. At comparable accuracy, CoGCo at 15% exceeding VisionZip at 35%, while reducing TTFT by 29.5%. These results establish a more favorable accuracy–latency trade-off at comparable predictive performance. We further measure peak GPU memory across visual encoding, compression, and LLM prefill for CoGCo and FlashVID.
Table R7. Peak GPU memory by stage (GB).
CoGCo's compression-stage peak and overall measured peak matches FlashVID. At 15% retention, CoGCo requires 15.4 GB during LLM prefill, compared with 16.9 GB for FlashVID at 25%, while achieving comparable average accuracy. Finally, we measure codec-sidecar preparation separately from model inference. H.264 re-encoding takes 1,453 ms per video, v.s. 1,383 ms for direct decoding, an additional 70 ms. The prepared codec metadata is reusable across questions, while the online compression operations remain included in the reported model TTFT.