- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CoVisco, a unified image-video vision encoder designed for efficient visual token compression. It introduces learnable abstract tokens to capture segment-level semantics while preserving fine-grained patch tokens within the encoder. An alternating attention mechanism enables cross-segment communication through abstract tokens, and a lightweight token selector supports flexible compression ratios for downstream MLLMs. Experiments demonstrate promising video understanding performance with substantially fewer visual tokens exposed to the language model.
Critical strengths
-
The paper addresses an important problem in efficient video understanding. The proposed design of maintaining both abstract and fine-grained tokens is reasonable, providing a flexible trade-off between compact semantic representations and detailed visual information.
-
CoVisco achieves competitive or better performance than several reference encoders while exposing significantly fewer visual tokens to the LLM. The evaluation also covers image classification, multimodal retrieval, and video understanding, demonstrating the potential of the unified encoder.
Critical weaknesses
-
The abstract suggests that "existing approaches primarily perform visual token compression after the vision encoder and before the language model". However, substantial prior research has already explored compression before or within vision encoders, as well as within LLMs. Recent approaches such as PVC, EarlyTom, and MeToM further demonstrate that visual token compression has been extensively investigated at different stages of the processing pipeline. Although the paper acknowledges some encoder-internal approaches, its characterization of prior work remains overly broad and does not clearly establish what fundamental limitation CoVisco addresses beyond existing methods.
-
Insufficient comparisons with recent token compression methods. The experimental comparisons mainly focus on conventional vision encoders such as CLIP and SigLIP, along with OneVision-Encoder. While these baselines are useful for evaluating representation quality, they are insufficient to demonstrate the advantages of the proposed compression architecture. Several more closely related approaches, including PVC (CVPR 2025), EarlyTom (CVPR 2026), MeToM (CVPR 2026), and Video-XL-Pro, have explored efficient visual token compression using different mechanisms. Comparisons with representative recent methods under comparable token and computational budgets would provide stronger evidence for the effectiveness of CoVisco.
-
The paper introduces several components, including learnable abstract tokens, alternating segmented attention, and a token selector, but their individual contributions are not adequately isolated. In particular, it remains unclear how much improvement comes from abstract-mediated cross-segment communication, whether the proposed token selector outperforms simpler selection strategies, and how the number of abstract tokens affects performance.
-
Although codec-native processing is emphasized in the paper's title and motivation, the proposed abstract-token architecture and segmented attention mechanism do not fundamentally depend on codec information. The same architecture supports uniform frame sampling, which achieves comparable or even better performance on several benchmarks. Consequently, the specific contribution of codec-based processing remains unclear, particularly given its close relationship to OneVision-Encoder. A more systematic comparison between codec-based and uniform sampling under matched computational and token budgets would help clarify its role and benefits.
[1] PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models
[2] EarlyTom : Early Token Compression Completes Fast Video Understanding
[3] MeToM: Metadata-Guided Token Merging for Efficient Video LLMs
[4] Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understanding
Questions for authors
I encourage the authors to clarify the distinctions between CoVisco and recent encoder-internal compression methods, include stronger and more relevant baselines, and conduct targeted ablations to isolate the contributions of the proposed components. In particular, demonstrating the independent benefits of codec-native processing and the advantages of abstract-token compression under controlled computational budgets would substantially strengthen the paper.