- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Context-aware Visual Token Compression (CVTC), a training-free visual token compression method for MLLMs. The method addresses the limitations of existing token pruning and merging approaches that rely primarily on individual token importance or raw feature similarity, potentially causing information loss or incorrect merging across distinct semantic regions. CVTC introduces a two-stage compression framework. First, Spatial Context Coarsening (SCC) constructs a spatial graph and performs context-aware greedy merging through iterative graph contraction. Second, Text-conditioned Context Coarsening (TCC) incorporates text-conditioned relevance to preserve task-critical visual details while further compressing the spatial clusters. Experiments on multiple MLLM backbones and visual understanding benchmarks demonstrate that CVTC achieves competitive performance under different token budgets, with particularly notable improvements under aggressive compression ratios.
Critical strengths
The paper evaluates CVTC on three representative MLLM architectures, covering fixed-resolution, multi-resolution, and dynamic-resolution visual tokenization. The experimental results demonstrate competitive accuracy across various benchmarks, particularly under aggressive token compression. The ablation studies on the spatial and text-conditioned components, as well as graph topology, provide additional evidence supporting the effectiveness of the proposed design.
Critical weaknesses
The proposed contextual greedy merging requires iterative graph contraction, including repeated neighbor matching, graph updates, and cluster feature aggregation. This process introduces additional computational complexity that may become increasingly expensive as the number of visual tokens grows. In Table 4, even on LLaVA-v1.5-7B with only 576 original visual tokens, CVTC exhibits higher prefill latency than several competing methods. The paper does not provide sufficient evidence regarding the scalability of this iterative procedure to substantially longer visual sequences, such as those encountered in high-resolution images.
Questions for authors
-
How does the inference efficiency of CVTC compare with existing methods on Qwen2.5-VL-7B? Table 4 reports efficiency results only on LLaVA-v1.5-7B, which has a relatively short visual token sequence. It is important to understand how the computational overhead scales with longer visual sequences. Could the authors provide prefill latency and end-to-end inference latency comparisons on Qwen2.5-VL-7B under the same token budgets as competing methods?
-
What explains the inconsistent experimental results reported in Tables 5 and 6? Tables 5 and 6 report different results for the full CVTC method under the same LLaVA-v1.5-7B setting with 64 retained tokens. Specifically, Table 5 reports an MME score of 1674 and a three-benchmark average retention of 93.6%, whereas Table 6 reports an MME score of 1640 and an average retention of 92.9%. Could the authors clarify the reasons for these discrepancies?