acceptodds
Under review as a conference paper at ICLR 2027

QCPruner: Query-Conditioned Population Coverage for Visual Token Pruning

Abstract

The high visual-token load in multimodal large language models (MLLMs) motivates aggressive training-free pruning, but under a prescribed final token budget, deciding which visual evidence survives is critical to preserving downstream performance. Existing methods rank tokens, diversify selected subsets, or combine relevance with coverage, while the direct conditioning of visual-to-visual population coverage by contextual per-visual query utility remains underexplored. We introduce QCPruner, a training-free method that uses contextual query utility to condition population-level visual-token coverage. Using keyword-matched query anchors, QCPruner fuses two cross-modal cues into utility and incorporates it into a visual-affinity-based coverage objective, where utility modulates representative suitability and, in the full formulation, also weights target importance. The resulting nonnegative facility-location objective is monotone and submodular, retains the standard 1-1/e greedy guarantee, and requires no model training or parameter updates. Across LLaVA-1.5, LLaVA-NeXT, LLaVA-Video, and Qwen2.5-VL, QCPruner consistently preserves strong downstream performance under matched token budgets, with the clearest gains appearing under aggressive compression. At 32 of 576 tokens on LLaVA-1.5-7B, it retains 96.1% of unpruned performance, versus 93.9% for the strongest evaluated baseline. At 256 of 1296 tokens on Qwen2.5-VL-7B, the corresponding values are 96.7% and 92.5%. Role-wise ablations show that representative-side query conditioning accounts for most of the gain over visual-only coverage, while target-side weighting provides a smaller, setting-dependent refinement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Visual Token Pruning, Efficient Inference, Training-Free Pruning, Vision-Language Models, Query-Aware Token Selection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VYX4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes QCPruner, a training-free visual token pruning method that integrates query-conditioned utility into a facility-location coverage objective. The method combines hidden-state similarity and query–key alignment to guide token selection, and demonstrates strong performance retention across four MLLM backbones under aggressive compression.

Critical strengths
  1. The paper addresses a practical problem and achieves strong performance across multiple MLLMs and benchmarks, particularly under aggressive token compression.

  2. The experimental evaluation is fairly comprehensive, including component ablations, controlled selector comparisons, pruning-layer analysis, and computational cost measurements.

Critical weaknesses
  1. The QCPruner-Early comparison in Table 3 does not isolate the effect of pruning depth, since this variant also changes the selection objective and removes query conditioning. I would like to see a comparison using the same visual-only coverage objective at different pruning layers.

  2. The necessity of bilateral utility weighting is not sufficiently justified. Table 4 shows that representative-only weighting already achieves nearly identical performance on Qwen2.5-VL, while the theoretical guarantee is inherited from standard facility-location optimization.

  3. Table 5 measures the effect of removing high-utility tokens on output KL divergence, but does not directly validate whether QCPruner preserves query-relevant visual evidence.

  4. The paper lacks controlled experiments on relational questions, negation, prompt paraphrases, and multilingual queries, where unigram extraction may miss important semantic information.

  5. The accuracy–efficiency trade-off needs further evaluation. In Table 3, QCPruner requires 11.1 TFLOPs and 127.5 ms of prefill compared with DivPrune's 5.0 TFLOPs and 41.8 ms; accuracy–latency curves across different budgets and backbones would be more informative.

Questions for authors

pls see weakness

Reviewer_wQjeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

QCPruner selects a fixed number of visual tokens using query-conditioned facility-location coverage. At a post-layer-7 hook, keyword-matched prompt anchors provide hidden-state similarity and pre-softmax query-key alignment cues; their fused utility weights both representative suitability and target importance. The nonnegative objective admits the standard greedy submodular guarantee for coverage, rather than downstream accuracy. Across four 7B image/video backbones, the method preserves strong performance at matched final budgets, including 96.1% average relative performance at 32/576 LLaVA-1.5 tokens and 96.7% at 256/1296 Qwen2.5-VL tokens. The empirical study is substantial, although some comparisons and calibration details need clarification.

Critical strengths
  1. The formulation makes a useful distinction between selection roles. Representative utility affects which retained tokens cover the population, whereas target utility changes which discarded evidence receives priority. This provides a concrete extension of population coverage beyond the separate text and visual targets used by MMTok [1].

  2. The evaluation covers diverse settings and aggressive compression. Standard images, native high-resolution inputs, and video are evaluated across twelve budgets, with complete task-level tables. The large Qwen improvement at 128 tokens is especially convincing as a retention result, while individual regressions are openly reported.

  3. The controls help identify what contributes. The paper includes matched selector transplants, single-cue and anchor ablations, representative/target weighting controls, and paired token-removal interventions. It correctly identifies representative-side weighting as the main contributor and acknowledges the smaller, setting-dependent benefit of target weighting.

  4. The scope and cost trade-off are unusually clear. Calibration is conducted on separate benchmarks, and the paper distinguishes complete systems from transplanted backends. It explicitly avoids claiming compute-matched superiority and reports the extra dense-layer processing and selector complexity.

Critical weaknesses
  1. Important coverage-oriented comparisons are missing. CoIn [2] combines visual-text alignment, saliency, and volume-based coverage, while SCOPE [3] combines saliency with population-coverage gains but appears only in related work. I would prefer a focused matched-budget comparison on a shared image backbone to better establish the advance beyond the evaluated MMTok and DPP formulations.

  2. How stable is keyword conditioning when the question changes? English unigram extraction and max-over-anchor scoring may miss relational or referential constraints. A small paired test of equivalent paraphrases/translations and relation-sensitive changes, comparing matched anchors with full-prompt conditioning, would address a different issue from the existing keyword-count sweep.

  3. The calibration accuracy needs a precise definition. Appendix C.2 uses the full 1,500-example MMStar set, yet 0.3294 and 0.6018 cannot be rounded integer correct counts divided by 1,500. What denominators and aggregation produced these values, including the 0.3294 versus 0.3293 difference used to select the LLaVA floor?

  4. I would be careful with the early/full comparison. Section 4.4 says it isolates pruning depth, but Appendix A.2 also changes the query cues and facility-location selector to a visual-only norm prior and self-novelty heuristic. The difference therefore reflects a change of pipeline and cannot be attributed to depth alone.

[1] Sixun Dong, Juhua Hu, Mian Zhang, Ming Yin, Yanjie Fu, and Qi Qian. MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs. ICLR, 2026. Primary source.

[2] Chenxi Du, Yongheng Deng, Jiani Liu, Yujia Zhang, Xi Chen, and Ju Ren. CoIn: Coverage and Informativeness-Guided Token Reduction for Efficient Large Multimodal Models. CVPR, 2026, pp. 10492-10501. Primary source.

[3] Jinhong Deng, Wen Li, Joey Tianyi Zhou, and Yang He. SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs. NeurIPS, 2025. Primary source.

Questions for authors

see the weaknesses above.

Reviewer_E25MOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes QCPruner, a training-free method for visual-token pruning in multimodal large language models. It combines hidden-state similarity and pre-softmax query–key alignment at keyword-matched prompt positions to estimate query-dependent visual-token utility. This utility weights target importance and representative suitability in a facility-location coverage objective, optimized through greedy selection. Experiments across four image and video backbones report improved performance retention at matched final token budgets.

Critical strengths
  1. Although QCPruner builds on existing relevance-aware and coverage-based token selection methods, the paper provides a thorough discussion of related work and clearly distinguishes its formulation from prior approaches such as MMTok [1] and CDPruner [2]. Its integration of contextual query utility into visual-to-visual population coverage offers a useful new approach to visual-token pruning, supported by controlled selector comparisons.
  2. The paper provides controlled ablations that distinguish the contributions of query weighting and subset selection. The experiments cover four backbones and twelve token-budget settings, demonstrating strong performance retention across image and video tasks, particularly under aggressive compression.

[1] Dong S, Hu J, Zhang M, et al. MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs. ICLR, 2026. arXiv:2508.18264.

[2] Zhang Q, Liu M, Li L, et al. Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs. NeurIPS, 2025. arXiv:2506.10967.

Critical weaknesses

1. Efficiency comparisons with closely related methods are incomplete.

The compute-and-latency comparison in Table 3 covers only LLaVA-NeXT-7B at a final budget of 320 visual tokens and does not include MMTok [1] or CDPruner [2], although both are included in the performance evaluation. Reporting their computational cost and prefill latency under the same evaluation setting, including pruning overheads, would provide a more complete assessment of QCPruner's performance–cost trade-off against these closely related methods.

Questions for authors

same as weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.