- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies training-free visual token pruning for multimodal LLMs. The key motivation is that different tasks rely on complementary image-driven and query-driven evidence, while sparse subset pruning can additionally disrupt the positional and content integrity of the visual sequence. To address these issues, the paper proposes Bifocal, which combines query-focused fine evidence, image-intrinsic fine evidence, and pooled coarse context before the LLM, followed by a second-stage attention-based pruning inside the LLM. Experiments across multiple VLM backbones and token budgets show strong results, especially on fine-grained grounding tasks.
Critical strengths
-
The paper is well-written and organized, and the figures are helpful for understanding the motivation and the proposed mixed-granularity design.
-
The motivation is clear and intuitive, especially the observation that sparse pruning can further damage both the positional and content integrity of the original visual sequence.
-
The additional visualizations and ablation studies are helpful for understanding the behavior of the proposed method and the contribution of its individual components.
Critical weaknesses
- My main concern is the generality of the proposed design and whether the current formulation fully follows from the paper’s own observations. The current method and analysis are largely developed around single-image setting, particularly with the three evidence types relying on separate scoring functions tailored to their respective roles. However, in more complex multimodal scenarios (e.g. multi-image, video, text-intensive multimodal reasoning tasks), the composition and interaction of visual evidence may differ substantially across images, text contexts, or temporal contexts, making it unclear whether the method design can generalize effectively.
- The evaluation could be further extended to more recent reasoning-oriented VLMs. The current experiments focus on LLaVA-1.5-7B, LLaVA-NeXT-7B, and Qwen2.5-VL-7B, while it remains unclear how well the proposed pruning strategy transfers to models with stronger multimodal reasoning behavior, such as Qwen3-VL-Thinking. Particularly, evaluating whether the selected visual evidence remains sufficient throughout longer decoding trajectories would provide a more complete picture of the method’s generality.
- The experimental results are not analyzed sufficiently. In Table 1, all baselines are reported for the 192- and 128-token settings, whereas FastV, PDrop, PruMerge+, and VisionZip are omitted from the 64-token setting without explanation. The paper should clarify the reason for these omissions and provide a more consistent comparison across budgets and backbones.
Questions for authors
- Have the authors considered using task-dependent token allocations for different types of visual evidence? Since the paper shows that different tasks favor different proportions of image-driven and query-driven evidence, it would be interesting to know whether allocating different numbers of coarse, image-intrinsic, and query-focused tokens according to the task type could further improve performance.
7 comments
Response to Reviewer VAWv (Part 2/4)
- Experiment 3: Performance on multi-image, video and text-intensive tasks
We further evaluate Qwen2.5-VL and Qwen3-VL on the following tasks:
- Multi-image: MuirBench
- Video: MVBench, Charades-STA
- Text-intensive reasoning: MMMU val, MathVista, TreeBench
Table 1. Qwen2.5-VL-7B, 25% tokens.
| Method | MuirBench | MVBench | Charades-STA [email protected] | Charades-STA mIoU | MMMU val | MathVista | TreeBench | Avg |
|---|---|---|---|---|---|---|---|---|
| Baseline | 45.8 | 65.6 | 31.5 | 30.6 | 50.7 | 68.1 | 40.2 | 100.0 |
| FastV | 40.6 | 62.2 | 28.0 | 28.2 | 47.9 | 51.7 | 35.3 | 89.0 |
| VisionZip | 44.0 | 62.9 | 26.9 | 27.4 | 47.8 | 62.9 | 38.8 | 92.9 |
| DART | 43.4 | 60.1 | 28.0 | 28.1 | 46.3 | 50.6 | 36.3 | 89.0 |
| HiPrune | 44.2 | 62.9 | 27.0 | 27.6 | 48.7 | 62.8 | 39.8 | 93.7 |
| EADP | 42.2 | 61.9 | 26.6 | 26.9 | 47.7 | 58.3 | 39.0 | 90.8 |
| VLM-Pruner | 43.2 | 62.4 | 25.5 | 25.9 | 47.3 | 59.4 | 40.5 | 90.9 |
| Bifocal | 44.2 | 63.4 | 28.2 | 28.6 | 48.9 | 65.4 | 41.2 | 95.9 |
Table 2. Qwen3-VL-8B-Instruct, 25% tokens.
| Method | MuirBench | MVBench | Charades-STA [email protected] | Charades-STA mIoU | MMMU val | MathVista | TreeBench | Avg |
|---|---|---|---|---|---|---|---|---|
| Baseline | 52.2 | 68.6 | 57.7 | 50.5 | 52.1 | 64.7 | 44.0 | 100.0 |
| FastV | 47.6 | 63.1 | 50.5 | 45.6 | 48.7 | 46.0 | 40.2 | 88.3 |
| VisionZip | 50.7 | 67.0 | 55.7 | 49.1 | 51.6 | 58.0 | 41.7 | 95.9 |
| DART | 49.0 | 65.8 | 51.9 | 46.9 | 49.8 | 50.8 | 40.7 | 91.6 |
| HiPrune | 50.4 | 67.5 | 55.4 | 48.9 | 50.3 | 59.0 | 41.7 | 95.7 |
| EADP | 51.6 | 65.2 | 54.6 | 48.5 | 51.3 | 53.7 | 42.5 | 94.7 |
| VLM-Pruner | 51.3 | 65.5 | 53.7 | 48.5 | 50.3 | 50.0 | 41.2 | 92.9 |
| Bifocal | 50.9 | 68.2 | 56.6 | 49.9 | 50.8 | 58.7 | 43.0 | 97.0 |
On the additional multi-image, video, and text-intensive tasks, Bifocal still leads on both models. However, for multi-image and video tasks, acceleration involves not only token redundancy. Key-image selection, memory, and other issues are equally critical, but these go beyond the scope of our method.
Response to Reviewer VAWv (Part 3/4)
W2: The evaluation could be further extended to more recent reasoning-oriented VLMs. The current experiments focus on LLaVA-1.5-7B, LLaVA-NeXT-7B, and Qwen2.5-VL-7B, while it remains unclear how well the proposed pruning strategy transfers to models with stronger multimodal reasoning behavior, such as Qwen3-VL-Thinking. Particularly, evaluating whether the selected visual evidence remains sufficient throughout longer decoding trajectories would provide a more complete picture of the method's generality.
We thank the reviewer for this very interesting suggestion. We explore it from two angles. First, we have added comparisons of Bifocal on Qwen3-VL-8B-Instruct in W1, and here we further add Qwen3-VL-8B-Thinking with thinking enabled. Second, to explore the generalization of our method during long decoding, we compare CoT reasoning with direct answering on MathVista for Qwen2.5-VL-7B and Qwen3-VL-8B-Instruct.
- Experiment 1: Qwen3-VL-8B-Thinking
With 25% tokens and at most 2048 generated tokens, the results are:
| Method | GQA | TextVQA | DocVQA | POPE | Gen | RefCOCOg test | REC |
|---|---|---|---|---|---|---|---|
| Baseline | 57.4 | 76.7 | 92.9 | 90.8 | 100.0 | 88.4 | 100.0 |
| FastV | 49.0 | 64.4 | 54.2 | 83.1 | 79.8 | 59.6 | 67.4 |
| VisionZip | 53.6 | 73.2 | 80.5 | 87.8 | 93.0 | 73.6 | 83.3 |
| HiPrune | 50.4 | 74.3 | 83.3 | 87.5 | 92.6 | 75.4 | 85.3 |
| EADP | 55.4 | 72.6 | 76.3 | 88.9 | 92.8 | 64.4 | 72.9 |
| Bifocal | 56.6 | 75.4 | 84.8 | 89.6 | 96.7 | 80.4 | 91.0 |
- Experiment 2: Long decoding
All results are on the full MathVista testmini with 25% tokens. Direct outputs only the option letter or a number, Reasoning reasons first and then answers (at most 4096 tokens), and Thinking uses Qwen3-VL-8B-Thinking (at most 8192 tokens):
| Method | Qwen2.5-VL-7B Direct | Qwen2.5-VL-7B Reasoning | Qwen3-VL-8B Direct | Qwen3-VL-8B Reasoning | Qwen3-VL-8B Thinking | Avg |
|---|---|---|---|---|---|---|
| Baseline | 68.1 | 69.9 | 64.7 | 72.3 | 75.8 | 100.0 |
| FastV | 51.7 | 50.3 | 46.0 | 45.5 | 52.1 | 70.1 |
| DART | 50.6 | 47.8 | 50.8 | 53.7 | 55.3 | 73.7 |
| EADP | 58.3 | 56.3 | 53.7 | 58.6 | 60.7 | 82.1 |
| VLM-Pruner | 59.4 | 54.7 | 50.0 | 55.2 | 58.0 | 79.1 |
| Bifocal | 65.4 | 60.9 | 58.7 | 59.8 | 59.8 | 87.1 |
In thinking mode, as the output grows longer, the overall ability of pruning methods drops, while Bifocal remains competitive.
W3: The experimental results are not analyzed sufficiently. In Table 1, all baselines are reported for the 192- and 128-token settings, whereas FastV, PDrop, PruMerge+, and VisionZip are omitted from the 64-token setting without explanation. The paper should clarify the reason for these omissions and provide a more consistent comparison across budgets and backbones.
FastV, PDrop, PruMerge+, and VisionZip are relatively early methods. Due to the page limit, we removed some of the earlier methods with relatively weaker performance from the 64-token setting. We have added these four methods to the 64-token setting of Table 1 in the revised paper. Below is the complete table with the venue of each method. Gen and REC follow Tables 1 and 2 of the paper. They average the following benchmarks relative to the unpruned model (%), with [email protected] for REC:
- Gen: VQAv2, GQA, MMB, OK-VQA, SQA-I, AI2D, TextVQA, OCRBench, DocVQA, ChartQA, POPE.
- REC: RefCOCO val, testA and testB, RefCOCO+ val, testA (R+-tA) and testB (R+-tB), RefCOCOg val and test (Rg-t).
Table 1. General and grounding benchmarks on LLaVA-1.5-7B, Average Tokens = 64. methods are those omitted from Table 1 of the paper.
| Method | Venue | VQA$^{\mathrm{v2}}$ | GQA | MMB | VQA | VQA^{\mathrm{T}}\ | DocVQA | POPE | R+-tA | Rg-t | Gen | REC |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| No Pruning | – | 78.5 | 61.9 | 64.1 | 53.5 | 46.0 | 21.5 | 85.9 | 59.4 | 48.4 | 100.0 | 100.0 |
| ECCV 2024 | 56.4 | 46.4 | 50.5 | 37.0 | 31.2 | 10.5 | 36.4 | 0.2 | 0.4 | 70.4 | 0.8 | |
| CVPR 2025 | 68.6 | 52.4 | 55.5 | 42.6 | 38.9 | 14.4 | 73.9 | 3.4 | 2.2 | 85.7 | 5.3 | |
| CVPR 2025 | 72.5 | 55.1 | 60.2 | 47.7 | 40.9 | 15.6 | 77.0 | 3.7 | 3.4 | 90.1 | 7.4 | |
| ICCV 2025 | 71.4 | 55.2 | 59.1 | 46.9 | 37.8 | 13.0 | 75.8 | 1.9 | 2.0 | 87.1 | 4.4 | |
| DART | EMNLP 2025 | 65.4 | 51.1 | 54.8 | 42.5 | 29.3 | 11.7 | 59.0 | 0.8 | 1.1 | 77.8 | 2.5 |
| Nüwa | ICLR 2026 | 74.6 | 58.6 | 62.0 | 49.4 | 39.9 | 15.5 | 79.8 | 33.9 | 24.3 | 87.4 | 54.9 |
| SID | ICLR 2026 | 74.2 | 57.3 | 59.5 | 48.4 | 38.0 | 13.8 | 84.3 | 2.6 | 2.2 | 89.8 | 5.4 |
| VLM-Pruner | CVPR 2026 | 69.9 | 54.4 | 57.7 | 43.0 | 32.8 | 11.9 | 69.1 | 1.2 | 1.3 | 82.2 | 3.3 |
| HiPrune | ACL Findings 2026 | 69.1 | 53.6 | 59.5 | 46.3 | 39.9 | 16.1 | 73.0 | 1.1 | 0.9 | 87.9 | 2.6 |
| EADP | ECCV 2026 | 75.3 | 58.1 | 60.7 | 47.9 | 37.6 | 14.4 | 85.6 | 3.2 | 2.3 | 90.0 | 6.2 |
| Bifocal | – | 75.3 | 58.5 | 62.7 | 46.8 | 40.9 | 17.0 | 81.7 | 41.9 | 30.4 | 90.6 | 66.7 |
Across backbones, all baselines are reproduced with their official code, so on Qwen2.5-VL we include those whose official implementations support Qwen. We further compare them on Qwen3-VL-8B (Appendix I.2), and compare the pre-LLM baselines with Bifocal using only its Stage 1 under the same per-layer budget (Appendix I.6).
Response to Reviewer VAWv (Part 1/4)
We thank the reviewer for the thorough review and insightful suggestions. We address each concern below. Newly added or substantially revised text in the manuscript is highlighted in blue. Below, Q, I and C denote query-focused evidence, image-intrinsic evidence and coarse context. With the additional experiments, Bifocal shows its generalization across 4 models, multiple budgets and 26 benchmarks covering 7 types of multimodal tasks, although some of these tasks go beyond the scope of this paper.
We understand the reviewer's concern and explain it from three aspects: how our observations lead to the design, whether the roles of the three kinds of evidence still hold in multi-image, video and text-intensive scenarios, and how Bifocal performs on these tasks.
(1) The observations, the method design and the three scoring functions are not independent. Our observations revolve around splitting the budget between two kinds of tokens, selected by the query and by the CLS token. They overlap little and each has a clear target: query tokens focus on objects, while CLS tokens are spread out and cover the image widely. The method therefore keeps these two kinds of selection and strengthens each around its target. For query tokens, we filter out the background more precisely to select target tokens. Based on these target tokens, we use the CLS attention to select further tokens that cover more information. The coarse tokens are obtained by aggregating the remaining tokens. Accordingly, query tokens aim at objects and improve REC, while CLS tokens help with questions about objects absent from the image and reduce hallucination.
(2) We thank the reviewer for this inspiring point and add further experiments on the role of token evidence in multimodal scenarios. On Qwen2.5-VL and Qwen3-VL, which support multi-image and video inputs, we select four benchmarks whose ground-truth target regions let us examine whether Q still keeps the content the question is about, and whether I still covers diverse content: the group grounding subset of MIG-Bench for multi-image (target: the target image and its box), Charades-STA for video (target: the frames within the queried time span), and TreeBench (target: the boxes of all instances in the question) and TextVQA for text-intensive tasks.
For each sample, we rank the tokens of all images or frames by the Q score and by the I score separately, take 25% of all visual tokens from each, and measure the fraction of the ground-truth region that is selected, ∣S∩G∣/∣G∣.
In these additional scenarios, Q still covers a larger fraction of the target, and the two selections share few tokens, so they are two different kinds of evidence. TextVQA is the exception because such questions rarely point to where the answer is (e.g., "what does the sign say?"). This is consistent with the analysis in Appendix B and does not contradict the conclusion.
On the multi-image and video tasks, under the same 25% budget, we compare giving the whole budget to Q, splitting it evenly between Q and I, and further adding C. We report the fraction of images or frames that receive at least half of their proportional share of tokens:
As the I evidence is added, the information coverage increases substantially.