- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MSDG-Prune uses update magnitudes to estimate visual-token saliency and update directions to form semantic groups. Query relevance guides token selection. The training-free method is evaluated across four MLLM backbones.
Critical strengths
The analysis offers useful insights into visual-token dynamics. The method requires no additional training and shows competitive average performance across several MLLMs.
Critical weaknesses
-
On NeXT-7B at 160 tokens, the average gain is largely driven by POPE, while four of seven tasks underperform PruneSID (Table 2). This weakens the claim that the method preserves visual evidence more broadly.
-
The comparison with EvoCut, a closely related method using representation dynamics, is limited to LLaVA-1.5. Its absence from the NeXT and Qwen experiments, despite supporting both models, leaves the benefit of MSDG-Prune’s specific design over existing dynamic pruning methods insufficiently established.
-
The reported 7.8× speedup applies to prefilling, while the scope of “Inference Time” in Table 5 remains unclear. Without explicit timing boundaries, generation lengths, and a breakdown covering vision encoding, pruning, and decoding, the actual end-to-end benefit is difficult to verify.
Questions for authors
-
On NeXT-7B at 160 tokens, the average gain is mainly driven by POPE, while four of seven tasks score below PruneSID (Table 2). How does this support the broader claim of preserving visual evidence?
-
EvoCut is missing from the NeXT and Qwen comparisons, although its original paper evaluates both models. Can the authors provide comparisons under matched settings to establish the benefit over this closely related method?
-
The timing boundaries in Table 5 are unclear. Does “Inference Time” include vision encoding, pruning, and decoding? Please clarify the generation length and provide a breakdown of end-to-end latency.
4 comments
Response to Reviewer DUqE (2/3)
Question 2: Additional Comparisons with EvoCut on LLaVA-NeXT and Qwen2.5-VL.
Thank you for suggesting a comparison with EvoCut [1], which also uses representation dynamics for token pruning. The published EvoCut results cover a different benchmark set for LLaVA-NeXT and different token retention ratios for Qwen2.5-VL. As no official implementation was publicly available, we reimplemented EvoCut and evaluated it under the same settings and token budgets as MSDG-Prune (Ours). As shown in Table DUqE-2. and Table DUqE-3, MSDG-Prune achieves higher RelAcc. than EvoCut across all six configurations, demonstrating the consistent performance advantage of our method.
Table DUqE-2. Qwen2.5-VL-7B Results.
| Method | GQA | MMB-en | MME | POPE | SQA | VQAv2 | HRB-8k | RelAcc. |
|---|---|---|---|---|---|---|---|---|
| Upper Bound (100%) | ||||||||
| Baseline | 60.9 | 83.9 | 2310 | 86.3 | 88.9 | 82.9 | 68.1 | 100.0% |
| Token Retention: 33.3% (↓66.7%) | ||||||||
| EvoCut | 57.6 | 80.5 | 2291 | 84.2 | 87.1 | 80.1 | 57 | 95.1% |
| Ours | 58.5 | 82.6 | 2332 | 84.7 | 87.8 | 80.5 | 61.9 | 97.2% (+2.1 pp) |
| Token Retention: 22.2% (↓77.8%) | ||||||||
| EvoCut | 57.3 | 79.5 | 2235 | 83.1 | 86.5 | 78.3 | 56.4 | 93.8% |
| Ours | 57.7 | 81.2 | 2297 | 84 | 87.3 | 78.9 | 61.9 | 96.1% (+2.3 pp) |
| Token Retention: 11.1% (↓88.9%) | ||||||||
| EvoCut | 52.8 | 76.6 | 2152 | 79.4 | 84.6 | 74.4 | 59.4 | 90.8% |
| Ours | 55.4 | 78.4 | 2178 | 81.3 | 86.3 | 75 | 59 | 92.4% (+1.6 pp) |
Table DUqE-3. LLaVA-NeXT-7B Results.
| Method | GQA | MMB-en | MME | POPE | SQA | VQAv2 | SEED-I | RelAcc. |
|---|---|---|---|---|---|---|---|---|
| Upper Bound (100%) | ||||||||
| Baseline | 64.2 | 67.9 | 1842 | 86.4 | 70.2 | 80.1 | 70.2 | 100.0% |
| Token Retention: 22.2% (↓77.8%) | ||||||||
| EvoCut | 63.2 | 66.3 | 1804 | 86.3 | 67.6 | 77.5 | 66.4 | 97.4% |
| Ours | 63.3 | 64.2 | 1805 | 87.6 | 68 | 77.6 | 67.5 | 97.5% (+0.1 pp) |
| Token Retention: 11.1% (↓88.9%) | ||||||||
| EvoCut | 60.9 | 63.7 | 1783 | 85.8 | 66.4 | 74.7 | 62.7 | 94.6% |
| Ours | 61 | 63.1 | 1747 | 87.6 | 67.5 | 74.8 | 64.4 | 95.1% (+0.5 pp) |
| Token Retention: 5.6% (↓94.4%) | ||||||||
| EvoCut | 59.4 | 62 | 1653 | 82.2 | 66.7 | 71.3 | 55.7 | 90.3% |
| Ours | 58 | 60.9 | 1627 | 87.1 | 68.5 | 71.5 | 60.9 | 91.9% (+1.6 pp) |
Table note. Values in parentheses indicate changes in RelAcc. relative to EvoCut at the same token retention ratio, measured in percentage points (pp). Higher is better for every reported score.
[1] H. Lu, F. Zhang, W. Jin, et al. EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models. arXiv:2606.01756, 2026.
Response to Reviewer DUqE (3/3)
Question 3: Clarification of inference time and detailed analysis of end-to-end latency
- Inference time is measured from when the input tensors are ready on the GPU to the completion of the final generated token. It includes vision encoding, visual projection, pruning, multimodal input preparation, LLM prefilling, autoregressive decoding, and other model-side overhead.
- Output length The original latency evaluation for MSDG-Prune was conducted on POPE [1] using natural EOS stopping. POPE requires binary yes/no answers; in this setting, each response consisted of one answer token ("yes" or "no") followed by an EOS token, giving a total output length of .
We conducted a stage-wise timing analysis of VisionZip, PruneSID and MSDG-Prune (ours) under the same evaluation configuration. Specifically, we measured inference latency of LLaVA-NeXT-7B on a single NVIDIA A800-80GB GPU with FP16 precision, batch size 1, and greedy decoding with KV caching. The pruning budget is 160 visual tokens (5.56% retention ratio). We evaluate EOS stopping (2 generated tokens, including EOS) and fixed-length generation of 8 tokens. Table DUqE-4. show these results.
Table DUqE-4. End-to-end latency breakdown on LLaVA-NeXT-7B (ms).
| Method | Output length | Request pre/ post-processing | Other overhead | Vision encoding | Pruning | Prefill | Decoding | Inference time | End-to-end latency |
|---|---|---|---|---|---|---|---|---|---|
| Baseline | L=2 | 36.341 | 1.939 | 24.07 | — | 211.456 | 21.27 | 258.735 | 295.076 |
| VisionZip | L=2 | 35.478 | 1.836 | 24.196 | 1.121 | 27.483 | 21.078 | 75.714 | 111.192 |
| PruneSID | L=2 | 36.509 | 2.321 | 23.801 | 17.15 | 27.589 | 20.046 | 90.907 | 127.416 |
| MSDG-Prune (Ours) | L=2 | 37.694 | 2.612 | 23.83 | 22.362 | 27.461 | 20.13 | 96.395 | 134.089 |
| Baseline | L=8 | 37.075 | 1.93 | 23.88 | — | 210.55 | 145 | 381.36 | 418.435 |
| VisionZip | L=8 | 37.327 | 1.921 | 24.295 | 1.181 | 27.497 | 144.742 | 199.636 | 236.963 |
| PruneSID | L=8 | 38.781 | 2.336 | 23.856 | 17.15 | 27.576 | 135.124 | 206.042 | 244.823 |
| MSDG-Prune (Ours) | L=8 | 38.004 | 2.605 | 24.14 | 22.368 | 27.523 | 138.71 | 215.346 | 253.35 |
Compared with the full model, MSDG-Prune reduces prefill latency from 211.46 to 27.46 ms under EOS stopping. Including the 22.36 ms pruning cost, inference time decreases from 258.74 to 96.40 ms, and end-to-end latency decreases from 295.08 to 134.09 ms. With a fixed output length of 8 tokens, autoregressive decoding accounts for a larger share of runtime, and the end-to-end speedup decreases from 2.20× to 1.65×.
Please note that the results for VisionZip and PruneSID in this table were obtained through re-evaluation and differ slightly from those reported in the manuscript, which were taken from the corresponding original papers. We will update the manuscript with these re-evaluated results.
[1] Y. Li, Y. Du, K. Zhou, et al. Evaluating Object Hallucination in Large Vision-Language Models. In EMNLP, 2023.
Dear Reviewer DUqE,
Thank you again for carefully reviewing our paper. As the discussion period closes on November 18, we wanted to kindly follow up on our response.
Your review directly motivated several new experiments. Following your suggestions, we summarized comparisons with PruneSID across all model configurations in our manuscript. We added comparisons with our reimplementation of EvoCut under matched evaluation settings on LLaVA-NeXT-7B and Qwen2.5-VL-7B. We also clarified the definition of “Inference Time” and generation lengths used in the original manuscript and conducted new timing experiments to provided a breakdown of end-to-end latency.
We hope that these results address your concerns. Thank you!
Response to Reviewer DUqE (1/3)
Thank you for reviewing our paper and for your valuable feedback. Below, we address your concerns point by point and we will incorporate the new results and corresponding clarifications into the revised manuscript. We would appreciate it if you could let us know whether your concerns are addressed by our response.
Question 1: Concerns about the generality of performance gains.
We also observed that the average gain on LLaVA-NeXT-7B at 160 tokens is mainly driven by POPE (Table 2), while our method and PruneSID achieve comparable overall performance across the other six benchmarks. The 160-token setting highlights the challenge of maintaining accurate object-presence recognition. On POPE, our method achieves an F1 score of 87.1, outperforming PruneSID (76.9) by 10.2 percentage points and VisionZip (74.8) by 12.3 percentage points. POPE asks whether a named object is present in the image, making the query directly informative about which visual evidence to retain. Our method combines query relevance with visual saliency to prioritize evidence for that object, while pruning within groups of similar tokens to reduce redundancy.
The broader claim is supported by results across additional models and tasks. The Table DUqE-1. summarizes the shared evaluation settings from the main experiments and Appendix, covering six models at three retention ratios. MSDG-Prune achieves higher aggregate RelAcc. than PruneSID in 17 of the 18 settings and matches it in the remaining one. The gains also extend beyond POPE. Across the 18 settings, excluding POPE yields a mean relative score improvement of 0.91% over PruneSID, computed as the arithmetic mean of (MSDG-Prune−PruneSID)/PruneSID over the non-POPE benchmark-level comparisons. At 11.1% retention, our method outperforms PruneSID on five of the six non-POPE benchmarks on Mini-Gemini-7B (Table 3) and four of the six on Qwen2.5-VL-7B (Table 4). These cross-model and non-POPE results support the preservation of useful visual evidence across multiple tasks and architectures.
Table DUqE-1. Benchmark-Level and Aggregate RelAcc. Win/Tie/Loss Counts Excluding LLaVA-NeXT-13B.
Table note. Benchmark-level comparisons count each model–retention–benchmark combination separately. RelAcc. comparisons count each model–retention setting once. Mean relative score improvement is computed as the arithmetic mean of 100×(Smethod−Sbaseline)/Sbaseline across all benchmark-level comparisons, including wins, ties, and losses.