- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TFPrune addresses the problem of visual-token redundancy in vision-language models. It proposes a training-free pruning framework that identifies redundant tokens based on their functional contributions and preserves their information through token aggregation. Experiments are conducted on three VLMs across nine benchmarks, comparing TFPrune with existing pruning methods. Results show that TFPrune maintains strong model performance under aggressive token compression while reducing inference costs.
Critical strengths
- The method considers the downstream effects of token removal rather than relying solely on attention scores or feature similarity.
- The combination of functional selection and information transfer effectively preserves visual information under aggressive compression.
- Experiments across multiple models and benchmarks demonstrate consistent improvements over existing pruning methods.
Critical weaknesses
- The evaluation lacks OCR-focused benchmarks (e.g., TextVQA, ChartQA, OCRBench), leaving the method's effectiveness on text-intensive visual understanding tasks unclear.
- Experiments lack more advanced architecture (e.g., Qwen3-VL, InternVL3.5) and stronger recent baselines (e.g., SparseVLM, VisionZip, CDPruner), limiting the evidence for generalizability and state-of-the-art performance.
- The end-to-end speedup is modest despite substantial token reduction, raising questions about the practical efficiency gains after accounting for pruning overhead.
Questions for authors
- Can the authors verify whether the functional score remains predictive of downstream performance under simultaneous token pruning, beyond the reported single-token intervention analysis?
- Are the default hyperparameter choices and text-probe selection strategy sufficiently justified, given the limited sensitivity analysis and the lack of evidence that these choices are optimal?
- Can the authors provide systematic end-to-end latency comparisons with competing pruning methods under matched token budgets and different compression ratios?
4 comments
Dear Reviewer boDB,
We have uploaded a revised manuscript with changes highlighted in blue. The revision incorporates the OCR-focused evaluation on TextVQA, ChartQA, and OCRBench (Table 18), the expanded sensitivity analysis covering text-probe strategies and additional hyperparameters (Appendix B.6), and the latency comparisons with other compression methods (Tables 13–14).
TFPrune achieves the highest scores among the evaluated compression methods on all three OCR benchmarks. The additional efficiency measurements report both fixed-token and proportional-budget settings, and Appendix C.6 derives the relationship between individual and simultaneous removal responses under fixed attention inputs.
We would appreciate your assessment of this additional evidence. In particular, it would be helpful to know which concerns remain most important to your evaluation, so that we can focus any further clarification before the discussion closes.
Thank you for your time and feedback.
Thank you for the detailed rebuttal and additional experiments. However, several concerns remain: (1) the OCR benchmark results appear inconsistent with those reported in prior work; (2) the Qwen3-VL evaluation lacks comparisons with competing methods; and (3) the end-to-end latency results show no clear advantage over existing methods. Therefore, I will maintain my original score.
Thank you for your re-evaluation and feedback. We clarify your concerns below:
- OCR benchmark: Our TextVQA, ChartQA and OCRBench tests run under lmms-eval. Divergent evaluation frameworks, prompts and hyperparameters may cause mismatches with prior work.
- Qwen3-VL-8B lacks baselines: Existing compression baselines have no public implementation or reported results on Qwen3-VL-8B, ly state this limitation in the revision.so we cannot run them for comparison.
- End-to-end latency: While our latency advantage over baselines is small, TFPrune preserves far better task performance. Many competing methods gain more speed by sacrificing accuracy.
All clarifications are included in our revised paper. Thank you again.
We thank the reviewer for the constructive feedback and recognition of TFPrune’s downstream-aware design and visual information preservation under aggressive compression. Below, we address your concerns with additional experiments and clarifications. We hope this evidence resolves your concerns and you will consider raising your score.
Q1: Can add a benchmark based on OCR?
A1: Thank you for the suggestion. We added TextVQA, ChartQA, and OCRBench experiments on LLaVA-1.5-7B to evaluate text-intensive visual understanding. TFPrune achieves the highest score among all evaluated compression methods on all three benchmarks (below), confirming that it retains useful textual information in such tasks.
Table: OCR-focused benchmark results (LLaVA-1.5-7B).
Q2: Evaluate on more architectures and compare with more baselines (SparseVLM, VisionZip, CDPruner).
A2: Thank you for this suggestion. TFPrune has been evaluated on three VLM architectures. We additionally adapted it to Qwen3VL-8B and evaluated POPE, MME, and GQA at 11.2% token retention (below). Our comparisons also include strong recent baselines (MMTok and AnchorPrune). Their original papers report improvements over SparseVLM, VisionZip, and CDPruner in evaluated settings. TFPrune outperforms them, indirectly demonstrating its competitiveness. If direct comparisons are required, we will add them in the revision.
Table: Qwen3VL-8B performance (11.2% tokens retained).
Q3: Verify whether the functional score predicts downstream performance under joint token pruning, beyond single-token LOO analysis.
A3: Thank you for raising this question. Single-token LOO responses are not generally additive. However, their relationship to joint deletion can be characterized exactly. For a fixed attention head and query, let D be the deleted set and sD=Σi∈Dai its total attention mass. With the original queries, keys, and values fixed, the joint response satisfies ΔD=Σi∈D[(1−ai)/(1−sD)]Δi.
Thus, single-token responses recover the exact local joint-deletion response via set-dependent reweighting. Summing them without this correction incurs an error bounded by ‖ΔD−Σi∈DΔi‖≤[1/(1−sD)]Σi∈D(sD−ai)‖Δi‖≤[sD/(1−sD)]Σi∈D‖Δi‖.
Q4: Could the authors expand the sensitivity analysis and clarify whether the default hyperparameters and text-probe strategy are optimal?
A4: Thanks for the suggestion. We do not claim that the default configuration is globally optimal or optimal for every dataset. Its purpose is to provide a shared operating point across tasks. We agree broader empirical justification is needed, as probe-count sensitivity alone does not establish robustness to probe selection. We expanded the sensitivity study to cover:
On COCO at K=128, the default is within 2.1% of the best observed CIDEr score. Varying α changes CIDEr by at most 0.2% relatively. Probe selection matters more on some tasks (e.g., on POPE, random-P scores 2.0 points below the default last-P strategy). Since text lengths and input distributions vary across datasets, a shared configuration may not maximize all metrics; we will provide extended sensitivity results in the revision.
Q5: Provide systematic end-to-end latency comparisons with competing methods under matched token budgets and different compression ratios.
A5: Thanks for your suggestion. We added end-to-end measurements on LLaVA-NeXT-7B and Qwen2.5-VL-7B (* DivPrune excludes its ~10s PyWSD warm-up). These results further show the acceleration effect of TFPrune.
Table: Efficiency on LLaVA-NeXT-7B at 94.4% compression ratios.
Table: Efficiency on Qwen2.5-VL-7B at 22.3% visual-token retention.
Table: Comparison of efficiency under different numbers of visual tokens