acceptodds
Under review as a conference paper at ICLR 2027

TFPrune: Pruning Visual Redundancy while Preserving Token Functions for Efficient VLMs

Abstract

Large vision-language models (LVLMs) achieve strong multimodal reasoning by converting visual inputs into long token sequences processed by every subsequent LLM layer. High-resolution inputs further increase sequence length despite spatial redundancy, resulting in considerable inference costs. To reduce this overhead, numerous pruning methods have been proposed, many of which estimate redundancy from attention saliency or feature similarity. However, these scores do not assess whether the downstream effects of removed tokens can be reproduced by the retained set. To address this limitation, we propose TFPrune, a training-free, plug-and-play framework based on future functional substitutability. Specifically, TFPrune introduces an instruction-conditioned functional score that uses an exact softmax leave-one-out identity to quantify the counterfactual effect of removing each visual token on subsequent attention outputs without requiring an additional decoder-block forward pass. It selects a subset that minimizes the maximum functional discrepancy between each unselected token and its nearest retained representative, thereby preserving evidence that is difficult to substitute. Rather than simply discarding unselected tokens, TFPrune transfers their features to functionally matched representatives and uses accumulated transfer weights to correct post-compression attention. Experiments on LLaVA-1.5-7B, LLaVA-NeXT-7B, and Qwen2.5-VL-7B across nine benchmarks and multiple compression ratios demonstrate its effectiveness. Applied to LLaVA-1.5-7B, TFPrune retains only 11.1% of visual tokens while preserving 98.3% of full-model performance, achieving a favorable trade-off between compression and accuracy.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Token Compression, Vision Language Models, Functional Substitutability, Training-Free Compression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_boDBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TFPrune addresses the problem of visual-token redundancy in vision-language models. It proposes a training-free pruning framework that identifies redundant tokens based on their functional contributions and preserves their information through token aggregation. Experiments are conducted on three VLMs across nine benchmarks, comparing TFPrune with existing pruning methods. Results show that TFPrune maintains strong model performance under aggressive token compression while reducing inference costs.

Critical strengths
  1. The method considers the downstream effects of token removal rather than relying solely on attention scores or feature similarity.
  2. The combination of functional selection and information transfer effectively preserves visual information under aggressive compression.
  3. Experiments across multiple models and benchmarks demonstrate consistent improvements over existing pruning methods.
Critical weaknesses
  1. The evaluation lacks OCR-focused benchmarks (e.g., TextVQA, ChartQA, OCRBench), leaving the method's effectiveness on text-intensive visual understanding tasks unclear.
  2. Experiments lack more advanced architecture (e.g., Qwen3-VL, InternVL3.5) and stronger recent baselines (e.g., SparseVLM, VisionZip, CDPruner), limiting the evidence for generalizability and state-of-the-art performance.
  3. The end-to-end speedup is modest despite substantial token reduction, raising questions about the practical efficiency gains after accounting for pruning overhead.
Questions for authors
  1. Can the authors verify whether the functional score remains predictive of downstream performance under simultaneous token pruning, beyond the reported single-token intervention analysis?
  2. Are the default hyperparameter choices and text-probe selection strategy sufficiently justified, given the limited sensitivity analysis and the lack of evidence that these choices are optimal?
  3. Can the authors provide systematic end-to-end latency comparisons with competing pruning methods under matched token budgets and different compression ratios?
4 comments

We thank the reviewer for the constructive feedback and recognition of TFPrune’s downstream-aware design and visual information preservation under aggressive compression. Below, we address your concerns with additional experiments and clarifications. We hope this evidence resolves your concerns and you will consider raising your score.

Q1: Can add a benchmark based on OCR?

A1: Thank you for the suggestion. We added TextVQA, ChartQA, and OCRBench experiments on LLaVA-1.5-7B to evaluate text-intensive visual understanding. TFPrune achieves the highest score among all evaluated compression methods on all three benchmarks (below), confirming that it retains useful textual information in such tasks.

Table: OCR-focused benchmark results (LLaVA-1.5-7B).

MethodRetained tokensTextVQAChartQAOCRBench
Full model57646.0218.2031.50
FastV12842.2017.1629.60
DivPrune12842.5317.4029.20
MMTok12844.3117.2830.40
TFPrune12845.1017.8030.90

Q2: Evaluate on more architectures and compare with more baselines (SparseVLM, VisionZip, CDPruner).

A2: Thank you for this suggestion. TFPrune has been evaluated on three VLM architectures. We additionally adapted it to Qwen3VL-8B and evaluated POPE, MME, and GQA at 11.2% token retention (below). Our comparisons also include strong recent baselines (MMTok and AnchorPrune). Their original papers report improvements over SparseVLM, VisionZip, and CDPruner in evaluated settings. TFPrune outperforms them, indirectly demonstrating its competitiveness. If direct comparisons are required, we will add them in the revision.

Table: Qwen3VL-8B performance (11.2% tokens retained).

ModelPOPEMMEGQA
Full88.32174561.58
TFPrune83.15158455.14

Q3: Verify whether the functional score predicts downstream performance under joint token pruning, beyond single-token LOO analysis.

A3: Thank you for raising this question. Single-token LOO responses are not generally additive. However, their relationship to joint deletion can be characterized exactly. For a fixed attention head and query, let DD be the deleted set and sD=Σi∈Dais_D = Σ_{i∈D} a_i its total attention mass. With the original queries, keys, and values fixed, the joint response satisfies ΔD=Σi∈D[(1−ai)/(1−sD)]ΔiΔ_D = Σ_{i∈D} [(1−a_i)/(1−s_D)] Δ_i.

Thus, single-token responses recover the exact local joint-deletion response via set-dependent reweighting. Summing them without this correction incurs an error bounded by ‖ΔD−Σi∈DΔi‖≤[1/(1−sD)]Σi∈D(sD−ai)‖Δi‖≤[sD/(1−sD)]Σi∈D‖Δi‖‖Δ_D − Σ_{i∈D} Δ_i‖ ≤ [1/(1−s_D)] Σ_{i∈D}(s_D−a_i)‖Δ_i‖ ≤ [s_D/(1−s_D)] Σ_{i∈D}‖Δ_i‖.

Q4: Could the authors expand the sensitivity analysis and clarify whether the default hyperparameters and text-probe strategy are optimal?

A4: Thanks for the suggestion. We do not claim that the default configuration is globally optimal or optimal for every dataset. Its purpose is to provide a shared operating point across tasks. We agree broader empirical justification is needed, as probe-count sensitivity alone does not establish robustness to probe selection. We expanded the sensitivity study to cover:

  • α ∈ {4, 8, 16},
  • numbers of heads ∈ {4, 8, all},
  • response dimensions ∈ {32, 64, 128},
  • probe strategies: last-P, uniform-P, random-P, all-post-image, all-text, and pre-image,
  • probe counts ∈ {1, 8, 16, 32, 64},
  • random seeds ∈ {17, 42, 123}.

On COCO at K=128, the default is within 2.1% of the best observed CIDEr score. Varying α changes CIDEr by at most 0.2% relatively. Probe selection matters more on some tasks (e.g., on POPE, random-P scores 2.0 points below the default last-P strategy). Since text lengths and input distributions vary across datasets, a shared configuration may not maximize all metrics; we will provide extended sensitivity results in the revision.

Q5: Provide systematic end-to-end latency comparisons with competing methods under matched token budgets and different compression ratios.

A5: Thanks for your suggestion. We added end-to-end measurements on LLaVA-NeXT-7B and Qwen2.5-VL-7B (* DivPrune excludes its ~10s PyWSD warm-up). These results further show the acceleration effect of TFPrune.

Table: Efficiency on LLaVA-NeXT-7B at 94.4% compression ratios.

MethodEnd-to-end latency (ms)Speedup
Full model1572.91.00×
FastV1357.41.16×
DivPrune*829.31.90×
MMTok1061.21.48×
TFPrune858.21.83×

Table: Efficiency on Qwen2.5-VL-7B at 22.3% visual-token retention.

MethodEnd-to-end latency (ms)Speedup
Full model2123.21.00×
DivPrune*1681.01.26×
MMTok1728.81.23×
TFPrune1756.91.21×

Table: Comparison of efficiency under different numbers of visual tokens

ModelRetained tokensEnd-to-end latency (ms)Speedup over full
LLaVA-NeXT-7B320970.91.62×
LLaVA-NeXT-7B160858.21.83×
Qwen2.5-VL-7B1281704.01.25×
Qwen2.5-VL-7B641582.41.34×

Dear Reviewer boDB,

We have uploaded a revised manuscript with changes highlighted in blue. The revision incorporates the OCR-focused evaluation on TextVQA, ChartQA, and OCRBench (Table 18), the expanded sensitivity analysis covering text-probe strategies and additional hyperparameters (Appendix B.6), and the latency comparisons with other compression methods (Tables 13–14).

TFPrune achieves the highest scores among the evaluated compression methods on all three OCR benchmarks. The additional efficiency measurements report both fixed-token and proportional-budget settings, and Appendix C.6 derives the relationship between individual and simultaneous removal responses under fixed attention inputs.

We would appreciate your assessment of this additional evidence. In particular, it would be helpful to know which concerns remain most important to your evaluation, so that we can focus any further clarification before the discussion closes.

Thank you for your time and feedback.

Thank you for the detailed rebuttal and additional experiments. However, several concerns remain: (1) the OCR benchmark results appear inconsistent with those reported in prior work; (2) the Qwen3-VL evaluation lacks comparisons with competing methods; and (3) the end-to-end latency results show no clear advantage over existing methods. Therefore, I will maintain my original score.

Thank you for your re-evaluation and feedback. We clarify your concerns below:

  • OCR benchmark: Our TextVQA, ChartQA and OCRBench tests run under lmms-eval. Divergent evaluation frameworks, prompts and hyperparameters may cause mismatches with prior work.
  • Qwen3-VL-8B lacks baselines: Existing compression baselines have no public implementation or reported results on Qwen3-VL-8B, ly state this limitation in the revision.so we cannot run them for comparison.
  • End-to-end latency: While our latency advantage over baselines is small, TFPrune preserves far better task performance. Many competing methods gain more speed by sacrificing accuracy.

All clarifications are included in our revised paper. Thank you again.

Reviewer_sfaZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TFPrune, a training-free visual token compression framework for large vision-language models. The central idea is to measure visual token redundancy through functional substitutability, rather than relying solely on attention saliency or feature similarity. Specifically, the method constructs instruction-conditioned functional representations using a softmax leave-one-out identity to estimate how removing each visual token affects the next decoder block's attention outputs at selected text positions. These representations are combined with auxiliary value information and used for minimax representative selection via farthest-first traversal. Unselected tokens are subsequently aggregated into their representatives using functionally fitted transfer weights, while accumulated weights are used to calibrate downstream attention. The method is evaluated on three VLM architectures and nine benchmarks. At 11.1% visual token retention, TFPrune preserves 98.3% of the full-model normalized average on LLaVA-1.5-7B and reports a 1.268× end-to-end speedup on COCO captioning. Additional experiments, ablations, and an intervention-based correlation analysis support the proposed approach.

Critical strengths
  1. Strong empirical performance across models and compression budgets. The paper demonstrates consistently competitive performance across three VLMs and nine benchmarks, particularly under aggressive compression. On LLaVA-1.5-7B, TFPrune preserves 98.3% of full-model performance using only 64 of 576 visual tokens, outperforming the strongest reported baseline by 4.7 percentage points in normalized average performance. Similar improvements are observed on LLaVA-NeXT and Qwen2.5-VL. These results support the practical utility and architectural applicability of the approach.
  2. A coherent formulation connecting token selection and information preservation. The proposed functional representation is used for both representative selection and feature transport, providing a conceptually unified compression framework. The exact single-token removal identity avoids performing a separate decoder-block forward pass for each candidate deletion. The ablations support the importance of transport and attention calibration, while the intervention analysis in Appendix B.1 provides additional evidence that functional-response dissimilarity is associated with downstream hidden-state changes.
Critical weaknesses
  1. The central novelty is the use of leave-one-out attention responses to characterize functional substitutability. However, Table 4 does not include an ablation that removes the counterfactual response while retaining the auxiliary value descriptor, minimax selection, functional transport, and attention calibration.
  2. The principal end-to-end efficiency results are reported only for LLaVA-1.5-7B. However, TFPrune incurs additional costs for response construction, projection, farthest-first selection, feature aggregation, and KV-cache updates. These costs may become particularly significant for the longer visual sequences of LLaVA-NeXT and the dynamic-resolution inputs of Qwen2.5-VL.
  3. While Table 4 demonstrates the benefit of functional transport, it does not isolate the contribution of the proposed functionally fitted transfer weights. In particular, it remains unclear whether the improvement primarily comes from aggregating discarded token features or from the specific weighting strategy in Eq. (4). A comparison with simple uniform-weight aggregation under the same token assignments would help establish the necessity of the proposed weighting mechanism.
Questions for authors
  1. Could the authors report an ablation that uses only the auxiliary value representation (without the leave-one-out response), while keeping minimax selection, functional transport, and attention calibration unchanged? Additionally, how does the proposed metric compare with attention-based or hidden-feature-based selection when all methods use the same transport mechanism?
  2. Could the authors provide end-to-end latency, pruning overhead on LLaVA-NeXT-7B and Qwen2.5-VL-7B, especially under the most aggressive compression settings?
  3. For the FastV comparison, could the authors clarify the exact compression boundaries and report whether the relative performance and efficiency advantages persist under matched pruning depths?
  4. Could the authors compare the proposed functionally fitted transfer weights with uniform aggregation weights while keeping the selected tokens and assignment groups unchanged? This would help clarify whether the proposed weighting strategy provides additional benefits beyond feature aggregation itself.
2 comments

We would like to thank the reviewer for the valuable time and insightful comments. Our detailed responses are below.

Q1: How does TFPrune perform with value-only representations and other components unchanged, or with attention- or feature-based selection under the same transport mechanism?

A1: Thanks for your suggestion. We added the experiments on LLaVA-1.5-7B with 64 retained visual tokens. The auxiliary value representation removes the LOO response while keeping the minimax selection, transport, and attention-calibration procedures. The table results show that the full TFPrune method using LOO performs better across the four tasks. To compare alternative metrics, we replace the functional metric with hidden-feature similarity or attention-based values while retaining the same compression and transport framework. Random selection chooses representatives without these scores or LOO. Full TFPrune achieves the highest score on all four tasks. We will add these ablation experiments to the revision.

Table: Ablation of the functional representation and token-selection strategy on LLaVA-1.5-7B with 64 retained visual tokens.

MethodCOCOGQAPOPESQA
Auxiliary value representation108.7459.6883.2068.36
Hidden-feature similarity103.6758.5183.6466.41
Attention-based metric100.3554.0379.0365.35
Random selection106.7458.3882.3766.36
Functional representation (TFPrune)109.5659.8384.2968.52

Q2: Could the authors provide end-to-end latency, pruning overhead on LLaVA-NeXT-7B and Qwen2.5-VL-7B, especially under the most aggressive compression settings?

A2: Thanks for your suggestion. We added experiments on LLaVA-NeXT-7B and Qwen2.5-VL-7B (below). For LLaVA-NeXT-7B, under the most aggressive test budget of 160 tokens, TFPrune reduces end-to-end latency by 45.4%, showing significant acceleration for longer visual sequences. For Qwen2.5-VL-7B, retaining 22.3% of the visual tokens reduces end-to-end latency from 2123.2 to 1756.9 ms/sample, corresponding to a 1.21× speedup. We also provide aggressive fixed-budget measurements. TFPrune achieves net acceleration over the full model on both additional architectures, including all compression operations.

Table: Efficiency on LLaVA-NeXT-7B.

ModelEnd-to-end latency (ms)Speedup
Full model1572.91.00×
TFPrune (320 tokens)970.91.62×
TFPrune (160 tokens)858.21.83×

Table: Efficiency on Qwen2.5-VL-7B.

ModelEnd-to-end latency (ms)Speedup
Full model2123.21.00×
TFPrune (128 tokens)1704.01.25×
TFPrune (64 tokens)1582.41.34×

Q3: For the FastV comparison, could the authors clarify the exact compression boundaries and report whether the relative performance and efficiency advantages persist under matched pruning depths?

A3: Thank you for requesting this clarification. FastV and TFPrune use the same compression boundary. They both compress immediately after the first decoder block, and the second decoder block first receives the shortened sequence. Thus, the reported comparisons between FastV and TFPrune already use matched pruning depths.

Q4: Could the authors compare the proposed functionally fitted transfer weights with uniform aggregation weights while keeping the selected tokens and assignment groups unchanged?

A4: Thank you for the suggestion. We added a uniform-weight baseline on LLaVA-1.5-7B with 64 retained visual tokens. Uniform aggregation sets all member weights to 1, yielding a simple group mean and an attention mass equal to the group size. Results show that the proposed weighting scheme provides additional benefits beyond uniform aggregation of discarded tokens. We will add these ablation experiments to the revision.

Table: Ablation on aggregation weights and transport design (LLaVA-1.5-7B, 64 tokens).

MethodCOCOGQAPOPESQA
Full109.5659.8384.2968.52
Uniform aggregation weights107.4358.2683.0666.33
w/o attention bias106.7359.1280.1567.46
w/o functional transport103.1658.7377.6165.72

Dear Reviewer sfaZ,

Thank you again for your constructive suggestions. We have uploaded a revised manuscript with changes highlighted in blue.

Appendix B.9 now includes the requested value-only and alternative-selection ablations (Table 16), as well as the uniform-weight comparison with the selected tokens and assignment groups held fixed (Table 17). The full functional representation improves all four evaluated task scores over the value-only variant, and fitted transfer weights improve all four scores over uniform aggregation.

We have also added TTFT and end-to-end measurements on LLaVA-NeXT-7B and Qwen2.5-VL-7B in Appendix B.8 and expanded the ablation configuration details in Appendix C.9.

We would appreciate your feedback on whether these additions clarify the contributions of the response representation and fitted transfer weights. Please let us know if any aspect of the experimental controls or efficiency evaluation would benefit from further clarification.

Reviewer_7EmGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TFPrune, a training-free visual token compression framework for large vision-language models (LVLMs) grounded in future functional substitutability. Operating at an early decoder boundary (after the first decoder block by default), TFPrune proceeds in three steps: (1) Functional Response Construction, which uses an exact softmax leave-one-out (LOO) identity in a Gaussian-projected value-output space to compute the counterfactual effect of removing each visual token on the subsequent block's attention outputs at selected text positions (fif_i), concatenated with an unweighted projected value descriptor (pip_i) to form a functional vector ϕi\phi_i; (2) Functional Cover Selection, which projects ϕi\phi_i via a Rademacher matrix and applies Gonzalez's farthest-first traversal to minimize the worst-case directional covering radius (kk-center objective); and (3) Functional Transport and Attention Calibration, which fits nonnegative least-squares scalar weights αi\alpha_i between unselected tokens and their nearest retained representatives to perform weighted feature/KV-cache aggregation, followed by log-mass proportional attention calibration (log⁡mj\log m_j) in subsequent layers. Experiments on LLaVA-1.5-7B, LLaVA-NeXT-7B, and Qwen2.5-VL-7B across nine VQA and captioning benchmarks evaluate its accuracy retention and KV-cache reduction.

Critical strengths
  1. Clean Leave-One-Out Formulation & Unified Selection-Transport Pipeline: Leveraging the exact softmax renormalization identity Δtih=atih1−atih(uih−yth)\Delta_{ti}^h = \frac{a_{ti}^h}{1 - a_{ti}^h}(u_i^h - y_t^h) in a low-dimensional projected value space cleverly captures both attention weight and value distinctiveness without requiring NN separate forward passes. Connecting kk-center directional coverage with nonnegative least-squares transport weights (Proposition 2) provides a cohesive mathematical pipeline for joint pruning and merging.
  2. Strong Accuracy Retention Across Architectures and Aggressive Budgets: Across LLaVA-1.5-7B, high-resolution LLaVA-NeXT-7B (up to 2,880 tokens), and dynamic-resolution Qwen2.5-VL-7B on nine benchmarks, TFPrune consistently achieves higher accuracy retention than drop-only pruning baselines, preserving 98.3% on LLaVA-1.5 at 11.1% retention (K=64K=64) and 93.7% on LLaVA-NeXT at 5.6% retention (K=160K=160).
Critical weaknesses
  1. Negligible (or Negative) Wall-Clock Speedups Undermine the Core Purpose of Token Compression:

    • The fundamental goal of visual token compression is inference acceleration, yet TFPrune's heavy overhead—computing explicit attention across query heads, materializing O(HsPNvr)\mathcal{O}(H_s P N_v r) difference tensors, and executing O(N2s+NK)\mathcal{O}(N^2 s + NK) kk-center traversal and transport fitting—largely cancels out its sequence-reduction gains. In Table 5 (LLaVA-1.5-7B), removing 66.7% of tokens (K=192K=192) incurs 129.82 ms of pruning overhead and yields a negligible 1.023x end-to-end speedup (896.67 ms vs. 917.60 ms native); even removing 88.9% of tokens (K=64K=64) yields only a 1.268x speedup.
    • Worse still, in Table 11 (K=64K=64, 88.9% tokens removed), TFPrune is 11.2% slower than the uncompressed native model on MMBench (2.475 s vs. 2.225 s) and achieves virtually zero speedup (only 1.4%–3.0% faster) on POPE, COCO, MME, and GQA. If pruning nearly 90% of visual tokens fails to deliver meaningful wall-clock acceleration, the practical utility of the compression mechanism is highly questionable. Moreover, the paper omits latency/TTFT evaluations on high-resolution LLaVA-NeXT (2,880 tokens) and Qwen2.5-VL, as well as latency comparisons against recent baselines (DivPrune, MMTok, ZOO-Prune, HiPrune).
  2. Questionable Compatibility with FlashAttention:

    • The paper never mentions FlashAttention. First, constructing the LOO functional response (Eqs. 8 and 12) requires explicitly materializing attention probabilities atjha_{tj}^h and O(HsPNvr)\mathcal{O}(H_s P N_v r) value-difference tensors. Second, injecting dynamic token-specific log-mass biases (log⁡mj\log m_j, Eq. 27) into the attention logits of all subsequent decoder layers conflicts with standard FlashAttention-2 kernels, which do not support arbitrary per-key additive bias vectors and typically force a fallback to slower attention implementations.
  3. Disconnect Between Cross-Layer Motivation and Shallow Single-Token Approximation:

    • While Section 1 motivates TFPrune via the attention shift between shallow (Layers 2–4) and deep layers (Layers 16–24), TFPrune actually prunes after Decoder Block 1 (OBSERVE_LAYERS=1) using only Block 2's attention operators, which remain too shallow to foresee deep cross-modal fusion. Furthermore, the exact LOO identity holds only for removing a single token while fixing the remaining N−1N-1 tokens; under simultaneous removal of 88.9%–94.4% of tokens, the softmax denominator shifts drastically, invalidating the single-token LOO approximation.
Questions for authors
  1. FlashAttention Compatibility: Is TFPrune compatible with FlashAttention-2? Specifically, does injecting the dynamic additive bias log⁡mj\log m_j (Eq. 27) into every subsequent decoder layer force a fallback to slower non-FlashAttention kernels, and was FlashAttention enabled in Tables 5 and 11?
  2. Practical Acceleration Value: Since visual token compression aims to accelerate inference, how do the authors justify the practical utility of TFPrune when removing 66.7% of tokens yields only a 1.023x speedup (Table 5) and removing 88.9% of tokens yields only 1.4%–3.0% speedup on GQA/POPE/MME/COCO and an 11.2% slowdown on MMBench (Table 11)? Please also report prefill (TTFT) and end-to-end latency on LLaVA-NeXT (2,880 tokens) and Qwen2.5-VL.
2 comments

We sincerely appreciate the reviewer’s detailed, constructive feedback. We are pleased you noticed our LOO formulation and unified selection-transport pipeline, which preserves strong accuracy across diverse architectures and compression budgets. We believe our responses address your concerns and respectfully ask you to reconsider the rating given our efforts and contributions. Below are our detailed replies:

Q1: Is TFPrune compatible with FlashAttention-2?

A1: Thanks for pointing this out. In the table of the paper, FlashAttention-2 was disabled. However, TFPrune can be adapted to it. Specifically, the function response construction requires attention probabilities only for the selected text probe at one compression boundary, not the full attention matrix. For ctih=atih/(1−atih)c_ti^h=a_ti^h/(1-a_ti^h), the head summation response can be expressed as fti=Σhctihuih−Σhctihythf_ti=Σ_hc_ti^hu_i^h-Σ_hc_ti^hy_t^h. This avoids the need for explicitly calculating the [Hs,P,Nv,r][H_s,P,N_v,r] difference tensor. As the reviewer notes, the additive bias log⁡mj\log m_j needs specific treatment. Let pj=softmax(b)jp_j=softmax(b)_j for the original attention logits, with the original causal and padding semantics preserved. Define B=Σjpjmjvj,ρ=ΣjpjmjB=Σ_jp_jm_jv_j,ρ=Σ_jp_jm_j. Then B/ρ=Σjsoftmax(b+log⁡m)jvjB/ρ=Σ_jsoftmax(b+\log m)_jv_j. Thus, the calibrated output can be computed using augmented values [mjvj,mj][m_jv_j,m_j], followed by per-head division before the output projection.

We evaluated this mode on Qwen2.5-VL-7B (K=64, 3 repeats). The FlashAttention path runs without an explicit mass-bias mask and reduces latency by ~6.0% relative to the original path, showing that TFPrune adapts to FlashAttention-2 with superior speed.

Table: Efficiency on Qwen2.5-VL.

ModeCIDErEnd-to-end time (ms)
SDPA38.811263.4
SDPA with TFPrune37.871167.9
FlashAttention-239.041203.4
FlashAttention-2 with TFPrune37.571128.4

Q2: TFPrune's speedups seem modest with occasional slowdowns. Please report TTFT and end-to-end latency on LLaVA-NeXT and Qwen2.5-VL.

A2: Thanks for pointing this out. We found original timing results in Table 11 were disturbed by sudden and unrelated GPU processes. We apologize for this measurement error and reran these experiments. Updated results show consistent positive end-to-end speedups, and the previously observed MMBench performance drop disappears. The corrected results are shown below. We will replace Table 11 accordingly.

DatasetNative (K=576)FastVTFPrune
mmbench_en_dev2.344 ± 0.0422.354 ± 0.0942.253 ± 0.038

We also extended the efficiency evaluation to LLaVA-NeXT-7B and Qwen2.5-VL-7B (below). TFPrune achieves 1.62×–1.83× end-to-end speedup on LLaVA-NeXT, demonstrating a larger net benefit with longer visual sequences. On Qwen2.5-VL, it achieves 1.25×–1.34× speedup at the tested budgets. These results further demonstrate the practical utility of TFPrune.

Table: Efficiency on LLaVA-NeXT-7B and Qwen2.5-VL-7B.

ModelTTFT (ms)End-to-end (ms)Speedup
LLaVA-NeXT-7B, full797.81572.91.00×
LLaVA-NeXT-7B, 320297.8970.91.62×
LLaVA-NeXT-7B, 160214.4858.21.83×
Qwen2.5-VL-7B, full314.12123.21.00×
Qwen2.5-VL-7B, 128131.81704.01.25×
Qwen2.5-VL-7B, 6496.81582.41.34×

We acknowledge that in Table 5, at large token budgets the acceleration is less pronounced (1.023× at K=192). However, this also reflects the trade-off between actual performance and efficiency. Therefore, an appropriate token budget should be chosen based on actual acceleration and performance requirements.

Q3: How can shallow single-token LOO responses capture deeper cross-modal effects and remain informative under aggressive simultaneous token pruning?

A3: Thanks for pointing this out. Our aim is to identify useful visual evidence at an early compression boundary, while leaving more subsequent computation available for acceleration. Cross-layer attention shifts motivate us to go beyond shallow attention magnitude. Tokens with low early attention may still be critical downstream. TFPrune uses instruction-conditioned attention responses, which encode both attention weights and value distinctiveness. These features guide which tokens to retain and how discarded information can be represented by the retained tokens. The correlation analysis in Table 7 provides empirical support for this proxy under our intervention setting.

Single-token LOO responses are not generally additive. However, their relationship to joint deletion can be characterized exactly. For a fixed attention head and query, let DD be the deleted set and sD=Σi∈Dais_D=Σ_{i∈D}a_i its total attention mass. With the original queries, keys, and values fixed, the joint response satisfies ΔD=Σi∈D[(1−ai)/(1−sD)]ΔiΔ_D=Σ_{i∈D}[(1−a_i)/(1−s_D)]Δ_i.

Thus, single-token responses recover the exact local joint-deletion response via set-dependent reweighting. Summing them without this correction incurs an error bounded by ‖ΔD−Σi∈DΔi‖≤[1/(1−sD)]Σi∈D(sD−ai)‖Δi‖≤[sD/(1−sD)]Σi∈D‖Δi‖‖Δ_D−Σ_{i∈D} Δ_i‖≤[1/(1−s_D)]Σ_{i∈D}(s_D−a_i)‖Δ_i‖≤[s_D/(1−s_D)]Σ_{i∈D}‖Δ_i‖.

Dear Reviewer 7EmG,

We have uploaded a revised manuscript with changes highlighted in blue. The revision incorporates the additional evidence discussed in our response:

  • TTFT and end-to-end latency on LLaVA-NeXT-7B and Qwen2.5-VL-7B, with end-to-end speedups of up to 1.83× and 1.34×, respectively (Figure 4 and Table 12).
  • A FlashAttention-2-compatible calibration construction and measured backend comparison (Appendix C.7 and Table 15).
  • The exact relationship between individual and simultaneous removal responses for a fixed attention operation, together with the associated approximation bound (Appendix C.6).

We would appreciate your assessment of these additions. We would be glad to clarify any outstanding technical points before the discussion closes.

Thank you for your time.

Comments on the paper

2 comments
Area_Chair_6zAaOpenReview ↗

ICLR 2027 Significance Reminder

Dear Reviewers,

Please review the authors' rebuttal to your comments and the other reviews and to respond.

Please remember to update your ratings and final justification if your concerns have been addressed. If ratings are not updated, clearly explain any remaining concerns in your final justification.

As a reminder, the discussion period ends on November 18th, 11:59 PM AoE.

Best, Your AC

Dear Area Chair,

TFPrune is a training-free framework for visual-token compression in VLMs. Its central contribution is a functional substitutability criterion. It removes tokens whose replacement leaves the model response largely unchanged, instead of removing tokens only because their features are similar. Across three VLMs and nine benchmarks, TFPrune improves inference efficiency while maintaining strong performance and outperforming recent baselines. We thank the reviewers for their thoughtful feedback and the Area Chair for coordinating the discussion. Below, we summarize the experiments and clarifications added to the uploaded revision in response to reviewer questions.

  1. Efficiency and baselines (Reviewers 7EmG, sfaZ, boDB). Tables 12 to 14 add measurements of time to first token and end-to-end latency including compression overhead on LLaVA-NeXT-7B and Qwen2.5-VL-7B, with speedups of up to 1.83× and 1.34× respectively. Appendix A.3 clarifies that FastV and TFPrune both compress after the first decoder block. We also corrected the MMBench timing result (Table 11).

  2. FlashAttention-2 compatibility (Reviewer 7EmG). Appendix C.7 describes an augmented-value construction for implementing attention calibration with FlashAttention-2. Table 15 reports that TFPrune has lower latency with FlashAttention-2 than with SDPA.

  3. Removal response and fitted weights (Reviewer sfaZ). New ablations (Table 16) compare the full functional representation with value-only, feature-based, attention-based, and random selection. Table 17 evaluates uniform aggregation with the selected tokens and assignment groups held fixed. The full configuration achieves higher scores across all four evaluated tasks in both comparisons.

  4. Shallow responses and simultaneous removal (Reviewers 7EmG, boDB). In the single token intervention study, functional response dissimilarity correlates more strongly than feature dissimilarity with the hidden state change at block 24 on all nine datasets (Table 7). For simultaneous removal, Appendix C.6 derives the exact set-dependent relationship between individual and joint removal responses for fixed attention inputs and provides a bound on the error of their unweighted sum.

  5. Text focused tasks, architectures, and sensitivity (Reviewer boDB). Table 18 adds TextVQA, ChartQA, and OCRBench. Appendix B.6 expands sensitivity analysis to six factors. Our reply to Reviewer boDB also reports Qwen3-VL-8B with low visual-token retention.

During the discussion period, we have not received follow-up comments from Reviewers 7EmG and sfaZ. Reviewer boDB shared additional feedback, with remaining concerns on OCR benchmark inconsistencies (potentially arising from evaluation framework differences), unavailable public baseline implementations for Qwen3-VL-8B, and modest end-to-end latency advantages. We have posted point-by-point clarifications in the corresponding review thread and supplemented discussions on evaluation variance and accuracy–latency tradeoffs in the blue-marked revised manuscript. We hope this summary helps the Area Chair and reviewers locate the supporting evidence.

Thank you for your time and consideration.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.