acceptodds
Under review as a conference paper at ICLR 2027

Language-Aware Efficient Adaptive Filtering for Visual Token Selection via Truncated Probing and Hierarchies

Abstract

Large vision-language models (VLMs) process hundreds or thousands of visual tokens, many of which are redundant for a given question. Existing training-free reduction methods often apply uniform transformations or rank patches independently using a single importance signal, making it difficult to preserve both spatial coverage and task-relevant detail under aggressive compression. We introduce LEAF, a training-free framework that jointly determines _how_ the image should be partitioned, _which_ patches are relevant, and _how much detail_ each region requires. LEAF combines a saliency-guided adaptive hierarchy with question-aware relevance extracted through truncated language-model probing, then allocates the token budget using a coverage-detail objective that balances regional coverage, patch relevance, and feature heterogeneity. Across four frozen VLMs and nine benchmarks, LEAF achieves the highest normalized average performance at all evaluated model-budget settings. On Qwen2.5-VL, it preserves 96.08% of full-token performance at 10% retention, compared with 90.76% for the strongest competing method, and achieves a BD-Token of -75.94% relative to random retention across the evaluated budget range, demonstrating effective visual-token reduction without training or backbone modification.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Token Pruning, Vision-Language Models, Efficient Multimodal Inference, Adaptive Visual Hierarchy

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_23ftOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LEAF, a training-free visual token selection method that combines adaptive spatial partitioning with a language-conditioned token selection mechanism. The method utilizes an auxiliary visual encoder to construct a saliency-guided hierarchical structure while extracting query-relevant scores through the forward pass of a truncated Large Language Model (LLM). Subsequently, a hierarchy-aware selector allocates the token budget based on spatial coverage, task relevance, and local feature heterogeneity. Experiments across various Vision-Language Models (VLMs) and visual understanding benchmarks demonstrate highly competitive performance, particularly in high-compression scenarios.

Critical strengths
  • Strong empirical performance under aggressive compression. LEAF achieves the highest normalized average performance among the evaluated methods across four frozen VLMs and the tested token budgets. On Qwen2.5-VL at 10% token retention, it preserves 96.08% of full-token performance, compared with 90.76% for the strongest evaluated baseline. These results provide substantive evidence that the method better preserves task performance under stringent visual-token budgets.

  • Comprehensive and coherent framework design. LEAF integrates saliency-guided spatial partitioning, question-aware relevance probing, and hierarchy-aware budget allocation into a unified training-free pipeline. These components address three complementary decisions: how to partition the image, which visual evidence is relevant, and how much detail each region requires. This coordinated design provides a systematic framework for balancing regional coverage and task-relevant detail under constrained token budgets.

Critical weaknesses
  • Insufficient justification for adaptive spatial partitioning. LEAF uses regional average saliency to determine whether further partitioning is needed, although high saliency does not necessarily indicate greater within-region complexity. Comparisons among QTMT split types do not isolate the benefit of adaptive partitioning, as simpler alternatives, such as fixed uniform partitioning with comparable granularity, are not evaluated.

  • Weak baselines for hierarchical budget allocation. The w/o S3 ablation exhausts higher-ranked leaves before considering others, encouraging excessive budget concentration. Its substantial degradation therefore does not convincingly establish the superiority of the proposed allocation mechanism over competitive alternatives, such as global relevance-based top-k selection, uniform regional allocation, or relevance-weighted regional budgets.

  • Unclear practical efficiency and unverified serving integration. LEAF adds an auxiliary DeiT-S encoder and nine full-sequence decoder blocks before rerunning inference on the compressed sequence, resulting in higher FLOPs than competing methods at matched token budgets. Its probing requires explicit Q/K replay beyond standard FlashAttention execution, although the backbone can still use fused attention kernels. Integration with vLLM/SGLang, including scheduling and KV-cache handling, is not demonstrated.

  • Insufficient validation of spatial coverage. LEAF measures regional coverage through feature similarity, potentially treating visually similar but spatially distinct regions as substitutes. The coverage objective does not explicitly account for their different locations or instance identities. Without targeted quantitative analysis of repeated objects and spatial relationships, it remains unclear whether this mechanism reliably preserves the fine-grained evidence required for counting and spatial reasoning.

Questions for authors
  1. Adaptive spatial partitioning. What justifies using regional average saliency to determine whether further subdivision is needed, given that saliency does not necessarily reflect within-region complexity? Can you compare QTMT with fixed uniform partitioning?

  2. Practical efficiency and serving integration. Can you clarify which stages retain fused attention kernels, quantify the additional cost of Q/K replay, and indicate whether the two-pass pipeline has been tested in vLLM or SGLang, including scheduling and KV-cache handling?

  3. Preservation of spatially distinct evidence. How does the feature-similarity-based coverage mechanism avoid treating visually similar but spatially distinct object instances as interchangeable? Can you provide targeted quantitative evidence on counting or spatial reasoning that isolates the effect of the coverage term on preserving these distinct instances?

Other questions please refer to weakness.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 23ft (Part 1/3): Adaptive Partitioning and Budget Allocation

We thank Reviewer 23ft for the thoughtful and detailed review, and for recognizing LEAF's strong empirical performance under aggressive compression and its coherent framework design. We address each question below.

W1 / Q1. Why mean saliency for splitting, and comparison with a uniform partition.

What the split criterion measures. LEAF's saliency is not an attention or objectness score. It is the normalized projection of each location's centered DeiT-S feature onto the leading principal direction of feature variation (Eqs. 10–13). Low mean saliency means a region is close to the image's dominant background mode and is treated as homogeneous; high mean saliency indicates structured content, where finer leaves help. We agree that mean saliency is a proxy for within-region complexity, so LEAF does not rely on it alone: the split geometry follows the saliency variation inside the region (anisotropy and center concentration, Eqs. 15–16), and the number of tokens per leaf is set by its feature heterogeneity in the S3 selector, independently of saliency.

Uniform partition. As suggested, we replace QTMT with a fixed uniform 18×18 grid (324 regions, matched to QTMT's average of 332 leaves), keeping S2 and S3 unchanged (LLaVA-1.5-7B, 128 tokens):

PartitionGQAMMBMMEPOPE
Patch-level leaves (576, no hierarchy)58.7462.121739.2885.08
Uniform 18×18 grid (matched granularity)58.9661.451738.6784.94
QT only (adaptive depth, fixed split shape)58.9962.371752.1385.82
QTMT (LEAF)59.3363.141764.9686.37

At matched granularity, adaptive partitioning beats the uniform grid on all four benchmarks (+0.37 GQA, +1.69 MMB, +26.3 MME, +1.43 POPE). Building the hierarchy from pixel gradients, a direct measure of local complexity, is also worse than DeiT saliency (58.83 / 62.21 / 1746.26 / 85.22; Fig. 4(a)), which suggests semantic structure is a better split cue than texture. LEAF is also stable in the threshold α\alpha that turns mean saliency into a split decision: across α∈{0.1,0.15,0.2,0.3,0.4}\alpha \in \lbrace 0.1, 0.15, 0.2, 0.3, 0.4 \rbrace, GQA stays within 59.02–59.34, MMB 63.06–63.20, MME 1754.6–1765.0 and POPE 85.59–86.41, all above the uniform grid. We have added both results to Appendix C.5 and Table 7 in Section 4.2.

W2. Stronger alternatives to hierarchy-aware budget allocation.

We agree that "w/o S3" is a step-necessity ablation, not a strong allocation baseline. As suggested, we compare allocation rules that use the same QTMT leaves and the same question-aware patch scores as LEAF and change only how the 128-token budget is distributed (LLaVA-1.5-7B; Avg. = mean retention relative to the full-token model):

Allocation ruleGQAMMBMMEPOPEAvg. (%)
Leaf-exhausting greedy ("w/o S3")50.0647.511407.9069.5977.73
Global relevance top-kk (no hierarchy)58.7962.371746.4687.0096.61
Uniform regional allocation59.2862.801744.0985.9496.64
Relevance-weighted regional budgets58.9463.491748.9086.6197.03
LEAF (coverage–detail selector)59.3363.141764.9686.3797.19

Uniform allocation gives every leaf an equal share of the budget; relevance-weighted budgets make each leaf's share proportional to its relevance rj=max⁡i∈Ljqir_j=\max\nolimits_{i\in L_j} q_i. Both apportion the budget with the D'Hondt rule, cap each leaf at its size, and keep the highest-scoring patches within each leaf.

All three alternatives are far stronger than "w/o S3", confirming the reviewer's point. LEAF still has the best average and is best on GQA and MME (+16 MME over the best alternative). Relevance-weighted budgets are slightly better on MMB (+0.35) and global top-kk on POPE (+0.63). POPE asks about a single named object, so the most relevant patches suffice; LEAF's advantage comes from questions that need evidence from several regions, which is what the coverage term targets. We have added this comparison to Appendix C.5 and Table 7 in Section 4.2.

AuthorsOpenReview ↗

Response to Reviewer 23ft (Part 2/3): Fused Kernels, Replay Cost, and Serving

W3 / Q2. Fused kernels, Q/K replay cost, and serving integration.

Which stages use fused attention kernels. All of them, except for one small matrix product.

  • Probe (layers 0–8, full sequence): standard fused attention (PyTorch SDPA) with output_attentions=False, so no attention matrix is materialized.
  • Compressed pass (all 32 layers, prefill and decoding): unchanged backbone, fused attention.
  • Q/K replay: forward hooks capture the layer-8 query and key projections that the fused probe already computes, so nothing is recomputed. The replay applies RoPE and computes only the question-row logits at that single layer (Lq×LL_q \times L per head instead of L×LL \times L), followed by a softmax. This is the only non-fused computation.
  • DeiT-S saliency and QTMT: run on the CPU, overlapped with the GPU probe.

Cost of Q/K replay. LLaVA-1.5-7B, fp16, batch 1, A100, median of 100 runs; LL = 35 system + 576 visual + LqL_q question tokens:

LqL_qReplay (ms)Probe w/o replay (ms)Probe w/ replay (ms)Full prefill (ms)Replay GFLOPs
320.2821.9122.1479.080.17
640.3022.7423.0180.900.35
1280.4123.6224.0384.550.78
2560.5229.1029.79102.621.82

Replay adds 0.2–0.7 ms (1.0–2.4%) to the probe and less than 0.1% of its FLOPs. The main cost of LEAF is the truncated probe itself (about 28% of a full-length prefill), not replay.

FLOPs versus latency. We agree that at a matched token budget LEAF costs more FLOPs than lighter pruning methods. In the revision we report Bjøntegaard deltas against random retention over 2–40% retention on Qwen2.5-VL-7B (new table in Appendix C.3):

VisionZipDivPruneZOO-PruneLEAF
BD-FLOPs (%) ↓−2.09−4.31−3.39+33.51
BD-Latency (%) ↓+0.18+2.75+16.26−0.14

LEAF is the only method with lower end-to-end latency than random retention at matched performance, because the CPU-side hierarchy is hidden behind the probe and LEAF needs far fewer tokens for the same accuracy (BD-Token −75.94%). On LLaVA-1.5-7B (Table 14), LEAF with 64 tokens is both more accurate on MMBench and faster in prefill than VisionZip, DivPrune and ZOO-Prune with 128 tokens (62.11% / 73.24 ms vs. 62.00–62.03% / 77.43–94.32 ms).

vLLM/SGLang. We have not integrated LEAF into vLLM or SGLang, and we state this as a limitation in the revised Section 5 and Appendix D. The two-pass design maps onto these engines without changing their attention kernels or KV-cache manager:

  1. The probe is a truncated 9-layer prefill whose KV is not written to the paged cache, and the replay is a small kernel on projections it already produces.
  2. The selected visual embeddings then enter the engine as an ordinary, shorter multimodal request, so its paged KV cache shrinks in proportion to the compression and decoding is unchanged.
  3. For the scheduler, the probe is one extra prefill-like job (about 9/32 of a full prefill) that can be batched or chunked with other prefills.

The main engineering work is the two-phase request lifecycle; no change to fused attention or paged KV handling is needed.

AuthorsOpenReview ↗

Response to Reviewer 23ft (Part 3/3): Spatially Distinct Instances

W4 / Q3. Preserving visually similar but spatially distinct instances.

LEAF's coverage term does not, by itself, decide whether a second instance is selected. Three properties of the selector keep distinct instances from being treated as interchangeable:

  1. Candidates are spatial units. The selector chooses among QTMT leaves, which are contiguous regions; instances at different locations generally fall into different leaves, scored separately.
  2. Each leaf keeps its own relevance term. An inactive leaf jj has gain Gj=ΔCj+λωijG_j = \Delta C_j + \lambda \omega_{i_j} (Eq. 9). If a look-alike instance was already selected, only the coverage part ΔCj\Delta C_j shrinks; the importance ωij\omega_{i_j} comes from the LLM probe on position-encoded tokens, so a second relevant instance still competes on its own relevance.
  3. Concentrating on one instance is discouraged. Extra patches from an opened leaf receive only the decayed detail gain λδjωij\lambda \delta_j \omega_{i_j} with δj=dj/tjγ\delta_j = d_j/t_j^{\gamma}, so opening another relevant instance's leaf is usually preferred.

Targeted evidence. We evaluate SEED-Bench categories that require distinct instances and their locations: Instance Counting (2,447 questions), Instance Location (978) and Spatial Relation (657), on LLaVA-1.5-7B with the same evaluator for all methods (Count / Loc. / Spatial rel.):

Method128 tokens64 tokens
ZOO-Prune54.80 / 54.60 / 45.97—
CDPruner56.40 / 54.91 / 47.0354.52 / 53.89 / 45.81
MMTok54.64 / 54.91 / 47.3453.45 / 52.86 / 45.05
Global top-kk (no hierarchy, no coverage)56.48 / 56.44 / 47.79—
LEAF, coverage only57.13 / 55.83 / 48.10—
LEAF w/o coverage56.76 / 56.85 / 47.3456.44 / 54.81 / 46.73
LEAF56.60 / 56.34 / 47.3456.44 / 54.50 / 47.34

Coverage on vs. off on the main benchmarks (GQA / MMB / MME / POPE)*:

128 tokens64 tokens
LEAF w/o coverage59.38 / 63.32 / 1756.34 / 86.4257.69 / 62.20 / 1691.41 / 85.95
LEAF59.45 / 62.97 / 1760.18 / 86.4158.18 / 62.37 / 1699.28 / 86.08

Three observations. (i) Coverage does not merge instances: removing it changes counting by at most 0.16 points, and the coverage-only selector has the highest counting and spatial-relation accuracy at 128 tokens. (ii) Coverage matters more under tight budgets: at 128 tokens the two variants are within noise, while at 64 tokens coverage improves all four main benchmarks and spatial relation (+0.61). (iii) LEAF matches or exceeds ZOO-Prune, CDPruner and MMTok on all three categories, with the largest margins at 64 tokens (+1.9 to +3.0 counting, +1.5 to +2.3 spatial relation).

We agree that coverage does not model instance identity explicitly: two instances inside the same leaf, or a distant instance with low question relevance, can still be under-represented. We have added this discussion and the tables to Appendix C.6, and note position-aware similarity (adding normalized leaf coordinates to sjus_{ju}) as a natural extension.

*Both variants re-run in the same environment; full-LEAF numbers are within 0.3 points (5 on MME) of Table 1.

We hope these responses address the reviewer's concerns, and we would be happy to clarify anything further.

Reviewer_eHtXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LEAF, a training-free visual token selector. The framework combines saliency maps from a small external encoder (DeiT-S) to build adaptive quadtree/hierarchical leaf regions, alongside question-to-visual attention extracted from early LLM layers (layer 8, with positional debiasing) to measure patch relevance. A greedy facility-location algorithm then allocates the token budget across leaves by balancing coverage, relevance, and heterogeneity, after which the selected original tokens undergo a full forward pass. The method is evaluated across multiple VLMs to show performance retention under compression.

Critical strengths
  • Comprehensive Specification and Detailed Ablations: The method is well-formulated, complete with clear algorithmic flow and extensive ablations covering stages, saliency sources, probe layers, and split strategies.

  • Strong Performance under Aggressive Compression: Shows noticeable accuracy retention under extreme compression regimes (e.g., 2%-10% retention) on recent backbones like Qwen2.5-VL.

Critical weaknesses
  • Missing Comparisons with Recent Baselines (e.g., VisionTrim): The primary baselines evaluated (VisionZip, DivPrune, ZOO-Prune) represent relatively older methods. The paper omits comparisons with multiple recent representative baselines (such as VisionTrim, MMTok, and CDPruner). Failing to compare against these newer methods under a unified protocol leaves the empirical validation incomplete and renders the claimed state-of-the-art performance unconvincing.

  • Insufficient Computational Overhead and Latency Analysis: LEAF incurs non-trivial computational and temporal overhead from running an external vision encoder and probing the first 8 LLM layers, yet the evaluation lacks a thorough analysis of its overall cost-effectiveness given these additional expenditures.

  • Marginal Gains and Questionable Baseline Numbers: At moderate budgets (e.g., 128 or 192 tokens), LEAF's lead over strong baselines like ZOO-Prune is marginal, and even trails on certain shared benchmarks. While the paper claims to re-run baselines using official code, several baseline table entries match published numbers in the ZOO-Prune paper to two decimal places, raising doubts about whether experiments were re-run under a unified protocol.

Questions for authors
  • Comparison with Recent Baselines: Can you evaluate LEAF against recent representative baselines like VisionTrim, CDPruner, and MMTok to more thoroughly validate its effectiveness?

  • Compute and Latency Analysis: Can you provide a comprehensive cost-effectiveness comparison against baselines that fully accounts for the overhead of the external encoder and early-layer probing?

  • Clarification on Baseline Reproduction: Could you clarify which baseline rows were re-run versus copied from literature, and re-evaluate all baselines under an identical evaluator setup?

2 comments
AuthorsOpenReview ↗

Response to Reviewer eHtX (Part 1/2): Recent Baselines and Computational Cost

We thank Reviewer eHtX for the careful review and for recognizing the comprehensive specification and detailed ablations of LEAF, as well as its strong performance under aggressive compression. We address each concern below.

Q1. Comparison with recent baselines.

As requested, we ran VisionTrim (ICLR'26), CDPruner (NeurIPS'25) and MMTok (ICLR'26) with their official code, and additionally ApET (CVPR'26), on LLaVA-1.5-7B with the same evaluator and nine benchmarks as Table 1 (Avg. = mean retention relative to the full-token model):

Method192 tokens128 tokens64 tokens
ZOO-Prune (CVPR'26)98.3997.3395.01
ApET (CVPR'26)97.5996.6894.75
VisionTrim (ICLR'26)†97.1995.1790.06
MMTok (ICLR'26)98.3797.3595.43
CDPruner (NeurIPS'25)98.2297.3495.61
LEAF98.4197.6096.33

LEAF has the best Avg. at every budget. At 192 tokens all strong methods are close to the full-token ceiling and the margins are small (+0.02 pp over ZOO-Prune, +0.04 over MMTok, +0.19 over CDPruner); the margin over the best baseline grows to +0.25 pp at 128 tokens and +0.72 pp at 64 tokens, consistent with LEAF's design goal of preserving evidence under aggressive compression. At 64 tokens LEAF also has the highest accuracy of all methods on SEED-Bench Instance Counting, Instance Location and Spatial Relation (see our response to Reviewer 23ft), and on Qwen2.5-VL-7B at 10% retention it leads the strongest baseline by 4.78 pp under our own baseline runs (Q3 below).

†Our runs of VisionTrim's released code are below the numbers reported in its paper, mainly at 64 tokens; we report our runs for a consistent protocol and will update them if the authors' settings differ.

Per-benchmark results for all methods are in the new Appendix C.7 and summarized in Table 4.

Q2. Cost-effectiveness including the external encoder and probe.

We agree that LEAF's selection cost must be counted in full. Its two evidence branches run concurrently: DeiT-S saliency and hierarchy construction on the CPU, and the truncated probe on the GPU. On LLaVA-1.5-7B (MMBench, 128 tokens, batch 1), this overlap reduces evidence extraction from 55.17 to 39.91 ms, cuts prefill latency by 15.1%, and raises throughput by 12.5%.

End-to-end cost on LLaVA-1.5-7B (MMBench; all overheads included, Table 14):

TokensMethodMMBPrefill (ms)E2E (s)TFLOPs
576Full64.70159.14929.083.65
128VisionZip62.0077.43442.831.29
128DivPrune62.0385.07472.821.29
128ZOO-Prune61.7794.32516.451.30
128LEAF63.1487.05486.692.31
64LEAF62.1173.24424.171.99

LEAF uses more FLOPs because of the probe, but its measured latency lies between the baselines. LEAF with 64 tokens is both more accurate and faster than every baseline with 128 tokens.

Cost at matched accuracy on Qwen2.5-VL-7B (Bjøntegaard deltas over 2–40% retention, relative to random retention; negative = cheaper):

MethodBD-TokenBD-FLOPsBD-Latency
VisionZip−13.97%−2.09%+0.18%
DivPrune−25.23%−4.31%+2.75%
ZOO-Prune−26.91%−3.39%+16.26%
LEAF−75.94%+33.51%−0.14%

LEAF executes more arithmetic at matched accuracy, but because it needs far fewer tokens and overlaps its branches, its wall-clock cost at matched accuracy is on par with random retention and lower than every baseline. Peak GPU memory of LEAF (15.37 GB) equals the full-token model (15.36 GB) and is 7.6 GB below ZOO-Prune (23.01 GB), since the hierarchy runs on the CPU.

Further reducing the cost. We explored two extensions (new Appendix E):

  • In-LLM selection reuses the probe's computation instead of restarting from layer 0. On Qwen2.5-VL it lowers TFLOPs by 9.2% / 6.9% at 20% / 10% retention while raising Avg. from 97.72 to 98.95% and from 96.08 to 96.99%.
  • Adaptive budget stops allocation once the selected evidence is sufficient. On LLaVA-1.5-7B ScienceQA it uses 125 / 88 tokens on average instead of 192 / 128 while improving accuracy (69.16 vs 68.12; 69.26 vs 68.32).

We added these BD metrics and the cost breakdown to Appendix C.3 (summarized in Table 5).

AuthorsOpenReview ↗

Response to Reviewer eHtX (Part 2/2): Baseline Provenance and Gains at Moderate Budgets

Q3. Which baseline rows were re-run versus taken from the literature?

We thank the reviewer for checking this closely, and we apologize that Appendix B.3 did not state it. To be fully transparent:

  • Taken from published results (ZOO-Prune, Kim et al., 2026; same checkpoints, budgets, and benchmarks): the full-token rows of all four models; VisionZip and DivPrune on LLaVA-1.5-7B/13B and LLaVA-NeXT-7B; all LLaVA-1.5-13B baselines; ZOO-Prune on LLaVA-NeXT at 640 tokens; and VisionZip, DivPrune, and ZOO-Prune on Qwen2.5-VL at 20% and 10%.
  • Run by us with the official code and our evaluator: all LEAF rows, ZOO-Prune on LLaVA-1.5-7B and LLaVA-NeXT at 320/160 tokens, random retention, ApET, and all baselines at the extended Qwen2.5-VL ratios (Table 12).

We have revised Appendix B.3 to state this, and each table now marks every row's source.

To address the concern about a unified protocol, the table below uses our own runs for every baseline on Qwen2.5-VL-7B (same lmms-eval commit, same full-token reference):

Method20%: GQA / MMB / MME / POPEAvg.10%: GQA / MMB / MME / POPEAvg.
VisionZip54.30 / 76.12 / 2026 / 83.1290.9450.54 / 72.08 / 1853 / 78.2384.91
DivPrune58.53 / 79.55 / 2200 / 83.6995.7555.71 / 75.77 / 2083 / 79.2290.91
ZOO-Prune59.22 / 78.09 / 2151 / 85.5295.6056.51 / 73.63 / 1990 / 83.4990.83
ApET (CVPR'26)58.70 / 79.61 / 2183 / 85.1596.0856.36 / 74.06 / 1983 / 85.1591.30
LEAF58.44 / 81.27 / 2257 / 86.7197.7257.22 / 78.52 / 2219 / 87.0296.08

LEAF remains first under our own runs, leading the best baseline by +1.64 and +4.78 pp at 20% and 10% (vs. +1.39 and +5.32 pp in the submission, which did not include ApET). The remaining published rows (VisionZip and DivPrune on LLaVA, the LLaVA-1.5-13B baselines, and ZOO-Prune on LLaVA-NeXT at 640 tokens) use the same checkpoints, budgets and benchmarks as our setup; wherever we re-ran a baseline ourselves, the ranking was unchanged. These rows are marked as published in the revised tables.

W3. Marginal gains at moderate budgets.

We agree that at 192 tokens LEAF is close to the strongest baselines; at mild compression most methods are near the full-token ceiling (≥ 98%). LEAF's advantage grows as the budget shrinks: over the best baseline in the paper, +0.02 / +0.27 / +1.32 pp at 192 / 128 / 64 tokens on LLaVA-1.5-7B, +0.21 / +0.52 / +0.61 pp on 13B, +0.95 / +0.98 / +1.28 pp at 640 / 320 / 160 tokens on LLaVA-NeXT, and +5.32 pp on Qwen2.5-VL at 10% (+4.78 pp under our own baseline runs, Q3). LEAF ranks first in Avg. at every evaluated model–budget setting, including against the newly added CDPruner, MMTok, VisionTrim and ApET (Part 1). We acknowledge that LEAF is not the best on every individual benchmark at moderate budgets (e.g., ZOO-Prune on POPE and DivPrune on VQAv2 at 192 and 128 tokens); its advantage lies in the average over nine benchmarks, and it widens as the budget shrinks.

We hope these clarifications and additional results address the reviewer's concerns, and we would be glad to answer any further questions.

Reviewer_5JMsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This appears to exceed the nine-page main-text limit. I request that the Area Chair verify compliance with the submission requirements and consider desk rejection if the violation is confirmed.

Critical strengths

None

Critical weaknesses

None

Questions for authors

None

1 comment
AuthorsOpenReview ↗

Page-Limit Issue Corrected in Revised PDF

We thank the reviewer for raising this. The reviewer is correct: in the previous submitted version, the last four lines of the Conclusion (Section 5) extend onto page 10. This was caused by a small last-minute edit to one figure before the deadline, after which we failed to recheck the layout, and no content was added beyond the limit. We sincerely apologize for the oversight.

We have uploaded a revised PDF whose main text is within the 10-page limit permitted during the discussion period. The original content is unchanged; the additional space is used only for material added in response to the reviews, which is marked in blue. As the review does not yet include comments on the method or the experiments, we would sincerely welcome any technical feedback and would be glad to address it during the discussion period.

Comments on the paper

1 comment

We sincerely thank the Area Chair and all reviewers for their time and feedback. We are encouraged that the reviewers found the paper clear and easy to follow (eHtX, 23ft), the method well-formulated with a clear algorithmic flow and extensive ablations (eHtX), the framework comprehensive and coherent, addressing three complementary decisions (how to partition the image, which evidence is relevant, and how much detail each region requires) (23ft), and the empirical results strong under aggressive compression, e.g., 96.08% retention on Qwen2.5-VL at 10% tokens versus 90.76% for the strongest baseline (eHtX, 23ft).

The reviewers also raised several important points, which we address individually below. The main additions in the revised PDF are:

  1. Recent baselines (eHtX). CDPruner, MMTok, VisionTrim and ApET on LLaVA-1.5-7B at 192 / 128 / 64 tokens over nine benchmarks, with official code and our evaluator. LEAF has the best average at every budget, and its margin grows as the budget shrinks (new Appendix C.7).
  2. Baseline provenance (eHtX). Appendix B.3 now states which rows were run by us and which are taken from published results, every table marks each row's source, and the Qwen2.5-VL comparison is repeated with our own runs of every baseline.
  3. Cost and serving (eHtX, 23ft). Matched-accuracy cost (BD-Token, BD-FLOPs, BD-Latency), the isolated cost of Q/K replay, which stages use fused attention, and how the two-pass design maps onto vLLM/SGLang, stated as a limitation (Appendix C.3, Section 5).
  4. Partitioning and allocation (23ft). QTMT versus a fixed uniform grid at matched granularity, sensitivity to the split threshold, and three stronger allocation baselines (global top-k, uniform regional, relevance-weighted regional) (Appendix C.5 and Table 7 in Section 4.2).
  5. Spatially distinct instances (23ft). SEED-Bench Instance Counting, Instance Location and Spatial Relation for LEAF, its coverage ablations and the new baselines (new Appendix C.6).
  6. Page limit (5JMs). The main text is within the 10-page limit allowed during the discussion period; the original content is unchanged, and the additional page holds only rebuttal additions, marked in blue.

All changes are marked in the revised PDF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.