- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a training-free, window-aware visual-token pruning method. It uses a minimum quota for each spatial region, adaptive budget allocation, and diversity-aware selection to preserve visual evidence while reducing computation. The authors evaluate the efficiency–performance trade-off on three 7B vision-language models and multiple benchmarks. However, direct cross-model evidence for hallucination mitigation remains incomplete.
Critical strengths
The three base models are LLaVA-1.5-7B, LLaVA-NeXT-7B, and Qwen2.5-VL-7B.
Critical weaknesses
-
The paper should clarify how CLS attention is obtained for Qwen2.5-VL, how it allocates tokens when the number of windows exceeds the budget, and what the end-to-end latency is. Direct hallucination results across all three backbones are also incomplete.
-
At the same 64-token budget, WinD improves aggregate performance retention only from 96.0% to 96.7%, while prefill is slightly slower (64.3 vs. 62.6 ms). Its advantage at matched end-to-end latency remains unclear.
-
The research could aim to prune tokens that contribute to hallucinations and thereby substantially reduce the hallucination rate. If the method merely ensures that pruning does not increase hallucinations, I see limited value in its hallucination-mitigation contribution.
Questions for authors
See weakness
7 comments
Response to Reviewer toA7 (2/2): Practical latency–quality trade-off and scope of hallucination mitigation
W2. Practical latency–quality trade-off.
At 64 retained visual tokens, WinD preserves 96.7% of the unpruned model's average performance, compared with 96.0% for FlowCut, at an additional prefill cost of 1.7 ms. The table summarizes this trade-off for both LLaVA backbones. All differences are relative to FlowCut, and prefill includes the complete token-selection procedure.
| Backbone | Tokens | ΔAvg. (pp) | FlowCut prefill (ms) | WinD prefill (ms) | Overhead (ms) |
|---|---|---|---|---|---|
| LLaVA-1.5-7B | 64 | +0.7 | 62.6 | 64.3 | +1.7 (2.7%) |
| LLaVA-NeXT-7B | 160 | +0.1 | 63.8 | 65.3 | +1.5 (2.4%) |
The captioning results show the practical benefit at the 64-token budget. On LLaVA-1.5-7B, the share of captions containing at least one hallucinated object (CHAIR-s) falls from 58.1% with FlowCut to 51.3% with WinD, a reduction of 6.8 percentage points. CHAIR-i also falls from 14.9% to 12.2%, and POPE F1 rises from 81.9 to 82.9. On LLaVA-NeXT-7B, the aggregate scores are nearly identical at 160 tokens, while POPE F1 rises from 79.9 to 85.1 with WinD. These comparisons quantify the change in quality and prefill latency at a fixed token budget.
W3. Scope and value of hallucination mitigation.
We agree that WinD is not designed to eliminate the intrinsic hallucinations of the unpruned model. Our claim is narrower: it mitigates the additional hallucination introduced by aggressive visual-token pruning. We will revise the paper to consistently use the term “pruning-induced hallucination mitigation” and avoid suggesting that WinD is a general-purpose hallucination-removal method.
Visual tokens cannot generally be divided into a fixed set of hallucination-causing and non-hallucination-causing tokens. Their usefulness depends on the image, prompt, and interactions within the model. In particular, attention-sink tokens are not necessarily harmful by themselves; the problem is that global top-k pruning can over-preserve them, thereby discarding spatially distributed evidence required to ground the response. Directly removing sink tokens may therefore also eliminate useful contextual information. WinD instead reduces this selection bias by preserving regional coverage and semantic diversity.
We believe this remains practically important: an efficiency method should not amplify hallucination as a side effect of compression. As shown by the matched-budget results above, WinD better preserves hallucination-related performance than global pruning while removing up to 88.9% of visual tokens. Thus, the contribution is not that token pruning itself solves all model hallucinations, but that substantial computational savings can be obtained with less additional reliability degradation. Methods for reducing the intrinsic hallucinations of the original model are complementary to, rather than replaced by, WinD.
Follow-up on our author response
Thank you again for your feedback. We added POPE F1 results across all three backbones and clarified the latency–quality trade-off. At 160 retained tokens on LLaVA-NeXT-7B, WinD improves POPE F1 by 5.2 points over FlowCut with 1.5 ms of additional prefill latency. If you have a chance to read our response, we would appreciate hearing whether the added results address your concerns about the practical benefit of WinD.
Response to Reviewer toA7 (1/2): Implementation clarifications and cross-backbone hallucination evaluation
W1. Implementation clarifications and cross-backbone hallucination evaluation.
We thank the reviewer for identifying several details that were insufficiently described. First, Qwen2.5-VL does not provide a CLS token. For this architecture, we compute each visual token's importance by averaging the attention from the global tokens to that visual token. Thus, WinD uses a model-appropriate global-attention signal rather than requiring a specific CLS-token architecture.
Second, the entropy-derived window size s^l is an initial proposal rather than an unconditional choice. At pruning layer l, the implementation selects sl⋆=min{s∈S∣s≥s^l,;Ml(s)≤Npresl}, where Ml(s) counts only windows containing at least one currently surviving token. Therefore, if the proposed partition produces more active windows than the available budget, neighboring regions are merged by selecting the closest larger feasible window size before applying the quota allocation in Eq. (4). This guarantees Ml(sl⋆)≤Npresl, so assigning at least one token to every active window remains feasible. We will add this omitted feasibility step to the method description.
Third, the prefill latency reported in Table 2 includes the complete pruning procedure, including dynamic partitioning, quota allocation, and diversity-aware selection. It measures the model-side inference pipeline and excludes disk I/O and CPU image loading. We will clarify this timing boundary and provide the component-level overhead breakdown.
Finally, to provide direct hallucination evidence across all three evaluated VLM backbones, we report matched POPE F1 results under the most aggressive compression setting using the same evaluation protocol:
POPE F1 (↑)
Under matched token budgets, WinD improves POPE F1 over FlowCut by 1.0, 5.2, and 0.3 points on LLaVA-1.5-7B, LLaVA-NeXT-7B, and Qwen2.5-VL-7B, respectively. These results complement the detailed CHAIR and HallusionBench analyses on LLaVA-1.5-7B and show that the mitigation of pruning-induced hallucination is not limited to a single VLM backbone.