acceptodds
Withdrawn from ICLR 2027

ReCo:Training-Free Visual Token Pruning via Relational Transition Coreset in Large Vision-Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large Vision-Language Models (LVLMs) incur substantial inference overhead and KV cache costs because visual inputs are represented as long, dense token sequences. Among existing pruning methods, attention-based approaches compress high-dimensional interactions into scalar saliency scores that tend to exhibit instability across layers and attention heads, whereas redundancy elimination methods often equate unimodal feature similarity with cross-modal semantic substitutability, risking the loss of fine-grained visual evidence. Both paradigms typically evaluate tokens within an isolated representation space either prior to or following cross-modal projection, failing to utilize the information conveyed by pairwise relational transitions across the projection. We propose ReCo, a training-free visual token pruning framework based on relational transition coresets. ReCo jointly characterizes pre-projection visual relations, post-projection language-aligned relations, and projection-induced relational shifts, integrating them into a unified relational transition kernel. Visual token pruning is subsequently formulated as a submodular facility-location problem, where a greedy selector constructs a compact coreset by maximizing marginal relational coverage. Experiments on LLaVA-1.5, LLaVA-NeXT, and Qwen2.5-VL demonstrate that ReCo achieves competitive performance across both fixed-length and dynamic-resolution inputs. It maintains 91.02% of the performance on Qwen2.5-VL even under an aggressive 90% visual token reduction.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Vision-Language Models, Visual Token Pruning, Relational Transition, Efficient Inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6FttOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReCo, a training-free visual token pruning method for LVLMs. Its motivation is that the multimodal projector reshapes how visual tokens relate to one another: about a quarter of each token's nearest neighbors change after projection. ReCo therefore measures token similarity in three ways (before the projector, after the projector, and how each token's relations change across the projector), combines them into a single similarity matrix, and greedily selects the subset of tokens that best "covers" all the others. Experiments on LLaVA-1.5-7B, LLaVA-NeXT-7B and Qwen2.5-VL-7B show that ReCo retains 95.48% of full-model performance on LLaVA-1.5-7B with 64 tokens (88.9% pruned), 96.00% on LLaVA-NeXT-7B with 320 tokens (88.9% pruned), and 91.02% on Qwen2.5-VL-7B with 10% of tokens.

Critical strengths

Simple and cheap to deploy. On LLaVA-1.5, its pruning stage takes about 10–24 ms, far less than ZOO-Prune (about 73–88 ms), giving lower time-to-first-token than the compared methods.

Ablation of the selection objective. With the similarity matrix fixed, the coverage-based selection beats both independent Top-K ranking and k-center selection at every budget and on every dataset (Table 8), showing that the selection objective itself contributes.

Critical weaknesses

Gains on LLaVA are within noise, and the new "relation change" component contributes little. On LLaVA-1.5, ReCo beats ZOO-Prune by only 0.06–0.28 points (Table 1); on LLaVA-NeXT it is worse than ZOO-Prune at 320 and 160 tokens (Table 2).

Gains on Qwen relies on a insufficient baseline. Strong diversity baselines are missing on Qwen.

Questions for authors

Significance on LLaVA. Please report mean ± std or paired bootstrap confidence intervals against ZOO-Prune on LLaVA-1.5 and LLaVA-NeXT, to show whether the 0.06–0.28 point gains exceed noise.

ZOO-Prune on Qwen. Why does its TextVQA collapse to 32–46? Also add DivPrune and VisionZip on Qwen2.5-VL under the same pipeline.

Reviewer_kUgbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ReCo, a training-free visual token pruning method for LVLMs. It builds three token-to-token similarity kernels: one from the vision encoder features, one from the projector outputs, and one from how each token's similarity profile shifts across the projector. These kernels are normalized and summed, and tokens are selected by greedily maximizing a facility-location objective under the token budget. Experiments on LLaVA-1.5, LLaVA-NeXT and Qwen2.5-VL (plus Qwen3-VL in the appendix) show average retention comparable to or better than recent pruning methods, along with low pruning overhead on LLaVA-1.5.

Critical strengths
  1. The method is simple and training-free, and treating pruning as facility-location selection gives a clean set-level objective. Table 8 shows that this objective really matters, since replacing it with Top-K or k-center on the same kernel hurts a lot.

  2. The evaluation covers fixed-length, AnyRes and dynamic-resolution models, and the gains on Qwen2.5-VL at 10% retention are fairly large. The selection step is also cheap on LLaVA-1.5 compared with ZOO-Prune and PruneSID.

Critical weaknesses
  1. The paper's main idea, modeling the relational shift across the projector, adds little in its own ablation: the Pre-only variant in Table 4 is within about one point of full ReCo and already on par with the best baseline. The gains therefore seem to come mostly from facility-location selection rather than from the "cross-modal" transition, and that framing is a stretch anyway, since the method never uses the text and the projector is a per-token MLP.

  2. Several of the most relevant baselines are missing, such as VisionZip (cited but not compared), CDPruner and DART, and the Qwen experiments compare against only three methods. Against the strongest included baseline (ZOO-Prune), the margins are small on LLaVA-1.5 and ReCo falls behind on LLaVA-NeXT at 320 and 160 tokens, so the empirical advantage is not clearly established.

Questions for authors
  1. Could you report the Pre-only facility-location variant on LLaVA-NeXT and Qwen2.5-VL? A clear benefit from the shift kernel there would change my view of the contribution.
  2. How does ReCo compare with VisionZip, CDPruner and DART under the same budgets?
  3. Clarification: given the O(N³) shift kernel, what are the latency and memory on long sequences (LLaVA-NeXT, high-resolution Qwen2.5-VL), and how do they compare with the unpruned model?
  4. Clarification: were the baseline numbers reproduced or taken from the original papers? ZOO-Prune's TextVQA score on Qwen2.5-VL looks unusually low.
Reviewer_xHGtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReCo, a training-free visual token pruning method for large vision-language models. It combines relational information before and after cross-modal projection with a representation of projection-induced relational shifts. These relations are fused into a kernel, and greedy optimization of a facility-location objective selects a representative token subset before language-model inference. Experiments evaluate performance retention and inference efficiency across multiple VLM backbones.

Critical strengths

Competitive performance across multiple backbones. Experiments cover seven benchmarks and several LLaVA and Qwen models, demonstrating competitive performance retention under aggressive token compression.

Useful controlled analyses. The paper includes component and selection-objective ablations, together with TTFT and complete generation latency measurements, providing evidence for its design choices.

Critical weaknesses
  • Limited technical novelty. Similarity-based redundancy estimation and coverage-based token selection are established ideas, including facility-location compression in FLoC and multimodal coverage in MMTok.

  • Modest gains from Shift. In Table 4, Pre-only and Pre+Post variants using the same facility-location selector already perform close to full ReCo on LLaVA-1.5.

  • Unverified framework compatibility. The paper does not demonstrate integration with vLLM or SGLang, or compare against an unpruned model using the same optimized backend. It remains unclear whether the reported acceleration persists when existing attention and serving optimizations are enabled and integration overhead is included.

  • Missing long-form evaluation. The benchmarks predominantly involve multiple-choice, binary, or short-answer tasks. It provides neither dedicated long-form evaluations nor output-length statistics. Prior work shows that visual pruning can increase generation length and substantially weaken latency gains, a concern that the current evaluations do not resolve.

Questions for authors
  1. What distinguishes ReCo from existing coverage-based and facility-location-based token selection methods beyond the construction and combination of relational kernels?

  2. On long-form generation tasks, how does ReCo affect answer quality, generated-token counts, and complete generation latency when models generate naturally to EOS under matched decoding settings and sufficiently large generation limits?

Others please refer to weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.