acceptodds
Withdrawn from ICLR 2027

ResizeBench: Revealing the Limits of Resizing for Visual Token Compression

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-token compression methods are typically evaluated against one another on standard VQA benchmarks, without comparison to pixel-level resizing. Yet on modern variable-resolution VLMs, naive image resizing matches or outperforms methods published over the past two years, while reducing both vision-encoder computation and the number of visual tokens processed by the LLM. We show that this result reflects a limitation of existing evaluations: most benchmark questions remain answerable from coarse visual information and rarely test localized fine-grained evidence. We introduce ResizeBench, a benchmark of 1,458 resolution-critical samples designed to measure this missing regime. Each sample is associated with a verified resolution barrier, the image-area budget at which its answer-relevant evidence becomes unrecoverable through resizing. We mine candidate barriers from a sweep of 387K samples and verify them using two frontier VLMs, answer filtering, and factuality checking. Across 15 token-compression methods, three VLM backbones, and 7.2M generated answers, the standard-suite and ResizeBench leaderboards exhibit weak agreement. Resizing performs strongly on standard benchmarks but degrades sharply when answers depend on localized detail, where selective compression provides substantial gains. The highest-ranked compression method also varies across backbones, indicating that token compression remains an open problem with substantial method–model dependence and no universally effective strategy. These results establish resizing as an essential matched-budget baseline and resolution-critical evaluation as a necessary complement to standard VQA benchmarks.

Primary area:
datasets and benchmarks
Keywords:
Token compression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pQNWOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces ResizeBench: benchmark composed from 1458 resolution-critical vqa samples designed to evaluate visual token compression methods. The authors show that simple image resizing is highly competitive with token pruning on standard VLM benchmarks, but performs substantially worse on samples requiring fine-grained visual information. Experiments across three VLMs and 15 compression methods reveal considerable differences between standard benchmark rankings and ResizeBench rankings.

Critical strengths
  1. The paper addresses an important evaluation issue in efficient VLMs: the trade-off between reducing computational cost and preserving fine-grained visual information.

  2. The experimental evaluation is extensive, covering numerous compression methods, three VLM backbones, and multiple token budgets.

  3. The per-sample resolution-barrier protocol offers a systematic way to analyze sensitivity to visual resolution, and the resulting benchmark could serve as a useful complementary diagnostic.

Critical weaknesses
  1. Limited insight of the central observation. The paper observation that most images can be processed at reduced resolution, while fine-grained tasks such as OCR and small-object recognition require higher resolution, is already well established. Benchmarks such as V* explicitly address fine-grained visual recognition. The paper includes V* in its standard suite but does not sufficiently explain why existing fine-grained evaluations are inadequate or what fundamentally new capability ResizeBench measures.

  2. Important missing related work. The paper overlooks several directly relevant studies. In particular, CARES [1] explicitly studies the minimum sufficient resolution for individual image-question pairs, using multi-resolution evaluation to establish supervision. This substantially overlaps with the paper's motivation and resolution-barrier formulation, although the two works have different objectives. VisionThink [2] and Awares [3] also demonstrates that resizing is an effective compression strategy and dynamically requests higher resolution when necessary. These omissions weaken the novelty claims.

  3. Incomplete evaluation of relevant efficiency strategies. The benchmark evaluates numerous pruning approaches but omits adaptive-resolution and crop-retrieval approaches such as the previus mentioned methods [1,2,3]. These methods explicitly address the central trade-off studied in this paper. Although their variable-budget behavior requires an appropriate evaluation protocol, excluding this family limits the benchmark's applicability to modern visual token efficiency methods. I belive that even if the evaluation do not yeild the same results as pruning.

  4. Benchmark construction favors pruning over resizing by design. ResizeBench is constructed by selecting examples that become incorrect under resizing at particular resolutions. Consequently, the finding that resizing performs poorly compared with approaches that first encode the full-resolution image is partly a consequence of the selection procedure. While such examples are useful for stress-testing information preservation, this is not sufficient to establish the practical superiority of one compression strategy over another. On less brouder scope, its a focus on spesific cases where resize is not sufficient, and that could be a good observation of when and why to use prunning over resize methods like the mentioned in points 2 and 3.

Less Critical but still to adress
  1. Missing combined resizing and pruning experiments. The experiments compare resizing against pruning, but do not appear to investigate their combination. Since these approaches act at different stages of the pipeline, they are complementary rather than mutually exclusive. Evaluating pruning on moderately resized inputs would clarify whether the observed gains are genuinely attributable to token selection and whether hybrid strategies offer better accuracy-efficiency trade-offs. Furthermore, comparisons are based on final token counts rather than measured end-to-end computational cost.

  2. Limited model coverage. Only three relatively small open-weight VLMs are evaluated. No proprietary frontier VLM is included in the compression evaluation, despite such models being used in benchmark verification. Moreover, alternative high-resolution architectures such as InternVL3.5, which supports dynamic high-resolution processing and resolution-aware token routing, are absent. This limits the generality of the conclusions.

[1] CARES : Context-Aware Resolution Selector for VLMs (ACL 2026)

[2] VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning (NeurIPS 2025)

[3] Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs (DocInsights 2026)

Questions for authors

How does ResizeBench differ conceptually and empirically from the resolution-sensitivity analysis in CARES (ACL 2026), and from existing fine-grained benchmarks such as V*?

Why were adaptive-resolution and crop-retrieval methods, particularly CARES, VisionThink, and AwaRes, excluded? Could the benchmark support an average-token-budget evaluation protocol to accommodate methods with input-dependent computational cost?

Have the authors evaluated combined resizing and token pruning? For example, could moderate downsampling followed by selective pruning achieve a better accuracy-latency trade-off than either technique alone?

Given that samples are selected specifically because resizing fails, how do the authors distinguish a meaningful method-ranking difference from the selection bias introduced by benchmark construction?

Can the authors extend the evaluation to stronger or more diverse architectures, such as InternVL3.5, and report actual latency or FLOPs alongside token-retention ratios?

Reviewer_m5KvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes ResizeBench, a resolution-critical benchmark for VLMs that evaluates token compression at the per-sample budget. It shows that naive pixel resizing, as a missing baseline in prior work, matches or beats recent compression methods on standard benchmarks. Experimental results demonstrate that the two leaderboards are nearly independent, that resizing fails sharply on localized fine-grained evidence where selective compression wins, and that the best method varies across backbones.

Critical strengths
  1. The paper identifies resizing as an important missing baseline for visual token compression, and demonstrates this through a large-scale study, showing that resizing can be a strong compression baseline on standard benchmarks.

  2. The paper introduces a dataset, ResizeBench, for evaluating whether compression methods can preserve localized fine-grained visual evidence.

Critical weaknesses
  1. The paper states that "resizing should be a matched-budget baseline in every token-compression study." However, in the main experiments the budget is matched only along the dimension of visual-token keep ratio. In practice, because different methods compress at different stages of the pipeline, a fair budget cannot be computed from the token keep ratio alone.

  2. All evaluated models are around 4B parameters, which is quite small. As far as I know, stronger model capability may lead to better compression results for some methods.

Questions for authors
  1. I agree with the claim that token compression methods perform better on questions requiring localized key information, while resizing may work better for questions that only need global information. Similarly, VoCo-LLaMA [1] reports naive compression methods like average pooling, whose effect may be comparable to resizing. Is there a comparison with pooling-based or similar methods?

  2. The comparison is conducted at matched token budgets, but the methods differ substantially in where computation is saved. Could you report end-to-end latency or FLOPs alongside the token counts?

[1] VoCo-LLaMA: Towards Vision Compression with Large Language Models, CVPR 2025

Reviewer_vZTHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that visual token compression methods for VLMs are almost never compared against the simplest baseline available on modern variable-resolution models: resizing the input image so that it yields the same number of visual tokens. Using a shared harness covering 15 published compression methods (10 post-ViT, 4 in-LLM, plus a uniform-sampling control), three ~4B backbones (Qwen3-VL-4B, Qwen3.5-4B, Gemma-4-E4B), five keep ratios, and about 7.2M generated answers, the authors show that matched-budget resizing is highly competitive on a seven-benchmark standard suite, ranking first in ten of fifteen model–budget settings. They attribute this to standard benchmarks mostly asking questions answerable from coarse visual information, and introduce ResizeBench, 1,458 "resolution-critical" samples, each with a per-sample resolution barrier. The barrier is found by sweeping 387K samples from 39 tasks over ten resize scales with Qwen3-VL-4B, detecting a clean correct-to-incorrect transition, and keeping only samples on which two frontier VLMs (Claude, GPT) also fail at the barrier resolution, followed by answer-matching and LLM-judge filtering. Each sample is evaluated only at its own barrier budget. On ResizeBench, resizing falls to last place on both Qwen models while selective compression retains far more accuracy. Rankings on the standard suite and ResizeBench agree only weakly (Kendall τb\tau_b of 0.15–0.34 on Qwen3.5-4B), and the best method changes across backbones. The authors recommend that compression papers always report matched-budget resizing and add a resolution-critical evaluation.

Critical strengths
  1. It exposes a missing baseline with a large, uniform evaluation. Showing that plain resizing matches or beats many recent compression methods on the benchmarks those methods are usually evaluated on is a practically important message for the community. The scale and uniformity of the comparison (15 methods, 3 backbones, 5 budgets, one harness and scoring protocol) make this finding more convincing than any single-paper comparison, and the recommendation to always report matched-budget resizing is easy to adopt.

  2. The reporting is unusually careful and transparent. The paper discloses where each baseline deviates from its original implementation, re-scores everything on full-resolution-correct samples (Appendix E), provides bootstrap intervals for rank displacements (Table F2), and explicitly warns against over-reading single rows. This makes the paper's own limitations visible to the reader.

Critical weaknesses
  1. ResizeBench is selected to defeat resizing, so the headline result is largely true by construction. Samples are kept only if Qwen3-VL fails on the resized image at the barrier and frontier models also fail on that resized image. Resizing therefore ranking last on ResizeBench, particularly on Qwen3-VL (17.6% relative accuracy in Table E1), is the selection criterion rather than an empirical finding. Post-ViT methods additionally see full-resolution encoder features, so they have access to information the resized input lacks by design. The benchmark thus measures complementarity to resizing rather than "resolution-critical" ability in general. Consistent with this, the effect largely disappears on the model not used for mining: on Gemma-4 resize reaches 79.4% relative accuracy, above most selectors (Table E1). Table 2 also omits the resize row entirely, which hides this.

  2. The "weak agreement" claim lacks a noise ceiling. Each budget is evaluated on a different, small cohort (86 samples at b80, 132 at b70), and Table F2 itself notes that only 4 of 14 displacement intervals exclude zero. With many methods separated by a few points, low Kendall τb\tau_b is expected even if the two benchmarks measured the same thing. The paper does not report the split-half reliability of ResizeBench rankings, the τb\tau_b among the seven standard benchmarks themselves (Table F1 already shows AI2D and SEED-Bench near zero or negative), or τb\tau_b with resize excluded (resize is last on ResizeBench by construction and mechanically lowers agreement). Without these, "the two leaderboards measure distinct aspects" is not established.

  3. The baselines are re-implementations, not the published methods. Several are substantial adaptations: CDPruner without its CLIP relevance model, AnchorPrune with a substituted importance prior, PruMerge without a CLS token, FLoC as "our local facility-location baseline control" rather than a port, FastV and MustDrop implemented through the PriorTR machinery, and all in-LLM methods pruning at a fixed layer 3 on Qwen. Claims that resizing beats "methods published at CVPR, NeurIPS, ICCV…" and that the best method depends on the backbone may partly reflect these adaptation choices rather than the methods themselves.

  4. Benchmark construction and verification are under-documented.

    • The frontier verifiers are only named as "Claude, GPT," without versions.
    • The 100-sample human verification reports no outcome (no agreement or error rate).
    • The final record-validity filter reshapes the barrier distribution without explanation (b40 shrinks from 1,127 to 477 while b50 goes from 501 to 498).
    • Qwen3-VL, the mining model, answers only 92.2% of ResizeBench correctly at full resolution, although full-resolution correctness was a selection criterion. This suggests the scoring or prompting differs between mining and evaluation.
    • The samples come from public benchmarks that the evaluated models may have been trained on.
  5. The efficiency argument is not measured. The paper repeatedly argues that resizing also saves vision-encoder computation and that token-matched comparison is "conservative for resizing," but it reports no FLOPs, latency, or memory for any method. Since the practical recommendation depends on the accuracy–cost trade-off, at least wall-clock and FLOP measurements at matched budgets are needed.

  6. Limited scope and some reporting inconsistencies. All three backbones are ~4B models, so it is unclear whether the conclusions hold for larger VLMs, which may be more robust to resizing. Some numbers cannot be reconciled across the paper. For example, the text gives resize 26.7% on Qwen3.5 and 18.5% on Qwen3-VL, while Table E1 gives 31.0% and 17.6%, and neither appears in Table 2. MustDrop is also missing from the Qwen3-VL suite. Finally, accuracy "improving from b40 to b80" compares different sample sets and is not a trend.

Questions for authors
  1. Reliability of the divergence claim. Please report (a) split-half or bootstrap reliability of the ResizeBench leaderboard at each budget, (b) the pairwise τb\tau_b among the seven standard benchmarks as a reference for "weak" agreement, and (c) suite-vs-ResizeBench τb\tau_b with resize excluded. If ResizeBench is about as self-consistent as its agreement with the suite, the main claim would need to be revised.

  2. Selection circularity. How would the conclusions change on a resolution-critical set that is not mined via resize failure (e.g., samples whose answer-relevant region is small, identified by annotation or by failure under a different degradation)? Please add the resize row to Table 2, and explain why resize is competitive on Gemma-4 if ResizeBench captures a model-independent property.

  3. Baseline fidelity. For methods whose official code supports Qwen-family models, do the official implementations reproduce the rankings of your adaptations? How sensitive are the in-LLM rankings to the choice of pruning layer?

  4. Construction details. Which frontier model versions were used? What was the outcome of the 100-sample human check? Why does the final filter remove most b40 samples but almost no b50 samples? Why does Qwen3-VL answer only 92.2% correctly at full resolution despite the full-resolution correctness requirement?

  5. Measured efficiency. Can you report FLOPs and end-to-end latency for resize, post-ViT and in-LLM methods at matched budgets on at least one backbone, so that the recommended trade-off is quantified?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.