- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces ResizeBench: benchmark composed from 1458 resolution-critical vqa samples designed to evaluate visual token compression methods. The authors show that simple image resizing is highly competitive with token pruning on standard VLM benchmarks, but performs substantially worse on samples requiring fine-grained visual information. Experiments across three VLMs and 15 compression methods reveal considerable differences between standard benchmark rankings and ResizeBench rankings.
Critical strengths
-
The paper addresses an important evaluation issue in efficient VLMs: the trade-off between reducing computational cost and preserving fine-grained visual information.
-
The experimental evaluation is extensive, covering numerous compression methods, three VLM backbones, and multiple token budgets.
-
The per-sample resolution-barrier protocol offers a systematic way to analyze sensitivity to visual resolution, and the resulting benchmark could serve as a useful complementary diagnostic.
Critical weaknesses
-
Limited insight of the central observation. The paper observation that most images can be processed at reduced resolution, while fine-grained tasks such as OCR and small-object recognition require higher resolution, is already well established. Benchmarks such as V* explicitly address fine-grained visual recognition. The paper includes V* in its standard suite but does not sufficiently explain why existing fine-grained evaluations are inadequate or what fundamentally new capability ResizeBench measures.
-
Important missing related work. The paper overlooks several directly relevant studies. In particular, CARES [1] explicitly studies the minimum sufficient resolution for individual image-question pairs, using multi-resolution evaluation to establish supervision. This substantially overlaps with the paper's motivation and resolution-barrier formulation, although the two works have different objectives. VisionThink [2] and Awares [3] also demonstrates that resizing is an effective compression strategy and dynamically requests higher resolution when necessary. These omissions weaken the novelty claims.
-
Incomplete evaluation of relevant efficiency strategies. The benchmark evaluates numerous pruning approaches but omits adaptive-resolution and crop-retrieval approaches such as the previus mentioned methods [1,2,3]. These methods explicitly address the central trade-off studied in this paper. Although their variable-budget behavior requires an appropriate evaluation protocol, excluding this family limits the benchmark's applicability to modern visual token efficiency methods. I belive that even if the evaluation do not yeild the same results as pruning.
-
Benchmark construction favors pruning over resizing by design. ResizeBench is constructed by selecting examples that become incorrect under resizing at particular resolutions. Consequently, the finding that resizing performs poorly compared with approaches that first encode the full-resolution image is partly a consequence of the selection procedure. While such examples are useful for stress-testing information preservation, this is not sufficient to establish the practical superiority of one compression strategy over another. On less brouder scope, its a focus on spesific cases where resize is not sufficient, and that could be a good observation of when and why to use prunning over resize methods like the mentioned in points 2 and 3.
Less Critical but still to adress
-
Missing combined resizing and pruning experiments. The experiments compare resizing against pruning, but do not appear to investigate their combination. Since these approaches act at different stages of the pipeline, they are complementary rather than mutually exclusive. Evaluating pruning on moderately resized inputs would clarify whether the observed gains are genuinely attributable to token selection and whether hybrid strategies offer better accuracy-efficiency trade-offs. Furthermore, comparisons are based on final token counts rather than measured end-to-end computational cost.
-
Limited model coverage. Only three relatively small open-weight VLMs are evaluated. No proprietary frontier VLM is included in the compression evaluation, despite such models being used in benchmark verification. Moreover, alternative high-resolution architectures such as InternVL3.5, which supports dynamic high-resolution processing and resolution-aware token routing, are absent. This limits the generality of the conclusions.
[1] CARES : Context-Aware Resolution Selector for VLMs (ACL 2026)
[2] VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning (NeurIPS 2025)
[3] Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs (DocInsights 2026)
Questions for authors
How does ResizeBench differ conceptually and empirically from the resolution-sensitivity analysis in CARES (ACL 2026), and from existing fine-grained benchmarks such as V*?
Why were adaptive-resolution and crop-retrieval methods, particularly CARES, VisionThink, and AwaRes, excluded? Could the benchmark support an average-token-budget evaluation protocol to accommodate methods with input-dependent computational cost?
Have the authors evaluated combined resizing and token pruning? For example, could moderate downsampling followed by selective pruning achieve a better accuracy-latency trade-off than either technique alone?
Given that samples are selected specifically because resizing fails, how do the authors distinguish a meaningful method-ranking difference from the selection bias introduced by benchmark construction?
Can the authors extend the evaluation to stronger or more diverse architectures, such as InternVL3.5, and report actual latency or FLOPs alongside token-retention ratios?