- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper evaluates the benefits of two existing “thinking with images” methods, ViCrop and HiDe, from the perspective of token costs. It reports that the gains claimed by both methods diminish when comparisons are made under matched token budgets. The paper also examines how crop accuracy and the placement of cropped regions affect performance.
Critical strengths
- The paper identifies unfair comparisons between existing “thinking with images” methods and their baselines in terms of token costs, and develops a comprehensive evaluation and associated metrics under constrained token budgets.
- The paper explains why accurate bounding boxes are important for improving visual understanding.
Critical weaknesses
- The writing is difficult to follow. Many sentences are hard to understand. For example: “At low budgets on a backbone that can spend the same tokens on resolution, neither gain survives: ViCrop’s once the answering call is matched, HiDe’s once its localization is charged.”
- Reallocating a multi-round token budget to a single baseline call does not constitute a fair comparison. In my view, pooling the input tokens used across ViCrop’s multiple rounds and allocating that budget to a single baseline call raises the baseline’s visual processing capacity, but does not establish that ViCrop itself is less effective. Under this setup, the two methods receive images at different resolutions, so differences in performance may arise from image resolution rather than the methods themselves. The baseline’s performance is thus tied to the artificially imposed resolution limit: relaxing that limit can improve its results without demonstrating an inherent advantage over ViCrop.
Questions for authors
- How could the effects of different token budgets be evaluated while ensuring a fair comparison in terms of input image resolution?
4 comments
Response to authors
Thank you for your response and the additional experiments, which have partially addressed my concerns. However, as noted in Weakness 1, I still find much of the manuscript difficult to read and understand. Substantial restructuring and revision are needed before it meets the standard for acceptance.
Revised PDF uploaded
Thank you for reading the revision and for the clear verdict. We took "substantial restructuring" literally: the revised PDF just uploaded rewrites the main text for readability, with no change to results or claims (summary in our comment to all reviewers). In brief, the prose is about 30% shorter, with confidence intervals moved into the tables. The findings are organized as three regimes stated in the abstract and introduction. Sections 4–6 have one subsection per finding, each titled with its conclusion. If specific passages are still hard to follow, we would be grateful to know which ones.
Response to authors
Thank you for the authors' efforts. I have read the revised paper, the long sentences have been improved. However, I believe that beyond the revisions to wording and length, the overall structure of the paper also needs to be revised. I think this requires time and effort to polish, rather than making some textual edits within just a few days. I maintain my score.
Response to Reviewer ZGo9
Thank you for the precise objection. Your strengths state the starting point: published comparisons give the method more tokens than its baseline. Your concern calls for two comparisons, placement at fixed resolution and budget allocation; the submission ran the first on LLaVA-1.5, and the revision adds it on the three Qwen models. On two points we respectfully differ.
1. The main comparison does not pool ViCrop's rounds. The baseline receives only the tokens of ViCrop's answering call, the main image plus the crop, bracketed from below and above (ratios 0.92–0.99 and 1.03–1.11). ViCrop's two localization passes are not charged, which favours ViCrop (Section 3.2, as in the submission), and Appendix A.4 checks that the brackets also enclose the answering call's FLOPs. Pooling across calls appears only in the total-compute comparisons (Section 5.3; Appendix B.3 for HiDe).
2. At fixed resolution, the crop's placement wins. We do not claim that ViCrop is ineffective in itself. On LLaVA-1.5, whose resolution is fixed, ViCrop's crop beats centered, random and repeated second views by 18.9 to 21.5 points at identical tokens, and two higher-resolution tiles by 13.1; the submission concluded that "its published gain stands" (now Section 4.1, Table 8). On Qwen (Table 10) the main image is pixel-identical and only the second view changes. R costs exactly the crop's tokens; D and T are within 2% of them on Qwen2.5-VL and 1–13% above them on Qwen3-VL.
Selected rows; crop minus condition, points ± rounded 95% paired-bootstrap interval half-width (HR-Bench resampled by question), * p < 0.05 uncorrected. Over all 18 cells the crop is significantly better than D in 4, T in 9 and R in 9, with one borderline loss against D on Qwen3-VL-8B, HR-8K b256 (−2.88, p ≈ 0.05). On Qwen3-VL-8B this series uses localization layer 26, re-selected by hit rate, which favours the crop (Appendix B.9).
3. What the resolution comparison establishes. We agree that raising the baseline's resolution changes the information it receives. G tests how to allocate answering tokens; it does not isolate localization at fixed resolution. R does: the main image is pixel-identical, and the random box has ViCrop's dimensions and exactly its tokens. On Qwen3-VL-8B, V*Bench b256, ViCrop gains +14.7±6.8 over R but −1.0±7.3 against G (Table 13: 61.3% at 346 tokens against 62.3% at 328). Placement helps without the crop beating resolution, and the interval against G does not establish equivalence. The asymmetry lies in the published comparison, which gives the method a second image while holding the baseline at the original cap.
Our conclusions concern the tested budgets. ViCrop's published LLaVA-1.5 gain stands, HiDe keeps a gain at its published 16,384-token setting (+4.00 [0.50, 7.62] on HR-Bench-4K, Appendix B.11), and saturation alone does not make a crop pay (Appendix I.2).
4. Your question: how to vary the budget while keeping resolution fair. Three designs:
5. Beyond "accurate boxes are important". Two results go further. Under pasting, accuracy is non-monotone in box position: a box beside the target scores 7.8 to 10.8 points below a distant one (Section 6.1, Figure 11). And Table 3 states in which regimes a crop is worth its tokens at the tested budgets.
Writing. The sentence you quoted is gone. Its two claims now stand separately, each with its condition: Section 4.2 opens with "Given a choice, the whole image at higher resolution usually wins", and the HiDe result is the third row of Table 3.