- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a pipeline that can generates annotations for fine-grained visual perception based on VLMs. The pipeline first locate target objects based on the difference between descriptions of local regions and whole images. VLMs then zoom-in the target region and generate correct answers. The pipeline has been applied to the construction of a challenging benchmark and annotations for RL.
Critical strengths
The proposed annotation generation pipeline can reduce labelling costs and improves VLMs' performance on fine-grained visual perception.
Critical weaknesses
- The motivation is not very clear. Considering that this paper is focused on fine-grained visual perception, which is a crowded research topic, I think it is better to clearly claim the key weakness of existing methods. However, I failed to find any claim in the manuscript.
- This paper is more like an engineering pipeline than a research paper. The proposed method, which firstly localize the target and then zooms-in and finally generates the answer, can be easily expected without any theoretical or experimental analysis.
- The RL experiments are limited to Qwen models. It is better to test at least one other VLM family.
Questions for authors
- I hope the authors can first answer the core question that, what problems of existing methods can be solved by the proposed method.
- The authors are suggested to justify the comparison between the results of RL training on ZoomInc and ZwZ. Are every experimental settings the same, including images and RL algorithms, except the ground-truth?
- This paper augues that existing benchmarks rarely filter out language-prior shortcuts. The authors are suggested to provide evidences for such claim, such as some QA examples.
8 comments
Response to W2
Response to W2 (Research Contribution and Experimental Analysis Beyond Engineering)
We believe scalable data construction engines could also be a core research contribution in modern vision and multimodal learning, as exemplified by foundational works whose core innovation is the data engine itself, such as SAM [7], Depth Anything [8], LAION-5B [9], and ShareGPT4V [10]. As shown in our response to W1 & Q1, ZoomInc goes far beyond simply chaining "localize, zoom, and answer": by explicitly contrasting what VLMs see in the full image versus the zoomed crop, it solves three key problems that naive pipelines miss: generating genuinely hard yet accurate QAs, referring to targets uniquely on clean images without red boxes, and preventing shortcut guesses.
Moreover, instead of merely proposing a pipeline, our paper provides systematic experimental analyses and empirical findings: (1) Different VLMs make consistent errors when fine details are unclear (Sec. 5.2, Figure 3): across 28 diverse VLMs, 71.7% of wrong answers fall into our full-image misprediction distractors, showing that different VLM architectures share common perception blind spots; (2) Experimental verification and superiority over prior work (Sec. 6): our ablation study (Table 4 in the paper) confirms that both incremental evidence and dedicated distractors are effective, and our RL comparisons (Table 3 in the paper) demonstrate that ZoomInc consistently outperforms ZwZ across all 14 benchmarks on both 4B and 9B models.
Reference
[1] Wu, Penghao, and Saining Xie. "V*: Guided visual search as a core mechanism in multimodal llms." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
[2] Wang, Wenbin, et al. "Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 8. 2025.
[3] Wang, Haochen, et al. "Traceable evidence enhanced visual grounded reasoning: Evaluation and method." International Conference on Learning Representations. Vol. 2026. 2026.
[4] Wei, Lai, et al. "Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception." arXiv preprint arXiv:2602.11858 (2026).
[5] Yuan, Qianhao, et al. "Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation." arXiv preprint arXiv:2605.18740 (2026).
[6] Chen, Lin, et al. "Are we on the right way for evaluating large vision-language models?." Advances in Neural Information Processing Systems 37 (2024): 27056-27087.
[7] Kirillov, Alexander, et al. "Segment anything." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2023.
[8] Yang, Lihe, et al. "Depth anything: Unleashing the power of large-scale unlabeled data." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2024.
[9] Schuhmann, Christoph, et al. "LAION-5B: An open large-scale dataset for training next generation image-text models." Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track. 2022.
[10] Chen, Lin, et al. "ShareGPT4V: Improving large multi-modal models with better captions." European Conference on Computer Vision (ECCV). 2024.
Response to Q2 & Q3 & W3
Response to Q2 (Experimental Settings and Controlled Comparison with ZwZ)
In the paper, our RL comparison between ZoomInc and ZwZ (ZwZ Original) uses the exact same 9K source images, base models, RL algorithm, and training hyperparameters (details are shown in Appendix G). The only differences came from following ZwZ's official default recipe: ZwZ overlays red bounding boxes with box-referring prompts on the images and uses multiple-choice format and rewards for counting and OCR, whereas ZoomInc uses clean images and open-ended counting/OCR rewards.
To ensure a strictly aligned comparison, we additionally trained ZwZ Align on Qwen3.5-9B: (1) Identical visual inputs: we removed all red bounding-box overlays and box-referring instructions so ZwZ Align uses the exact same full images as ZoomInc; (2) Aligned format and reward: we converted ZwZ's counting and OCR questions from multiple-choice to open-ended format and trained with the exact same reward functions as ZoomInc:
| Model | ZoomBench | V* | HR-4K | HR-8K | MME-RW | TreeBench | VisualProbe | CV-Bench | MMStar | MMVP | BabyVision | PerceptionBench | O3-Bench | Avg. (Held-out) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-9B | 50.30 | 86.91 | 82.13 | 79.63 | 58.52 | 49.88 | 44.66 | 86.27 | 71.73 | 60.00 | 15.98 | 34.20 | 56.52 | 59.75 |
| + ZwZ Original | 58.93 | 90.58 | 85.50 | 82.38 | 61.70 | 47.90 | 52.62 | 88.16 | 73.27 | 63.33 | 17.27 | 34.13 | 57.10 | 62.53 |
| + ZwZ Align | 57.86 | 91.62 | 85.25 | 82.38 | 61.23 | 49.38 | 51.26 | 88.27 | 73.27 | 67.33 | 18.04 | 34.07 | 58.26 | 62.94 |
| + ZoomInc | 60.36 | 93.19 | 90.25 | 85.50 | 63.31 | 53.33 | 56.31 | 88.58 | 73.87 | 70.00 | 17.53 | 39.00 | 68.12 | 66.10 |
Under identical clean images, aligned question rewards, and identical RL hyperparameters, ZoomInc still outperforms ZwZ Align on 12 of 13 held-out benchmarks and by +3.16 points on average (66.10 vs. 62.94). We will add this controlled comparison to the revised paper.
Response to Q3 (Concrete QA Examples of Language-Prior)
First, as verified from their official papers and Table 1 in paper, existing fine-grained benchmarks such as ZoomBench, V* Bench, HR-Bench, and TreeBench do not include a blind-filtering stage during construction.
To directly show that language-prior shortcuts exist in these benchmarks, we removed both the image and all multiple-choice options, feeding only the question stem to Qwen3.5-4B and Qwen3.5-9B in an open-ended blind setting. Even without seeing the image or the candidate options, both models directly output the exact ground-truth answers through common-sense priors alone (note that the cases below are just a few illustrative examples instead of an exhaustive list):
| Benchmark | Question Stem (Fed Without Image and Without Options) | Ground Truth | Blind Output (Qwen3.5-4B / 9B) |
|---|---|---|---|
| ZoomBench | What type of jewelry is the person wearing on their finger? | Ring | Ring / Ring |
| ZoomBench | What is the person doing with the watering can? | Water plants | Watering plants / Watering plants |
| V* Bench | What is the cartoon character on the clock. | Mickey Mouse | Mickey Mouse / Mickey Mouse |
| V* Bench | What is the color of the lock? | Silver | Silver / Silver |
| HR-Bench | Who is the founder of the company that makes the square white adapter shown in the image? | Steve Jobs | Steve Jobs / Steve Jobs |
| HR-Bench | What is the material of the ceiling beams in the room? | Wood | Wood / Wood |
When multiple-choice options are further provided in standard evaluation, guessing such questions without resolving the visual detail becomes even easier. By contrast, ZoomInc's blind shortcut filter (Sec. 3.3; see rejected candidates in Appendix D.2) explicitly removes such questions so that answering requires visual inspection. We will include these concrete examples in the revised paper.
Response to W3 (RL Experiments on Another VLM Family)
Thank you for the suggestion. We are currently running RL training on another non-Qwen VLM family and will report the results in a follow-up comment as soon as training and evaluation finish.
Response to W1 & Q1
Hello, Reviewer zU3h, thank you for the constructive review on our work. Below, we address each of your weaknesses and questions point by point.
Response to W1 & Q1 (Key Weaknesses of Existing Methods Solved by ZoomInc)
Existing fine-grained perception benchmarks and automated data synthesis methods face three key limitations, which ZoomInc directly addresses:
Manual curation is costly, while full-image synthesis is unreliable.
Problem of existing methods:
Existing fine-grained benchmarks such as V* Bench[1], HR-Bench[2], and TreeBench[3] rely on manual annotation or heavy human editing of model drafts (Table 1 in the paper), keeping dataset sizes small (only hundreds of samples) and impractical for large-scale RL training. Meanwhile, directly prompting VLMs to generate QAs from full images is unreliable: if a detail is already easy for the VLM to see in the full image, the resulting question is too simple; if a detail is truly tiny, the VLM itself often fails to see it clearly and generates wrong answers.
How ZoomInc solves it:
By contrasting global and zoomed descriptions, ZoomInc generates questions strictly from incremental details (absent in the full image but verified by multi-model consensus on the zoomed crop), ensuring both challenging questions and reliable ground truth at scale (Table 4 in the paper: 60.96 --> 58.57 held-out average without incremental evidence).
Crop-based synthesis suffers from referential ambiguity and binds perception to bounding-box overlays.
Problem of existing methods:
Recent scalable pipelines such as ZwZ [4] and Vision-OPD [5] synthesize QAs solely from local micro-crops and transfer them to full images. Because these questions are generated from the crop alone without seeing the full image, it is often unclear which object they refer to in the full image (e.g., asking "What color are the person's shoes?" when multiple people appear). Consequently, these pipelines must overlay red bounding boxes on training images to specify targets, creating a train-test visual mismatch (test images have no oracle boxes).
Also, we find that red-box training binds the model's perception to box prompts, as shown by evaluating Qwen3.5-9B variants without vs. with ground-truth red bounding boxes at test time. The Table below shows that ZwZ relies heavily on test-time red boxes:
How ZoomInc solves it:
As detailed in Sec. 3.1 of the paper, ZoomInc generates globally unique natural-language referring expressions and verifies them via multi-model grounding consensus. This enables end-to-end RL training directly on clean, unmodified images without bounding-box prompts: rather than binding perception to given boxes (See the Table above), models learn to autonomously locate small targets from natural language and recognize fine details, outperforming ZwZ (+3.57 held-out avg, see Table 3 in paper).
Existing methods lack blind-shortcut filtering and a dedicated distractor construction.
Problem of existing methods:
Although language-prior leakage is well documented in general VLM evaluation (e.g., MMStar [6]), existing fine-grained benchmarks and data pipelines (including ZoomBench, V* Bench, HR-Bench, and TreeBench; Table 1 in the paper; some cases are shown in Response to Q3) neither apply blind shortcut filtering nor construct targeted distractors. This may allow models to eliminate generic multiple-choice options without actually resolving the tiny target detail, leading to inflated benchmark scores and false-positive RL rewards from lucky guesses.
How ZoomInc solves it:
As detailed in Sec. 3.3 of the paper, ZoomInc applies a blind shortcut filter and builds multi-source hard distractors. This leads to (1) Challenging benchmark: even the strongest frontier model scores only 61.08 on ZoomInc Bench (Table 2 in the paper), with 71.7% of incorrect choices across 28 VLMs drawn to full-image misprediction distractors (Figure 3 in the paper); (2) Effective RL supervision: our RL ablation confirms the effectiveness of these hard distractors: removing them drops ZoomInc Bench by 8.96 points and the 10-benchmark held-out average from 60.96 to 59.14 (Table 4 in the paper).
We will reorganize the Introduction in the revised paper to foreground these limitations of existing methods upfront and make the overall motivation clearer.