acceptodds
Desk rejected at ICLR 2027

ZoomInc: Mining Global-to-Zoom Perception Gaps for Fine-Grained Visual Supervision

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Fine-grained visual perception requires Vision-Language Models (VLMs) to locate and interpret subtle evidence within complex scenes. Constructing supervision for this ability is challenging: questions must have correct, unambiguous answers that depend on actual fine-grained visual details rather than coarse cues or language priors. Verifying these properties makes manual annotation labor-intensive and automatic generation difficult for models that themselves struggle with fine-grained perception. Yet the same models can often recognize such details once the region is zoomed in. We exploit this global-to-zoom perception gap and introduce ZoomInc, a data construction pipeline that turns incremental details revealed only by zooming into questions about the full image. It verifies each answer on the zoomed view with multiple models, ensures that each question remains unambiguous in the full image and cannot be answered from language priors, and specifically generates high-quality distractors for multiple-choice questions. With this pipeline and human-only screening (no manual QA editing), we build ZoomInc Bench, a 1,000-question bilingual benchmark on which the best of 28 VLMs achieves only 61.08. Failure analysis reveals that models favor a particular type of distractor when answering multiple-choice questions incorrectly. Beyond evaluation, training on just 9K ZoomInc samples substantially improves the visual perception capabilities of our 4B/9B models, enabling them to outperform several hundred-billion-parameter models across various perception benchmarks. The trained 9B model achieves SoTA performance on several benchmarks, such as 90.25 on HR-Bench 4K. These results establish global-to-zoom perception gaps as a practical basis for scaling fine-grained visual supervision for both evaluation and training. Pipeline code and data will be released after publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Models, Multimodal Large Language Models, Fine-Grained Visual Perception, Synthetic Data Generation, Multimodal Benchmarking, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_zU3hOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a pipeline that can generates annotations for fine-grained visual perception based on VLMs. The pipeline first locate target objects based on the difference between descriptions of local regions and whole images. VLMs then zoom-in the target region and generate correct answers. The pipeline has been applied to the construction of a challenging benchmark and annotations for RL.

Critical strengths

The proposed annotation generation pipeline can reduce labelling costs and improves VLMs' performance on fine-grained visual perception.

Critical weaknesses
  1. The motivation is not very clear. Considering that this paper is focused on fine-grained visual perception, which is a crowded research topic, I think it is better to clearly claim the key weakness of existing methods. However, I failed to find any claim in the manuscript.
  2. This paper is more like an engineering pipeline than a research paper. The proposed method, which firstly localize the target and then zooms-in and finally generates the answer, can be easily expected without any theoretical or experimental analysis.
  3. The RL experiments are limited to Qwen models. It is better to test at least one other VLM family.
Questions for authors
  1. I hope the authors can first answer the core question that, what problems of existing methods can be solved by the proposed method.
  2. The authors are suggested to justify the comparison between the results of RL training on ZoomInc and ZwZ. Are every experimental settings the same, including images and RL algorithms, except the ground-truth?
  3. This paper augues that existing benchmarks rarely filter out language-prior shortcuts. The authors are suggested to provide evidences for such claim, such as some QA examples.
8 comments
AuthorsOpenReview ↗

Response to W1 & Q1

Hello, Reviewer zU3h, thank you for the constructive review on our work. Below, we address each of your weaknesses and questions point by point.

Response to W1 & Q1 (Key Weaknesses of Existing Methods Solved by ZoomInc)

Existing fine-grained perception benchmarks and automated data synthesis methods face three key limitations, which ZoomInc directly addresses:

  1. Manual curation is costly, while full-image synthesis is unreliable.

    • Problem of existing methods:

      Existing fine-grained benchmarks such as V* Bench[1], HR-Bench[2], and TreeBench[3] rely on manual annotation or heavy human editing of model drafts (Table 1 in the paper), keeping dataset sizes small (only hundreds of samples) and impractical for large-scale RL training. Meanwhile, directly prompting VLMs to generate QAs from full images is unreliable: if a detail is already easy for the VLM to see in the full image, the resulting question is too simple; if a detail is truly tiny, the VLM itself often fails to see it clearly and generates wrong answers.

    • How ZoomInc solves it:

      By contrasting global and zoomed descriptions, ZoomInc generates questions strictly from incremental details (absent in the full image but verified by multi-model consensus on the zoomed crop), ensuring both challenging questions and reliable ground truth at scale (Table 4 in the paper: 60.96 --> 58.57 held-out average without incremental evidence).

  2. Crop-based synthesis suffers from referential ambiguity and binds perception to bounding-box overlays.

    • Problem of existing methods:

      Recent scalable pipelines such as ZwZ [4] and Vision-OPD [5] synthesize QAs solely from local micro-crops and transfer them to full images. Because these questions are generated from the crop alone without seeing the full image, it is often unclear which object they refer to in the full image (e.g., asking "What color are the person's shoes?" when multiple people appear). Consequently, these pipelines must overlay red bounding boxes on training images to specify targets, creating a train-test visual mismatch (test images have no oracle boxes).

      Also, we find that red-box training binds the model's perception to box prompts, as shown by evaluating Qwen3.5-9B variants without vs. with ground-truth red bounding boxes at test time. The Table below shows that ZwZ relies heavily on test-time red boxes:

      ModelZoomBench (Clean)ZoomBench (w/ Box)ΔBox\Delta_{\text{Box}}ZoomInc Bench (Clean)ZoomInc Bench (w/ Box)ΔBox\Delta_{\text{Box}}
      Qwen3.5-9B (Base)50.3050.30+0.0032.1340.54+8.41
      + ZwZ Original58.9361.30+2.3744.0756.69+12.62
      + ZoomInc60.3659.05-1.3155.4862.17+6.69
    • How ZoomInc solves it:

      As detailed in Sec. 3.1 of the paper, ZoomInc generates globally unique natural-language referring expressions and verifies them via multi-model grounding consensus. This enables end-to-end RL training directly on clean, unmodified images without bounding-box prompts: rather than binding perception to given boxes (See the Table above), models learn to autonomously locate small targets from natural language and recognize fine details, outperforming ZwZ (+3.57 held-out avg, see Table 3 in paper).

  3. Existing methods lack blind-shortcut filtering and a dedicated distractor construction.

    • Problem of existing methods:

      Although language-prior leakage is well documented in general VLM evaluation (e.g., MMStar [6]), existing fine-grained benchmarks and data pipelines (including ZoomBench, V* Bench, HR-Bench, and TreeBench; Table 1 in the paper; some cases are shown in Response to Q3) neither apply blind shortcut filtering nor construct targeted distractors. This may allow models to eliminate generic multiple-choice options without actually resolving the tiny target detail, leading to inflated benchmark scores and false-positive RL rewards from lucky guesses.

    • How ZoomInc solves it:

      As detailed in Sec. 3.3 of the paper, ZoomInc applies a blind shortcut filter and builds multi-source hard distractors. This leads to (1) Challenging benchmark: even the strongest frontier model scores only 61.08 on ZoomInc Bench (Table 2 in the paper), with 71.7% of incorrect choices across 28 VLMs drawn to full-image misprediction distractors (Figure 3 in the paper); (2) Effective RL supervision: our RL ablation confirms the effectiveness of these hard distractors: removing them drops ZoomInc Bench by 8.96 points and the 10-benchmark held-out average from 60.96 to 59.14 (Table 4 in the paper).

We will reorganize the Introduction in the revised paper to foreground these limitations of existing methods upfront and make the overall motivation clearer.

AuthorsOpenReview ↗

Response to W2

Response to W2 (Research Contribution and Experimental Analysis Beyond Engineering)

We believe scalable data construction engines could also be a core research contribution in modern vision and multimodal learning, as exemplified by foundational works whose core innovation is the data engine itself, such as SAM [7], Depth Anything [8], LAION-5B [9], and ShareGPT4V [10]. As shown in our response to W1 & Q1, ZoomInc goes far beyond simply chaining "localize, zoom, and answer": by explicitly contrasting what VLMs see in the full image versus the zoomed crop, it solves three key problems that naive pipelines miss: generating genuinely hard yet accurate QAs, referring to targets uniquely on clean images without red boxes, and preventing shortcut guesses.

Moreover, instead of merely proposing a pipeline, our paper provides systematic experimental analyses and empirical findings: (1) Different VLMs make consistent errors when fine details are unclear (Sec. 5.2, Figure 3): across 28 diverse VLMs, 71.7% of wrong answers fall into our full-image misprediction distractors, showing that different VLM architectures share common perception blind spots; (2) Experimental verification and superiority over prior work (Sec. 6): our ablation study (Table 4 in the paper) confirms that both incremental evidence and dedicated distractors are effective, and our RL comparisons (Table 3 in the paper) demonstrate that ZoomInc consistently outperforms ZwZ across all 14 benchmarks on both 4B and 9B models.


Reference

[1] Wu, Penghao, and Saining Xie. "V*: Guided visual search as a core mechanism in multimodal llms." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

[2] Wang, Wenbin, et al. "Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 8. 2025.

[3] Wang, Haochen, et al. "Traceable evidence enhanced visual grounded reasoning: Evaluation and method." International Conference on Learning Representations. Vol. 2026. 2026.

[4] Wei, Lai, et al. "Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception." arXiv preprint arXiv:2602.11858 (2026).

[5] Yuan, Qianhao, et al. "Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation." arXiv preprint arXiv:2605.18740 (2026).

[6] Chen, Lin, et al. "Are we on the right way for evaluating large vision-language models?." Advances in Neural Information Processing Systems 37 (2024): 27056-27087.

[7] Kirillov, Alexander, et al. "Segment anything." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2023.

[8] Yang, Lihe, et al. "Depth anything: Unleashing the power of large-scale unlabeled data." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2024.

[9] Schuhmann, Christoph, et al. "LAION-5B: An open large-scale dataset for training next generation image-text models." Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track. 2022.

[10] Chen, Lin, et al. "ShareGPT4V: Improving large multi-modal models with better captions." European Conference on Computer Vision (ECCV). 2024.

AuthorsOpenReview ↗

Response to Q2 & Q3 & W3

Response to Q2 (Experimental Settings and Controlled Comparison with ZwZ)

In the paper, our RL comparison between ZoomInc and ZwZ (ZwZ Original) uses the exact same 9K source images, base models, RL algorithm, and training hyperparameters (details are shown in Appendix G). The only differences came from following ZwZ's official default recipe: ZwZ overlays red bounding boxes with box-referring prompts on the images and uses multiple-choice format and rewards for counting and OCR, whereas ZoomInc uses clean images and open-ended counting/OCR rewards.

To ensure a strictly aligned comparison, we additionally trained ZwZ Align on Qwen3.5-9B: (1) Identical visual inputs: we removed all red bounding-box overlays and box-referring instructions so ZwZ Align uses the exact same full images as ZoomInc; (2) Aligned format and reward: we converted ZwZ's counting and OCR questions from multiple-choice to open-ended format and trained with the exact same reward functions as ZoomInc:

ModelZoomBenchV*HR-4KHR-8KMME-RWTreeBenchVisualProbeCV-BenchMMStarMMVPBabyVisionPerceptionBenchO3-BenchAvg. (Held-out)
Qwen3.5-9B50.3086.9182.1379.6358.5249.8844.6686.2771.7360.0015.9834.2056.5259.75
+ ZwZ Original58.9390.5885.5082.3861.7047.9052.6288.1673.2763.3317.2734.1357.1062.53
+ ZwZ Align57.8691.6285.2582.3861.2349.3851.2688.2773.2767.3318.0434.0758.2662.94
+ ZoomInc60.3693.1990.2585.5063.3153.3356.3188.5873.8770.0017.5339.0068.1266.10

Under identical clean images, aligned question rewards, and identical RL hyperparameters, ZoomInc still outperforms ZwZ Align on 12 of 13 held-out benchmarks and by +3.16 points on average (66.10 vs. 62.94). We will add this controlled comparison to the revised paper.


Response to Q3 (Concrete QA Examples of Language-Prior)

First, as verified from their official papers and Table 1 in paper, existing fine-grained benchmarks such as ZoomBench, V* Bench, HR-Bench, and TreeBench do not include a blind-filtering stage during construction.

To directly show that language-prior shortcuts exist in these benchmarks, we removed both the image and all multiple-choice options, feeding only the question stem to Qwen3.5-4B and Qwen3.5-9B in an open-ended blind setting. Even without seeing the image or the candidate options, both models directly output the exact ground-truth answers through common-sense priors alone (note that the cases below are just a few illustrative examples instead of an exhaustive list):

BenchmarkQuestion Stem (Fed Without Image and Without Options)Ground TruthBlind Output (Qwen3.5-4B / 9B)
ZoomBenchWhat type of jewelry is the person wearing on their finger?RingRing / Ring
ZoomBenchWhat is the person doing with the watering can?Water plantsWatering plants / Watering plants
V* BenchWhat is the cartoon character on the clock.Mickey MouseMickey Mouse / Mickey Mouse
V* BenchWhat is the color of the lock?SilverSilver / Silver
HR-BenchWho is the founder of the company that makes the square white adapter shown in the image?Steve JobsSteve Jobs / Steve Jobs
HR-BenchWhat is the material of the ceiling beams in the room?WoodWood / Wood

When multiple-choice options are further provided in standard evaluation, guessing such questions without resolving the visual detail becomes even easier. By contrast, ZoomInc's blind shortcut filter (Sec. 3.3; see rejected candidates in Appendix D.2) explicitly removes such questions so that answering requires visual inspection. We will include these concrete examples in the revised paper.


Response to W3 (RL Experiments on Another VLM Family)

Thank you for the suggestion. We are currently running RL training on another non-Qwen VLM family and will report the results in a follow-up comment as soon as training and evaluation finish.

Reviewer_87KLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ZoomInc, a method for constructing fine-grained visual question answering data by comparing descriptions of full images with those of zoomed-in, localized views. To improve data quality, the approach incorporates multi-model verification and the generation of challenging distractors. The authors also establish a corresponding evaluation benchmark and demonstrate the effectiveness of the data across various visual understanding tasks through training with GRPO.

Critical strengths
  1. The research problem holds practical value. The paper focuses on local details within a full image that are easily overlooked—encompassing tasks such as attribute recognition, counting, and text recognition—and carries real-world significance for fine-grained visual understanding.

  2. The data construction process is quite comprehensive. It integrates differential analysis, multi-model validation, filtering of "blind-guess" items, and the generation of challenging distractors, with targeted design choices addressing both question answerability and the difficulty level of the options.

Critical weaknesses
  1. The novelty of the proposed method is somewhat limited. The overall workflow is similar to the framework established by ZwZ—which involves local question-answer generation, verification, and reinforcement learning-based training—with the primary new contributions focusing on mining incremental evidence from differences between global- and zoom-view descriptions, as well as constructing distractors from multiple sources.

  2. The interpretation of the description-based selection criterion is not fully established. Table 4 supports the usefulness of this criterion. However, details omitted from a full-image description may reflect selective captioning rather than a failure of perception. It remains unclear how reliably description discrepancies correspond to actual differences in question-answering performance between full-image and zoomed-in views.

Questions for authors
  1. Control conditions for training comparisons. Appendix G indicates that ZoomInc and ZwZ employ different image presentation methods and reward rules. Could these differences be addressed—either through unified experimental settings or additional ablation studies—to clarify their impact on the results, thereby enabling a more precise assessment of the independent contribution of the question-construction strategy?

  2. Table 4 supports the overall distractor design, and Figure 3 analyzes distractor selection by source. Could the authors compare the full strategy against a variant using only incorrect answers produced by VLMs on full images, to quantify the additional training benefit of combining multiple sources?

  3. Splitting of the training and evaluation sets. Section 4 mentions that the evaluation Q&A pairs were excluded from the training pool. Were other Q&A pairs associated with these same evaluation images also excluded, to ensure that the training set is isolated from the ZoomInc Bench at the level of the original images?

5 comments
AuthorsOpenReview ↗

Response to W1

Hello, Reviewer 87KL, thank you for the constructive feedback on our work. Below, we address each of your weaknesses and questions point by point.


Response to W1 (Novelty and Fundamental Differences from ZwZ)

Thank you for raising this point. We would like to clarify that ZoomInc differs fundamentally from ZwZ in design. Specifically, the two methods differ in four core aspects:

(1) Natural-language referring vs. red-box binding: ZoomInc generates globally unique natural-language referring expressions verified via multi-model grounding (Sec. 3.1). This enables RL training directly on clean images, teaching models to locate small targets on their own. By contrast, ZwZ generates QAs from local crops alone without seeing the full image, so it is often unclear which object a question refers to in the full image (e.g., asking "What color are the person's shoes?" when multiple people appear). Thus, ZwZ has to draw red bounding boxes on training images to specify targets, creating a train-test mismatch (test images have no oracle boxes). To demonstrate how this affects inference, we evaluate the base Qwen3.5-9B, ZwZ-trained 9B, and ZoomInc-trained 9B on ZoomBench and ZoomInc Bench under two test settings: Clean (standard unmodified images) and w/ Box (drawing the ground-truth red box around the target region on test images that aligns the ZwZ-training setting):

ModelZoomBench (Clean)ZoomBench (w/ Box)ΔBox\Delta_{\text{Box}}ZoomInc Bench (Clean)ZoomInc Bench (w/ Box)ΔBox\Delta_{\text{Box}}
Qwen3.5-9B (Base)50.3050.30+0.0032.1340.54+8.41
+ ZwZ58.9361.30+2.3744.0756.69+12.62
+ ZoomInc60.3659.05-1.3155.4862.17+6.69

Adding ground-truth red boxes at inference time improves ZwZ the most (+2.37 on ZoomBench and +12.62 on ZoomInc Bench), whereas on standard Clean test images it falls well behind ZoomInc. This confirms that ZwZ binds perception heavily to oracle red-box prompts, while ZoomInc learns to locate fine-grained targets directly on the naive images.

(2) Blind-shortcut filtering: ZoomInc explicitly applies a blind shortcut filter (Sec. 3.3) to remove questions solvable without visual inspection, preventing inflated evaluation scores and lucky-guess RL rewards. By contrast, ZwZ does not perform blind filtering, leaving questions that models can answer from language prior. For example, on ZwZ's benchmark (ZoomBench), even when we remove both the image and all options and feed only the question stem, Qwen3.5-4B and 9B still output the exact ground-truth answers:

BenchmarkQuestion Stem (Fed Without Image and Without Options)Ground TruthBlind Output (Qwen3.5-4B / 9B)
ZoomBench (ZwZ)What type of jewelry is the person wearing on their finger?RingRing / Ring
ZoomBench (ZwZ)What is the person doing with the watering can?Water plantsWatering plants / Watering plants

(3) Dedicated distractor construction: ZoomInc constructs multi-source hard distractors (Sec. 3.3). As proven in Table 4 of the paper, these distractors are crucial for RL training: removing them drops ZoomInc Bench from 55.81 to 46.85 (-8.96) and the 10-benchmark held-out average from 60.96 to 59.14. By contrast, ZwZ has no dedicated distractor construction stage, which can let models eliminate easy options without resolving fine visual details.

(4) Genuinely hard fine-grained questions: ZoomInc extracts incremental facts by contrasting full-image and crop captions, ensuring that questions target subtle details missed in the full image. By contrast, ZwZ generates questions from crops alone without checking what is already visible in the full image. As shown in the Response to W2 table below (where all questions are open-ended, removing any distractor effect), models only answer 37 to 53 of ZoomInc questions on the full image, whereas they achieve 73 to 79 on ZwZ-style questions, demonstrating that ZoomInc generates much more challenging fine-grained questions than ZwZ even without distractors.

Together, these designs lead to substantial gains over ZwZ across all 14 benchmarks (Table 3 in the paper). We will add this statement to the revised paper.

AuthorsOpenReview ↗

Response to W2

Response to W2 (Description Discrepancies vs. Full-to-Zoom QA Performance)

Thank you for raising this point. We agree that a caption gap does not always mean a QA gap. To address this:

(1) Detailed captioning prompt: In our pipeline (Sec. 3.2), we instruct the VLM to describe the target region in as much detail as possible (and mark unreadable details as "unclear"), which helps reduce simple omissions from selective captioning.

(2) Comparison to another QA construction baseline: More importantly, to test whether caption differences lead to real QA gaps between full-image and zoomed-in views (and uncover fine-grained details better), we sample 300 regions from ZoomInc Bench and generate open-ended questions on the same regions in two ways:

  • Crop-only (ZwZ-style baseline): use ZwZ's prompt to instruct Qwen3.5-397B-A17B to generate questions directly from the crop view alone (This is just what ZwZ did);
  • ZoomInc Incremental: generate questions from incremental facts between the full-image and crop captions (This is just what ZoomInc did).

We check each QA pair manually to ensure the correctness. For both question sets, we evaluate some VLMs with both Full Image w/ Box (marking the region with a red box on the full image so models know where to look) and Zoomed Crop inputs. Using both views serves a clear purpose: testing 'Full Image w/ Box' checks whether models can already answer the question on the full image, while testing 'Zoomed Crop' checks whether the question becomes solvable after zooming in. Their score difference (Δ=Crop−Full w/ Box\Delta = \text{Crop} - \text{Full w/ Box}) directly measures the actual QA gap between full-image and zoomed-in views:

ModelCrop-only (Full w/ Box → Crop, Δ\Delta)ZoomInc Incremental (Full w/ Box → Crop, Δ\Delta)
Qwen3.5-4B73.00 → 85.33 (+12.33)36.67 → 88.00 (+51.33)
Qwen3.5-9B78.67 → 86.00 (+7.33)43.33 → 90.00 (+46.67)
Qwen3.5-35B-A3B75.00 → 85.33 (+10.33)51.00 → 91.00 (+40.00)
GLM-5.3-Flash (320B)76.67 → 89.00 (+12.33)53.33 → 94.00 (+40.67)

Both sets reach similarly high accuracy on 'Zoomed Crop', showing that all questions are clear once zoomed in. However, 'ZwZ-style Crop-only' questions are already largely solved on 'Full Image w/ Box' (73 to 79, Δ\Delta of only +7.33 to +12.33), whereas 'ZoomInc Incremental' questions drop to 36.67 to 53.33 on 'Full Image w/ Box' and increase by +40.00 to +51.33 points upon zooming (a 3x to 6x larger QA gap than Crop-only).

In short, while selective captioning may occur on individual items, description discrepancies reliably translate into large full-to-zoom QA gaps and find truly fine-grained details much more effectively. We will add this experiment and discussion to the revised paper.

AuthorsOpenReview ↗

Response to Q1 & Q2 & Q3

Response to Q1 (Controlled Comparison with ZwZ)

In our original comparison (ZwZ Original), we follow ZwZ's official setup (red-box overlays and multiple-choice rewards). To eliminate these differences and isolate the contribution of the question construction strategy, we train an aligned baseline (ZwZ Align) on the exact same 9K samples with two unified settings: (1) Same clean images: We remove all red-box overlays and box-referring prompts so ZwZ Align uses the exact same original images as ZoomInc. (2) Same rewards: We convert ZwZ's counting and OCR questions to open-ended format and use the exact same reward functions and RL settings as ZoomInc.

ModelZoomBenchV*HR-4KHR-8KMME-RWTreeBenchVisualProbeCV-BenchMMStarMMVPBabyVisionPerceptionBenchO3-BenchAvg. (Held-out)
Qwen3.5-9B50.3086.9182.1379.6358.5249.8844.6686.2771.7360.0015.9834.2056.5259.75
+ ZwZ Original58.9390.5885.5082.3861.7047.9052.6288.1673.2763.3317.2734.1357.1062.53
+ ZwZ Align57.8691.6285.2582.3861.2349.3851.2688.2773.2767.3318.0434.0758.2662.94
+ ZoomInc60.3693.1990.2585.5063.3153.3356.3188.5873.8770.0017.5339.0068.1266.10

Under identical clean images and unified rewards, ZoomInc still outperforms ZwZ Align on 12 of 13 held-out benchmarks (+3.16 points on average). This confirms that our improvements stem directly from our question construction strategy. We will add this controlled comparison to the revised paper.


Response to Q2 (Distractor Variant Using Only Full-Image VLM Errors)

Thank you for the suggestion. We are currently running the training experiment for this distractor ablation variant and will report the comparative results in a follow-up comment as soon as training and evaluation finish.


Response to Q3 (Splitting of Training and Evaluation Sets)

Thank you for raising this point. We clarify that there is no image leakage between the training set and the benchmark. In our implementation, after pairing the ZoomInc training pool with ZwZ data, all QA pairs associated with the images shown in ZoomInc Bench were completely filtered out. We will clarify this point in the revised paper.

Reviewer_Bm9HOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents ZoomInc, a data construction pipeline for improving fine-grained visual perception in VLMs. The key idea is to exploit the perception gap between full images and zoomed-in regions to generate questions targeting subtle visual details. The pipeline incorporates multi-model verification, language-prior filtering, and targeted distractor generation to improve data quality. Using this approach, the authors construct ZoomInc Bench, a bilingual benchmark of 1,000 questions evaluated on 28 VLMs. They further demonstrate that training on 9K ZoomInc samples improves the performance of 4B and 9B models across multiple perception and reasoning benchmarks, outperforming models trained on comparable ZwZ data.

Critical strengths
  1. The paper aims to address an important practical challenge in constructing high-quality supervision for fine-grained visual perception, where obtaining reliable annotations is particularly difficult.
  2. The motivation is clear and intuitive. Exploiting the perception gap between full images and zoomed-in regions provides an interesting approach to identifying challenging visual details for supervision.
  3. The use of multiple VLMs for grounding and answer verification helps reduce potential ambiguities and annotation errors, which is particularly important for automatically generated visual question-answering data.
Critical weaknesses
  1. The central global-to-zoom perception gap is not directly validated. The method identifies incremental evidence by comparing captions generated from full and zoomed-in images. However, the absence of a detail in the global caption does not necessarily mean that the model cannot perceive it in the original image.
  2. The reliability of the automatically generated ground-truth answers is not sufficiently established. Agreement among multiple VLMs does not guarantee correctness, as these models may share systematic errors. An independent human evaluation of answer accuracy would provide stronger evidence for the quality of both the benchmark and training data.
  3. The comparison with ZwZ is not fully controlled. Although the authors match the source images and training settings, ZwZ retains bounding-box overlays while ZoomInc uses original images. The two approaches also employ different reward definitions, making it difficult to attribute the observed improvements solely to the proposed data construction strategy.
  4. The experiments do not sufficiently disentangle the benefits of exploiting perception gaps from those of generating harder training questions. The ablation studies show that incremental evidence and dedicated distractors are useful, but do not establish whether the improvements specifically result from learning to recognize previously overlooked visual details.
Questions for authors

Please refer to the weaknesses.

4 comments
AuthorsOpenReview ↗

Response to W1 & W2

Hello, Reviewer Bm9H, thank you for the positive feedback on our work. Below, we address each of your weaknesses point by point.


Response to W1 (Validating the Global-to-Zoom Perception Gap)

Thank you for raising this point. We agree that a caption gap does not always mean a QA gap. To address this:

(1) Detailed captioning prompt: In our pipeline (Sec. 3.2), we instruct the VLM to describe the target region in as much detail as possible (and mark unreadable details as "unclear"), which helps reduce simple omissions during captioning.

(2) Comparison to another QA construction baseline: More importantly, to test whether caption differences uncover fine-grained details better, we sample 300 regions from ZoomInc Bench and generate open-ended questions on the same regions in two ways:

  • Crop-only (ZwZ-style baseline): use ZwZ's prompt to instruct Qwen3.5-397B-A17B to generate questions directly from the crop view alone (This is just what ZwZ did);
  • ZoomInc Incremental: generate questions from incremental facts between the full-image and crop captions (This is just what ZoomInc did).

We check each QA pair manually to ensure the correctness. For both question sets, we evaluate some VLMs with both Full Image w/ Box (marking the region with a red box on the full image so models know where to look) and Zoomed Crop inputs. Using both views serves a clear purpose: testing 'Full Image w/ Box' checks whether models can already answer the question on the full image, while testing 'Zoomed Crop' checks whether the question becomes solvable after zooming in. Their score difference (Δ=Crop−Full w/ Box\Delta = \text{Crop} - \text{Full w/ Box}) directly measures how much zooming helps:

ModelCrop-only (Full w/ Box → Crop, Δ\Delta)ZoomInc Incremental (Full w/ Box → Crop, Δ\Delta)
Qwen3.5-4B73.00 → 85.33 (+12.33)36.67 → 88.00 (+51.33)
Qwen3.5-9B78.67 → 86.00 (+7.33)43.33 → 90.00 (+46.67)
Qwen3.5-35B-A3B75.00 → 85.33 (+10.33)51.00 → 91.00 (+40.00)
GLM-5.3-Flash (320B)76.67 → 89.00 (+12.33)53.33 → 94.00 (+40.67)

Both sets reach similarly high accuracy on 'Zoomed Crop', showing that all questions are clear once zoomed in. However, 'ZwZ-style Crop-only' questions are already largely solved on 'Full Image w/ Box' (73 to 79, Δ\Delta of only +7.33 to +12.33), whereas 'ZoomInc Incremental' questions drop to 36.67 to 53.33 on 'Full Image w/ Box' and jump by +40.00 to +51.33 points upon zooming (a 3x to 6x larger gap than Crop-only).

In short, though a caption gap does not strictly equal a QA gap on every single item, comparing full-image and crop captions finds truly fine-grained details much more effectively.


Response to W2 (Reliability of GT Answer)

We process the benchmark and training data separately: (1) ZoomInc Bench (100% human-verified): In our implementation, ZoomInc Bench does not rely on model agreement alone. All benchmark samples are manually screened by two human reviewers to ensure the correctness; (2) ZoomInc Training Set: To keep the pipeline fully automated and scalable, the 9K training set is not manually checked and allows minor label noise. Still, our RL results directly prove the data quality: training on this training set improves the performance of Qwen3.5-4B and 9B model across multiple benchmarks (Table 3 in the paper).

AuthorsOpenReview ↗

Response to W3 & W4

Response to W3 (Controlled Comparison with ZwZ)

In our original comparison (ZwZ Original), we followed ZwZ's official setup (red-box overlays and multiple-choice rewards). To make the comparison strictly controlled, we train an aligned baseline (ZwZ Align) on the same 9K samples with two changes: (1) Same clean images: We remove all red-box overlays and box-referring prompts so ZwZ Align uses the exact same original images as ZoomInc. (2) Same rewards: We convert ZwZ's counting and OCR questions to open-ended format and use the exact same reward functions and RL settings as ZoomInc.

ModelZoomBenchV*HR-4KHR-8KMME-RWTreeBenchVisualProbeCV-BenchMMStarMMVPBabyVisionPerceptionBenchO3-BenchAvg. (Held-out)
Qwen3.5-9B50.3086.9182.1379.6358.5249.8844.6686.2771.7360.0015.9834.2056.5259.75
+ ZwZ Original58.9390.5885.5082.3861.7047.9052.6288.1673.2763.3317.2734.1357.1062.53
+ ZwZ Align57.8691.6285.2582.3861.2349.3851.2688.2773.2767.3318.0434.0758.2662.94
+ ZoomInc60.3693.1990.2585.5063.3153.3356.3188.5873.8770.0017.5339.0068.1266.10

Under identical clean images and aligned rewards, ZoomInc still outperforms ZwZ Align on 12 of 13 held-out benchmarks and by +3.16 points on average (66.10 vs. 62.94). We will add this controlled comparison to the revised paper.


Response to W4 (Disentangling Perception-Gap Learning from Question Difficulty)

Two pieces of evidence show that our models truly learn to recognize previously overlooked fine details, rather than merely benefiting from harder questions:

(1) Where the gains appear across benchmarks (Table 3 in the paper): If improvements are mainly from generic hard-question training, gains would be roughly uniform across benchmarks. Instead, gains are small on general perception (e.g., +2.03 on 4B and +2.31 on 9B for CV-Bench), but much larger on fine-grained perception benchmarks, such as VisualProbe (+13.98 on 4B, +11.65 on 9B), ZoomBench (+11.00 / +10.06), V* Bench (+8.37 / +6.28), and HR-Bench 4K (+6.00 / +8.12), which require finding small visual details.

(2) Direct test on the 300 open-ended sets from Response to W1: We further evaluate our ZoomInc-trained 4B and 9B models on the same 300 regions from W1. Because all questions here are open-ended (no multiple-choice options to eliminate), improvements directly reflect the visual recognition capability:

ModelZoomInc Incremental (Full w/ Box)ZoomInc Incremental (Zoomed Crop)Crop-only (Full w/ Box)Crop-only (Zoomed Crop)
Qwen3.5-4B36.6788.0073.0085.33
+ ZoomInc (4B)51.67 (+15.00)89.33 (+1.33)78.00 (+5.00)86.67 (+1.34)
Qwen3.5-9B43.3390.0078.6786.00
+ ZoomInc (9B)57.67 (+14.34)91.33 (+1.33)82.33 (+3.66)86.33 (+0.33)

After ZoomInc training, accuracy on 'Zoomed Crop' changes very little (+0.33 to +1.34), whereas accuracy on 'Full Image w/ Box' increases by +15.00 points (4B) and +14.34 points (9B) on ZoomInc Incremental. This directly shows that models learn to recognize subtle details on the full image that they previously missed without zooming.

Thank you for the thoughtful and detailed response. I appreciate the authors' efforts and believe the paper makes a meaningful contribution to fine-grained visual supervision. Overall, I have a positive view of this work, although I still consider it closer to a weak accept than a strong accept. I will therefore maintain my original rating.

Thank your sincerely for your positive feedback to our work :)

Reviewer_7jqXOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ZoomInc, an automated pipeline for constructing fine-grained visual supervision by exploiting the perception gap between global images and zoomed-in regions. The pipeline extracts incremental visual details from paired views, generates questions based on these details, verifies answer reliability using multiple VLMs, filters questions that can be answered through language priors, and constructs challenging distractors from multiple sources. Based on this pipeline, the authors introduce a bilingual benchmark containing 1,000 questions and demonstrate the effectiveness of ZoomInc data for reinforcement learning. Training Qwen3.5-4B and Qwen3.5-9B on only 9K samples yields consistent improvements across multiple perception benchmarks, with the 9B model achieving an average score of 66.10 on 13 held-out benchmarks.

Critical strengths
  • A practical and well-motivated data construction approach. The key insight of exploiting global-to-zoom perception gaps is intuitive and effective. By combining incremental evidence extraction, multi-model answer verification, blind filtering, and multi-source distractor generation, the proposed pipeline addresses several important challenges in automatically constructing reliable fine-grained visual supervision.
  • Strong empirical results with limited training data. Training on only 9K samples brings substantial improvements over both the base models and models trained on ZwZ data. Notably, the trained Qwen3.5-9B achieves an average score of 66.10 across 13 held-out benchmarks, outperforming Qwen3.5-397B-A17B (65.55) and matching GLM-5.3-Flash (66.07). These results suggest that carefully constructed supervision can improve fine-grained perception without requiring substantially larger models or datasets.
  • Useful ablation studies and error analysis. The ablations demonstrate that both incremental evidence extraction and dedicated distractor construction contribute to the training gains. The analysis of model errors also provides useful insights into how VLMs rely on scene-plausible guesses when fine-grained visual evidence is difficult to resolve.
Critical weaknesses
  • Potential image-level data leakage between training and benchmark sets. Section 4 states that the 1,000 benchmark QA pairs are excluded from the training pool, but it is unclear whether the split is performed at the QA level or the source-image level. Since the generated pool contains multiple questions per image, other questions associated with benchmark images may remain in the training set. If so, the held-in ZoomInc Bench results could be inflated by exposure to the same images during training. The authors should clarify the splitting protocol and ideally ensure image-level separation.
  • The comparison with ZwZ may contain a confounding factor. Although the authors match the source images and training settings, the ZwZ images retain bounding-box overlays, whereas ZoomInc uses the original images. Consequently, the actual visual inputs are not identical, and the observed performance differences cannot be attributed exclusively to question construction. A controlled comparison using the same image presentation for both datasets would make the conclusions more convincing.
  • The benchmark does not fully isolate autonomous zoom-based perception. According to Table 1, ZoomInc Bench provides both the full image and the evidence region. This setup evaluates fine-grained recognition with access to a localized crop, but does not directly establish whether a model can independently identify the relevant region and decide when to zoom in. An evaluation using only the full image, alongside the current dual-view setting, would better distinguish localization and evidence-discovery capabilities from recognition given an already-provided crop.
Questions for authors
  • Can the authors clarify whether the training and benchmark sets are separated by source image or only by QA pair? If the split is QA-level, could the main results be reproduced using an image-disjoint split?
  • Can the authors provide a controlled comparison between ZoomInc and ZwZ using identical image inputs, particularly with respect to bounding-box overlays, to isolate the contribution of the proposed question construction strategy?
  • How much of the benchmark performance depends on the availability of the evidence crop? Could the authors report results under both full-image-only and full-image-plus-crop settings to clarify what capabilities ZoomInc Bench measures?
3 comments

Hello, Reviewer 7jqX, thank you for the positive feedback on our work. Below, we address each of your weaknesses and questions point by point.


Response to W1 & Q1 (Image Leak between training set and benchmark)

Thank you for raising this point. We clarify that there is no image leakage between the training set and the benchmark. In our implementation, after pairing the ZoomInc training pool with ZwZ data, all QA pairs associated with the images shown in ZoomInc Bench were completely filtered out. We will clarify this point in the revised paper.


Response to W2 & Q2 (Controlled Comparison between ZoomInc and ZwZ)

Thank you for raising this point. In our original comparison (ZwZ Original), we retained ZwZ's official recipe: red bounding-box overlays, box-referring prompts, and multiple-choice rewards.

To eliminate both visual-input and reward differences as potential confounders, we conducted a strictly controlled experiment (ZwZ Align) on the same 9K samples: (1) Identical visual inputs: we remove all red bounding-box overlays and box-referring instructions so ZwZ Align uses the same full images as ZoomInc; (2) Aligned format and reward: we converted ZwZ's counting and OCR questions from multiple-choice to open-ended format and trained with the exact same reward functions as ZoomInc under identical GRPO settings. The results are shown as follows:

ModelZoomBenchV*HR-4KHR-8KMME-RWTreeBenchVisualProbeCV-BenchMMStarMMVPBabyVisionPerceptionBenchO3-BenchAvg. (Held-out)
Qwen3.5-9B50.3086.9182.1379.6358.5249.8844.6686.2771.7360.0015.9834.2056.5259.75
+ ZwZ Original58.9390.5885.5082.3861.7047.9052.6288.1673.2763.3317.2734.1357.1062.53
+ ZwZ Align57.8691.6285.2582.3861.2349.3851.2688.2773.2767.3318.0434.0758.2662.94
+ ZoomInc60.3693.1990.2585.5063.3153.3356.3188.5873.8770.0017.5339.0068.1266.10

Under identical clean images and aligned rewards, ZoomInc still outperforms ZwZ Align on 12 of 13 held-out benchmarks and by +3.16 points on average (66.10 vs. 62.94). We will add this controlled comparison to the revised paper.


Response to W3 & Q3 (Clarifying "Dual-View" and Multi-View Results)

We apologize for the confusion caused by our unclear wording of Dual-view in Table 1. In Table 1, Dual-view describes what the dataset provides (similar to ZoomBench, each sample includes both the full image and the localized evidence crop with its paired local question for diagnostic analysis), not the model input during standard evaluation. All ZoomInc Bench results reported in our paper are already evaluated under the Full Image Only setting, where models receive only the full image and global question.

Following your suggestion, we compare three view settings on ZoomInc Bench across four models from 4B to 320B: (1) Full Image Only (default setting: full image + global question), (2) Full Image + Crop (full image + ground-truth evidence crop + global question), and (3) Crop + Local Question (evidence crop alone + local question):

ModelFull Image Only (Paper Default)Full Image + CropCrop + Local Question
Qwen3.5-4B30.0373.7382.65
Qwen3.5-9B32.1380.2490.78
Qwen3.5-35B-A3B38.0183.7990.51
GLM-5.3-Flash (320B)45.5489.8693.86

Providing the evidence crop boosts accuracy by +45 points (Full Image Only vs. Full Image + Crop), and asking the local question directly on the crop further reaches ~90. This confirms that: (1) the questions are unambiguous and solvable once zoomed in, (2) our default Full Image Only evaluation primarily bottlenecks autonomous evidence localization and high-resolution discovery capabilites of existing VLMs. We will clarify Dual-view in Table 1 and include this multi-view analysis in the revised paper.

Thank you for the detailed response. The clarification that all QA pairs associated with ZoomInc Bench source images were excluded from training addresses my concern about image-level leakage. The new ZwZ Align comparison also addresses my concern about the bounding-box overlays.

The response addresses my main concerns, and I am raising my rating from 3 to 4.

Thank you sincerely for your acknowledgment of our work :)

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank our reviewers for their time and effort in providing thorough and constructive feedback on our work.


We highlight our contributions as follows:

  • ZoomInc is an automated fine-grained visual data construction pipeline that exploits the global-to-zoom perception gap: it converts incremental details revealed only by zooming into questions about the full image, uses globally unique natural-language referring expressions, verifies answers by multi-model consensus on the zoomed view, filters language-prior shortcuts with blind verification, and constructs multi-source distractors.
  • ZoomInc Bench is a challenging fine-grained visual perception benchmark of 1,000 questions whose questions and answers are entirely model-authored, without labor-intensive human annotation: human reviewers only accept or reject candidates. The error analysis shows that incorrect choices concentrate on distractors mined from full-image errors, revealing confusions shared across models.
  • ZoomInc training data serves as effective RL supervision: with only 9K samples, ZoomInc consistently outperforms ZwZ across all 14 benchmarks on Qwen3.5-4B and Qwen3.5-9B, and the benefits also generalize to Gemma-4B.

As suggested by the reviewers, we revised our manuscript as follows (all revisions are highlighted in blue):

Clarified Introduction:

  • Refined the Introduction to foreground the key limitations of existing methods (costly manual curation and unreliable full-image synthesis; referring ambiguity and red-box binding in crop-based synthesis; lack of blind-shortcut filtering and dedicated distractors design) and how ZoomInc addresses each of them, as suggested by Reviewers zU3h and 87KL.

Additional Results:

  • Added the controlled ZwZ Align baseline for both Qwen3.5-4B and Qwen3.5-9B to Table 3 and Section 6.1, as suggested by Reviewers 7jqX, Bm9H, 87KL, and zU3h.
  • Added a new Section 6.3 and Table 5 validating that caption gaps correspond to actual full-to-zoom QA gaps, as suggested by Reviewers Bm9H and 87KL.
  • Added ZoomInc Bench results under the Full Image Only, Full Image + Crop, and Crop + Local Question settings in Section 5.1 and Appendix G.1, as suggested by Reviewer 7jqX.
  • Added RL results of ZoomInc and ZwZ on Gemma-4-E4B-it in Section 6.1 and Appendix I.1, as suggested by Reviewer zU3h.
  • Added the distractor ablation using only full-image VLM errors in Section 6.2 and Appendix I.2, as suggested by Reviewer 87KL.
  • Added the evaluation of ZwZ-trained models with and without test-time oracle red boxes in Appendix I.3, as suggested by Reviewers 87KL and zU3h.
  • Added concrete examples of language-prior shortcuts in existing fine-grained benchmarks in Appendix D, as suggested by Reviewer zU3h.

Other Modifications:

  • Clarified in Section 4 that all QA pairs associated with ZoomInc Bench images are excluded from the training pool, as suggested by Reviewers 7jqX and 87KL.
  • Clarified the Dual-view column in Table 1: the evidence region is provided as metadata, and models receive only the full image during evaluation, as suggested by Reviewer 7jqX.
  • Added the training settings of ZwZ Align in Appendix H, as suggested by Reviewers 7jqX, Bm9H, 87KL, and zU3h.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.