acceptodds
Under review as a conference paper at ICLR 2027

Exact Visual Grounding Strengthens Visual Information Utilization in Vision-Language Models

Abstract

Modern vision-language models (VLMs) achieve strong semantic reasoning yet remain unreliable on questions requiring precise visual evidence, such as exact counts, distances, or geometric relations. Broad multimodal pre-training provides rich semantic supervision, but rarely requires models to ground their predictions in exact visual evidence, leaving room for semantic and language priors to dominate when precise perception is required. To address this limitation, we introduce Exact Visual Grounding (EVG), a framework that provides explicit supervision for grounding answers in precisely controlled visual properties. EVG adopts an answer-first construction: it first samples a target answer, then renders an image whose relevant visual property exactly realizes that answer, and finally generates the corresponding question. This yields annotation-free supervision across 12 Color, Counting, and Geometric Measurement tasks, together with matched missing-object examples. Empirically, we find that training on these tasks consistently improves downstream multimodal performance. For example, on Qwen3.5-9B, EVG raises the macro-average across nine benchmarks from 60.06% to 63.90%, with similar gains across different model scales and backbones. To understand how EVG alters visual information processing, module grafting shows that the learned changes are distributed across both the vision and language stacks, while linear probing finds that task-relevant visual properties become more linearly accessible in middle-to-late language layers. Together, these results show that exact visual grounding can complement multimodal training by strengthening how VLMs extract and utilize fine-grained visual evidence.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision-language models, precise visual perception, synthetic data, visual grounding

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_mZ1MOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Exact Visual Grounding (EVG), an answer-first pipeline for synthesizing perception-centric VQA data. For each task and difficulty level, it samples a target answer, renders a synthetic scene whose queried property exactly realizes that answer, and then fills in a question template. The result is annotation-free supervision across 12 Color, Counting, and Geometric Measurement tasks, plus matched missing-object negatives. Models are fully fine-tuned with a curriculum, and checkpoints are selected on a held-out Vero-600K panel. On Qwen3.5-9B, EVG raises a nine-benchmark macro-average from 60.06 to 63.90, and gains are also reported for Qwen3.5-4B, Gemma-4-12B, and the agentic DeepEyesV2-7B. Module grafting and layer-wise linear probing are used to argue that the changes span both the vision and language stacks and make visual properties more linearly accessible in middle-to-late LM layers.

Critical strengths
  • Exact, cheap, and controllable supervision. Labels are fixed by the rendering parameters, so the pipeline produces noise-free supervision without annotators or upstream detectors. The calibrated answer distributions, randomized nuisance factors, matched missing-object negatives, and explicit validity checks show care in closing obvious shortcuts. The full recipe runs in about 10 hours on one GPU and ships with the generator, so it is easy to extend and reproduce.
  • Targets a real and well-documented failure mode. The method directly addresses VLMs’ tendency to answer from priors rather than from pixels. The largest gains appear where that tendency is exposed: VLMs Are Biased (+10.1) and VLMs Are Blind (+5.4). The qualitative cases, such as the added star on the New Zealand flag and the 10-column Sudoku grid, show a real shift from memorized defaults to reading the image.
  • Broad empirical coverage with useful controls. Results span two Qwen scales, a second backbone, and an agentic model, where the higher crop-tool rate is an interesting behavioral side effect. The paper also includes task-family, curriculum, and missing-ratio ablations, a same-recipe re-implementation of Rizzoli et al., and the explicit-visual-question subset analysis, which together go beyond headline numbers.
Critical weaknesses
  • Novelty and the scope of “transfer” are overstated. Answer-first sampling is essentially standard procedural generation with a calibrated answer prior, and closely related work is missing. Euclid trains purely on synthetic geometric data with multi-stage curriculum training and reports generalization to novel shapes. VisOnlyQA releases 70k synthetic training instances and finds that fine-tuning gains are confined to certain tasks and models. In addition, the largest gains come from VLMs Are Blind and VLMs Are Biased, whose items (line intersections, grid rows/columns, counting stars or columns) closely mirror EVG’s Counting templates, so these results are better described as near-domain than as downstream transfer.
  • Base models appear to be evaluated far from their operating point. The reported base scores for Qwen3.5-9B (50.78 MMMU-Val, 37.69 MMMU-Pro) and Qwen3.5-4B (47.11, 33.64) are roughly 28–33 points below the model card’s figures of 78.4/70.1 and 77.6/66.3 on MMMU/MMMU-Pro, and Qwen3.5 runs in thinking mode by default. Qwen3.5-4B also scores only 1.44% on VLMs Are Biased. EVG trains on short, format-constrained answers, so part of the gain may come from fixing formatting, response length, or answer parsing under this harness rather than from better perception. Whether the gains survive under the recommended inference settings is central to the claim and currently unknown.
  • The mechanistic analyses do not support the causal story. The probes are trained on the same 12 tasks used for fine-tuning, at the last prompt token right before the answer. Higher late-layer decodability is therefore largely expected after supervising those answers, and it says little about why unrelated benchmarks improve. No control tasks are reported despite citing Hewitt & Liang, and a 20-iteration probe conflates accessibility with ease of optimization. The two analyses also conflict: grafting places the LM contribution mainly in layers 0–7 and almost none in layers 24–31 (86.89 vs. 86.50), while probing places the divergence after layer 16, and neither this tension nor the non-additive stack grafts (3.43 + 2.50 > 5.37) is addressed.
Questions for authors

N/A

6 comments
AuthorsOpenReview ↗

Authors' Rebuttal (1/4)

We thank the reviewer for their valuable feedback.


W1. Novelty and the scope of "transfer" are overstated. Answer-first sampling is essentially standard procedural generation with a calibrated answer prior, and closely related work is missing. Euclid trains purely on synthetic geometric data with multi-stage curriculum training and reports generalization to novel shapes. VisOnlyQA releases 70k synthetic training instances and finds that fine-tuning gains are confined to certain tasks and models. In addition, the largest gains come from VLMs Are Blind and VLMs Are Biased, whose items (line intersections, grid rows/columns, counting stars or columns) closely mirror EVG's Counting templates, so these results are better described as near-domain than as downstream transfer.

A1. We thank the reviewer for raising concerns about the scope. We do not claim that answer-first sampling alone is a new procedural-generation primitive. Our contribution is the EVG framework, which combines exact target realization, calibrated answer distributions, 12 complementary perception tasks, three difficulty levels, matched missing-object cases, and curriculum training. We will clarify this scope and expand the related-work discussion to include Euclid [1], which studies synthetic geometric supervision and curriculum learning, and VisOnlyQA [2], which provides 70K synthetic examples for geometric perception.

We also add three equal-budget SFT controls using VisOnlyQA [2], STARE [3], and Zebra-CoT [4], with the same Qwen3.5-9B backbone and training recipe used for our method except for the difficulty curriculum. We evaluate all checkpoints on the held-out EVG test set and the same nine VQA benchmarks.

We additionally include a controlled Euclid [1] baseline on the nine VQA benchmarks, using the same backbone and optimization setup.

Table 1: Results on the held-out EVG test set. EM denotes exact-match accuracy, and MRE denotes mean relative error. All values are percentages.

Training dataD1 EMD2 EMD3 EMMean EMNumeric MRE ↓
Base40.7029.2020.5030.1342.13
VisOnlyQA39.5827.3315.6727.5331.98
STARE39.7525.5015.6726.9730.61
Zebra-CoT36.1723.7513.7524.5634.02
EVG56.4048.6038.6047.8710.67

Table 2: Results on the nine VQA benchmarks. Macro denotes the unweighted average across the nine benchmarks. All values are percentages.

Training dataMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro
Base50.7869.2284.9186.2862.3847.9114.8386.5037.6960.06
Euclid55.4455.0460.5284.0851.6210.112.6685.1040.3549.44
VisOnlyQA56.7869.9587.3786.8067.2551.2124.9383.3544.3963.56
STARE57.1171.2187.0887.6867.7954.2921.9882.5743.8263.73
Zebra-CoT58.7870.5988.1286.0068.3454.5122.2781.8744.2863.86
EVG56.8971.3787.0586.9665.6649.2324.9391.8741.1663.90

Notably, Euclid exhibits a clear degradation in general-purpose visual-language performance after fine-tuning, consistent with the Euclid paper's observation that its geometry-focused training can induce out-of-domain forgetting. This limitation likely arises from Euclid's narrowly synthetic, geometry-centric data construction, which has limited semantic and visual diversity. In contrast, our EVG encompasses diverse colors, shapes, distractors and visual layouts, which significantly increases the visual diversity.

As shown in Tables 1 and 2, our method retains the highest macro-average across the nine VQA benchmarks and outperforms the VisOnlyQA, STARE, and Zebra-CoT controls by a large margin on the held-out EVG test set. These three controls achieve comparable general VQA performance. These datasets emphasize different capabilities: VisOnlyQA uses coarse labels for geometric perception, STARE focuses on spatial transformations and visual simulations, and Zebra-CoT covers broad visual reasoning over puzzles, diagrams, planning, and strategy games. EVG directly provides exact supervision for fine-grained visual properties across color, counting, and geometric measurement.

This distinction is reflected in precise-perception performance. Relative to Base, fine-tuning on VisOnlyQA, STARE, and Zebra-CoT changes mean exact-match accuracy on the held-out EVG test set by -2.60, -3.16, and -5.57 percentage points, respectively, whereas our method improves it by 17.73 percentage points. Fine-tuning on the three alternative datasets also reduces accuracy on VLMs Are Blind from the Base score of 86.50% to 81.87%-83.35%, whereas our method increases it to 91.87%. Thus, improvements in broad VQA performance do not by themselves reproduce the precise-perception gains obtained with our method.

AuthorsOpenReview ↗

Authors' Rebuttal (2/4)

(A1 continued)

To further examine whether these precise-perception benefits complement broader vision-centric training, we conduct an additional experiment with Zebra-CoT. We train sequentially on Zebra-CoT and EVG, allocating half of the total training steps to each dataset while keeping the total training-step budget the same as the Zebra-CoT baseline. We denote this sequential training setup as Z+E.

Table 3: Results on the held-out EVG test set for Zebra-CoT and Z+E.

Training dataD1 EMD2 EMD3 EMMean EMNumeric MRE ↓
Zebra-CoT36.1723.7513.7524.5634.02
Z+E61.5032.7520.0038.0818.53

Table 4: Results on the nine VQA benchmarks for Zebra-CoT and Z+E.

Training dataMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro
Zebra-CoT58.7870.5988.1286.0068.3454.5122.2781.8744.2863.86
Z+E59.2271.9988.2887.4467.7555.1625.3686.1444.5765.10

At the same total training-step budget, Z+E raises the nine-benchmark macro-average from 63.86% to 65.10% (+1.24 percentage points), improving eight of the nine benchmarks over Zebra-CoT alone. On the held-out EVG test set, mean exact-match accuracy increases from 24.56% to 38.08%, while numerical MRE decreases from 34.02% to 18.53%. Notably, VLMs Are Blind accuracy recovers from 81.87% to 86.14%, substantially mitigating the decline after Zebra-CoT training and bringing performance close to the Base score of 86.50%.

These results highlight EVG's distinct contribution to precise visual perception and show that this supervision can also complement Zebra-CoT training.

Regarding transfer, VLMs Are Blind and VLMs Are Biased evaluate fine-grained visual perception and reliance on visual evidence, which are central to EVG's training objectives. Improvements on these independently constructed benchmarks show that the targeted capabilities extend beyond our generated training tasks. Moreover, neither benchmark is limited to counting. VLMs Are Blind also tests overlap and contact, circled-letter recognition, path connectivity, and nested spatial structure. VLMs Are Biased additionally covers equality and attribute identification across optical illusions, flags, logos, animals, chess pieces, game boards, and patterned grids. Improvement also remains after excluding both benchmarks: EVG raises the macro-average over the other seven benchmarks from 62.74% to 65.47% (+2.74 pp). We will therefore state the main conclusion more precisely: EVG primarily strengthens exact low-level visual perception, and this targeted capability also benefits broader VQA tasks.

[1] Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions. arXiv 2024.

[2] VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information. COLM 2025.

[3] Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations. ICLR 2026.

[4] Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning. ICLR 2026.

AuthorsOpenReview ↗

Authors' Rebuttal (3/4)

W2. Base models appear to be evaluated far from their operating point. The reported base scores for Qwen3.5-9B (50.78 MMMU-Val, 37.69 MMMU-Pro) and Qwen3.5-4B (47.11, 33.64) are roughly 28–33 points below the model card's figures of 78.4/70.1 and 77.6/66.3 on MMMU/MMMU-Pro, and Qwen3.5 runs in thinking mode by default. Qwen3.5-4B also scores only 1.44% on VLMs Are Biased. EVG trains on short, format-constrained answers, so part of the gain may come from fixing formatting, response length, or answer parsing under this harness rather than from better perception. Whether the gains survive under the recommended inference settings is central to the claim and currently unknown.

A2. We thank the reviewer for pointing out the discrepancy between our reported scores and the model card. Our original evaluations use the default configurations of the standard VLM evaluation platform LMMs-Eval, where the MMMU/MMMU-Pro tasks are evaluated in non-thinking mode, and the models are required to directly output the answers. Both Base and EVG achieve a 100% answer-parsing success rate on MMMU-Val and MMMU-Pro under these settings. Therefore, the accuracy gains cannot be explained by a reduction in answer-parsing failures. Although the complete evaluation settings behind the model-card scores were unavailable to us, we obtained comparable baseline results using thinking mode, greedy decoding, and a maximum output budget of 32,768 tokens. Below, we report Qwen3.5-9B results under these settings on MMMU/MMMU-Pro and BLINK-Twice, a reasoning benchmark focused on visual perception [1]:

Reasoning BenchmarkBaseEVG
MMMU-Val78.3378.33
MMMU-Pro71.3372.37
BLINK-Twice41.4943.60

As shown by the results, the gains are indeed smaller in thinking mode than in the non-thinking settings. However, EVG improves accuracy on MMMU-Pro and BLINK-Twice despite being trained exclusively on non-thinking SFT data. These results suggest that improvements in exact visual perception can also benefit reasoning.

Moreover, frontier performance on MMMU is approaching the reported expert reference: the 2026 AI Index reports 88.2% accuracy for the leading model, within 0.4 percentage points of the best human expert reference, implying limited space for further improvement [2]. Prior work [1] also shows that reasoning can support visual perception: such inference-time reasoning may partly compensate for perceptual weaknesses, helping explain the smaller additional gains from EVG in thinking mode.

Non-thinking inference also remains important for applications with tight latency and token budgets. Qwen3-VL explicitly supports both thinking and non-thinking variants to address different application requirements [3]. Furthermore, MME-CoT finds that chain-of-thought prompting can reduce accuracy on perception-heavy tasks through overthinking, reinforcing the value of evaluating direct-answer performance [4]. Therefore, the larger gains observed with EVG in non-thinking mode are practically relevant and directly support our central claim that exact visual supervision strengthens the extraction of fine-grained visual evidence. At the same time, the positive results in thinking mode show that this benefit is not limited to direct-answer inference.

In the revised manuscript, we will include these thinking-mode results and the corresponding analysis.

[1] BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception. NeurIPS 2025.

[2] The 2026 AI Index Report. Stanford HAI 2026.

[3] Qwen3-VL Technical Report. arXiv 2025.

[4] MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency. ICML 2025.

Reviewer_xGabOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

1.The authors introduce EVG data, a synthetically generated VQA dataset that includes tasks such as counting, color recognition, and ... 2.They show that training/fine-tuning the Qwen 3.5 9B model on this data leads to an improvement of around 3% on visual benchmarks, and around 6% for the 4B model. 3.They run experiments to understand the contribution of each module of the model to the improvements. 4.They use a three-stage training process, starting with easier tasks and finishing with harder ones. 5. They show that this curriculum learning outperforms randomized ordering by about 2%.

Critical strengths

The paper includes many ablations, interpretability analyses, and experiments. They show improvements on 9 SOTA visual benchmarks by training on EVG data. Generating EVG data is straightforward and cheap.

Critical weaknesses

1.They only compare their data with Rizzoli et al., and not with other general VQA datasets of the same size, such as Zebra-CoT, or synthetic spatial data such as STARE (“Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations”). These papers, particularly STARE, also include VQAs involving objects, counting, colors and....

  1. The agentic improvement on V* is less than 1%, but the paper’s claims about it sound overstated.

  2. Why is the increase in tool-call rate considered a positive outcome in the paper? A higher tool-call rate increases cost and latency if the calls are unnecessary and do not yield significant improvement. The paper does not show examples where additional cropping was actually helpful.

  3. There are no ablations on the data size. Does increasing data size lead to better models?

  4. All experiments use the same seed. Have you tried different seeds?

Questions for authors

Please address the weaknesses listed above.

The improvement gets smaller as the models get larger (within the Qwen family). Have you tried larger Qwen models?

Do different seeds lead to the same percentage improvements? V* improvement is about 1 percent that could be noise.

The paper only does SFT, since EVG has exact answers, have you tried RL with verifiable reward instead?

8 comments
AuthorsOpenReview ↗

Authors' Rebuttal (1/4)

We thank the reviewer for their valuable feedback.


W1. They only compare their data with Rizzoli et al., and not with other general VQA datasets of the same size, such as Zebra-CoT, or synthetic spatial data such as STARE ("Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations"). These papers, particularly STARE, also include VQAs involving objects, counting, colors and....

A1. We thank the reviewer for this helpful advice. To test whether the gains can instead be explained by generic additional SFT, we add three equal-budget SFT controls using VisOnlyQA [1], STARE [2], and Zebra-CoT [3], with the same Qwen3.5-9B backbone and training recipe used for our method except for the difficulty curriculum. We evaluate all checkpoints on the held-out EVG test set and the same nine VQA benchmarks.

Table 1: Results on the held-out EVG test set. EM denotes exact-match accuracy, and MRE denotes mean relative error. All values are percentages.

Training dataD1 EMD2 EMD3 EMMean EMNumeric MRE ↓
Base40.7029.2020.5030.1342.13
VisOnlyQA39.5827.3315.6727.5331.98
STARE39.7525.5015.6726.9730.61
Zebra-CoT36.1723.7513.7524.5634.02
EVG56.4048.6038.6047.8710.67

Table 2: Results on the nine VQA benchmarks. Macro denotes the unweighted average across the nine benchmarks. All values are percentages.

Training dataMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro
Base50.7869.2284.9186.2862.3847.9114.8386.5037.6960.06
VisOnlyQA56.7869.9587.3786.8067.2551.2124.9383.3544.3963.56
STARE57.1171.2187.0887.6867.7954.2921.9882.5743.8263.73
Zebra-CoT58.7870.5988.1286.0068.3454.5122.2781.8744.2863.86
EVG56.8971.3787.0586.9665.6649.2324.9391.8741.1663.90

As shown in Tables 1 and 2, our method retains the highest macro-average across the nine VQA benchmarks and outperforms the three alternative training approaches by a large margin on the held-out EVG test set. The alternatives achieve comparable general VQA performance. These datasets emphasize different capabilities: VisOnlyQA uses coarse labels for geometric perception, STARE focuses on spatial transformations and visual simulations, and Zebra-CoT covers broad visual reasoning over puzzles, diagrams, planning, and strategy games. EVG directly provides exact supervision for fine-grained visual properties across color, counting, and geometric measurement.

This distinction is reflected in precise-perception performance. Relative to Base, fine-tuning on VisOnlyQA, STARE, and Zebra-CoT changes mean exact-match accuracy on the held-out EVG test set by -2.60, -3.16, and -5.57 percentage points, respectively, whereas our method improves it by 17.73 percentage points. Fine-tuning on the three alternative datasets also reduces accuracy on VLMs Are Blind from the Base score of 86.50% to 81.87%-83.35%, whereas our method increases it to 91.87%. Thus, improvements in broad VQA performance do not by themselves reproduce the precise-perception gains obtained with our method.

AuthorsOpenReview ↗

Authors' Rebuttal (2/4)

(A1 continued)

To further examine whether these precise-perception benefits complement broader vision-centric training, we conduct an additional experiment with Zebra-CoT. We train sequentially on Zebra-CoT and EVG, allocating half of the total training steps to each dataset while keeping the total training-step budget the same as the Zebra-CoT baseline. We denote this sequential training setup as Z+E.

Table 3: Results on the held-out EVG test set for Zebra-CoT and Z+E.

Training dataD1 EMD2 EMD3 EMMean EMNumeric MRE ↓
Zebra-CoT36.1723.7513.7524.5634.02
Z+E61.5032.7520.0038.0818.53

Table 4: Results on the nine VQA benchmarks for Zebra-CoT and Z+E.

Training dataMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro
Zebra-CoT58.7870.5988.1286.0068.3454.5122.2781.8744.2863.86
Z+E59.2271.9988.2887.4467.7555.1625.3686.1444.5765.10

At the same total training-step budget, Z+E raises the nine-benchmark macro-average from 63.86% to 65.10% (+1.24 percentage points), improving eight of the nine benchmarks over Zebra-CoT alone. On the held-out EVG test set, mean exact-match accuracy increases from 24.56% to 38.08%, while numerical MRE decreases from 34.02% to 18.53%. Notably, VLMs Are Blind accuracy recovers from 81.87% to 86.14%, substantially mitigating the decline after Zebra-CoT training and bringing performance close to the Base score of 86.50%.

These results highlight EVG's distinct contribution to precise visual perception and show that this supervision can also complement Zebra-CoT training.

[1] VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information. COLM 2025.

[2] Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations. ICLR 2026.

[3] Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning. ICLR 2026.


W2. The agentic improvement on V* is less than 1%, but the paper's claims about it sound overstated.

A2. We thank the reviewer for this comment. This experiment examines whether exact visual grounding provides complementary benefits when a model already supports active visual inspection. DeepEyesV2 can acquire additional visual evidence through cropping and zooming, allowing us to evaluate whether EVG remains useful in this setting.

Adding EVG improves HRBench-4K from 71.75% to 74.00% and V*-Bench from 81.68% to 82.20%. We acknowledge that the gain on V*-Bench is modest. These results suggest that precise visual grounding may remain complementary to existing agentic visual inspection capabilities. We will make this experimental motivation and the scope of the conclusion more explicit in the revised manuscript.


W3. Why is the increase in tool-call rate considered a positive outcome in the paper? A higher tool-call rate increases cost and latency if the calls are unnecessary and do not yield significant improvement. The paper does not show examples where additional cropping was actually helpful.

A3. We thank the reviewer for this question. In V*-Bench and HRBench-4K, answering correctly often requires identifying fine-grained details in high-resolution images. Cropping allows the model to inspect local regions that may be difficult to resolve from the overall image. Our reported metric measures the frequency of valid, executable crop requests. Its increase indicates that the model more actively seeks additional visual evidence, which aligns with EVG's objective of strengthening reliance on precise visual information. Together with the accompanying accuracy gains, we view this as an encouraging sign of more active visual inspection.

We will rename "Tool Rate" to "Valid Crop Rate," define it explicitly, and describe the finding as increased visual evidence acquisition accompanying improved task accuracy. In practical deployment, the preferred cropping frequency depends on the application's accuracy, cost, and latency requirements.

AuthorsOpenReview ↗

Authors' Rebuttal (3/4)

W4. There are no ablations on the data size. Does increasing data size lead to better models?

A4. We thank the reviewer for raising the role of data scale. We add an ablation evaluating performance as the number of EVG training steps and training examples increases.

Training StepsTraining ExamplesMacroMean EMNumeric MRE ↓
1016061.2728.9432.52
2032061.4430.8629.39
3048062.9433.6126.18
4064063.7734.0623.11
5080063.6935.9421.34
1001,60064.3436.7220.41
1502,40064.4739.6716.18
2003,20064.3740.3914.69
2504,00064.5841.2814.44
3004,80064.6540.6116.20

Performance improves rapidly at smaller data sizes and then approaches a plateau as more examples are used. This pattern appears in both the nine-benchmark macro-average and mean exact-match accuracy on the held-out EVG test set.

For checkpoint selection, we use the held-out general-VQA panel sampled from Vero-600K to monitor whether the model retains general multimodal capability while learning from the targeted exact-perception supervision. Rather than selecting the checkpoint after the maximum possible number of EVG updates, we select the checkpoint with the best score on this panel. The panel is used only for early stopping and checkpoint selection. Under this protocol, increasing the nominal dataset size does not necessarily increase the amount of data actually used for training, because training stops after two consecutive decreases in validation accuracy. This selection balances improvements in exact perception with the preservation of general multimodal capability.


W5. All experiments use the same seed. Have you tried different seeds?

A5. We thank the reviewer for the advice regarding multiple seeds. We carried out 3 more experiments on Qwen3.5-9B with seeds 43, 44 and 45, using the same recipe as before. Here are the results on the 9 benchmarks:

BenchmarkBaseEVG (paper)EVG seeds 43/44/45 (mean±SD)
MMMU-Val50.7856.8958.00±0.58
MMStar69.2271.3770.82±0.84
AI2D84.9187.0587.49±0.10
ChartQA86.2886.9687.43±0.29
BLINK62.3865.6666.94±0.30
HallusionBench47.9149.2351.43±0.76
VLMs Are Biased14.8324.9325.56±1.51
VLMs Are Blind86.5091.8788.74±0.75
MMMU-Pro37.6941.1643.45±0.55
Macro-average60.0663.9064.43±0.64

Across the three training seeds, EVG obtains a macro-average of 64.43±0.64, compared with 60.06 for Base. Importantly, the macro-average remains above Base for every individual seed, with the least favorable seed still improving over Base by 3.55 pp. The three-seed mean is also close to the paper-reported EVG result, showing that the overall improvement does not disappear when the training seed changes. In the revised manuscript, we will add the multi-seed results.


Q1. The improvement gets smaller as the models get larger (within the Qwen family). Have you tried larger Qwen models?

AQ1. Our conclusion is that EVG remains effective across the evaluated model scales, while the magnitude of improvement can vary with model size. The original paper included only Qwen3.5-4B and Qwen3.5-9B, which is insufficient to establish a scaling law.

We have now extended EVG to Qwen3.8-27B. The new result does not follow a monotonic decrease: the nine-benchmark gain is +3.98 pp at 27B, slightly larger than the +3.84 pp gain at 9B.

Qwen modelBase macroEVG macroΔ (pp)
Qwen3.5-4B54.9760.55+5.58
Qwen3.5-9B60.0663.90+3.84
Qwen3.8-27B62.4666.43+3.98

On an independently generated held-out EVG set, the 27B model also shows substantial improvements at all three difficulty levels:

ModelD1D2D3Mean
Base47.6731.0019.7532.81
EVG62.5051.6741.6751.94
Gain (pp)+14.83+20.67+21.92+19.14

The mean relative error on numerical tasks decreases from 23.00% to 8.58%.

The downstream gains are also consistent across all nine benchmarks:

ModelMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro average
Base58.6770.1686.1486.9666.3053.8517.8981.6240.5262.46
EVG60.3370.8286.4387.6068.8454.5135.7086.9146.7666.43
Gain (pp)+1.67+0.66+0.29+0.64+2.54+0.66+17.82+5.29+6.24+3.98

We will revise the paper to state that EVG benefits all evaluated Qwen scales, while the gain magnitude is model- and benchmark-dependent. The 27B results show that EVG's effectiveness persists at a substantially larger scale and does not exhibit a monotonic decline with model size.

Reviewer_K5LWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EVG, a framework that constructs 12 categories of synthetic visual training tasks by first determining the answer and subsequently generating the corresponding image. The authors employ standard SFT with a curriculum learning strategy to enhance the capabilities of color, counting, and geometric information perception. Experiments across multiple models and downstream benchmarks demonstrate that the approach improves multimodal performance.

Critical strengths
  1. The paper proposes a simple and scalable method for constructing synthetic data, capable of generating visual training samples with precise ground truth.
  2. EVG improves performance across multiple downstream benchmarks for Qwen3.5-9B. It demonstrates effective generalization across several VLMs, indicating that the benefits of the approach extend well beyond the specific synthetic training tasks themselves.
  3. Beyond conducting ablation studies on task categories and curriculum learning, the paper employs techniques such as module grafting and linear probing to analyze internal model changes. These analyses provide valuable evidence for understanding how precise visual supervision influences the representation and utilization of visual information.
Critical weaknesses
  1. The paper lacks a comparison with other synthetic visual data generation methods.
  2. The experiments have not yet evaluated the method’s effectiveness on models with larger parameter. The largest model tested is 12B, which remains unclear whether EVG can yield similar benefits for larger-scale models, such as Qwen3.8-27B, Gemma-4-31B, and Muse-Glimmer-30B.
  3. Although EVG ensures the numerical accuracy of answers through programmatic generation, the paper has not sufficiently verified whether these precise values ​​remain consistently discernible given actual image resolutions and the visual encoding process. It raises questions regarding the true efficacy of these precise supervisory signals.
  4. While EVG improves overall downstream performance, the gains on certain tasks are marginal. Performance on ChartQA increased by only 0.68 percentage points.
Questions for authors

Refer to Critical Weaknesses.

6 comments
AuthorsOpenReview ↗

Authors' Rebuttal (1/3)

We thank the reviewer for their valuable feedback.


W1. The paper lacks a comparison with other synthetic visual data generation methods.

A1. We thank the reviewer for raising this helpful question. We would like to first clarify that we compare EVG with the method of Rizzoli et al. in Appendix I. Here we also add three equal-budget SFT controls using VisOnlyQA [1], STARE [2], and Zebra-CoT [3], with the same Qwen3.5-9B backbone and training recipe used for our method except for the difficulty curriculum. We evaluate all checkpoints on the held-out EVG test set and the same nine VQA benchmarks.

Table 1: Results on the held-out EVG test set. EM denotes exact-match accuracy, and MRE denotes mean relative error. All values are percentages.

Training dataD1 EMD2 EMD3 EMMean EMNumeric MRE ↓
Base40.7029.2020.5030.1342.13
VisOnlyQA39.5827.3315.6727.5331.98
STARE39.7525.5015.6726.9730.61
Zebra-CoT36.1723.7513.7524.5634.02
EVG56.4048.6038.6047.8710.67

Table 2: Results on the nine VQA benchmarks. Macro denotes the unweighted average across the nine benchmarks. All values are percentages.

Training dataMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro
Base50.7869.2284.9186.2862.3847.9114.8386.5037.6960.06
VisOnlyQA56.7869.9587.3786.8067.2551.2124.9383.3544.3963.56
STARE57.1171.2187.0887.6867.7954.2921.9882.5743.8263.73
Zebra-CoT58.7870.5988.1286.0068.3454.5122.2781.8744.2863.86
EVG56.8971.3787.0586.9665.6649.2324.9391.8741.1663.90

As shown in Tables 1 and 2, our method retains the highest macro-average across the nine VQA benchmarks and outperforms the three alternative training approaches by a large margin on the held-out EVG test set. The alternatives achieve comparable general VQA performance. These datasets emphasize different capabilities: VisOnlyQA uses coarse labels for geometric perception, STARE focuses on spatial transformations and visual simulations, and Zebra-CoT covers broad visual reasoning over puzzles, diagrams, planning, and strategy games. EVG directly provides exact supervision for fine-grained visual properties across color, counting, and geometric measurement.

This distinction is reflected in precise-perception performance. Relative to Base, fine-tuning on VisOnlyQA, STARE, and Zebra-CoT changes mean exact-match accuracy on the held-out EVG test set by -2.60, -3.16, and -5.57 percentage points, respectively, whereas our method improves it by 17.73 percentage points. Fine-tuning on the three alternative datasets also reduces accuracy on VLMs Are Blind from the Base score of 86.50% to 81.87%-83.35%, whereas our method increases it to 91.87%. Thus, improvements in broad VQA performance do not by themselves reproduce the precise-perception gains obtained with our method.

AuthorsOpenReview ↗

Authors' Rebuttal (2/3)

(A1 continued)

To further examine whether these precise-perception benefits complement broader vision-centric training, we conduct an additional experiment with Zebra-CoT. We train sequentially on Zebra-CoT and EVG, allocating half of the total training steps to each dataset while keeping the total training-step budget the same as the Zebra-CoT baseline. We denote this sequential training setup as Z+E.

Table 3: Results on the held-out EVG test set for Zebra-CoT and Z+E.

Training dataD1 EMD2 EMD3 EMMean EMNumeric MRE ↓
Zebra-CoT36.1723.7513.7524.5634.02
Z+E61.5032.7520.0038.0818.53

Table 4: Results on the nine VQA benchmarks for Zebra-CoT and Z+E.

Training dataMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro
Zebra-CoT58.7870.5988.1286.0068.3454.5122.2781.8744.2863.86
Z+E59.2271.9988.2887.4467.7555.1625.3686.1444.5765.10

At the same total training-step budget, Z+E raises the nine-benchmark macro-average from 63.86% to 65.10% (+1.24 percentage points), improving eight of the nine benchmarks over Zebra-CoT alone. On the held-out EVG test set, mean exact-match accuracy increases from 24.56% to 38.08%, while numerical MRE decreases from 34.02% to 18.53%. Notably, VLMs Are Blind accuracy recovers from 81.87% to 86.14%, substantially mitigating the decline after Zebra-CoT training and bringing performance close to the Base score of 86.50%.

These results highlight EVG's distinct contribution to precise visual perception and show that this supervision can also complement Zebra-CoT training.

[1] VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information. COLM 2025.

[2] Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations. ICLR 2026.

[3] Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning. ICLR 2026.


W2. The experiments have not yet evaluated the method's effectiveness on models with larger parameter. The largest model tested is 12B, which remains unclear whether EVG can yield similar benefits for larger-scale models, such as Qwen3.8-27B, Gemma-4-31B, and Muse-Glimmer-30B.

A2. We agree that evaluation at a larger scale is important. We therefore extended EVG to Qwen3.8-27B and evaluated its held-out EVG accuracy and downstream performance on nine multimodal benchmarks.

On an independently generated held-out EVG set containing 3,600 examples across 12 tasks and three difficulty levels, EVG consistently improves exact-match accuracy:

Held-out EVGBaseEVGΔ (pp)
D147.6762.50+14.83
D231.0051.67+20.67
D319.7541.67+21.92
Mean32.8151.94+19.14

The mean relative error on numerical tasks also decreases from 23.00% to 8.58%. The improvement therefore extends beyond output formatting and reflects substantially more accurate visual measurements.

We further evaluate both checkpoints on the same nine downstream benchmarks:

BenchmarkBaseEVGΔ (pp)
MMMU-Val58.6760.33+1.67
MMStar70.1670.82+0.66
AI2D86.1486.43+0.29
ChartQA86.9687.60+0.64
BLINK66.3068.84+2.54
HallusionBench53.8554.51+0.66
VLMs Are Biased17.8935.70+17.82
VLMs Are Blind81.6286.91+5.29
MMMU-Pro40.5246.76+6.24
Macro average62.4666.43+3.98

EVG improves all nine benchmarks and raises the macro average by 3.98 pp. Thus, its benefits persist at the 27B scale, improving both exact visual grounding and general downstream multimodal performance. In the revised manuscript, we will extend the evaluation to larger models.

AuthorsOpenReview ↗

Authors' Rebuttal (3/3)

W3. Although EVG ensures the numerical accuracy of answers through programmatic generation, the paper has not sufficiently verified whether these precise values remain consistently discernible given actual image resolutions and the visual encoding process. It raises questions regarding the true efficacy of these precise supervisory signals.

A3. We thank the reviewer for raising the distinction between numerical correctness and perceptual recoverability. In EVG, "exact" refers to ground-truth construction, while the required answer precision is explicitly controlled for each task. For example, angles between rays are sampled at 30°/15°/5° increments for D1/D2/D3; rectangle and ellipse ratios use one-decimal targets, with integer-pixel dimensions checked for consistency at that precision. Normalized positions and lengths use increments of 1 on a 0–100 scale or 10 on a 0–1000 scale, both corresponding to 1% of the image width. Thus, these tasks use discrete target values with specified granularity.

The generator also incorporates rendering and geometric safeguards, including 2× supersampling with antialiasing, foreground-background contrast checks, and task-specific separation constraints. Detailed measures used to reduce rendering ambiguity can be found in the code provided in the supplementary material.

Regarding supervisory efficacy, our existing held-out evaluation provides complementary evidence: at D3, the macro-average numerical relative error decreases from 44.5% to 12.0% after EVG training (Table 10), including reductions from 85.7% to 2.8% for point position and from 32.3% to 7.5% for rectangle aspect ratio. These results support the practical usefulness of the supervision under the evaluated pipeline. We will make the per-task quantization explicit and clarify the distinction between exact label construction and perceptual recoverability.


W4. While EVG improves overall downstream performance, the gains on certain tasks are marginal. Performance on ChartQA increased by only 0.68 percentage points.

A4. We thank the reviewer for noting the modest improvement on ChartQA. VQA benchmarks typically assess a combination of capabilities. For example, ChartQA requires not only fine-grained visual perception but also text recognition, chart interpretation, numerical computation, and multi-step reasoning. Improving visual perception alone therefore does not necessarily lead to equally large gains in overall task accuracy.

EVG specifically targets the extraction and utilization of precise visual evidence through supervision on color, counting, and geometric measurement. Consistent with this focus, EVG yields larger improvements on benchmarks that directly test fine-grained perception and reliance on visual evidence, including VLMs Are Blind (+5.37 percentage points) and VLMs Are Biased (+10.10 percentage points). On benchmarks that combine perception with other capabilities, such as ChartQA, the improvement is more modest. We will clarify that EVG's main contribution is to strengthen fine-grained visual perception, and this targeted improvement can also benefit broader VQA tasks, with gains varying across benchmarks according to the capabilities they assess.

Reviewer_RqkcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Exact Visual Grounding (EVG) to improve precise visual perception in VLMs. EVG generates synthetic training data using an answer-first approach, covering 12 tasks involving color, counting, and geometry. The method improves performance across multiple benchmarks and model backbones. The authors also use module grafting and linear probing to investigate how EVG affects visual information processing.

Critical strengths
  1. The paper addresses an important limitation of current VLMs: their difficulty in utilizing precise visual evidence rather than relying on semantic priors.
  2. The answer-first data generation approach is simple, scalable, and provides accurate supervision without manual annotation.
  3. The improvements across different models and benchmarks are encouraging. The additional representation analyses also provide useful insights beyond benchmark performance.
Critical weaknesses
  1. The novelty over existing synthetic visual reasoning training is somewhat limited. It is unclear whether the gains come specifically from exact grounding or simply from additional synthetic training data.
  2. The generalization from synthetic images to complex real-world scenes needs stronger validation.
  3. The mechanistic analysis is not fully convincing. Better linear probe performance does not necessarily imply that the model actually uses the encoded visual information when making predictions.
Questions for authors

I do not have questions to ask to the authors (all addressed in the weaknesses)

7 comments
AuthorsOpenReview ↗

Authors' Rebuttal (1/3)

We thank the reviewer for their valuable feedback.


W1. The novelty over existing synthetic visual reasoning training is somewhat limited. It is unclear whether the gains come specifically from exact grounding or simply from additional synthetic training data.

A1. We thank the reviewer and agree that procedural generation and synthetic visual-reasoning supervision are established ideas. We will revise the paper to make clear that our novelty claim is not synthetic rendering itself. EVG's intended contribution is more concrete: a target-first framework that samples an exact answer before rendering and programmatically realizes it across 12 complementary low-level visual tasks, together with calibrated answer distributions, three difficulty levels, matched missing-object cases, and curriculum training. We will expand the related-work discussion and position EVG relative to prior synthetic-training approaches without claiming that synthetic visual reasoning training itself is new.

To test whether the gains can instead be explained by generic additional SFT, we add three equal-budget SFT controls using VisOnlyQA [1], STARE [2], and Zebra-CoT [3], with the same Qwen3.5-9B backbone and training recipe used for our method except for the difficulty curriculum. We evaluate all checkpoints on the held-out EVG test set and the same nine VQA benchmarks.

Table 1: Results on the held-out EVG test set. EM denotes exact-match accuracy, and MRE denotes mean relative error. All values are percentages.

Training dataD1 EMD2 EMD3 EMMean EMNumeric MRE ↓
Base40.7029.2020.5030.1342.13
VisOnlyQA39.5827.3315.6727.5331.98
STARE39.7525.5015.6726.9730.61
Zebra-CoT36.1723.7513.7524.5634.02
EVG56.4048.6038.6047.8710.67

Table 2: Results on the nine VQA benchmarks. Macro denotes the unweighted average across the nine benchmarks. All values are percentages.

Training dataMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro
Base50.7869.2284.9186.2862.3847.9114.8386.5037.6960.06
VisOnlyQA56.7869.9587.3786.8067.2551.2124.9383.3544.3963.56
STARE57.1171.2187.0887.6867.7954.2921.9882.5743.8263.73
Zebra-CoT58.7870.5988.1286.0068.3454.5122.2781.8744.2863.86
EVG56.8971.3787.0586.9665.6649.2324.9391.8741.1663.90

As shown in Tables 1 and 2, our method retains the highest macro-average across the nine VQA benchmarks and outperforms the three alternative training approaches by a large margin on the held-out EVG test set. The alternatives achieve comparable general VQA performance. These datasets emphasize different capabilities: VisOnlyQA uses coarse labels for geometric perception, STARE focuses on spatial transformations and visual simulations, and Zebra-CoT covers broad visual reasoning over puzzles, diagrams, planning, and strategy games. EVG directly provides exact supervision for fine-grained visual properties across color, counting, and geometric measurement.

This distinction is reflected in precise-perception performance. Relative to Base, fine-tuning on VisOnlyQA, STARE, and Zebra-CoT changes mean exact-match accuracy on the held-out EVG test set by -2.60, -3.16, and -5.57 percentage points, respectively, whereas our method improves it by 17.73 percentage points. Fine-tuning on the three alternative datasets also reduces accuracy on VLMs Are Blind from the Base score of 86.50% to 81.87%-83.35%, whereas our method increases it to 91.87%. Thus, improvements in broad VQA performance do not by themselves reproduce the precise-perception gains obtained with our method.

AuthorsOpenReview ↗

Authors' Rebuttal (2/3)

(A1 continued)

To further examine whether these precise-perception benefits complement broader vision-centric training, we conduct an additional experiment with Zebra-CoT. We train sequentially on Zebra-CoT and EVG, allocating half of the total training steps to each dataset while keeping the total training-step budget the same as the Zebra-CoT baseline. We denote this sequential training setup as Z+E.

Table 3: Results on the held-out EVG test set for Zebra-CoT and Z+E.

Training dataD1 EMD2 EMD3 EMMean EMNumeric MRE ↓
Zebra-CoT36.1723.7513.7524.5634.02
Z+E61.5032.7520.0038.0818.53

Table 4: Results on the nine VQA benchmarks for Zebra-CoT and Z+E.

Training dataMMMU-ValMMStarAI2DChartQABLINKHallusionBenchVLMs Are BiasedVLMs Are BlindMMMU-ProMacro
Zebra-CoT58.7870.5988.1286.0068.3454.5122.2781.8744.2863.86
Z+E59.2271.9988.2887.4467.7555.1625.3686.1444.5765.10

At the same total training-step budget, Z+E raises the nine-benchmark macro-average from 63.86% to 65.10% (+1.24 percentage points), improving eight of the nine benchmarks over Zebra-CoT alone. On the held-out EVG test set, mean exact-match accuracy increases from 24.56% to 38.08%, while numerical MRE decreases from 34.02% to 18.53%. Notably, VLMs Are Blind accuracy recovers from 81.87% to 86.14%, substantially mitigating the decline after Zebra-CoT training and bringing performance close to the Base score of 86.50%.

These results highlight EVG's distinct contribution to precise visual perception and show that this supervision can also complement Zebra-CoT training.

[1] VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Information. COLM 2025.

[2] Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations. ICLR 2026.

[3] Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning. ICLR 2026.


W2. The generalization from synthetic images to complex real-world scenes needs stronger validation.

A2. We thank the reviewer for highlighting the importance of real-world settings. We would like to clarify that the benchmarks evaluated in the paper already include real-world imagery. In particular, BLINK includes natural-image tasks such as relative depth estimation and visual correspondence, while HRBench-4K and V*-Bench evaluate fine-grained attribute recognition and spatial relations in high-resolution real-world scenes.

To further evaluate transfer to real-world scenes, we add RealWorldQA, which tests spatial understanding using real-world images, including photographs captured from vehicles:

ModelRealWorldQA accuracy
Qwen3.5-9B Base75.69
EVG76.99

We acknowledge that the real-world gains are modest. Nevertheless, EVG fine-tuning uses only synthetic images, so the improvements provide evidence that exact visual supervision can benefit perception beyond the generated scenes. We will clarify that our results support modest transfer to the evaluated real-world benchmarks in the revised manuscript.

AuthorsOpenReview ↗

Authors' Rebuttal (3/3)

W3. The mechanistic analysis is not fully convincing. Better linear probe performance does not necessarily imply that the model actually uses the encoded visual information when making predictions.

A3. We thank the reviewer for raising this distinction. We would like to clarify that the paper does not make a causal claim: we do not claim that a property being decodable by a linear probe implies that the model causally uses that property during answer generation. Our probe measures fixed-budget linear accessibility in hidden states, and we will clarify that the probing and behavioral analyses are observational in the revised manuscript.

To make this representation-level interpretation more rigorous, we introduce task-specific control tasks [1]. A high probe score can reflect the probe fitting patterns in a finite probing dataset, rather than recovering the intended visual concept from the representation. For each target task, we therefore train the same probe, with the same hidden states, split, layer, classifier, and fixed optimization budget, to predict a matched competing visual property (for example, a distractor's visual property that the question does not ask about). We report selectivity, defined as the difference between target and control accuracy. If the target remains easier to decode than the control under this matched protocol, the result is less likely to reflect the probe simply learning a generic visual prediction task. We also include no-image controls to test whether the effect depends on visual input rather than question wording or task priors.

This analysis is useful even though it does not establish causal decoder use. It asks whether EVG changes what is represented in the model's hidden states. Under the same probing setup as in the paper, EVG has higher full-image selectivity than Base for line length (0.367 vs. 0.254), angle (0.484 vs. 0.230), rectangle aspect ratio (0.317 vs. 0.132), and ellipse ratio (0.177 vs. 0.065). These advantages become much smaller or disappear in the no-image condition. Our original probing results already show higher target-task accuracy for EVG on 11 of the 12 tasks at the final layer. The additional selectivity and no-image results strengthen confidence in interpreting these gains as improved access to task-relevant visual information.

However, we acknowledge that the probing analysis has an inherent limitation: it reads each property from a single hidden state using a fixed probe. This setup is less suitable for relational information such as relative rotation, which depends on multiple objects and may be distributed across their representations. A weak relative-rotation probe therefore does not show that EVG fails to represent this relation. It shows only that this relation is not readily recovered by the probe used here. Similar limitations have been reported in recent VLM probing work [2]. We therefore limit our conclusion to visual properties that are readily decoded under the probing setup used in the paper.

[1] Designing and Interpreting Probes with Control Tasks. EMNLP 2019.

[2] Probing Visual Concepts in Lightweight Vision-Language Models for Automated Driving. TMLR 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

General Response

Dear Reviewers, AC, SAC, and PC,

Thank you for your time and effort in reviewing and discussing our paper. We have posted point-by-point responses and follow-ups to the reviewers and summarize the main points below.


Paper Strengths

[Strengths]: The reviewers recognize that the paper addresses an important limitation of current VLMs: their difficulty in extracting precise visual evidence rather than relying on semantic priors (Rqkc, mZ1M). They find EVG simple, scalable, inexpensive, and controllable, with exact supervision generated without manual annotation (Rqkc, K5LW, xGab, mZ1M). They also view the improvements across multiple models and benchmarks as encouraging and highlight the extensive ablations and representation analyses as useful evidence beyond aggregate benchmark performance (Rqkc, K5LW, xGab, mZ1M).


Additional Experiments for Concerns

Our primary objective is to strengthen exact visual perception, with broader VQA improvements as an observed benefit rather than a claim of outperforming broad visual-reasoning datasets. We conduct the following experiments.

  • [Controlled Synthetic-Data Comparisons and Complementarity] (Rqkc, K5LW, xGab, mZ1M): Under an equal-budget setup, we compare EVG with VisOnlyQA, STARE, and Zebra-CoT. EVG achieves 47.87% mean exact-match accuracy on the held-out EVG set, compared with 24.56%-27.53% for the alternatives, while retaining the highest nine-benchmark Macro. At the same total training-step budget, combining Zebra-CoT with EVG raises Macro from 63.86% to 65.10% and improves eight of nine benchmarks, showing that the two datasets strengthen complementary aspects of VLM performance.

  • [Larger-Scale Model] (K5LW, xGab): On Qwen3.8-27B, EVG raises held-out mean exact-match accuracy from 32.81% to 51.94%, reduces Numeric MRE from 23.00% to 8.58%, and improves all nine downstream benchmarks, increasing Macro from 62.46% to 66.43%. The benefit therefore persists at the largest evaluated scale, while its magnitude remains model- and benchmark-dependent.

  • [Training Exposure and Robustness] (xGab): A sweep from 160 to 4,800 training examples shows rapid initial improvement followed by a plateau. Across three additional training seeds, EVG obtains a Macro of 64.43 ± 0.64, compared with 60.06 for the base model, and every seed improves over the base model.

  • [Inference Settings and Answer Parsing] (mZ1M): Under the original non-thinking evaluation, both the base and EVG models achieve 100% answer-parsing success on MMMU-Val and MMMU-Pro, ruling out reduced parsing failures as the source of the gains. Under thinking-mode inference, EVG matches the base model on MMMU-Val and improves MMMU-Pro from 71.33% to 72.37% and BLINK-Twice from 41.49% to 43.60%, showing that the benefit is not limited to direct-answer inference.

  • [Probing Controls and Robustness] (Rqkc, mZ1M): We introduce matched task-specific controls and report selectivity, defined as target accuracy minus control accuracy. EVG has higher selectivity than the base model on all four evaluated properties. Increasing the probe optimization budget from 20 to 100 or 2,000 iterations does not change the main conclusions shown in the paper. We also clarify that probing measures linear accessibility rather than causal use during answer generation.

  • [Real-Image and Agentic Evaluation] (Rqkc, xGab): We clarify that the existing evaluation already includes natural and high-resolution real-world images in BLINK, HRBench-4K, and V*-Bench. On the additional RealWorldQA evaluation, EVG improves accuracy from 75.69% to 76.99% despite training only on synthetic images. The agentic results further show that exact visual supervision remains beneficial when the model can actively crop and zoom.


Revision During Rebuttal

  • [Introduction and Related Work]: We clarify that the contribution is the complete EVG framework rather than synthetic rendering or answer-first sampling alone, and expand the discussion of related synthetic-training methods.

  • [Sec. 3 and Generated Task Specification]: We clarify the task-specific answer granularity and the rendering and geometric safeguards used to keep the target values visually discernible.

  • [Sec. 4 and Evaluation Appendices]: We add the controlled synthetic-data comparisons, Zebra-CoT complementarity experiment, Qwen3.8-27B evaluation, training-exposure ablation, multi-seed results, and reasoning-oriented inference results.

  • [Sec. 4.4]: We clarify the purpose of the agentic evaluation, rename "Tool Rate" to "Valid Crop Rate," and limit the conclusion to the observed accuracy and visual-inspection behavior.

  • [Sec. 5 and the Probing Appendix]: We clarify the interpretation of probing, add matched control tasks and selectivity results, and report robustness to larger probe optimization budgets.

All changes are highlighted in blue in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.