- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces EvidenceConnect, which uses cropped region views to generate visual question-answering data requiring evidence composition across multiple regions, and filters samples through cross-model answer verification and region-removal tests. The authors use approximately 20.4K examples to train three sizes of Qwen models with GRPO, producing PerceptVLM. Experiments cover 26 benchmarks across six task categories, with improvements over the corresponding base models at all three scales. Despite using only single-image training data, the models also improve on multi-image long-context understanding and agent tasks.
Critical strengths
- The data construction pipeline is fairly complete, and the authors carefully consider answer reliability, dependence on multi-region evidence, and referential ambiguity. Cross-model answer verification and region-removal tests can improve data quality, making the method practically useful.
- The evaluation is broad, with improvements at all three model scales. Training only on single-image data also improves multi-image long-context understanding and agent tasks, showing good cross-task transfer.
Critical weaknesses
- Although the authors explain that colored outlines reduce referential ambiguity, they do not report the proportions of marked and unmarked training examples or directly evaluate the effect of region markers on downstream performance. Moreover, comparisons with other single-image datasets provide indirect evidence for multi-region supervision, but differences in data quality and question distributions remain confounding factors; controlled ablations within the proposed data construction pipeline would better distinguish the contributions of region markers and multi-region evidence composition.
- The paper primarily compares against three general-purpose models, without representative strong multimodal or domain-specialized models, and external baseline results are missing on some benchmarks, leaving its claims of frontier-level competitiveness insufficiently supported.
Questions for authors
- What proportions of the training samples have region markers and have no markers? Downstream evaluations usually do not provide these markers. Could this difference affect generalization? Have the authors compared performance after training with and without region markers, keeping other training conditions the same?
- Comparisons with other datasets also involve differences in data quality and question types, making it difficult to determine how much of the improvement comes from multi-region evidence composition. Could the authors compare single-region and multi-region supervision within their own data construction pipeline, keeping the source images, verification procedures, data size, and training budget as consistent as possible?
- What was the basis for selecting these three external reference models? Could the authors provide results for stronger general-purpose multimodal models or relevant domain-specialized models under the same or comparable evaluation conditions to support the paper’s claims of frontier-level competitiveness?
8 comments
Response to Reviewer i7Ls (2-1)
Q2. The paper primarily compares against three general-purpose models, without representative strong multimodal or domain-specialized models, and external baseline results are missing on some benchmarks, leaving its claims of frontier-level competitiveness insufficiently supported.
A2. Thank you for the advice. Evaluating external models across our suite of 25 benchmarks spanning six task categories incurs substantial API and inference costs, which constrained both the number of models and the completeness of their evaluation in the original submission. We therefore initially prioritized three general-purpose multimodal models to provide references across these diverse tasks. To address your concern, we have now expanded the model comparisons and completed the previously missing results for the original reference models. We have also revised the competitiveness claims to specify the models, tasks, and evaluation settings supported by these results.
(1) Broader external model comparisons.
We initially selected general-purpose multimodal models because our evaluation spans six task categories. DeepSeek-V4.1-Flash and GLM-5.3-Flash provide references from two open-model families, while GPT-5.6-Luna provides a cost-effective proprietary reference that enabled broad evaluation coverage. This selection prioritized coverage and diversity of model families. To broaden the comparison, we have added MiMo-V2.6-Pro, Step5, and Claude-Haiku-5.5 (closed-source). In the expanded comparison, PerceptVLM remains competitive, achieving the highest scores among the reported configurations on 7 of the 15 benchmarks. PerceptVLM-27B achieves a mean score of 54.72, exceeding MiMo-V2.6-Pro (high: 47.30) and Step5 (low: 43.21; high: 41.96). Notably, PerceptVLM-27B exceeds the strongest external result in this table by 5.23 points on VStar, 4.47 points on MindCube-Tiny, and 4.13 points on ViewSpatial-Bench. These results have now been incorporated into the revised manuscript, summarized in Section 4.2 and detailed in Appendix A.5 (Table 8).
| Benchmark | MiMo-V2.6-Pro high | Step5 low | Step5 high | Haiku-5.5 low | Haiku-5.5 high | PerceptVLM-4B | PerceptVLM-9B | PerceptVLM-27B |
|---|---|---|---|---|---|---|---|---|
| Chartography | 5.00 | 9.00 | 10.00 | 35.00 | 37.00 | 14.00 | 13.00 | 12.00 |
| VStar | 80.10 | 82.20 | 82.72 | 86.39 | 85.86 | 89.00 | 90.58 | 91.62 |
| BabyVision | 26.55 | 29.64 | 27.32 | 34.54 | 42.01 | 18.56 | 21.39 | 28.35 |
| O3-Bench | 72.46 | 69.57 | 71.30 | 75.07 | 81.45 | 63.77 | 67.83 | 72.46 |
| TreeBench | 57.78 | 55.80 | 60.25 | 60.74 | 62.47 | 51.36 | 51.60 | 56.05 |
| MMStar | 77.07 | 78.40 | 78.33 | 75.93 | 76.93 | 76.27 | 78.27 | 79.07 |
| ERQA | 51.75 | 47.75 | 44.50 | 62.25 | 59.25 | 52.00 | 55.50 | 61.75 |
| MMSI-Bench | 25.90 | 16.80 | 12.30 | 41.60 | 44.10 | 32.30 | 33.90 | 37.10 |
| MindCube-Tiny | 25.24 | 24.57 | 19.05 | 50.29 | 39.43 | 53.05 | 52.57 | 54.76 |
| ZoomBench | 52.54 | 51.24 | 49.82 | 58.46 | 59.53 | 58.22 | 59.53 | 61.78 |
| CountQA | 53.40 | 36.19 | 34.88 | 67.34 | 69.70 | 47.19 | 48.43 | 54.32 |
| Perception-Bench | 41.70 | 40.80 | 40.33 | 54.83 | 57.03 | 37.80 | 38.89 | 44.93 |
| SPAR-Bench | 47.47 | 14.96 | 11.69 | 48.60 | 47.20 | 45.69 | 47.23 | 50.78 |
| ViewSpatial-Bench | 38.53 | 34.17 | 30.65 | 46.50 | 46.38 | 45.26 | 45.27 | 50.63 |
| MIMIC | 53.98 | 57.13 | 56.33 | 55.93 | 56.50 | 66.10 | 64.98 | 65.16 |
| Avg | 47.30 | 43.21 | 41.96 | 56.90 | 57.66 | 50.04 | 51.26 | 54.72 |
Response to Reviewer i7Ls (2-2)
(2) Comparison with a domain-specialized training approach.
We would like to clarify that our goal is to identify and strengthen a shared underlying capability, connecting visual evidence, that supports multiple task domains. Domain-specific data synthesis can be effective, but developing separate synthesis and verification procedures for each domain can require substantial additional effort. EvidenceConnect targets this shared capability through a unified, single-image data synthesis pipeline, aiming to improve performance across domains without separately curating training data for each one.
To assess this approach against domain-specialized supervision, Section 4.3 (Table 2) has already compared EvidenceConnect with ZwZ [1], which targets fine-grained visual perception through supervision grounded in individual regions. Using the same Qwen3.5-4B backbone and GRPO recipe, training on 20.7K EvidenceConnect examples achieves a mean score of 53.86 across 11 benchmarks, compared with 49.86 when training on 73K ZwZ examples, a 4.00-point improvement with fewer training examples. These results support our premise that explicitly training models to connect evidence across regions provides an effective route to broad performance gains, even when compared with supervision specifically designed for fine-grained perception.
(3) Completion of the original baseline results and scope of conclusions.
We have filled in the previously missing entries for the three original reference models in Section 4.2 (Table 1). These completed results, together with the expanded comparisons, provide a broader basis for assessing absolute performance. Separately, comparisons with the corresponding 4B, 9B, and 27B Qwen baselines establish the gains attributable to EvidenceConnect within each backbone.
[1] Wei, Lai, et al. "Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception." ICML 2026.
Response to Reviewer i7Ls (1)
Q1. Although the authors explain that colored outlines reduce referential ambiguity, they do not report the proportions of marked and unmarked training examples or directly evaluate the effect of region markers on downstream performance. Moreover, comparisons with other single-image datasets provide indirect evidence for multi-region supervision, but differences in data quality and question distributions remain confounding factors; controlled ablations within the proposed data construction pipeline would better distinguish the contributions of region markers and multi-region evidence composition.
A1. Thank you for the suggestions. We provide the requested statistics and two controlled ablation studies. Both studies use the same Qwen3.5-4B initialization and GRPO configuration, with 7K training examples for efficiency and two epochs of training.
(1) Region markers.
Among the 20,662 examples used to train the 4B model, 20,624 (99.82%) contain at least one region marker, while 38 (0.18%) use unmarked images. Counting each referenced region once per question, 71,485 of 71,615 evidence regions (99.82%) are marked. As explained in Section 3.3, these markers disambiguate the intended instances when several objects in the full image could match the same description. The model must still perceive and combine the visual evidence within the referenced regions. We have included these in the revised version, as reported in Appendix A.1.
To directly assess the effect of markers, we constructed paired marked and unmarked versions of the same 7K examples for training, preserving the source images, target evidence regions, and underlying tasks. For the unmarked version, we removed the outlines and replaced color-based references in the questions and answer options, where applicable, with natural-language descriptions, followed by answer verification. The unmarked variant achieves a mean score of 52.93, compared with 49.41 for the base model (+3.52 points). The marked variant achieves 54.08, improving over the base model by 4.67 points and over the unmarked variant by 1.15 points. These results show that the training data remain beneficial without markers, while retaining markers during training yields higher downstream performance even though no such markers are added at evaluation time. We attribute this gap to clearer visual references. Colored outlines directly identify the intended objects, whereas descriptions may be ambiguous among similar objects or reveal the queried attribute. Under RLVR, this ambiguity introduces reward noise by penalizing the model for selecting a plausible but unintended region. Markers reduce this confound, focusing training on locating relevant regions and combining their evidence. The gains persist without markers at evaluation, suggesting improved composition ability rather than reliance on outlines. This supports the transferability of marker-based supervision beyond the marked training inputs. We have updated our submission in Section 4.4, with details provided in Appendix A.1 (Table 4).
(2) Multi-region evidence composition.
We constructed single-region and multi-region supervision within our own pipeline, using the same source images, question-generation model, and answer-verification procedure. Both variants use a consistent region-reference format and the matched training settings described above.
In the comparison, single-region supervision achieves a mean score of 52.23, while multi-region supervision achieves 54.08, a gain of 1.86 points, with improvements on 9 of the 12 benchmarks. Because single-region and multi-region supervision necessarily involve different questions, this comparison evaluates the supervision procedure as a whole. With source images, answer verification, data size, and training settings controlled, the results provide direct evidence that requiring the model to compose evidence across regions improves downstream performance over single-region supervision. These have now been incorporated into Section 4.4, as summarized therein and detailed in Appendix A.1 (Table 4).