- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses perception under insufficient evidence, where a single glance at the image does not suffice to output a box, mask or count, and formulates perception as an agentic evidence-seeking process. Two data generation pipelines are introduced: (i) evidence present in the image but not resolvable at a glance (tiny targets in high-resolution images, I-spy and spot-the-difference scenes), and (ii) evidence beyond the image (real or GPT-Image-2-synthesized group photographs of public figures and anime characters, with queries rewritten into multi-hop form). They yield two training sets (EviRover-SFT-5K, EviRover-RL-12K) and EviLens, a human-verified benchmark of 688 instances covering localization, recognition, spot-the-difference, segmentation and counting. EviRover (Qwen3-VL-4B) is trained by SFT on trajectories distilled from proprietary models, then by agentic RL (EMA-GRPO) with ten tools (web and image search, cropping, SAM3, etc.). It is evaluated on EviLens and WebEyes against proprietary and open-source models, and on conventional perception (ReasonSeg, RefCOCOg) and general multimodal benchmarks against its backbone.
Critical strengths
- Relevant formulation. Treating perception as evidence-seeking addresses a real limitation of one-shot grounding (tiny targets, identities requiring external knowledge). The output can be checked directly against annotations, which gives clean RL rewards.
- Data and benchmark contribution. The paper provides two scalable generation pipelines and EviLens, a human-verified benchmark that fills a gap in existing perception benchmarks. Code, models and data are released.
- Strong gains for a small model. The 4B model improves over its backbone by about 30 points on EviLens and reaches proprietary-level performance on several categories. Gains transfer to WebEyes, without degrading conventional perception benchmarks.
Critical weaknesses
- Evaluation protocol and baselines do not support the main claims.
- Unclear protocol. Table 2 does not state which models are evaluated directly and which with the agentic framework. Do proprietary models get the same ten tools and 35-call budget, with native tools (e.g., built-in web search) disabled? How are refusals to identify real individuals scored?
- Missing open-source agentic baselines. The open-source baselines are non-agentic, so their low scores are expected. Assessing the contribution of training requires the backbone with the same tools but no training, and at least one trained open-source agent (e.g., DeepEyesV2, Mini-o3, MMSearch-R1, WebWatcher or Seg-ReSearch). On WebEyes, baseline numbers are copied from the original paper, possibly with a different tool setup.
- Proprietary comparison. SFT trajectories are distilled from Gemini-3.0-Pro and Seed-2.0-Pro, yet these teachers are absent from Table 2. Since EviRover is trained to imitate them, reaching proprietary-level performance is less surprising than presented, and the teachers should be reported as the natural reference point. In addition, multi-hop queries are rewritten until Seed-1.8 can no longer answer without tools, so the benchmark is calibrated against one of the compared baselines. Finally, the claim that EviRover “surpasses most proprietary models” on spot-the-difference is overstated: it outperforms only 2 of 7 in micro-F1.
- Data construction, contamination and leakage.
- The paper does not explain how instances are allocated between SFT-5K, RL-12K and EviLens, nor whether images, identities or events overlap across them. Since EviLens is generated by the same pipelines as the training data, this is critical to rule out train–test contamination. Gains on EviLens partly reflect in-distribution fit, leaving WebEyes as the only external test.
- For real group photos collected from the web, reverse image search may retrieve the source page and its caption, revealing the answer without visual reasoning. Results should be reported separately for real and synthesized images.
- Several details are missing: the task composition and real/synthetic ratio of each set, how ground-truth boxes and masks are obtained (identity verification yields names, not spatial annotations), and what the manual verification of EviLens consists of.
- Limited novelty and missing positioning.
- The approach combines existing ingredients: SFT plus agentic RL with crop and search tools, close to WebEyes/Pixel-Searcher and Seg-ReSearch. EMA-GRPO is borrowed, and sequence-mean reduction is a known option; the “tempered” variant in Appendix A.4 is never specified.
- On-policy distillation and learning with privileged information are not discussed, although the setting naturally provides privileged information (evidence-centered crops, the original query naming the target, the multi-hop entity chain, ground-truth identities and boxes). Vision-OPD (Yuan et al., NeurIPS 2026) distills a crop-conditioned teacher into a full-image student through on-policy self-distillation, internalizing the benefit of zooming into a single forward pass. This is a direct alternative to agentic exploration for in-image evidence. More generally, on-policy distillation addresses the off-policy mismatch of SFT on proprietary trajectories (acknowledged in Appendix A.1 and handled by heuristic downsampling), while providing dense supervision that complements the sparse outcome reward.
- Insufficient analysis to support the claims.
- Few ablations. Beyond SFT vs. RL, nothing isolates the tools, the reward design, the reduction choice, or real vs. synthetic data.
- Missing analyses. There is no analysis of tool-use behavior, no cost or latency figures (up to 35 calls with live search), and no variance across runs. Live web search also makes results non-reproducible over time.
- Generalization claims. On general multimodal benchmarks (MMMU, MathVerse, BrowseComp-VL), EviRover is compared only with its backbone, so gains could come from any additional SFT+RL stage rather than from evidence-seeking. Whether tools are used at evaluation is also unclear.
Questions for authors
I find the formulation relevant and the gains over the backbone substantial. In the rebuttal, I would mainly expect answers to the following:
- Evaluation protocol and baselines: clarify the evaluation setting of each model in Table 2, and report results for the backbone with tools but no training, a trained open-source agent, and the teacher models (see Weakness 1).
- Data construction and contamination: how instances are allocated between SFT-5K, RL-12K and EviLens, and whether images, identities or events overlap across them.
- Leakage through reverse image search: results on EviLens reported separately for real and synthesized images.
- Positioning: what does EviRover bring beyond search-based perception approaches (WebEyes/Pixel-Searcher, Seg-ReSearch) and on-policy distillation with privileged information (Vision-OPD) for in-image evidence?
10 comments
W1.2 Missing open-source agentic baselines. The open-source baselines are non-agentic, so their low scores are expected. Assessing the contribution of training requires (1)the backbone with the same tools but no training, and (2)at least one trained open-source agent (e.g., DeepEyesV2, Mini-o3, MMSearch-R1, WebWatcher or Seg-ReSearch). (3)On WebEyes, baseline numbers are copied from the original paper, possibly with a different tool setup. Q1. Report results for the backbone with tools but no training, a trained open-source agent.
A1.2. Thank you for your comment.
(1) We attempted to evaluate the untrained Qwen3-VL-4B-Instruct backbone using the EviRover agentic harness, but it failed to use the tools and scored zero across all five tasks under this protocol. We therefore report its direct-answering results from Table 2 below, with the inference protocol explicitly indicated for each model.
| Model | Protocol | Localization IoU / [email protected] | Recognition IoU / [email protected] | Spot Diff micro-F1 / macro-F1 | Segmentation gIoU / cIoU | Counting Acc. (%) |
|---|---|---|---|---|---|---|
| Qwen3-VL-4B-Instruct | Direct | 0.276 / 0.251 | 0.265 / 0.231 | 0.005 / 0.008 | 0.203 / 0.279 | 19.2 |
| EviRover | Agentic | 0.444 / 0.500 | 0.648 / 0.687 | 0.171 / 0.266 | 0.686 / 0.708 | 50.0 |
(2) We additionally evaluated five trained open-source agents on EviLens using their respective tool protocols. Seg-ReSearch was not included because its trained model checkpoint has not been publicly released. Since these models are trained with specific tool-use interfaces, we retain their original tool implementations and inference protocols to preserve their intended tool-use behavior. All models are evaluated using the same EviLens tasks and scoring metrics.
| Model | Localization IoU / [email protected] | Recognition IoU / [email protected] | Spot Diff micro-F1 / macro-F1 | Segmentation gIoU / cIoU | Counting accuracy |
|---|---|---|---|---|---|
| Mini-o3-7B | 0.078 / 0.007 | 0.075 / 0.044 | 0.000 / 0.000 | 0.015 / 0.073 | 9.6 |
| DeepEyes-7B | 0.095 / 0.064 | 0.391 / 0.390 | 0.082 / 0.107 | 0.258 / 0.206 | 28.2 |
| DeepEyesV2-7B | 0.027 / 0.000 | 0.152 / 0.110 | 0.000 / 0.000 | 0.113 / 0.185 | 23.7 |
| MMSearch-R1-7B | 0.036 / 0.036 | 0.000 / 0.000 | 0.023 / 0.029 | 0.000 / 0.000 | 31.4 |
| WebWatcher-32B | 0.027 / 0.014 | 0.221 / 0.225 | 0.048 / 0.061 | 0.127 / 0.343 | 48.2 |
These agents are primarily trained to produce text-based final answers rather than perception predictions such as bounding boxes and segmentation masks, which may contribute to their limited performance on EviLens. In contrast, counting is a VQA-style task more closely aligned with their training objectives. Nevertheless, our 4B EviRover achieves the highest counting accuracy (50.0%) among these five agentic baselines, including substantially larger models.
(3) For WebEyes, the baseline results in our manuscript were taken from the original WebEyes paper. To address potential differences in tool configurations, we conducted additional evaluations under two workflows. Specifically, Pixel-Searcher is a training-free workflow originally evaluated with Qwen3-VL-8B-Instruct. We retain this reference configuration and evaluate both Qwen3-VL-8B-Instruct and EviRover model under the Pixel-Searcher workflow and the EviRover harness.
| Model | Workflow | Grounding IoU / [email protected] | Segmentation gIoU / cIoU | VQA accuracy |
|---|---|---|---|---|
| Qwen3-VL-8B-Instruct | Pixel-Searcher | 30.7 / 33.5 | 34.0 / 29.3 | 43.5 |
| EviRover | Pixel-Searcher | 34.2 / 38.0 | 44.0 / 31.8 | 41.0 |
| Qwen3-VL-8B-Instruct | EviRover harness | 32.3 / 38.0 | 34.7 / 30.3 | 40.2 |
| EviRover | EviRover harness | 40.0 / 48.5 | 49.5 / 38.0 | 40.3 |
Pixel-Searcher uses a preset workflow for retrieval, candidate-box generation, and final selection, whereas the EviRover harness allows the model to select its tool calls. Within each workflow, both models use the same tool implementations. These new results provide matched-workflow comparisons under explicitly specified tool configurations, enabling a more direct assessment of model performance across the two workflows.
W1.3 Proprietary comparison. SFT trajectories are distilled from (1)Gemini-3.0-Pro and Seed-2.0-Pro, yet these teachers are absent from Table 2. Since EviRover is trained to imitate them, reaching proprietary-level performance is less surprising than presented, and the teachers should be reported as the natural reference point. In addition, multi-hop queries are rewritten until Seed-1.8 can no longer answer without tools, so (2)the benchmark is calibrated against one of the compared baselines. Finally, the claim that EviRover “surpasses most proprietary models” on spot-the-difference is overstated: it outperforms only 2 of 7 in micro-F1. Q1. Report results for the teacher models.
A1.3 Thank you for your comment.
(1) As suggested, we have added the teacher models (Gemini-3.0-Pro and Seed-2.0-Pro) to Table 2. Below is the updated comparison on EviLens:
| Model | Localization IoU / [email protected] | Recognition IoU / [email protected] | Spot Diff micro-F1 / macro-F1 | Segmentation gIoU / cIoU | Counting accuracy |
|---|---|---|---|---|---|
| Gemini-3.0-Pro | 0.441 / 0.531 | 0.652 / 0.692 | 0.306 / 0.368 | 0.744 / 0.737 | 64.7 |
| Seed-2.0-Pro | 0.401 / 0.441 | 0.649 / 0.681 | 0.250 / 0.345 | 0.679 / 0.647 | 52.6 |
| EviRover | 0.444 / 0.500 | 0.648 / 0.687 | 0.171 / 0.266 | 0.686 / 0.708 | 50.0 |
Despite its smaller size, EviRover achieves performance close to its teachers on localization, recognition, and segmentation. Moreover, as shown in A1.2, EviRover substantially improves upon the directly evaluated backbone across all five task categories and outperforms the five evaluated trained open-source agents on EviLens. These results support the effectiveness of our training approach.
(2) The Seed-1.8 stopping criterion operates on a different task from the benchmark evaluation. During query construction, Seed-1.8 receives only the text of the multi-hop query and is asked to identify the target by name. In contrast, EviLens evaluates image-grounded perception, where models must locate or segment the target in an image and are scored using bounding-box or mask IoU. Thus, the stopping criterion is based on text-only identity recognition, rather than Seed-1.8's performance on the actual perception tasks. Given its role in benchmark construction, we will exclude Seed-1.8 from the main baseline comparison. We will also update the spot-the-difference performance statement to reflect the revised comparison results.
A1.1. Thank you for your comment.
(1) The evaluation protocols for the Table 2 rows are:
(2)All native tool capabilities, including built-in web search, were disabled for proprietary models. This use of proprietary models as direct baselines follows prior agentic visual-search evaluations [1–4]. The agentic rows used the EviRover harness and its 35-call limit. We also tried agentic inference for Seg-R1-7B, Seg-Zero-7B, Perception-R1, UniVG-R1, OneThinker-8B, and Qwen3-VL-4B-Instruct. However, these models failed to use the provided tools, and we therefore report their direct-inference results.
(3) A refusal that supplies no valid prediction remains in the evaluation set and receives a zero score for that item; for a box prediction, this means IoU = 0 and [email protected] = 0. In the proprietary-model runs, we found only one explicit refusal to identify a real person by name (GPT-5.6-Terra).
[1] Feng K, Zhang M, Chen S, et al. Gen-searcher: Reinforcing agentic search for image generation[J]. arXiv preprint arXiv:2603.28767, 2026.
[2] Zheng Z, Yang M, Hong J, et al. Deepeyes: Incentivizing "thinking with images" via reinforcement learning[C]//International Conference on Learning Representations. 2026.
[3] Lai X, Li J, Li W, et al. Mini-o3: Scaling up reasoning patterns and interaction turns for visual search[C]//International Conference on Learning Representations. 2026.
[4] Yang B, Sun X, Feng K, et al. From Web to Pixels: Bringing Agentic Search into Visual Perception[J]. arXiv preprint arXiv:2605.12497, 2026.