acceptodds
Withdrawn from ICLR 2027

EviRover: Reinforcing Agentic Perception Beyond a Glance

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual perception is conventionally formulated as a one-shot prediction from a single glance at the image, under the assumption that the image content and the model's parametric knowledge suffice to resolve the query. This assumption often fails in real-world scenarios that hinge on fine-grained visual details or require knowledge-intensive and up-to-date information. We term such cases perception under insufficient evidence and formulate perception as an agentic process that can obtain information beyond a single glance. To address the absence of data for this setting, we design two dedicated data generation pipelines, yielding EviRover-SFT-5K and EviRover-RL-12K for training. We further construct EviLens, a human-verified benchmark comprising 688 instances across five perception categories. Building on these data, we present EviRover, to our knowledge the first perception agent explicitly trained to resolve perceptual queries through interaction, using supervised fine-tuning followed by agentic reinforcement learning. Experiments show that the 4B EviRover outperforms its backbone by 30 points on average on EviLens, reaching performance comparable to advanced proprietary models. The gains transfer beyond EviLens to WebEyes, conventional perception benchmarks, and general multimodal benchmarks, including a 15-point improvement on BrowseComp-VL. All code, models, and data are released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Perception, Agent, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yjS7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses perception under insufficient evidence, where a single glance at the image does not suffice to output a box, mask or count, and formulates perception as an agentic evidence-seeking process. Two data generation pipelines are introduced: (i) evidence present in the image but not resolvable at a glance (tiny targets in high-resolution images, I-spy and spot-the-difference scenes), and (ii) evidence beyond the image (real or GPT-Image-2-synthesized group photographs of public figures and anime characters, with queries rewritten into multi-hop form). They yield two training sets (EviRover-SFT-5K, EviRover-RL-12K) and EviLens, a human-verified benchmark of 688 instances covering localization, recognition, spot-the-difference, segmentation and counting. EviRover (Qwen3-VL-4B) is trained by SFT on trajectories distilled from proprietary models, then by agentic RL (EMA-GRPO) with ten tools (web and image search, cropping, SAM3, etc.). It is evaluated on EviLens and WebEyes against proprietary and open-source models, and on conventional perception (ReasonSeg, RefCOCOg) and general multimodal benchmarks against its backbone.

Critical strengths
  • Relevant formulation. Treating perception as evidence-seeking addresses a real limitation of one-shot grounding (tiny targets, identities requiring external knowledge). The output can be checked directly against annotations, which gives clean RL rewards.
  • Data and benchmark contribution. The paper provides two scalable generation pipelines and EviLens, a human-verified benchmark that fills a gap in existing perception benchmarks. Code, models and data are released.
  • Strong gains for a small model. The 4B model improves over its backbone by about 30 points on EviLens and reaches proprietary-level performance on several categories. Gains transfer to WebEyes, without degrading conventional perception benchmarks.
Critical weaknesses
  • Evaluation protocol and baselines do not support the main claims.
    • Unclear protocol. Table 2 does not state which models are evaluated directly and which with the agentic framework. Do proprietary models get the same ten tools and 35-call budget, with native tools (e.g., built-in web search) disabled? How are refusals to identify real individuals scored?
    • Missing open-source agentic baselines. The open-source baselines are non-agentic, so their low scores are expected. Assessing the contribution of training requires the backbone with the same tools but no training, and at least one trained open-source agent (e.g., DeepEyesV2, Mini-o3, MMSearch-R1, WebWatcher or Seg-ReSearch). On WebEyes, baseline numbers are copied from the original paper, possibly with a different tool setup.
    • Proprietary comparison. SFT trajectories are distilled from Gemini-3.0-Pro and Seed-2.0-Pro, yet these teachers are absent from Table 2. Since EviRover is trained to imitate them, reaching proprietary-level performance is less surprising than presented, and the teachers should be reported as the natural reference point. In addition, multi-hop queries are rewritten until Seed-1.8 can no longer answer without tools, so the benchmark is calibrated against one of the compared baselines. Finally, the claim that EviRover “surpasses most proprietary models” on spot-the-difference is overstated: it outperforms only 2 of 7 in micro-F1.
  • Data construction, contamination and leakage.
    • The paper does not explain how instances are allocated between SFT-5K, RL-12K and EviLens, nor whether images, identities or events overlap across them. Since EviLens is generated by the same pipelines as the training data, this is critical to rule out train–test contamination. Gains on EviLens partly reflect in-distribution fit, leaving WebEyes as the only external test.
    • For real group photos collected from the web, reverse image search may retrieve the source page and its caption, revealing the answer without visual reasoning. Results should be reported separately for real and synthesized images.
    • Several details are missing: the task composition and real/synthetic ratio of each set, how ground-truth boxes and masks are obtained (identity verification yields names, not spatial annotations), and what the manual verification of EviLens consists of.
  • Limited novelty and missing positioning.
    • The approach combines existing ingredients: SFT plus agentic RL with crop and search tools, close to WebEyes/Pixel-Searcher and Seg-ReSearch. EMA-GRPO is borrowed, and sequence-mean reduction is a known option; the “tempered” variant in Appendix A.4 is never specified.
    • On-policy distillation and learning with privileged information are not discussed, although the setting naturally provides privileged information (evidence-centered crops, the original query naming the target, the multi-hop entity chain, ground-truth identities and boxes). Vision-OPD (Yuan et al., NeurIPS 2026) distills a crop-conditioned teacher into a full-image student through on-policy self-distillation, internalizing the benefit of zooming into a single forward pass. This is a direct alternative to agentic exploration for in-image evidence. More generally, on-policy distillation addresses the off-policy mismatch of SFT on proprietary trajectories (acknowledged in Appendix A.1 and handled by heuristic downsampling), while providing dense supervision that complements the sparse outcome reward.
  • Insufficient analysis to support the claims.
    • Few ablations. Beyond SFT vs. RL, nothing isolates the tools, the reward design, the reduction choice, or real vs. synthetic data.
    • Missing analyses. There is no analysis of tool-use behavior, no cost or latency figures (up to 35 calls with live search), and no variance across runs. Live web search also makes results non-reproducible over time.
    • Generalization claims. On general multimodal benchmarks (MMMU, MathVerse, BrowseComp-VL), EviRover is compared only with its backbone, so gains could come from any additional SFT+RL stage rather than from evidence-seeking. Whether tools are used at evaluation is also unclear.
Questions for authors

I find the formulation relevant and the gains over the backbone substantial. In the rebuttal, I would mainly expect answers to the following:

  • Evaluation protocol and baselines: clarify the evaluation setting of each model in Table 2, and report results for the backbone with tools but no training, a trained open-source agent, and the teacher models (see Weakness 1).
  • Data construction and contamination: how instances are allocated between SFT-5K, RL-12K and EviLens, and whether images, identities or events overlap across them.
  • Leakage through reverse image search: results on EviLens reported separately for real and synthesized images.
  • Positioning: what does EviRover bring beyond search-based perception approaches (WebEyes/Pixel-Searcher, Seg-ReSearch) and on-policy distillation with privileged information (Vision-OPD) for in-image evidence?
10 comments

W1.1 Evaluation protocol and baselines do not support the main claims. Unclear protocol. Table 2 does not (1)state which models are evaluated directly and which with the agentic framework. Do (2)proprietary models get the same ten tools and 35-call budget, with native tools (e.g., built-in web search) disabled? How are (3)refusals to identify real individuals scored? Q1. Evaluation protocol and baselines: clarify the evaluation setting of each model in Table 2.

A1.1. Thank you for your comment.

(1) The evaluation protocols for the Table 2 rows are:

ProtocolTable 2 rows
DirectGPT-5.6-Sol, GPT-5.6-Luna, GPT-5.6-Terra, Seed-2.1-Turbo, Seed-1.8, Qwen3.7-Plus, Gemini-3.5-Flash, Seg-R1-7B, Seg-Zero-7B, Perception-R1, UniVG-R1, Minimax-M3, OneThinker-8B, Qwen3-VL-4B-Instruct
AgenticSAM3-Agent, InternVL-3.5-8B, Qwen3-VL-4B-Instruct + SFT, EviRover

(2)All native tool capabilities, including built-in web search, were disabled for proprietary models. This use of proprietary models as direct baselines follows prior agentic visual-search evaluations [1–4]. The agentic rows used the EviRover harness and its 35-call limit. We also tried agentic inference for Seg-R1-7B, Seg-Zero-7B, Perception-R1, UniVG-R1, OneThinker-8B, and Qwen3-VL-4B-Instruct. However, these models failed to use the provided tools, and we therefore report their direct-inference results.

(3) A refusal that supplies no valid prediction remains in the evaluation set and receives a zero score for that item; for a box prediction, this means IoU = 0 and [email protected] = 0. In the proprietary-model runs, we found only one explicit refusal to identify a real person by name (GPT-5.6-Terra).

[1] Feng K, Zhang M, Chen S, et al. Gen-searcher: Reinforcing agentic search for image generation[J]. arXiv preprint arXiv:2603.28767, 2026.

[2] Zheng Z, Yang M, Hong J, et al. Deepeyes: Incentivizing "thinking with images" via reinforcement learning[C]//International Conference on Learning Representations. 2026.

[3] Lai X, Li J, Li W, et al. Mini-o3: Scaling up reasoning patterns and interaction turns for visual search[C]//International Conference on Learning Representations. 2026.

[4] Yang B, Sun X, Feng K, et al. From Web to Pixels: Bringing Agentic Search into Visual Perception[J]. arXiv preprint arXiv:2605.12497, 2026.

W1.2 Missing open-source agentic baselines. The open-source baselines are non-agentic, so their low scores are expected. Assessing the contribution of training requires (1)the backbone with the same tools but no training, and (2)at least one trained open-source agent (e.g., DeepEyesV2, Mini-o3, MMSearch-R1, WebWatcher or Seg-ReSearch). (3)On WebEyes, baseline numbers are copied from the original paper, possibly with a different tool setup. Q1. Report results for the backbone with tools but no training, a trained open-source agent.

A1.2. Thank you for your comment.

(1) We attempted to evaluate the untrained Qwen3-VL-4B-Instruct backbone using the EviRover agentic harness, but it failed to use the tools and scored zero across all five tasks under this protocol. We therefore report its direct-answering results from Table 2 below, with the inference protocol explicitly indicated for each model.

ModelProtocolLocalization IoU / [email protected]Recognition IoU / [email protected]Spot Diff micro-F1 / macro-F1Segmentation gIoU / cIoUCounting Acc. (%)
Qwen3-VL-4B-InstructDirect0.276 / 0.2510.265 / 0.2310.005 / 0.0080.203 / 0.27919.2
EviRoverAgentic0.444 / 0.5000.648 / 0.6870.171 / 0.2660.686 / 0.70850.0

(2) We additionally evaluated five trained open-source agents on EviLens using their respective tool protocols. Seg-ReSearch was not included because its trained model checkpoint has not been publicly released. Since these models are trained with specific tool-use interfaces, we retain their original tool implementations and inference protocols to preserve their intended tool-use behavior. All models are evaluated using the same EviLens tasks and scoring metrics.

ModelLocalization IoU / [email protected]Recognition IoU / [email protected]Spot Diff micro-F1 / macro-F1Segmentation gIoU / cIoUCounting accuracy
Mini-o3-7B0.078 / 0.0070.075 / 0.0440.000 / 0.0000.015 / 0.0739.6
DeepEyes-7B0.095 / 0.0640.391 / 0.3900.082 / 0.1070.258 / 0.20628.2
DeepEyesV2-7B0.027 / 0.0000.152 / 0.1100.000 / 0.0000.113 / 0.18523.7
MMSearch-R1-7B0.036 / 0.0360.000 / 0.0000.023 / 0.0290.000 / 0.00031.4
WebWatcher-32B0.027 / 0.0140.221 / 0.2250.048 / 0.0610.127 / 0.34348.2

These agents are primarily trained to produce text-based final answers rather than perception predictions such as bounding boxes and segmentation masks, which may contribute to their limited performance on EviLens. In contrast, counting is a VQA-style task more closely aligned with their training objectives. Nevertheless, our 4B EviRover achieves the highest counting accuracy (50.0%) among these five agentic baselines, including substantially larger models.

(3) For WebEyes, the baseline results in our manuscript were taken from the original WebEyes paper. To address potential differences in tool configurations, we conducted additional evaluations under two workflows. Specifically, Pixel-Searcher is a training-free workflow originally evaluated with Qwen3-VL-8B-Instruct. We retain this reference configuration and evaluate both Qwen3-VL-8B-Instruct and EviRover model under the Pixel-Searcher workflow and the EviRover harness.

ModelWorkflowGrounding IoU / [email protected]Segmentation gIoU / cIoUVQA accuracy
Qwen3-VL-8B-InstructPixel-Searcher30.7 / 33.534.0 / 29.343.5
EviRoverPixel-Searcher34.2 / 38.044.0 / 31.841.0
Qwen3-VL-8B-InstructEviRover harness32.3 / 38.034.7 / 30.340.2
EviRoverEviRover harness40.0 / 48.549.5 / 38.040.3

Pixel-Searcher uses a preset workflow for retrieval, candidate-box generation, and final selection, whereas the EviRover harness allows the model to select its tool calls. Within each workflow, both models use the same tool implementations. These new results provide matched-workflow comparisons under explicitly specified tool configurations, enabling a more direct assessment of model performance across the two workflows.

W1.3 Proprietary comparison. SFT trajectories are distilled from (1)Gemini-3.0-Pro and Seed-2.0-Pro, yet these teachers are absent from Table 2. Since EviRover is trained to imitate them, reaching proprietary-level performance is less surprising than presented, and the teachers should be reported as the natural reference point. In addition, multi-hop queries are rewritten until Seed-1.8 can no longer answer without tools, so (2)the benchmark is calibrated against one of the compared baselines. Finally, the claim that EviRover “surpasses most proprietary models” on spot-the-difference is overstated: it outperforms only 2 of 7 in micro-F1. Q1. Report results for the teacher models.

A1.3 Thank you for your comment.

(1) As suggested, we have added the teacher models (Gemini-3.0-Pro and Seed-2.0-Pro) to Table 2. Below is the updated comparison on EviLens:

ModelLocalization IoU / [email protected]Recognition IoU / [email protected]Spot Diff micro-F1 / macro-F1Segmentation gIoU / cIoUCounting accuracy
Gemini-3.0-Pro0.441 / 0.5310.652 / 0.6920.306 / 0.3680.744 / 0.73764.7
Seed-2.0-Pro0.401 / 0.4410.649 / 0.6810.250 / 0.3450.679 / 0.64752.6
EviRover0.444 / 0.5000.648 / 0.6870.171 / 0.2660.686 / 0.70850.0

Despite its smaller size, EviRover achieves performance close to its teachers on localization, recognition, and segmentation. Moreover, as shown in A1.2, EviRover substantially improves upon the directly evaluated backbone across all five task categories and outperforms the five evaluated trained open-source agents on EviLens. These results support the effectiveness of our training approach.

(2) The Seed-1.8 stopping criterion operates on a different task from the benchmark evaluation. During query construction, Seed-1.8 receives only the text of the multi-hop query and is asked to identify the target by name. In contrast, EviLens evaluates image-grounded perception, where models must locate or segment the target in an image and are scored using bounding-box or mask IoU. Thus, the stopping criterion is based on text-only identity recognition, rather than Seed-1.8's performance on the actual perception tasks. Given its role in benchmark construction, we will exclude Seed-1.8 from the main baseline comparison. We will also update the spot-the-difference performance statement to reflect the revised comparison results.

Reviewer_wAN2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work examines visual perception when resolving a query requires either closer examination of the image or information unavailable from the model’s internal knowledge. It introduces two data-generation pipelines, the EviRover-SFT-5K and EviRover-RL-12K training sets, and EviLens, a manually verified benchmark with 688 instances spanning five perception categories. Starting from Qwen3-VL-4B-Instruct, EviRover combines supervised fine-tuning with EMA-GRPO and task-dependent outcome rewards. Its tool interface supports visual inspection, external retrieval, and spatial prediction, producing boxes, masks, or counts. Table 2 shows a mean improvement of approximately 30.2 points over the backbone after expressing the five primary metrics on a common 0–100 scale. Further evaluations report gains on WebEyes, standard perception datasets, and broader multimodal benchmarks.

Critical strengths

1. A useful formulation supported by concrete data resources. The paper brings visual inspection and knowledge acquisition into a shared perception setting, with outputs that can be scored against spatial or numerical annotations. The distinction between visually unresolved targets and targets requiring external identity information is well motivated. EviLens extends evaluation across localization, recognition, difference detection, segmentation, and counting. The combination of this benchmark and the two training sets is a substantive contribution.

2. Large reported gains with evidence of transfer. On EviLens, localization IoU increases from 0.276 to 0.444 and segmentation gIoU from 0.203 to 0.686. The RL stage improves all five primary metrics beyond SFT, adding about 7.2 points on their equally weighted average. On WebEyes, EviRover obtains 40.0 grounding IoU and 49.5 segmentation gIoU, compared with 34.1 and 39.1 for Pixel-Searcher-8B. Tables 4–5 also show improvements over the backbone on conventional perception and general multimodal evaluations. Although comparison conditions require clarification, the breadth of these results supports the usefulness of the overall system.

Critical weaknesses

1. The experiments establish system improvements more clearly than they establish the proposed learning mechanism.

Section 3.3 assigns RL the role of learning which evidence is missing and how to obtain it. Table 2 provides a useful SFT-to-RL comparison, but approximately 23.0 of the overall 30.2 points are already obtained after SFT; the remaining 7.2 points do not by themselves identify what behavior RL changes. This distinction matters because Appendix A.5.1 already gives detailed guidance on when to crop, search, verify, and compare regions. Figure 4 describes the SFT data’s tool-use composition, and Figures 6–7 illustrate training-set trajectories; neither measures how the learned policy changes on held-out queries. Matched-tool evaluations of the backbone, SFT checkpoint, and RL checkpoint, together with targeted removal of visual inspection or external retrieval, would make the central interpretation substantially more convincing.

The paper does specify a common prompt, output conventions, and a 35-call limit for agentic methods in Section 4.1. However, Table 2 does not explicitly identify which rows use the direct or agentic protocol. There is also an ambiguity between direct segmentation from a predicted box and EviRover’s box-plus-point interface with iterative mask verification. A per-model configuration table would clarify whether the gains reflect training, additional tool access, or different prediction interfaces. Appendix A.5.2 further states that the browsing tool uses Qwen3.5-27B for summarization. Thus, the 4B designation describes the controller, while system-level resource comparisons should also account for the auxiliary model and actual tool usage. Since WebEyes baseline scores are imported from the original paper, their alignment with EviRover’s tool and inference settings also remains unclear.

2. EviLens needs stronger documentation of independence, annotation quality, and statistical uncertainty.

Section 3.2 builds the benchmark with the same pipelines used for training, but the supplied manuscript does not describe the splitting unit or checks for shared source images, near-duplicates, queries derived from the same image, or reused synthesis references. This is a gap in the evidence for generalization, not a finding that leakage occurred. Identity and event overlap should likewise be reported so that readers can distinguish generalization to new images from generalization to unseen entities or contexts.

The statement that every example was manually verified does not specify how ambiguous descriptions, identity facts, box and mask boundaries, or complete counts were adjudicated. The benchmark’s real/synthetic and person/anime composition is also unclear. Moreover, the query-rewriting procedure stops when Seed-1.8 fails without tools; this establishes difficulty relative to that construction model rather than demonstrating that all evaluated models require external evidence. Stratified results and a small audit of query uniqueness and factual correctness would help characterize what EviLens measures.

The narrow recognition lead, 0.648 versus 0.636 IoU, should be interpreted with uncertainty estimates. The difference-detection subset contains only 15 images and 79 annotated differences. Appendix A.3 explicitly reports eight rollouts per instance for this task, which helps characterize generation variability but does not increase the number of independent scenes. Paired uncertainty estimates that retain image-level clustering would be more informative than treating rollouts or individual differences as independent test examples.

3. The SAM3 fallback has an unsupported mathematical justification and an unquantified practical impact.

Appendix A.2.1 replaces a failed SAM3 output with the predicted box interpreted as a rectangular mask, and describes its IoU as a strict lower bound on the mask IoU SAM3 would have produced. No such ordering follows in general: a returned mask can overlap the target less accurately than the input rectangle. The fallback may be a reasonable heuristic, but it is not guaranteed to be a lower bound under the stated assumptions. The discussion also treats failure as unrelated to prediction quality without establishing whether failure depends on the proposed box or point prompts. Reporting the fallback frequency, failure categories, and sensitivity to an alternative failure-handling rule would reveal whether this issue is practically important. The current evidence does not establish that the policy exploits the fallback or that the main results are invalid.

Questions for authors
  1. Evaluation controls: For each Table 2 row, was the model evaluated directly or with tools? Were the backbone, SFT model, and RL model given identical prompts, tool schemas, call and token budgets, and segmentation point/verification interfaces? How were the imported WebEyes results aligned with the current evaluation, including the browsing summarizer?

  2. Learning mechanism: Under matched tool access, what changes between SFT and RL in tool selection, calls per query, and success by task? Can focused visual-inspection and external-retrieval ablations separate their contributions without changing the remaining evaluation protocol?

  3. Benchmark validity: How were SFT, RL, and EviLens separated and deduplicated, including synthesis references? What are the identity/event overlaps and source-type proportions? How were query uniqueness, factual claims, spatial labels, and counts checked, and how stable are the close rankings under image-level uncertainty analysis?

  4. Reward behavior: How often is the SAM3 fallback used, for what reasons, and with what dependence on predicted prompts? Is the same rule used during evaluation? Can the lower-bound assertion be corrected and the effect of the fallback quantified?

7 comments

W1. **(1)The experiments establish ... The paper does specify a common prompt, output conventions, and a 35-call limit for agentic methods in Section 4.1. However, Table 2 does not explicitly identify (3)which rows use the direct or agentic protocol. ... A per-model configuration table would clarify whether the gains reflect training, additional tool access, or different prediction interfaces. There is also an ambiguity between (4)direct segmentation from a predicted box and EviRover’s box-plus-point ...

(3) Table 2 uses the following inference protocols:

ProtocolTable 2 rows
DirectGPT-5.6-Sol, GPT-5.6-Luna, GPT-5.6-Terra, Seed-2.1-Turbo, Seed-1.8, Qwen3.7-Plus, Gemini-3.5-Flash, Seg-R1-7B, Seg-Zero-7B, Perception-R1, UniVG-R1, Minimax-M3, OneThinker-8B, Qwen3-VL-4B-Instruct
AgenticSAM3-Agent, InternVL-3.5-8B, Qwen3-VL-4B-Instruct + SFT, EviRover

Among the agentic rows, InternVL-3.5-8B, SAM3-Agent, Qwen3-VL-4B-Instruct + SFT, and EviRover use the EviRover harness. All other models use direct inference. We tried to use agentic inference for Seg-R1-7B, Seg-Zero-7B, Perception-R1, UniVG-R1, OneThinker-8B, Qwen3-VL-4B-Instruct, but the models failed to use tools. For GPT-5.6-Sol, GPT-5.6-Luna, GPT-5.6-Terra, Seed-2.1-Turbo, Seed-1.8, Qwen3.7-Plus, Gemini-3.5-Flash, we followed previous works [1,2,3,4] and used direct inference.

[1] Feng K, Zhang M, Chen S, et al. Gen-searcher: Reinforcing agentic search for image generation[J]. arXiv preprint arXiv:2603.28767, 2026.

[2] Zheng Z, Yang M, Hong J, et al. Deepeyes: Incentivizing" thinking with images" via reinforcement learning[C]//International Conference on Learning Representations. 2026, 2026: 126775-126798.

[3] Lai X, Li J, Li W, et al. Mini-o3: Scaling up reasoning patterns and interaction turns for visual search[C]//International Conference on Learning Representations. 2026, 2026: 76722-76746.

[4] Yang B, Sun X, Feng K, et al. From Web to Pixels: Bringing Agentic Search into Visual Perception[J]. arXiv preprint arXiv:2605.12497, 2026.

(4) Our box-plus-point interface follows OneThinker [5]. Because the baseline models did not reliably follow the box-plus-point output instructions, we asked them to output boxes. To examine the effect of box-only versus box-plus-point prompts on segmentation scores, we compared the two settings using EviRover on the EviLens segmentation subset:

SettingSegmentation gIoU / cIoU
box-plus-point (EviRover)68.630 / 70.750
box69.189 / 71.076

The two settings yield similar segmentation scores, so we consider the comparison fair.

We will clarify the differences between box-only and box-olus-point and the experimental results in the revised manuscript.

[5] Feng K, Zhang M, Li H, et al. Onethinker: All-in-one reasoning model for image and video[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 5432-5443.

W1. **(1)The experiments establish ... that the (5)browsing tool uses Qwen3.5-27B for summarization. ..., while system-level resource comparisons should also account for the auxiliary model and actual tool usage. Since (6)WebEyes baseline scores are imported from the original paper , their alignment with EviRover’s tool and inference settings also remains unclear.

(5) We clarify that the use of an auxiliary summarization model is also adopted in several prior works, including Gen-Searcher [1] (Qwen3-VL-30B-A3B-Instruct), MMSearch-R1 [6] (Qwen3-32B), and WebWatcher [7] (Qwen2.5-72B).

To investigate whether EviRover's performance depends on this larger auxiliary model, we conducted additional experiments using two alternative summarization models: DeepSeek-v4-flash and EviRover-4B itself. As shown in the table below, the three configurations achieve largely comparable performance across task categories.

Summary ModelLocalization IoU / [email protected]Recognition IoU / [email protected]Spot Diff micro-F1 / macro-F1Segmentation gIoU / cIoUCounting acc
DeepSeek-v4-flash0.448 / 0.5140.664 / 0.6710.172 / 0.2340.666 / 0.71951.3
EviRover-4B0.435 / 0.4810.675 / 0.7140.169 / 0.2670.655 / 0.70550.6
Qwen3.5-27B0.444 / 0.5000.648 / 0.6870.171 / 0.2660.686 / 0.70850.0

Importantly, using EviRover-4B itself for summarization eliminates the dependency on an additional large summarization model while preserving comparable performance. This demonstrates that EviRover's performance is not contingent on using the 27B auxiliary model.

Furthermore, the browsing tool was invoked only 30 times across all 688 evaluation samples, averaging just 0.044 calls per sample. This indicates that browsing-related summarization is infrequently required, limiting its contribution to the overall inference workload.

We will clarify the distinction between controller size and system-level resource requirements, and report the actual tool usage statistics in the revised manuscript.

[1] Feng K, Zhang M, Chen S, et al. Gen-searcher: Reinforcing agentic search for image generation[J]. arXiv preprint arXiv:2603.28767, 2026.

[6] Jiang D, Zhang R, Guo Z, et al. Mmsearch: Benchmarking the potential of large models as multi-modal search engines[J]. arXiv preprint arXiv:2409.12959, 2024.

[7] Geng X, Xia P, Zhang Z, et al. Webwatcher: Breaking new frontiers of vision-language deep research agent[C]//International Conference on Learning Representations. 2026, 2026: 61240-61271.

(6) To ensure a fair comparison, we conducted additional experiments evaluating Qwen3-VL-8B-Instruct and EviRover under both the Pixel-Searcher workflow and the EviRover harness.

For the Pixel-Searcher workflow, we followed the publicly released implementation. All four configurations were evaluated on the same 645 grounding/segmentation queries and 637 VQA queries, using the same evaluation environment and WebEyes scoring protocol. Segmentation masks were generated from the predicted boxes using SAM3.

As shown in the table below, EviRover consistently outperforms Qwen3-VL-8B-Instruct on grounding and segmentation under both workflows. Furthermore, the EviRover harness improves grounding and segmentation performance for both models, demonstrating the effectiveness of the agentic workflow.

ModelWorkflowGrounding IoU / [email protected]Segmentation gIoU / cIoUVQA accuracy
Qwen3-VL-8B-InstructPixel-Searcher30.7 / 33.534.0 / 29.343.5
EviRoverPixel-Searcher34.2 / 38.044.0 / 31.841.0
Qwen3-VL-8B-InstructEviRover harness32.3 / 38.034.7 / 30.340.2
EviRoverEviRover harness40.0 / 48.549.5 / 38.040.3

We note that the reproduced Pixel-Searcher scores differ somewhat from those reported in the original paper, despite following the publicly released implementation. More importantly, our new experiments ensure a fair and controlled comparison by evaluating both models under both workflows, using identical evaluation samples, tool configurations within each workflow, and scoring protocols. This directly addresses the concern regarding potential inconsistencies between the originally reported WebEyes baseline and EviRover's inference settings. We will include these controlled comparisons and clarify the corresponding settings in the revised manuscript.

Q1. Evaluation controls: For each Table 2 row, was the model evaluated directly or with tools? Were the backbone, SFT model, and RL model given identical prompts, tool schemas, call and token budgets, and segmentation point/verification interfaces? How were the imported WebEyes results aligned with the current evaluation, including the browsing summarizer?

AQ1. Thank you for your comment. We have answered these evaluation controls in detail in W1 (2–6).

(1) Evaluation protocols. As detailed in W1 (3), we clarify the direct and agentic inference protocols for every Table 2 row.

(2) Consistency of evaluation settings. As clarified in W1 (2), the untrained backbone failed to use the tools and scored zero under the agentic protocol, so its reported scores are the direct-answering results from Table 2. The SFT and RL checkpoints use identical prompts, tool schemas, decoding settings, call and token budgets, and segmentation point/verification interfaces. Both trained checkpoints have access to the same ten tools, with a maximum of 35 tool calls and a 256K token budget per rollout. Actual token consumption remained well below this limit:

Token usage statisticTokens
Median (P50)9K
95th percentile (P95)25K
Maximum69K
Configured budget256K

As detailed in W1 (4), we additionally examined the effect of box-only versus box-plus-point segmentation outputs and found comparable performance. Furthermore, the browsing summarizer (Qwen3.5-27B) is part of the shared agentic tool implementation and is identical across all models evaluated under the EviRover harness. W1 (5) further investigates the impact of different summarization models.

(3) Alignment with WebEyes results. As detailed in W1 (6), we conducted additional controlled experiments evaluating EviRover and Qwen3-VL-8B-Instruct under both the Pixel-Searcher workflow and the EviRover harness, using identical evaluation samples and scoring protocols. These experiments provide a controlled comparison beyond the imported WebEyes scores.

We will clarify these evaluation configurations and distinctions in the revised manuscript.

Q2. Learning mechanism: Under matched tool access, what changes between SFT and RL in tool selection, calls per query, and success by task? Can focused visual-inspection and external-retrieval ablations separate their contributions without changing the remaining evaluation protocol?

AQ2. Thank you for your comment. We have conducted additional experiments to investigate the learning mechanism under matched tool access, as detailed in W1 (1–2).

(1) Changes from SFT to RL. As shown in W1 (1), under identical evaluation settings, RL substantially improves tool-use efficiency, reducing the average number of calls per query from 11.42 to 5.96 (−47.8%) and tool-call errors from 181 to 38. Tool selection also shifts from repeated cropping toward candidate verification: crop calls decrease from 3,569 to 122, while verify calls increase from 350 to 1,098. As shown in W1 (2), RL also consistently outperforms SFT across all five task categories under identical tool access.

(2) Focused tool ablations. Yes. As detailed in W1 (2), we conducted controlled ablations by keeping the RL checkpoint fixed and separately disabling external retrieval or visual inspection, while preserving the remaining evaluation protocol, including prompts, decoding settings, and inference budgets. Removing external retrieval primarily reduces recognition, segmentation, and counting performance, whereas removing visual inspection substantially degrades localization performance. These results help distinguish the contributions of different evidence sources.

Together, these experiments provide evidence that RL improves both tool-use efficiency and task performance, while different tool categories contribute to different perception tasks.

Reviewer_QiSjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces a new approach to visual perception that enables multimodal models to actively gather missing evidence rather than relying on a single observation. To support training and evaluation, they introduce EviLens, a benchmark containing challenging perception tasks requiring fine-grained visual inspection or external knowledge

Critical strengths

S1. The paper offers self-curated evaluation set that could further be evaluated at the future works.

Critical weaknesses

W1. Limited contribution on the task expansion

  • Finding the small obj at the high resolution image [1,2] / Find the subtle difference from the images [3] / Multi modal external knowledge [4,5] those three tasks are already explored from the various research. Contribution of combining those into single training are limited than expanding to under-explored tasks.

W2. Benchmark data for high-resolution benchmark.

  • The task of high-reoslution VQA has well-established benchmarks [1,2] and models [6] to solve those tasks, which is not utilized as the baseline for the comparison at this paper.
[1] Wu, Penghao, and Saining Xie. "V*: Guided visual search as a core mechanism in multimodal llms." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
[2] Lai, Xin, et al. "Mini-o3: Scaling up reasoning patterns and interaction turns for visual search." International Conference on Learning Representations. Vol. 2026. 2026.
[3] Kim, Minkyu, Sangheon Lee, and Dongmin Park. "VLM-SubtleBench: How Far Are VLMs from Human-Level Subtle Comparative Reasoning?." arXiv preprint arXiv:2603.07888 (2026).
[4] Wu, Jinming, et al. "Mmsearch-r1: Incentivizing lmms to search." arXiv preprint arXiv:2506.20670 2.4 (2025).
[5] Geng, Xinyu, et al. "Webwatcher: Breaking new frontiers of vision-language deep research agent." International Conference on Learning Representations. Vol. 2026. 2026.
[6] Zheng, Ziwei, et al. "Deepeyes: Incentivizing" thinking with images" via reinforcement learning." International Conference on Learning Representations. Vol. 2026. 2026.
Questions for authors

Q1. Is there any quantative results that backs the model does not call the tools when the sufficient information is presented to the model?

  • The paper argues that "the model decides whether further evidence is needed.". For the argue to be true, there must be the results with the model automatically answers without tool-calling when the sufficient information is fed-in at the inference. Without this stats, I cannot agree with the statement as the model might only decide which evidence is needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.