- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes 3 methods to improve multimodal agentic search, including Factorized Adaptive Rollout to improve RL training efficency, evidence-verified reasoning to generate training data that encourages evidence verification, and self-summary to summarize webpages without a separate model. With these methods, SimpleSearch-VL achieves better performance than many competing baselines.
Critical strengths
- Good performance in Table 1
Critical weaknesses
- Unfair experimental setup - SimpleSearch-VL uses different training data, search pipeline, tools and compute compared to other methods in Table 1. I don't see any fair comparison that isolates the improvements from the proposed methods. How do we know the gains comes from these methods rather than simply better training data or search pipeline? Without a controlled comparison, it is hard to tell what actually contributes to the performance improvement.
- Limited novelty - FAR is just allocating more rollouts to unsuccessful prompts. Evidence-Verified Reasoning is mainly using teacher-generated trajectories and Gemini to filter and rewrite reasoning, which have already been explored in SenseNova-MARS, OpenSearch-VL, etc. Self-summary also seems more like a pipeline change rather than a new method. I don't clearly see what is technically new here or how much each of these methods actually contributes.
Questions for authors
- Why are some baseline results missing in Table 1? e.g. SenseNova-MARS-32B have result for LiveVQA and SimpleVQA. Why are they not in the table? Have all baseline results been checked? If results missing but model released, they should be evaluated using the same settings as SimpleSearch-VL for better comparison.
- Isn't FAR partly just benefiting from more rollouts? Figure 8(b) already shows that increasing rollout size from 4 to 6 gives a large improvement. How does FAR compare with fixed rollout sizes of 6 and 8 under the same training settings and compute budget?
- Is the improvement from evidence-verified reasoning or just better training data? Table 2 compares Direct SFT and Evidence-Aware SFT, but appendix C also mentions filtering trajectories and rewriting final answers. Are the same training examples and final answers used? How does this ablation isolate verification reasoning itself?
- Why does self-summary perform better than a dedicated summarizer in Table 3? What checkpoint and prompt are used for the dedicated summarizer? Are the generation settings matched?
- Table 4 compares H200 with H20 GPUs and uses different training data. How is this a fair comparison of training efficiency? Can you provide a fairer comparison that accounts for hardware differences and data construction costs?
10 comments
Response to Critical Weakness 1: Cross-system comparisons, data efficiency, and controlled evidence
Thank you for recognizing the performance in Table 1. Multimodal search agents typically construct their own training trajectories and adopt method-specific harness configurations and tool interfaces. Representative examples include:
- SenseNova-MARS combines FVQA, Pixel-Reasoner, and curated expert data for approximately 3K cold-start examples, followed by RL on FVQA, DeepEyes-4K, and Visual-Probe [1].
- Vision-DeepResearch constructs long-horizon trajectories from verified VQA, text-only QA, and synthesized fuzzy multi-hop questions, using 30K SFT trajectories and 15K RL examples for its 30B-A3B model [2].
- DeepMMSearch-R1 distills Gemini trajectories from InfoSeek into 10K SFT examples, then uses FVQA for RL [3].
- OpenSearch-VL combines Wikipedia-derived questions with LiveVQA, FVQA, and WebQA, producing 36,592 SFT trajectories and approximately 8K RL examples [4].
Our tool set is also deliberately simple: text search, reverse-image search, and webpage visiting. These functions are already covered by Vision-DeepResearch and OpenSearch-VL [2, 4]; OpenSearch-VL additionally provides OCR and image-enhancement tools [4]. These overlaps make the cited systems relevant comparators, while data construction and execution remain part of each method. Table 1 compares complete systems; matched backbone evaluations and component ablations address where our gains come from. We summarize the evidence along three dimensions:
1. Training efficiency and resource cost. Like many existing multimodal search agents, we draw our training questions from public QA sources, including FVQA, LiveVQA, WebQA, and Wikipedia-derived QA. Our data sources substantially overlap with those used by existing methods, particularly OpenSearch-VL. We use 5,193 versus 36,592 SFT trajectories, and 1,995 versus approximately 8K RL examples [4]. SFT+RL takes approximately 18 hours at 8B and 27 hours at 30B-A3B on one eight-H200 node (Table 5; configuration in Appendix A). We have also completed an 8B run on one eight-H20 node (96 GB per GPU), taking 10 hours for SFT and 27 hours for RL—37 hours in total.
2. Search performance and controlled evidence. Despite using fewer training examples, our 8B model has higher reported scores than OpenSearch-VL-30B-A3B on the five shared benchmarks in Table 1. For the within-system comparison, each Qwen3-VL agentic baseline and its trained counterpart use the same evaluation harness and protocol within that model scale. The six-benchmark average improves by 15.8 percentage points at 8B and 16.0 at 30B-A3B, reflecting the full post-training recipe. The component studies provide more targeted evidence: FAR (2×, 4–6) improves the three-benchmark average from 66.3 to 70.7, with the SFT initialization, RL data, and training steps held fixed (revised Table 20).
3. Model capacity without an auxiliary LLM. During search inference, the same 8B or 30B-A3B policy performs search planning, visual evidence verification, webpage summarization, and final-answer generation. Our self-summary design does not call a separate LLM to process retrieved evidence. Thus, the reported agent size is also the size of the model performing these reasoning and summarization tasks: an 8B result is not obtained by pairing an 8B agent with a larger auxiliary LLM. This makes the model capacity behind the results explicit while avoiding an additional summarizer deployment.
The SFT-initialized FAR control is summarized in Section 3.3, with full results in Appendix F.3 (Table 20).
References
[1] Yong Xien Chng et al. SenseSearch: Empowering Vision-Language Models with High-Resolution Agentic Search-Reasoning via Reinforcement Learning. In CVPR, 2026. SenseNova-MARS technical report: arXiv:2512.24330.
[2] Wenxuan Huang et al. Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Models. In ICML, 2026.
[3] Kartik Narayan et al. DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Search. arXiv preprint arXiv:2510.12801, 2025.
[4] Shuang Chen et al. OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents. arXiv preprint arXiv:2605.05185, 2026.
Response to Critical Weakness 2: Technical contributions of FAR, visual verification, and self-summary
Our contribution is a practical recipe for making multimodal search agents efficient to train, grounded in visual evidence, and simpler to deploy. We target three concrete bottlenecks: slow and uneven tool-interleaved rollouts, plausible but visually mismatched search results, and the overhead of a separately served webpage summarizer.
1. FAR: recovering useful RL signal while controlling rollout cost. Fixed sampling can leave hard prompts with only incorrect answers and no answer-level contrast, while a few long tool-call trajectories delay the entire step. FAR coordinates prompt expansion, selective within-prompt allocation, and stopping rules: it expands all-wrong groups breadth-first, masks later expanded slots once a correct answer appears, and can end generation once enough valid groups have been collected or the near-completion criterion is met. At most n_min trajectories per retained group enter the update, while the full completed group informs the leave-one-out baseline. Thus, FAR changes both where additional exploration is spent and how long the step waits, rather than merely assigning more rollouts to unsuccessful prompts.
Simple scheduling rules can yield substantive gains. DAPO's dynamic sampling uses oversampling and group filtering [1], while APRIL addresses rollout tails through over-provisioning, early termination, and resumption [2]. What matters is resolving a concrete bottleneck and saving training time without sacrificing accuracy, not rule complexity.
Figure 4(a) shows that 4–6 and 4–8 allocation recover 11.7% and 17.7% of initially all-wrong groups. In Figure 4(b)'s fixed-step direct-RL study, FAR (2×, 4–6) improves the three-benchmark average from 56.6 to 62.0, taking 165 seconds per step versus 192 for standard sampling. Separately, the same FAR configuration improves the three-benchmark average from 66.3 to 70.7 in the SFT-initialized control, holding the initial checkpoint, RL data, and training steps fixed (revised Table 20). These results support hard-group recovery and a better accuracy–time trade-off.
2. Evidence-verified reasoning: checking visual correspondence before trusting a source. A search result can provide a convincing title and URL while depicting a different object from the queried image or region. The contribution is to make this correspondence check explicit in the agent's training: inspect returned thumbnails, assess whether they support the current hypothesis, and decide whether further evidence is needed. Our audit and rewrite pipeline constructs trajectories around these decisions. The emphasis is on how the agent learns to use visual evidence, not the identity of the teacher or auditor; thumbnail access itself is also available in SenseNova-MARS [3].
Table 2 reports 61.0 → 64.4 (+3.4 points) for the evidence-aware SFT data pipeline. A separate control on the untuned checkpoint improves the four-benchmark average from 41.4 to 44.3 when thumbnails are enabled. These support the data pipeline and the value of retrieved visual evidence. Section 3.3.2 reports the untuned-policy control in Table 3.
3. Self-summary: removing a model dependency and sharing serving resources. A dedicated summarizer requires another model deployment and separates webpage processing from the search policy that uses its output. We instead route webpage summarization through the policy's own rollout engine. This removes the separate summary service, lets search and summary generation share serving resources, and gives both tasks access to the same task-adapted weights. Reasoning, summarization, and answering thus remain within the agent, without an auxiliary LLM. Shared task-adapted weights also offer a plausible explanation for the accuracy benefit.
In Table 4's 8B setting, self-summary achieves 69.8 versus 67.9 average accuracy with one policy-serving node instead of one agent node plus one summary node. At the same two-node budget, two self-summarizing agent replicas complete the evaluation workload in 358 versus 501 seconds (28.5% less time), with 69.7 versus 67.9 average accuracy. This measures a resource-allocation benefit, not just a simpler interface.
Together, these designs improve training signal, visual verification, and deployment simplicity. Their significance lies in these practical capabilities and the experimental evidence for them, rather than in adding more components to the system.
References
[1] Qiying Yu et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. In NeurIPS, 2025.
[2] Yuzhen Zhou et al. APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-Tail Generation. arXiv preprint arXiv:2509.18521, 2025.
[3] Yong Xien Chng et al. SenseSearch: Empowering Vision-Language Models with High-Resolution Agentic Search-Reasoning via Reinforcement Learning. In CVPR, 2026. SenseNova-MARS technical report: arXiv:2512.24330.
Overview of our response and manuscript revisions
Thank you for the careful review and for recognizing SimpleSearch-VL's performance in Table 1. We respond to your concerns in seven comments below:
Changes specific to your review are marked in red. Each reply points to the relevant sections and tables. Please feel free to ask follow-up questions under each comment; we will respond promptly.