acceptodds
Under review as a conference paper at ICLR 2027

Neos: Learning to Interleave Search and Visual Reasoning

10 views

Abstract

Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance for subsequent search, limiting their effectiveness in long-horizon settings. To address this gap, we present **Neos**, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions. To make this coupling learnable, Neos constructs training tasks with intermediate visual dependencies while filtering out textual shortcuts, and introduces a lightweight, trainable Multimodal Evidence Interface (MEI) that transforms retrieved content into goal-conditioned observations. At its core, an agent–environment co-evolution procedure treats MEI as a co-adaptive component of this loop. Specifically, reference-guided credit assignment uses verified successful trajectories to localize critical policy errors while preserving credit for useful decisions in failed searches, whereas rollout feedback selects informative local evidence-processing interactions for MEI supervised fine-tuning (SFT). By alternating policy reinforcement learning (RL) with MEI SFT, Neos allows the policy's evolving evidence requests to shape MEI training and the adapted MEI to reshape subsequent observations, closing the loop between evidence acquisition and evidence-conditioned action. Across ten benchmarks, experiments show that Neos-9B improves over its backbone under the same tool workflow by 17.4 percentage points on average, achieving leading performance among specialized multimodal deep search agents.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Model, Deep Search, Agentic Reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fT8pOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes three tricks to improve interleaved search and visual reasoning, including an evidence-dependent synthesis pipeline to generate training data, reference-guided credit assignment to improve RL training, and a Multimodal Evidence Interface (MEI) that uses a separate model to process retrieved text and images. With these tricks, Neos-9B reports much better performance than several competing baselines.

Critical strengths
  1. Good performance in Table 1
Critical weaknesses
  1. Severe clarity issue - The paper uses complicated terminology to describe relatively simple ideas without clearly explaining them. For example, MEI seems to be just another model that processes retrieved content before passing it to the main agent, with the two models trained alternately. Why not just explain it this way? Instead, the paper introduces confusing terms like "goal-conditioned observations" and "agent–environment co-evolution," making the method unnecessarily difficult to follow. Figures 2 and 3 are also unclear and require going back and forth between the figures and main text to understand them, which defeats the purpose of having figures. Please improve the clarity throughout the paper and avoid using unnecessary, confusing, or misleading terminology.
  2. Unfair experimental setup - Neos use different training data, search pipeline, tools and training compute compared to all the methods in Table 1. I don't see any experimental setup that compare these methods fairly. How do we know the improvements are from the proposed tricks, rather than simply different training data or a better search pipeline? In fact, Table 13 already shows that changing the workflow alone can make a huge difference. The data ablation in Table 2 only compares different data sources within Neos, which does not address this issue. It is already well known that better training data and more sophisticated search pipelines can improve performance. If that is where most of the improvement comes from, then what new insight does this paper actually provide? Without a fair comparison, it is hard to assess the real contribution of the proposed methods.
Questions for authors
  1. Most of the methods in Table 1 are open-source, e.g. SenseNova-MARS, OpenSearch-VL etc. Why not reevaluate these models on the benchmarks with missing results in table 1?
  2. Table 1 only show frontier models like GPT and Gemini under Direct Answer and Agentic Workflow settings. What about frontier models like GPT and Claude running in existing deep search agent frameworks? Why are these comparisons missing?
  3. How many search turns can Neos sustain? How well does it perform on longer tasks that require hundreds or even thousands of steps, instead of only 50 turns?
  4. The gap between RL-only and SFT->RL in Table 11 seem quite small. Several prior works suggest that RL can generalize better in OOD settings. Are there any evaluations on OOD tasks? Could the RL-only model actually perform better in these settings? Since we are training foundation models, we should care more about generalization rather than just getting better performance on a few benchmarks.
  5. Why do we need a separate model to process retrieved information? Why cant the main agent do this itself? Please provide an ablation comparing the two setups.
5 comments

Thank you for your thorough reading of our manuscript and your insightful analysis of our methodology. We appreciate your rigorous approach to scientific evaluation and have done our best to address your concerns below, as well as to present the additional experiments we have conducted, which are also incorporated in Appendix.E highlighted in purple for ease of reference.

W1: Severe clarity issue - The paper uses complicated terminology to describe relatively simple ideas without clearly explaining them. For example, MEI seems to be just another model that processes retrieved content before passing it to the main agent, with the two models trained alternately. Why not just explain it this way? Instead, the paper introduces confusing terms like "goal-conditioned observations" and "agent–environment co-evolution," making the method unnecessarily difficult to follow...

We sincerely thank the reviewer for this constructive feedback and acknowledge that the core mechanism of MEI can be simply understood as a preprocessing model for retrieved content that is trained alternately with the main agent; however, we would like to respectfully clarify that the specific terminology used is technically precise rather than unnecessarily complex, as "goal-conditioned observations" accurately captures that MEI explicitly filters and returns information conditioned on the search goal passed by the agent—a nuance that normal terms like "process retrieved information" would fail to convey—and "agent–environment co-evolution" precisely describes the coupled optimization dynamics arising from alternating training with MEI embedded in the environment, which aligns with established concepts in reinforcement learning literature. That said, we fully agree that our current exposition could be more accessible, and in the revised manuscript we will introduce these terms with clearer plain-language explanations upfront and fine-tune the figures in our paper, thereby preserving technical rigor while significantly improving readability.

W2: Unfair experimental setup - Neos use different training data, search pipeline, tools and training compute compared to all the methods in Table 1. I don't see any experimental setup that compare these methods fairly. How do we know the improvements are from the proposed tricks, rather than simply different training data or a better search pipeline? In fact, Table 13 already shows that changing the workflow alone can make a huge difference. The data ablation in Table 2 only compares different data sources within Neos, which does not address this issue...

Regarding the comparison with other methods in the main table: because online search evaluation is inherently sensitive to temporal shifts—leading to discrepancies when reproducing results via search APIs—and given that some baseline methods are not fully open-sourced, we followed the common practice in the search research community by reporting the original numbers from prior work. Nevertheless, we fully recognize the importance of comprehensive ablation studies for ensuring the rigor of our paper. To address your concerns, we have supplemented alignment experiments to the extent permitted by our available resources.

First, we conducted experiments to verify data fairness. Our training data includes questions from RedSearcher-MM as well as self-constructed questions; however, these were sampled as trajectories within the Neos workflow and used for SFT training. To ensure a fair comparison, we applied the same rejection sampling model to these questions under the RedSearcher-MM workflow, obtaining an SFT dataset of equivalent size for training and evaluation. Building upon the results in Table 2, we obtained the following ablation experiment results:

WorkflowQuestion sourceBC-VLMMSearch+FVQA
REDSearcher-MMREDSearcher-MM (1.1K)42.122.871.3
REDSearcher-MMOur synthetic data (1K)48.432.876.3
NeosREDSearcher-MM (1.1K)56.938.977.3
NeosOur synthetic data (1K)58.443.278.3

Using our synthetic data instead of REDSearcher-MM questions improves BC-VL, MMSearch+, and FVQA scores by 6.3, 10.0, and 5.0 points under the REDSearcher-MM workflow, and by 1.5, 4.3, and 1.0 points under Neos, respectively. Holding the question source fixed, Neos outperforms REDSearcher-MM by 10.0–14.8, 10.4–16.1, and 2.0–6.0 points on the three benchmarks.

We appreciate your suggestion. These experiments clearly and independently demonstrate that both our data construction approach and search pipeline make unique contributions to enhancing multimodal long-horizon search capabilities. Specifically, they provide support from two perspectives—task design and search workflow—for shaping the agent's ability to capture critical visual information during long-horizon search processes and guide subsequent reasoning steps.

Q1: Most of the methods in Table 1 are open-source, e.g. SenseNova-MARS, OpenSearch-VL etc. Why not reevaluate these models on the benchmarks with missing results in table 1?

For multimodal long-horizon search agents with open-source evaluation frameworks and models like SenseNova-MARS and OpenSearch-VL, we provide additional evaluations on benchmarks to make a more comprehensive comparison. Non-bold values indicate previously reported results.

ModelMMSearch+MM-BCBC-V3VisBrowseBC-VLVDRAvg.
SenseNova-MARS-32B18.316.514.824.939.425.823.3
OpenSearch-VL-30B17.418.315.527.841.133.525.6
Neos-9B47.639.537.053.363.231.445.3

Neos-9B achieves an average score of 45.3 across the six benchmarks, outperforming SenseNova-MARS-32B and OpenSearch-VL-30B by 22.0 and 19.7 points. These substantial gains demonstrate stronger overall multimodal long-horizon search performance despite its smaller model size.

Q2: Table 1 only show frontier models like GPT and Gemini under Direct Answer and Agentic Workflow settings. What about frontier models like GPT and Claude running in existing deep search agent frameworks? Why are these comparisons missing?

We fully agree that directly comparing agent workflows using frontier models provides a more intuitive assessment. In Table 13 of the appendix, we already compare REDSearcher-MM, a representative workflow that returns all retrieved information to the context, with our MEI-based workflow, highlighting the performance differences between these designs. Furthermore, within our limited available resources, we have added comparisons with other workflows featuring distinct designs, as shown below:

PolicyWorkflowBC-VLMM-BCMMSearch+
GPT-5.6SenseNova-MARS61.333.848.9
REDSearcher-MM64.235.553.5
OpenSearch-VL62.134.350.2
Neos66.459.565.9
Qwen3.5-35B-A3BSenseNova-MARS42.614.820.9
REDSearcher-MM45.414.521.6
OpenSearch-VL44.616.320.3
Neos51.429.028.1
Qwen3.5-9BSenseNova-MARS38.18.818.6
REDSearcher-MM38.69.819.3
OpenSearch-VL37.19.517.7
Neos43.622.824.8

Holding the policy model fixed, Neos consistently outperforms all three alternative workflows across all benchmarks. With GPT-5.6, it exceeds the strongest competing result on BC-VL, MM-BC, and MMSearch+ by 2.2, 24.0, and 12.4 points, respectively; corresponding gains are 6.0, 12.7, and 6.5 points with Qwen3.5-35B-A3B, and 5.0, 13.0, and 5.5 points with Qwen3.5-9B. These consistent improvements across frontier and open-source policies support the advantage of the Neos workflow beyond the capabilities of the underlying policy model alone.

Q3: How many search turns can Neos sustain? How well does it perform on longer tasks that require hundreds or even thousands of steps, instead of only 50 turns?

Thank you for providing the insight that industrial-grade agents may handle long-horizon tasks with more than hundreds of interaction turns. We have added an evaluation with a larger turn budget.

Maximum turnsBC-VLBC-V3FVQA
5063.237.083.3
8062.637.382.7
10063.637.883.3
15063.437.583.7

We found that excessively long interactions frequently exceed the configured context window. Since our method does not incorporate context compression, and considering both comparison fairness and the limited rebuttal period, we conducted an extended evaluation allowing up to 150 turns. The results show that increasing the turn budget yields modest performance gains for Neos. However, for long-tail samples, additional reasoning turns do not substantially improve accuracy.

Q4: The gap between RL-only and SFT->RL in Table 11 seem quite small. Several prior works suggest that RL can generalize better in OOD settings. Are there any evaluations on OOD tasks? Could the RL-only model actually perform better in these settings? Since we are training foundation models, we should care more about generalization rather than just getting better performance on a few benchmarks.

High-quality SFT trajectories provide a strong initialization, while agentic RL further develops the model’s ability to tackle challenging, long-horizon search tasks. As shown in Table 5, vanilla GRPO yields limited improvements, whereas our agentic RL approach effectively improves upon SFT, highlighting the complementary roles of the two training stages.

Thank you for your insightful suggestion. Although we train a specialized agent for multimodal deep search, investigating the out-of-distribution (OOD) generalization of RL-trained agents is also a valuable research direction. We have added experiments on several OOD benchmarks, including multimodal reasoning benchmarks.

TrainingWeMathMathVistaLogicVistaVisuLogicCharXiv-RQVisualToolBench APRVisualToolBench ARS
Qwen3.5-9B base82.4885.6073.3233.6071.904.3224.83
SFT only82.9185.4072.5434.8072.205.0323.27
RL only82.7785.7074.1134.9072.355.3821.70
SFT→RL (Neos-9B)83.0585.6074.8935.2072.505.7324.05

Compared with RL-only training, Neos-9B achieves higher scores on six of the seven OOD metrics, including gains of 0.78 points on LogicVista and 0.30 points on VisuLogic. On VisualToolBench, SFT→RL improves APR from 5.38 to 5.73 and ARS from 21.70 to 24.05, yielding gains of 0.35 and 2.35 points.

The results suggest that our training approach offers better overall generalization than RL alone. Although such specialized training may not produce OOD gains as substantial as those achieved by MixRL, we attribute the observed improvements partly to the task’s role in developing the model’s reasoning abilities during search, which can benefit generalization. The SFT-based cold start may also contribute to these gains.

Q5: Why do we need a separate model to process retrieved information? Why cant the main agent do this itself? Please provide an ablation comparing the two setups.

We compared policy-side summarization, policy-side image processing, their combination, and the standard MEI setup. MEI performs best across all three benchmarks, supporting the benefit of delegating evidence processing to a separate interface in our evaluated configuration. Moving this processing to the policy side does not eliminate its overhead: deploying an auxiliary agent entails a setup comparable to MEI, whereas having the main agent perform it directly adds latency. Our results also suggest that Neos’s specialized training can interfere with its self-summarization ability. Methods such as SimpleSearch-VL[1] instead seek to integrate this ability into the policy through explicit training, which introduces additional training requirements.

Evidence processingBC-VLMM-BCSimpleVQA
Policy summarizes pages59.936.078.7
Policy processes image candidates61.737.880.0
Policy handles both60.235.879.3
MEI handles both (standard Neos)63.239.581.3

[1] Dai, Ming, et al. "SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Search." arXiv preprint arXiv:2606.31504 (2026).

Reviewer_kncXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Neos, a multimodal deep search framework that aims to improve how agents use intermediate visual evidence to guide subsequent search decisions. The proposed approach consists of three main components: (1) an evidence-dependent data synthesis pipeline; (2) reference-guided credit assignment for RL; and (3) a trainable Multimodal Evidence Interface (MEI) with an agent-environment co-evolution strategy. Experiments across ten benchmarks show that trained Neos-9B achieves strong performance among specialized multimodal deep search agents. Additional ablation studies investigate the contributions of training data, reference-guided credit assignment, and agent–environment co-evolution.

Critical strengths
  1. Well-Motivated and Sound Framework. The proposed framework is well motivated, with a reasonable integration of its key components. Credit assignment and agent–environment co-evolution strategy are particularly interesting. And the methodology and implementation details are thoroughly described.

  2. Thorough Experimental Evaluation. The paper evaluates Neos against a broad range of recent and strong baselines across multiple benchmarks. Extensive ablation studies systematically examine the contributions of all major components, providing solid empirical support for their effectiveness.

Critical weaknesses
  1. Unclear Necessity and Efficiency of MEI. MEI is described as lightweight, but it relies on a separate 9B model, and its additional computational overhead is not sufficiently analyzed. More importantly, ablation studies only compare different MEI variants and training strategies, without a w/o MEI training variant that directly provides raw retrieved information to the agent. Such a comparison would be important to establish whether MEI is truly necessary and whether its benefits justify the additional complexity.
Questions for authors
  1. The proposed credit assignment depends on several hyperparameters in Eq.(3), including weighting parameters λd,λc,λp\lambda_d, \lambda_c, \lambda_p, the decay parameter γ\gamma, and floor parameters κe,κd\kappa_e, \kappa_d. While Table 10 reports their values, how were these values selected, and how sensitive is performance to their choices?

  2. The credit judge's severity scores si,ts_{i,t} and confidence scores ci,tc_{i,t} determine the turn-level credit weights wi,tw_{i,t} in Eq.(3). While Tables 15--16 validate the judge's turn-level labels against human annotations, they do not directly assess the reliability of these numerical scores. Could the authors provide additional validation of the accuracy of si,ts_{i,t} and ci,tc_{i,t}, for example through comparison with human ratings?

  3. Please clarify how the adapted MEI checkpoint is obtained and reused in Table 3. Specifically, please explain in detail the difference among "RL + Frozen adapted" setting, "SFT+adapted" setting and "alternating" setting.

  4. Table 1 compares Neos with results reported in prior work, potentially involving different evaluation protocols. For example, original DeepVoyager-VL [1] uses the same Vision-DeepResearch evaluation prompt but with Qwen3.6-35B-A3B as the judge, whereas Neos uses Qwen3.7-Flash. Could authors clarify the sensitivity of the choice of evaluation prompt and judge model.

  5. Previous works like DeepVoyager-VL's EventVoyage-VL [1] also constructs long-horizon search tasks with explicit intermediate visual dependencies through a structure-before-language synthesis pipeline. What are the key methodological differences and advantages of the proposed evidence-dependent data synthesis pipeline compared with EventVoyage-VL?

[1] Zhang, Huanyao, et al. "DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agents." arXiv preprint arXiv:2608.01827 (2026).

3 comments

Thank you for your careful review and for recognizing the motivation and soundness of our framework, particularly its credit assignment and agent–environment co-evolution strategy, as well as the thorough description of our methodology and implementation. We also appreciate your acknowledgment of our broad experimental evaluation and systematic ablation studies supporting the effectiveness of the key components. We address your concerns in detail below and incorporate the additional experiments and accompanying analyses into the Appendix.E of our revised manuscript. These additions are highlighted in dark teal for ease of reference.

W1: Unclear Necessity and Efficiency of MEI. MEI is described as lightweight, but it relies on a separate 9B model, and its additional computational overhead is not sufficiently analyzed. More importantly, ablation studies only compare different MEI variants and training strategies, without a w/o MEI training variant that directly provides raw retrieved information to the agent. Such a comparison would be important to establish whether MEI is truly necessary and whether its benefits justify the additional complexity.

MEI integrates goal-conditioned summarization as well as goal-based ranking and description of retrieved images. Compared to other approaches that rely on models with tens or even hundreds of billions of parameters, our approach is indeed lightweight. To address your concern regarding efficiency, we have added the following analysis: Based on 100 randomly sampled calls each for web visit, image search, and reverse image search, we measured and compared throughput and mean latency across three settings: using the Qwen3.7-Plus API as a substitute for MEI, the untrained Qwen3.5-9B base model, and our co-evolved MEI.

MEI modelThroughput (tokens/s)Mean latency (s)
Qwen3.7-Plus8.0732.04
Qwen3.5-9B42.354.96
Co-evolved MEI41.925.18

Regarding your concern about the lack of "a w/o MEI training variant that directly provides raw retrieved information to the agent," we have already included an ablation study in Appendix E (Table 13). This table compares our MEI-dependent workflow against the RedSearcher-MM workflow, which feeds all retrieved information directly to the policy model. We further add more comparisons of different search workflows in our reply to Reviewer fT8p's Q2.

Q1: The proposed credit assignment depends on several hyperparameters in Eq. (3), including weighting parameters λd\lambda_d, λc\lambda_c, λp\lambda_p, the decay parameter γ\gamma, and floor parameters κe\kappa_e, κd\kappa_d. While Table 10 reports their values, how were these values selected, and how sensitive is performance to their choices?

We sincerely thank you for your question regarding the choice of credit assignment parameters. Our current parameter settings were determined through extensive early-stage experimentation and empirical analysis. We will include an ablation study and detailed analysis of λ in the revised manuscript.

ConfigurationBC-VLFVQA
Higher productive weight, λp = 0.359.478.7
Higher corrective weight, λc = 0.562.481.7
Lower corrective weight, λc = 0.161.480.0
Higher downstream strength, λd = 0.858.977.3
Slower downstream decay, γ = 0.961.981.3
Default63.283.3

Deviating from the normal setting for any of these parameters leads to training instability or misdirected optimization. Specifically, an excessively high productive weight causes advantages to be overly concentrated; a similarly high corrective weight has the same effect, though with lower sensitivity. Conversely, if the corrective weight is set too low, effective turns fail to receive sufficient reward. Furthermore, both downstream strength and downstream decay directly affect how neutral turns are penalized during model optimization.

Q2: The credit judge's severity scores si,ts_{i,t} and confidence scores ci,tc_{i,t} determine the turn-level credit weights wi,tw_{i,t} in Eq. (3). While Tables 15–16 validate the judge's turn-level labels against human annotations, they do not directly assess the reliability of these numerical scores. Could the authors provide additional validation of the accuracy of si,ts_{i,t} and ci,tc_{i,t}, for example through comparison with human ratings?

Reusing the existing 60 failure trajectories and their manual category labels, two annotators classified the critical errors into mild, moderate, and severe levels, which correspond to minor search deviations, critical binding errors requiring explicit correction, and the failure of the main reasoning path, respectively. Inter-annotator agreement was evaluated using weighted κ. For the error events correctly identified by the judge, we computed the Spearman correlation between the raw severity scores and the human adjudication levels. Regarding confidence, within each of the three categories—Critical, Corrective, and Productive—we calculated the AUROC for distinguishing between correct and incorrect judgments. All metrics are reported with 95% confidence intervals estimated via trajectory-level bootstrapping.

AssessmentMetricResult [95% CI]
Human severity agreementLinear-weighted Cohen’s κ0.683 [0.521, 0.824]
Judge–human severity associationSpearman’s ρ0.791 [0.658, 0.886]
Confidence: CriticalWithin-class AUROC0.924 [0.847, 0.976]
Confidence: CorrectiveWithin-class AUROC0.763 [0.608, 0.893]
Confidence: ProductiveWithin-class AUROC0.709 [0.583, 0.827]

Human annotators showed substantial agreement on severity (weighted κ = 0.683), and the judge’s severity scores were strongly associated with human ratings among correctly identified error events (Spearman’s ρ = 0.791). Confidence distinguished correct from incorrect judgments most effectively for Critical calls (AUROC = 0.924), with moderate discrimination for Corrective and Productive calls (0.763 and 0.709). These results support the ordinal reliability of severity scores and the usefulness of confidence for identifying more reliable judgments.

Q3: Please clarify how the adapted MEI checkpoint is obtained and reused in Table 3. Specifically, please explain in detail the difference among "RL + Frozen adapted" setting, "SFT+adapted" setting and "alternating" setting.

We sincerely appreciate your insightful comments regarding the limitations of our ablation study. In response, we have revised the section to provide a more comprehensive explanation below and have included additional experimental analyses to demonstrate the benefits of our post-SFT design.

You can find the full introduction and more detailed experiments in our reply to Reviewer 4oJ8's Q3.

Q4: Table 1 compares Neos with results reported in prior work, potentially involving different evaluation protocols. For example, original DeepVoyager-VL [1] uses the same Vision-DeepResearch evaluation prompt but with Qwen3.6-35B-A3B as the judge, whereas Neos uses Qwen3.7-Flash. Could authors clarify the sensitivity of the choice of evaluation prompt and judge model.

Thank you for raising this question and we understand your concern regarding the potential inconsistency in semantic judgment when using different models as judges. We add a answer judge model ablation in our reply to Reviewer 4oJ8's Q4.

To address your concern regarding the sensitivity of the choice of evaluation prompt, we have conducted an additional experiment to assess the consistency of model performance under different answer-judge prompts. Specifically, we evaluated our Neos-9B model on the full BCVL benchmark (399 instances) and report the agreement rate in correctness judgments between the WebThinker and Vision-DeepResearch evaluation prompts:

Judge ModelVision-DeepResearch / WebThinker Agreement (%)
Qwen3.7-Flash63.2 / 63.6
Qwen3.6-35B-A3B63.2 / 62.9
DeepSeek-V4-Flash63.4 / 63.2

The results demonstrate that the judge models remain stable across different evaluation prompts, with discrepancies amounting to only 1–2 instances at most.

Q5: Previous works like DeepVoyager-VL's EventVoyage-VL [1] also constructs long-horizon search tasks with explicit intermediate visual dependencies through a structure-before-language synthesis pipeline. What are the key methodological differences and advantages of the proposed evidence-dependent data synthesis pipeline compared with EventVoyage-VL?

Thank you for raising this question. We are happy to clarify how our method differs from DeepVoyager-VL’s structure-before-language synthesis pipeline and explain our advantages. DeepVoyager-VL is an excellent contribution, and its EventVoyager-VL pipeline shares a similar goal with ours but takes a different starting point for data construction. It first associates entities with images and extracts visual features, then constructs questions requiring visual reasoning based on the relationships between these entities. This extensive image collection and processing before question construction can incur higher costs. In contrast, we first algorithmically construct a text-guided directed acyclic graph (DAG) as the scaffold for a multi-hop question. We then selectively search for images at intermediate nodes to introduce visual checkpoints into the question. Combining our additional verification with visual checkpoints and question sampling, our datapipeline could better ensure that the visual information is indispensable for guiding the subsequent reasoning step, rather than merely supplementary.

Reviewer_4oJ8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Neos, a multimodal deep-search framework that interleaves web search with visual reasoning. It constructs tasks requiring intermediate visual evidence and uses successful reference trajectories to assign turn-level credit, distinguishing critical errors from useful or corrective actions within failed searches. A trainable Multimodal Evidence Interface converts retrieved content into goal-conditioned observations and co-adapts with the search policy through alternating policy reinforcement learning and interface supervised fine-tuning. Across ten benchmarks, Neos-9B improves average performance by 17.4 percentage points over its backbone using the same tool workflow.

Critical strengths
  1. Reference-guided credit assignment. Neos preserves positive credit for useful and corrective actions within failed searches while targeting penalties for critical errors. Table 5 reports gains of 3.6–5.0 points over outcome-only GRPO across three benchmarks, supporting the value of this finer-grained supervision for learning long-horizon search.
  2. Co-adaptation supported by controlled comparisons. Neos alternates search-policy RL with evidence-interface SFT. Table 3 shows consistent improvements over training with an already adapted but frozen MEI under matched policy-update budgets, providing evidence that continued co-adaptation contributes beyond simply using a stronger evidence interface.
Critical weaknesses
  1. Limited validation of MEI self-training supervision. Figure 3 and Section 3.5 show that MEI learns from its own generated responses, retaining all valid calls from successful trajectories and selected calls from failures. However, final-answer correctness does not establish the accuracy of every intermediate response. The source faithfulness of responses retained solely through trajectory success is not directly evaluated, leaving uncertainty about whether the filtering adequately prevents erroneous local outputs from being reinforced.
  2. Uncertain robustness and cost–benefit of continued co-evolution. Table 3 includes a strong control using an already-adapted but frozen MEI, over which alternating updates yield gains of 0.7–1.0 percentage points across three benchmarks. These results support continued adaptation, but repeated-run uncertainty estimates are absent. Moreover, matching policy-update budgets does not establish comparable total training costs. Consequently, the stability of these incremental gains and whether they justify the additional training complexity remain insufficiently characterized.
Questions for authors
  1. Reliability of MEI self-training targets. Section 3.5 retains all valid MEI calls from successful trajectories, although final-answer correctness does not guarantee the accuracy of each intermediate response. Can the authors audit these retained responses for faithfulness to the retrieved sources, quantify unsupported or incorrect statements, and explain how such errors are controlled during subsequent MEI updates?
  2. Contribution of process-filtered MEI supervision. Section 3.5 additionally trains MEI on productive or corrective interactions from failed searches. Can the authors compare this approach with MEI training using successful trajectories alone, under matched policy-training conditions and MEI-update budgets? This would isolate the benefit of incorporating selected interactions from failures.
  3. Incremental value and cost of post-SFT training. Table 1 reports an average improvement from 58.7 after SFT to 60.7 for the full system. Can the authors clarify which MEI checkpoint is used for the SFT results and extend the existing training-stage comparisons to report total training costs and variation across training seeds for SFT-only, RL with a frozen adapted MEI, and full co-evolution? Including rollout generation, judge calls, and MEI updates would help assess whether the incremental gains justify the additional training cost.
  4. Reliability of answer-correctness judgments. Appendix C.2 uses Qwen3.7-Flash to evaluate answer correctness across ten benchmarks, whereas the human audit in Tables 15–16 concerns the credit judge. Can the authors report agreement between the answer judge and human assessments on a representative evaluation subset, and whether the main conclusions remain stable under an alternative answer judge?
3 comments

Thank you for your careful review and for recognizing the value of our reference-guided credit assignment in preserving credit for useful and corrective actions within failed searches. We also appreciate your acknowledgment that our controlled comparisons support the benefits of continued policy–MEI co-adaptation beyond using an already adapted but frozen evidence interface. We address your concerns in detail below and incorporate the additional experiments and accompanying analyses into the Appendix.E of our revised manuscript. These additions are highlighted in blue for ease of reference.

W1 / Q1: Limited validation of MEI self-training supervision. Figure 3 and Section 3.5 show that MEI learns from its own generated responses, retaining all valid calls from successful trajectories and selected calls from failures. However, final-answer correctness does not establish the accuracy of every intermediate response. The source faithfulness of responses retained solely through trajectory success is not directly evaluated, leaving uncertainty about whether the filtering adequately prevents erroneous local outputs from being reinforced.

We audited 180 trajectories with four coauthers, stratified by training stage and tool, plus 40 productive/corrective calls retained from failures. Each audit used the stored inputs and MEI outputs from the original call. Entries report supported / unsupported or unverifiable / contradicted or incorrect proportions (%).

Source of audited callsEarlyMiddleLate
WebVisit, successful trajectories83.4 / 9.2 / 7.488.7 / 3.5 / 7.890.2 / 5.8 / 4.0
ImageSearch, successful trajectories74.6 / 19.3 / 6.180.4 / 12.7 / 6.983.5 / 8.4 / 8.1
ReverseImageSearch, successful trajectories80.3 / 9.7 / 10.084.8 / 6.2 / 9.083.1 / 9.6 / 7.3
Retained calls from failed trajectories91.5 / 2.4 / 6.189.3 / 5.6 / 5.190.6 / 7.8 / 1.6

The proportion of supported outputs increased from early to late training by 6.8, 8.9, and 2.8 percentage points for WebVisit, ImageSearch, and ReverseImageSearch, while remaining around 90% for retained calls from failed trajectories.

During MEI training, we did not verify the correctness of each individual sample, primarily due to time and resource constraints. However, we attribute this to the distributional shift induced by co-evolution: specifically, the agent and MEI gradually adapt to each other throughout training, resulting in a progressively decreasing proportion of negative samples generated by MEI. We appreciate your suggestion for validation, and accordingly conducted this experiment. The results show that the accuracy of MEI outputs steadily improves over the course of co-evolutionary training, effectively corroborating this insight.

W2 / Q2: Uncertain robustness and cost–benefit of continued co-evolution. Table 3 includes a strong control using an already-adapted but frozen MEI, over which alternating updates yield gains of 0.7–1.0 percentage points across three benchmarks. These results support continued adaptation, but repeated-run uncertainty estimates are absent. Moreover, matching policy-update budgets does not establish comparable total training costs. Consequently, the stability of these incremental gains and whether they justify the additional training complexity remain insufficiently characterized.

We sincerely thank you for your careful reading of our MEI self-training design and your valuable suggestions. In response, we have conducted the following additional experiments: Starting from the same 9B SFT policy model and the post-SFT MEI model, and under identical hyperparameter settings (e.g., update batch size), we performed RL ablation studies on different training data selection strategies during MEI's RL co-evolution training. We can find that number of selected failure samples is 10%–25% of the number of correct samples. Based on these results, we conclude that selected failures serve as an effective supplement to diverse training samples and effectively reinforce the correct paradigm.

MEI supervisionBC-VLMM-BCFVQA
Successful trajectories only61.938.781.3
Successful + selected failed trajectories63.239.583.3

Q3: Incremental value and cost of post-SFT training. Table 1 reports an average improvement from 58.7 after SFT to 60.7 for the full system. Can the authors clarify which MEI checkpoint is used for the SFT results and extend the existing training-stage comparisons to report total training costs and variation across training seeds for SFT-only, RL with a frozen adapted MEI, and full co-evolution? Including rollout generation, judge calls, and MEI updates would help assess whether the incremental gains justify the additional training cost.

We provide a more comprehensive explanation below and have included additional experimental analyses to demonstrate the benefits of our post-SFT design.

The checkpoint definitions in Table 3. “Initial” and “Adapted” denote MEI before and after alternating training. SFT rows share the same policy; RL rows restart from that policy with the indicated MEI frozen. The frozen-adapted checkpoint comes from a separate alternating run. Alternating training starts with the SFT policy and initial MEI and updates both. Unless MEI itself is being ablated, evaluation—including the main table’s SFT row—uses the final adapted MEI.

Due to time and resource constraints, we have conducted one additional repeat of the ablation training and evaluation run to demonstrate the effectiveness of our training design (the left value denotes the original result, while the right value denotes the new result). We believe these results can address your concern regarding the absence of repeated-run uncertainty estimates.

PolicyMEIMMSearch+BC-V3VDR
SFTInitial42.8 / 42.433.7 / 33.328.0 / 27.6
SFTAdapted44.8 / 45.035.3 / 35.730.2 / 30.0
RLFrozen initial45.8 / 45.035.7 / 35.528.2 / 29.0
RLFrozen adapted46.9 / 46.336.3 / 35.730.4 / 29.8
AlternatingUpdated alternately47.6 / 48.237.0 / 37.331.4 / 30.8

This provides stronger support for our ablation findings: using a co-evolved MEI as a fixed interface improves long-horizon search performance compared with freezing the initial MEI, while alternating updates consistently yield further gains.

We also analyze the cost of co-evolution training with under our implementation details. To facilitate comparison, we report the amortized time of each component on a per-step basis, together with the per-sample costs of judge calls. Rollout generation time is averaged over interaction turns. For example, when the policy and MEI are updated once every 10 steps, the time required for each update is divided by 10. MEI inference, answer judging, and credit judging times are measured within a sample's rollout and normalized to the same per-step basis.

MetricRollout generationMEI inferencePolicy updatesMEI updatesAnswer judgeCredit judge
Amortized time (h/step); judge cost (USD/sample)1.890.200.100.050.001 h; $0.000150.08 h; $0.016

Rollout generation accounts for the largest share of training time and therefore incurs a substantially higher GPU-hour-based cost than the other components. However, Table 5 shows that outcome-only GRPO provides limited performance gains on its own. Together with our ablation results, this suggests that additional components, including judge calls and MEI updates, improve training effectiveness at relatively low additional cost.

Q4: Reliability of answer-correctness judgments. Appendix C.2 uses Qwen3.7-Flash to evaluate answer correctness across ten benchmarks, whereas the human audit in Tables 15–16 concerns the credit judge. Can the authors report agreement between the answer judge and human assessments on a representative evaluation subset, and whether the main conclusions remain stable under an alternative answer judge?

We randomly sampled 20 instances each from BCVL, MMBC, and VDR. Below, we report the agreement rates between answer-judge evaluations and human annotations across three benchmarks (the left value corresponds to the post-RL Neos model, while the right value corresponds to the untrained Qwen3.5-9B model):

Answer judgeBC-VLMM-BCVDR
Qwen3.7-Flash100 / 100100 / 100100 / 100
Qwen3.6-35B-A3B100 / 100100 / 100100 / 100
DeepSeekV4-Flash100 / 100100 / 10095 / 95

The results effectively demonstrate that the accuracy of answer judges remains stable across different data distributions and judge models. In the case of VDR, a subset of questions involves progressive sub-questions or dual-answer formats, which may occasionally lead to minor ambiguities when judges assess semantic equivalence for longer responses. However, such issues do not arise in our training data or other benchmarks. Moreover, even for VDR, the resulting discrepancy is negligible. We also add an evaluation prompt ablation in our reply to Reviewer kcnX's Q4.

Thank you for the authors' detailed responses. The supplementary experiments have addressed most of my concerns, and I have no further questions. Hope these new experiments and results can be included in the final manuscript, and I will keep my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.