acceptodds
Under review as a conference paper at ICLR 2027

MDS-ZERO: Self-Improving Multimodal Deep Search via Online QA Solver Co-training

Abstract

Multimodal large language models (MLLMs) have become strong visual reasoners, yet remain limited in open-world, multi-step search that requires grounding images, selecting tools, and integrating web evidence. Existing approaches typically construct and filter fixed offline corpora whose question distributions cannot adapt as the target deep-search agent improves. We present MDS-ZERO (Multimodal Deep Search with Zero precomputed multi-hop corpora), an online multimodal QA–Solver co-training framework for self-improving multimodal deep-search agents. Starting from image-grounded VQA seed contexts, a trainable QA Generator retrieves evidence and constructs verified multi-hop questions online. A separate deep-search Solver then selects tools, gathers evidence, and answers the generated questions. To focus generation on the Solver's learning frontier, we derive a Goldilocks reward for the QA Generator from the Solver's group-wise rollout success rate: questions solved by all rollouts or by none receive zero reward, whereas questions near the current frontier are favored. We optimize the two policies independently with GRPO, without sharing parameters or gradients. Across six multimodal deep-search benchmarks, MDS-ZERO improves a Qwen3-VL-8B Solver from 36.91% to 51.58% mean accuracy and achieves performance comparable to GPT-5 under the same agent workflow. The same framework also improves a newer Qwen3.6-27B Solver by 10.82 points. These results show that verified online question generation provides a policy-specific curriculum for continually improving multimodal deep-search agents.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Deep Search, MLLMs, Multi-hop Reasoning, Tool-Augmented Agents, Synthetic Data Generation, self-improving

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vAP7OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MDS-Zero jointly trains a QA Generator and a multimodal deep-search Solver using separate GRPO objectives. The Generator constructs evidence-grounded questions from image-based VQA seeds and receives a Goldilocks reward favoring approximately 50% Solver success. The method improves an 8B Solver from 36.91% to 51.58% across six benchmarks, with additional experiments on a 27B backbone.

Critical strengths

The paper reports substantial downstream improvements and includes a useful controlled comparison: training the Generator improves the four-benchmark average from 44.28% with frozen QA to 52.40%. This supports the value of adapting the generation policy under the reported configuration. The held-out checkpoint cross-play and difficulty-target ablation provide additional supporting evidence.

Critical weaknesses
  1. The connection between the difficulty reward and actual learning value is insufficiently established: A Solver success rate near 50% provides non-degenerate GRPO groups, but may reflect either useful difficulty or ambiguity, tool instability, and adjudication noise. The existing analyses support the reward’s empirical usefulness, but do not distinguish these causes or directly establish whether higher rewards predict greater learning gains. Consequently, optimizing the Generator with GRPO may reinforce sources of variability that do not improve reasoning capability. Robustness to such proxy exploitation remains unclear.

  2. The learned QA policy may overfit the fixed image-seed distribution:The Generator is optimized on a fixed collection of image-grounded seeds. Its improved reward may therefore reflect adaptation to specific entities and relation patterns, rather than a transferable ability to generate appropriately challenging questions. Can the trained Generator maintain this ability on a new image-seed collection without further optimization? This reinforces my concern about the objective: it is not sufficiently established whether the optimization learns generalizable difficulty calibration or mainly improves reward within a restricted seed distribution.

Questions for authors
  1. What patterns emerge in the questions produced by the trained QA Generator? How do they differ from those generated before training—for example, in question types, entity-relation chains, or recurring templates? Representative examples and a characterization of these changes would help explain what the Generator actually learns.

  2. How does adaptive QA compare with frozen QA generating 16/32 candidates per seed, followed by the same verification and difficulty filtering? Please compare downstream Solver performance and total cost, including candidate generation, verification, and Solver-based scoring.

6 comments

Reviewer: The connection between the difficulty reward and actual learning value is insufficiently established: A Solver success rate near 50% provides non-degenerate GRPO groups, but may reflect either useful difficulty or ambiguity, tool instability, and adjudication noise. The existing analyses support the reward’s empirical usefulness, but do not distinguish these causes or directly establish whether higher rewards predict greater learning gains. Consequently, optimizing the Generator with GRPO may reinforce sources of variability that do not improve reasoning capability. Robustness to such proxy exploitation remains unclear.

Our response:

Our success-rate reward is a difficulty signal within a quality-controlled pipeline. The fixed harness first checks evidence support, entity dependence, answer uniqueness and stability, and merged-question coherence (Table 8). Independent evaluations then examine the reliability of the tasks, solution paths, and scoring:

  • Question quality: the external LLM-based audit evaluates 512 tasks per Generator checkpoint. At Q50, 80.3% have supported evidence/chains, 83.0% have unique and stable answers, and 73.0% are judged to require the seed image (Table 9).
  • Solution paths: among 500 correct Solver trajectories from steps 30–40, 494 (98.8%) show image clues contributing to solving and 442 (88.4%) use the full fact chain. The audit also identifies nine key-fact shortcuts, three other reference-relation omissions, and one complete text-only solution with an unnecessary image branch; these exceptions are reported explicitly. Observed use describes the recorded paths, without establishing necessity across all possible solutions.
  • Human scoring audit: manual reassessment of 100 LLM-judged answers finds 98% agreement with the judge (Appendix D.4).
  • Finite-rollout sensitivity: on 100 sampled questions at their corresponding fixed Solver checkpoints, exhaustive 8→4 subsampling gives a mean within-question reward standard deviation of 15.1 on a 0–100 reporting scale and an expected positive-to-negative GRPO advantage flip rate of 18.3% after recomputing original Generator-group means. This measures sensitivity to the number of logged outcomes; it does not separate ambiguity, retrieval instability, and judge errors.
  • Downstream learning: updating the Generator improves the four-benchmark Solver mean from 44.3% to 52.4% relative to the original frozen-QA control under matched conditions (Table 4).

Additional fixed-Solver comparison. We compare 100 question pairs from the initial Generator Q0 and trained Generator Q50 using the same final Solver S50. Both questions in each pair have five construction hops. We report mean turns for successful solutions only.

SampleQuestion pairsMean S50 turns on Q0Mean S50 turns on Q50Q50 − Q0
Both questions are 5-hop; successful solutions only1005.78.1About +2.4

The fixed-Solver comparison shows a change in observed solving effort. It does not separate useful reasoning from extra retries or retrieval noise, and does not test whether higher-reward questions produce larger learning gains.

Additional held-out evaluation. We evaluated S0 and S50 on 200 questions generated from held-out seeds. Neither the questions nor their seeds were used in training. Multiple model judges screened the questions for both image dependence and multi-hop requirements.

Evaluation subsetQuestionsS0 solve rateS50 solve rateGain
Judged to require both the image and multi-hop search20035.5%58.0%+22.5 pp

The 22.5-point gain shows that Solver improvement extends to unseen questions judged to require both image use and multi-hop search.

Additional rollout-count experiment. We also ran co-training with 16 Solver rollouts per question and compared its final results with the eight-rollout run.

Solver rollouts per questionMMSearch-PlusHR-MMSearch-VLBrowseComp-VLMMSearch-VLMean
8 (Table 4)29.953.148.478.252.4
16 (additional co-training run)32.352.851.178.753.7

With 16 rollouts, mean accuracy is 53.7%, about 1.3 percentage points above the eight-rollout run. This adds evidence about final performance at a larger rollout count. Together, the independent quality checks, observed solution paths, human scoring audit, and matched downstream gains support the reward's usefulness beyond merely producing non-degenerate groups.

Reviewer: The learned QA policy may overfit the fixed image-seed distribution:The Generator is optimized on a fixed collection of image-grounded seeds. Its improved reward may therefore reflect adaptation to specific entities and relation patterns, rather than a transferable ability to generate appropriately challenging questions. Can the trained Generator maintain this ability on a new image-seed collection without further optimization? This reinforces my concern about the objective: it is not sufficiently established whether the optimization learns generalizable difficulty calibration or mainly improves reward within a restricted seed distribution.

Our response:

Thank you for raising this concern. Section 5.1 and Table 2 already evaluate Generator generalization on 128 held-out seed QA pairs, without further Generator optimization. Each frozen Generator checkpoint produces four questions per seed, and the same fixed Solver S50 evaluates each question with eight independent rollouts. With the seed contexts, Solver, and harness held fixed, the informative-group rate increases from 44.3% for Q0 to 56.8% for Q50 (+12.5 percentage points).

These results provide evidence that the learned difficulty calibration transfers to held-out seed contexts. We will clarify the training/held-out split and the scope of this evaluation in the revision.

Reviewer: What patterns emerge in the questions produced by the trained QA Generator? How do they differ from those generated before training—for example, in question types, entity-relation chains, or recurring templates? Representative examples and a characterization of these changes would help explain what the Generator actually learns.

Our response:

We added two analyses comparing Q0 and Q50, focusing on chain completion and how a fixed Solver answers their questions.

1. Completing five-hop questions. Both checkpoints target five hops. We report the output depths from 512 generation attempts per checkpoint.

QA checkpointNo valid extension2-hop3-hop4-hop5-hop
Q051526382264
Q5023453042372

The five-hop completion rate rises from 51.6% (264/512) to 72.7% (372/512), a gain of 21.1 percentage points. Attempts with no valid extension fall from 51 to 23.

2. Solver turns at the same construction depth. We compare 100 pairs of Q0 and Q50 questions using the same final Solver checkpoint, S50. Both questions in each pair have five construction hops. We report mean turns for successful solutions only.

ComparisonQuestion pairsS50 turns on Q0 questionsS50 turns on Q50 questionsQ50 − Q0
Both questions are 5-hop; successful solutions only1005.78.1About +2.4

S50 uses about 2.4 more turns on Q50 questions than on Q0 questions in these successful solutions.

Together, these results show that Q50 completes five-hop construction more often. At the same construction depth, its questions also take the fixed S50 Solver more turns to answer successfully. This suggests better construction ability and more challenging questions after training in the evaluated sample. Turn counts alone do not prove that every hop is necessary.

Reviewer_WTPsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Existing methods rely on fixed offline corpora, whose question distributions fail to adapt as multimodal deep-search agents improve. To address this, this paper proposes MDS-Zero, an online multimodal QA–Solver co-training framework. Starting from images, a trainable QA generator retrieves evidence and constructs verifiable multi-hop questions, while an independent deep-search Solver selects tools and answers them. A Goldilocks reward is designed based on the Solver's group-wise rollout success rate. Experiments show that MDS-Zero significantly improves agent performance and achieves results comparable to advanced closed-source models under the same workflow, validating that verifiable online question generation provides a continuously improving curriculum for these agents.

Critical strengths
  1. Clear motivation around VQA. As the Solver's capabilities improve, a fixed question distribution may gradually lose its training value. This paper proposes dynamically adjusting the question generation strategy in accordance with the Solver's capabilities.
  2. The two-stage training is easy to understand and reasonable. The QA Generator and the Solver are optimized online as two independent policies. The Generator learns to produce questions with training value, while the Solver learns to complete multi-hop reasoning using external tools.
  3. The RL reward design is intuitive. Question difficulty is defined based on the Solver's rollout success rate on the same question, encouraging the generation of questions that are neither completely unsolvable nor overly simple.
  4. MDS-Zero outperforms the corresponding Qwen3-VL-8B base model on all six multimodal deep-search benchmarks, improving average accuracy from 36.91% to 51.58%. In addition, experiments on Qwen3.6-27B also show further improvements, preliminarily demonstrating its scalability.
Critical weaknesses
  1. The core framework of this paper bears significant similarities to methods such as Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning, as both adopt a two-stage generation-and-solving training paradigm.
  2. MDS-Zero requires simultaneously maintaining the QA Generator and the Solver, and executing multiple Solver rollouts for each generated question. In addition, online search, web information processing, and Harness verification may also incur substantial time and computational overhead. The paper emphasizes that it does not require pre-built multi-hop QA corpora or an additional task-specific SFT stage; however, the actual cost of a single SFT run (training time and tool expenses) is not necessarily higher than that of the proposed method.
  3. The design of Equation (2), which uses 4 as the baseline, lacks extensive validation. The current analysis of the reward function's hyperparameters is insufficient. Comparing only two target success rates is not enough to verify the robustness of the reward objective and the number of rollouts.
  4. Dependence of the QA Generator on the Solver. The paper uses the Solver's answer success rate as the primary reward signal for the QA Generator. However, a moderate success rate does not necessarily imply that a question has reasonable difficulty or high training value.
Questions for authors
  1. Could the authors clarify the core differences between MDS-Zero and methods such as Agent0? Specifically, which core designs are essential to this paper?
  2. Could the authors show the training time and tool-related expenses of MDS-Zero, and what are the trade-offs between accuracy and cost compared with SFT?
  3. The Goldilocks reward relies solely on the Solver's success rate, but a moderate success rate may also result from ambiguous answers, incorrect facts, or unstable retrieval results. Have the authors evaluated issues such as the factual correctness of generated questions, evidence sufficiency, and the validity of multi-hop reasoning?
  4. What are the final Solver performance and the variance of the QA reward under different settings? Furthermore, could the authors analyze whether increasing the number of rollouts improves the stability of curriculum learning?
8 comments

Reviewer: The core framework of this paper bears significant similarities to methods such as Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning, as both adopt a two-stage generation-and-solving training paradigm. / Could the authors clarify the core differences between MDS-Zero and methods such as Agent0? Specifically, which core designs are essential to this paper?

Our response:

MDS-Zero and Agent0 have a similar co-training structure, but differ in how they build tasks and where they are used. Agent0 studies math and general reasoning with a Python tool; MDS-Zero studies multimodal open-web search.

  • Task construction: Our Generator starts from an image-grounded seed QA, searches the web, and adds one fact at a time. The harness checks the evidence and whether the answer is unique, then combines the hops without revealing intermediate answers.
  • Answer checks: Agent0 uses the most common answer from its own rollouts as the training label. We compare Solver answers with a reference answer checked by the harness.
  • Training: The Generator receives accept/reject feedback. We train on accepted search and proposal steps. Rejected attempts and harness observations do not contribute to the loss.
  • Difficulty feedback: Agent0's uncertainty reward uses agreement among its own answers. Our Goldilocks reward uses success against a reference answer checked with web evidence.
  • Update cycle: Agent0 updates its Curriculum Agent, then freezes it to generate a task pool for Executor training. In MDS-Zero, each iteration builds and scores new questions, then updates both models from the QA and Solver rollouts collected before either update.

Reviewer: MDS-Zero requires simultaneously maintaining the QA Generator and the Solver, and executing multiple Solver rollouts for each generated question. In addition, online search, web information processing, and Harness verification may also incur substantial time and computational overhead. The paper emphasizes that it does not require pre-built multi-hop QA corpora or an additional task-specific SFT stage; however, the actual cost of a single SFT run (training time and tool expenses) is not necessarily higher than that of the proposed method. / Could the authors show the training time and tool-related expenses of MDS-Zero, and what are the trade-offs between accuracy and cost compared with SFT?

Our response:

We do not claim that omitting SFT necessarily makes co-training cheaper than a single SFT run. The benefit is a curriculum that adapts to the current Solver's needs, without requiring a large pre-generated multi-hop corpus. In the original matched frozen-versus-adaptive QA experiment (Table 4), updating the QA policy improves the four-benchmark mean from 44.3% to 52.4%. Our main run attempts 3,200 online QA-generation trajectories over 50 iterations.

Using the published workload for the original eight-rollout main run in Table 14, the illustrative LLM/API accounting gives:

External-service componentEstimated USD
QA construction, verification, and filtering LLMsApproximately $500
QA search APIApproximately $17
Solver-training summaries and LLM judgingApproximately $270
Solver-training tool APIsApproximately $79–237
External-service subtotal, excluding GPUsApproximately $1,467–1,625

The separate GPU allocation is 16 H200 GPUs for 16:21:48, totaling 261.8 GPU-hours for both agents.

Reviewer: The design of Equation (2), which uses 4 as the baseline, lacks extensive validation. The current analysis of the reward function's hyperparameters is insufficient. Comparing only two target success rates is not enough to verify the robustness of the reward objective and the number of rollouts.

Our response:

Four is the target number of correct answers out of eight rollouts: a 50% success rate. It is not the GRPO baseline, which is the mean reward within the group. We choose 50% to favor questions with both correct and incorrect attempts, so GRPO has different outcomes to learn from.

Table 3 compares target success rates of 50% and 25% under the same 50-step training setup. The mean accuracy across four benchmarks is 52.4% versus 49.1%, supporting the default 50% target.

We have also added a sampling study using 100 logged questions and their eight original Solver outcomes. For each question, we check all 70 ways of choosing four outcomes, giving 7,000 subsets in total. Each subset is rescored using the same 50% target. The average reward standard deviation per question is 15.1 on a 0–100 reporting scale; training rewards are unchanged. We also resample each candidate's Solver outcomes and recompute its original Generator-group mean. Among 40 candidates with positive advantages, an expected 18.3% become negative after this resampling.

Additional rollout-count experiment. We also ran co-training with 16 Solver rollouts per question. The four-benchmark mean rises from 52.4% with eight rollouts to 53.7% with 16, an increase of about 1.3 percentage points.

Solver rollouts per questionMMSearch-PlusHR-MMSearch-VLBrowseComp-VLMMSearch-VLMean
8 (Table 4)29.953.148.478.252.4
16 (additional co-training run)32.352.851.178.753.7

This adds a direct final-performance comparison at a larger rollout count. Reward variance and training stability for the 16-rollout run are not measured by these final scores.

Reviewer_WqK2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MDS-Zero, an online QA–Solver co-training framework for multimodal deep search. Starting from image-grounded one-hop QA seeds, a trainable QA Generator searches for evidence and constructs multi-hop questions under a fixed verification harness. A separate Solver attempts each accepted question through multiple independent tool-use rollouts. The Solver’s group-wise success rate rewards the Generator for producing questions that are challenging but solvable, while mixed-outcome rollout groups train the Solver. On six benchmarks, the method improves a Qwen3-VL-8B agent from 36.91% to 51.58% mean accuracy. The paper also reports gains on a 27B backbone and analyzes the effect of updating the Generator rather than keeping it fixed.

Critical strengths
  1. The frozen-versus-adaptive Generator experiment directly tests the paper’s central claim. Under a matched training setup, updating the Generator raises the four-benchmark average from 44.28% to 52.40%. This provides stronger evidence for adaptive task generation than the final benchmark scores alone.
  2. The checkpoint cross-play analysis offers a plausible account of the gain. Against the final Solver, the trained Generator produces more questions with mixed correct and incorrect rollouts than the initial Generator (56.8% versus 44.3%). The difficulty-target ablation also suggests that reliably solvable questions are more useful than questions pushed close to universal failure.
Critical weaknesses
  1. A five-hop construction target does not establish that solving the resulting question requires five reasoning or retrieval steps. Since the reward is based on final-answer correctness, a question may receive a favorable difficulty score even when successful rollouts use a shortcut. The external audit judges only 73.0% of questions from the final Generator to require the seed image and 80.3% to have supported evidence and chains. These results leave uncertainty about how consistently the learned curriculum trains the intended multimodal, multi-hop behavior.
  2. The paper does not characterize training saturation or the effect of seed-pool coverage. The main results use the Solver after 50 co-training iterations, but Solver benchmark performance is not reported across training checkpoints or beyond iteration 50. The experiments also use one fixed pool of 1,019 one-hop seeds. It is therefore unclear whether performance is approaching a plateau, or how the pool size affects the rate and extent of improvement.
  3. Comparisons with previously reported specialist agents are not matched for tools or total cost. The method uses substantial online search, model-assisted verification, answer judging, and Solver tool calls. The paper establishes gains over its own matched baselines, but does not yet establish whether online co-training is more effective or efficient than an offline construction pipeline under the same end-to-end budget.
Questions for authors
  1. Effective reasoning depth and shortcuts: Among accepted questions targeting five hops, what fraction actually require multiple retrieval steps to solve? Could the authors audit successful Solver trajectories, quantify shortcut solutions, and separately report performance on questions verified to require both the seed image and multi-hop search?
  2. Construction depth: How do other target depths, such as N=3,7N=3, 7, and, if feasible, N=10N=10, affect Solver performance, the fraction of informative rollout groups, question quality, and cost? When increasing depth, how are the Generator and Solver interaction budgets adjusted?
  3. Saturation and seed-pool size: Could the authors report Solver benchmark performance throughout training and extend the run beyond 50 iterations to assess whether gains saturate? Under a matched training and tool-use budget, how does changing the one-hop seed-pool size affect learning speed and final performance?
6 comments

Reviewer: A five-hop construction target does not establish that solving the resulting question requires five reasoning or retrieval steps. Since the reward is based on final-answer correctness, a question may receive a favorable difficulty score even when successful rollouts use a shortcut. The external audit judges only 73.0% of questions from the final Generator to require the seed image and 80.3% to have supported evidence and chains. These results leave uncertainty about how consistently the learned curriculum trains the intended multimodal, multi-hop behavior.

Our response:

We agree that a five-hop construction target does not establish a minimum solution depth. To examine actual solving behavior, we audited 500 correct Solver trajectories, checking image use and fact-chain use on the same trajectories.

Table 9 judges whether a question needs the image (73.0%); this audit checks image use in correct solutions (98.8%). Even when the image is not strictly needed, using it can help train the Solver to understand visual clues and use them to guide search.

Image-use assessmentCountPercentage
Image clues observed to contribute to solving49498.8%
Complete text-only solution; image branch unnecessary10.2%
Indeterminate51.0%
Total500100%
Fact-chain assessmentCountPercentage
Confirmed key-fact shortcut91.8%
Valid image-grounded, multi-fact alternative solution40.8%
Other confirmed omissions of reference relations30.6%
Full fact chain observed to be used44288.4%
No additional fact relations; not applicable428.4%
Total500100%

Among the correct trajectories we checked, 98.8% used image clues and 88.4% used the full fact chain. We found key-fact shortcuts or other skipped links in the reference chain in 2.4%, and one complete text-only solution (0.2%). These results show that most checked solutions use the image and facts as intended.

Additional fixed-Solver comparison. We compare 100 question pairs from the initial Generator Q0 and trained Generator Q50 using the same final Solver S50. Both questions in each pair have five construction hops. We report mean turns for successful solutions only.

SampleQuestion pairsMean S50 turns on Q0Mean S50 turns on Q50Q50 − Q0
Both questions are 5-hop; successful solutions only1005.78.1About +2.4

In this successful sample, S50 takes about 2.4 more turns on Q50 questions at the same construction depth. This adds evidence about observed solving behavior after Generator training.

Reviewer: The paper does not characterize training saturation or the effect of seed-pool coverage. The main results use the Solver after 50 co-training iterations, but Solver benchmark performance is not reported across training checkpoints or beyond iteration 50. The experiments also use one fixed pool of 1,019 one-hop seeds. It is therefore unclear whether performance is approaching a plateau, or how the pool size affects the rate and extent of improvement. / Saturation and seed-pool size: Could the authors report Solver benchmark performance throughout training and extend the run beyond 50 iterations to assess whether gains saturate? Under a matched training and tool-use budget, how does changing the one-hop seed-pool size affect learning speed and final performance?

Our response:

Thank you for the suggestion. We report the Solver's accuracy (%) from step 0 to step 80 below.

StepBrowseComp-VLHR-MMSearch-VLMMSearch+MMSearch-VLMean
0 (Base)35.932.118.457.135.9
1041.442.021.364.742.3
2044.443.625.865.944.9
3047.651.828.174.150.4
4047.649.829.978.251.4
5048.453.129.978.252.4
6048.952.829.081.253.0
7051.147.225.378.250.5
8048.947.925.378.250.1

Among the reported checkpoints, the four-benchmark mean peaks at step 60 (53.0%), only 0.6 percentage points above step 50 (52.4%), and declines at steps 70–80. These results show limited additional gains from extending training beyond step 50.

We also ran a 490-seed experiment using only FVQA and Live VQA seeds. All other training settings and turn/token/tool-use limits were the same. Both runs were trained for 50 steps.

Seed poolSeedsBrowseComp-VLHR-MMSearch-VLMMSearch+MMSearch-VLMean
FVQA + Live VQA49047.644.927.270.647.6
Main mixed-source pool1,01948.453.129.978.252.4

Overall, the checkpoint results suggest that gains level off around steps 50–60. In this comparison, the smaller and less diverse seed pool gives lower final performance.

Reviewer: Comparisons with previously reported specialist agents are not matched for tools or total cost. The method uses substantial online search, model-assisted verification, answer judging, and Solver tool calls. The paper establishes gains over its own matched baselines, but does not yet establish whether online co-training is more effective or efficient than an offline construction pipeline under the same end-to-end budget.

Our response:

Thank you for raising this point. Strictly matched comparisons are difficult because specialists use different GPUs, external models, tools, and APIs. We therefore use MTA-Agent, whose verification-and-merging harness we adapt, as the closest reference. The LLM/API figures below are illustrative scenario estimates, not measured API bills; GPU resources are listed separately.

LLM/API component (USD)MTA-8BMDS-Zero-8B (original 8-rollout main run)
QA construction and verification LLMsApproximately $3,556*Approximately $500
QA search/tool APIsFees not separately itemized; 9.3 tool calls per generation sample reportedApproximately $17 for 17,000 QA searches
Solver-training summaries and LLM judgingNot separately itemizedApproximately $270
Solver-training tool APIsNot separately itemizedApproximately $79–237
  • MTA estimate: 12,699 retained multi-hop QA rows × 28 cents = $3,555.7, assuming its reported price is per retained QA rather than per complete generation chain.

GPU resources, separately: MTA-8B reports 8 H100 GPUs for 100 training steps; MDS-Zero uses 16 H200 GPUs for 50 co-training iterations, totaling 261.8 allocated GPU-hours over 16:21:48, including both agents.

We provide this comparison as a cost reference, not a matched-budget test. The original matched frozen-QA comparison in Table 4 supports the value of updating the Generator online.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comment to All Reviewers and the Area Chair (1/1)

We thank all reviewers for their careful feedback. We have added experiments and analyses to address the main concerns about MDS-Zero, alongside clarifications of the method and its costs. The main updates are summarized below.

  1. Image grounding and shortcuts. We audited 500 correct Solver trajectories: 98.8% used image clues and 88.4% used the full recorded fact chain. We also evaluated 200 questions generated from held-out seeds and screened by multiple model judges for image dependence and multi-hop requirements. Solver accuracy improves from 35.5% to 58.0% (+22.5 percentage points). We distinguish observed image use from strict necessity.

  2. What the Generator learns. We added realized-depth statistics for 3,200 generation attempts. Across 512 attempts per Generator checkpoint, five-hop completion improves from 51.6% to 72.7%. On 100 matched five-hop pairs, successful solutions with the same final Solver take 5.7 turns for questions from the initial Generator and 8.1 for the trained Generator. A separate blinded 100-pair LLM quality audit prefers the trained Generator in 67 pairs, the initial Generator in 19, with 14 ties.

  3. Seed coverage and generalization. A 490-seed pool reaches 47.6% mean accuracy, compared with 52.4% for the main 1,019-seed pool under the same training settings and configured tool-use limits. This changes both pool size and source mix. We also clarify the existing Generator evaluation on 128 held-out seeds: with the Solver fixed, the informative-group rate increases from 44.3% to 56.8%.

  4. Training duration. We add a checkpoint study covering steps 0–80. Mean accuracy is 52.4% at step 50 and peaks at 53.0% at step 60, with no sustained improvement through step 80.

  5. Stronger frozen-QA control. We added a separate offline control using a frozen Generator with 16 QA candidates per seed and dynamic sampling during Solver training. It reaches 50.4% mean accuracy, compared with 52.4% for the main co-training run. The original frozen-QA control in Table 4 remains 44.3%; the proposed method still constructs its questions online. We report 6,400 candidates generated, 4,128 evaluated, and 1,600 used for updates.

  6. Reward feedback and rollout count. We clarify that four is the 50% target success count for eight Solver rollouts, not the GRPO baseline. Exhaustive 8-to-4 subsampling on 100 logged questions gives 7,000 subsets and quantifies changes in rewards and update directions. An additional co-training run with 16 Solver rollouts per question reaches 53.7%, versus 52.4% with eight rollouts. These analyses address feedback sensitivity and final performance separately.

  7. Method and cost clarifications. We explain that Agent0 follows block-alternating evolution: it trains the Curriculum Agent against a frozen Executor, then separately generates and filters a task pool before sampling new trajectories to train the Executor. In contrast, MDS-Zero updates both policies within each iteration from the same freshly collected, correctness-scored rollout groups, making its contribution a same-batch, correctness-grounded co-training formulation rather than generic co-evolution. We also report external-service workload and the original run's 261.8 H200 GPU-hours, and label price estimates as illustrative, without claiming matched-budget superiority.

Further details and tables are provided in our point-by-point responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.