- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MDS-Zero jointly trains a QA Generator and a multimodal deep-search Solver using separate GRPO objectives. The Generator constructs evidence-grounded questions from image-based VQA seeds and receives a Goldilocks reward favoring approximately 50% Solver success. The method improves an 8B Solver from 36.91% to 51.58% across six benchmarks, with additional experiments on a 27B backbone.
Critical strengths
The paper reports substantial downstream improvements and includes a useful controlled comparison: training the Generator improves the four-benchmark average from 44.28% with frozen QA to 52.40%. This supports the value of adapting the generation policy under the reported configuration. The held-out checkpoint cross-play and difficulty-target ablation provide additional supporting evidence.
Critical weaknesses
-
The connection between the difficulty reward and actual learning value is insufficiently established: A Solver success rate near 50% provides non-degenerate GRPO groups, but may reflect either useful difficulty or ambiguity, tool instability, and adjudication noise. The existing analyses support the reward’s empirical usefulness, but do not distinguish these causes or directly establish whether higher rewards predict greater learning gains. Consequently, optimizing the Generator with GRPO may reinforce sources of variability that do not improve reasoning capability. Robustness to such proxy exploitation remains unclear.
-
The learned QA policy may overfit the fixed image-seed distribution:The Generator is optimized on a fixed collection of image-grounded seeds. Its improved reward may therefore reflect adaptation to specific entities and relation patterns, rather than a transferable ability to generate appropriately challenging questions. Can the trained Generator maintain this ability on a new image-seed collection without further optimization? This reinforces my concern about the objective: it is not sufficiently established whether the optimization learns generalizable difficulty calibration or mainly improves reward within a restricted seed distribution.
Questions for authors
-
What patterns emerge in the questions produced by the trained QA Generator? How do they differ from those generated before training—for example, in question types, entity-relation chains, or recurring templates? Representative examples and a characterization of these changes would help explain what the Generator actually learns.
-
How does adaptive QA compare with frozen QA generating 16/32 candidates per seed, followed by the same verification and difficulty filtering? Please compare downstream Solver performance and total cost, including candidate generation, verification, and Solver-based scoring.
6 comments
Reviewer: The learned QA policy may overfit the fixed image-seed distribution:The Generator is optimized on a fixed collection of image-grounded seeds. Its improved reward may therefore reflect adaptation to specific entities and relation patterns, rather than a transferable ability to generate appropriately challenging questions. Can the trained Generator maintain this ability on a new image-seed collection without further optimization? This reinforces my concern about the objective: it is not sufficiently established whether the optimization learns generalizable difficulty calibration or mainly improves reward within a restricted seed distribution.
Our response:
Thank you for raising this concern. Section 5.1 and Table 2 already evaluate Generator generalization on 128 held-out seed QA pairs, without further Generator optimization. Each frozen Generator checkpoint produces four questions per seed, and the same fixed Solver S50 evaluates each question with eight independent rollouts. With the seed contexts, Solver, and harness held fixed, the informative-group rate increases from 44.3% for Q0 to 56.8% for Q50 (+12.5 percentage points).
These results provide evidence that the learned difficulty calibration transfers to held-out seed contexts. We will clarify the training/held-out split and the scope of this evaluation in the revision.
Reviewer: What patterns emerge in the questions produced by the trained QA Generator? How do they differ from those generated before training—for example, in question types, entity-relation chains, or recurring templates? Representative examples and a characterization of these changes would help explain what the Generator actually learns.
Our response:
We added two analyses comparing Q0 and Q50, focusing on chain completion and how a fixed Solver answers their questions.
1. Completing five-hop questions. Both checkpoints target five hops. We report the output depths from 512 generation attempts per checkpoint.
| QA checkpoint | No valid extension | 2-hop | 3-hop | 4-hop | 5-hop |
|---|---|---|---|---|---|
| Q0 | 51 | 52 | 63 | 82 | 264 |
| Q50 | 23 | 45 | 30 | 42 | 372 |
The five-hop completion rate rises from 51.6% (264/512) to 72.7% (372/512), a gain of 21.1 percentage points. Attempts with no valid extension fall from 51 to 23.
2. Solver turns at the same construction depth. We compare 100 pairs of Q0 and Q50 questions using the same final Solver checkpoint, S50. Both questions in each pair have five construction hops. We report mean turns for successful solutions only.
| Comparison | Question pairs | S50 turns on Q0 questions | S50 turns on Q50 questions | Q50 − Q0 |
|---|---|---|---|---|
| Both questions are 5-hop; successful solutions only | 100 | 5.7 | 8.1 | About +2.4 |
S50 uses about 2.4 more turns on Q50 questions than on Q0 questions in these successful solutions.
Together, these results show that Q50 completes five-hop construction more often. At the same construction depth, its questions also take the fixed S50 Solver more turns to answer successfully. This suggests better construction ability and more challenging questions after training in the evaluated sample. Turn counts alone do not prove that every hop is necessary.
Reviewer: The connection between the difficulty reward and actual learning value is insufficiently established: A Solver success rate near 50% provides non-degenerate GRPO groups, but may reflect either useful difficulty or ambiguity, tool instability, and adjudication noise. The existing analyses support the reward’s empirical usefulness, but do not distinguish these causes or directly establish whether higher rewards predict greater learning gains. Consequently, optimizing the Generator with GRPO may reinforce sources of variability that do not improve reasoning capability. Robustness to such proxy exploitation remains unclear.
Our response:
Our success-rate reward is a difficulty signal within a quality-controlled pipeline. The fixed harness first checks evidence support, entity dependence, answer uniqueness and stability, and merged-question coherence (Table 8). Independent evaluations then examine the reliability of the tasks, solution paths, and scoring:
Additional fixed-Solver comparison. We compare 100 question pairs from the initial Generator Q0 and trained Generator Q50 using the same final Solver S50. Both questions in each pair have five construction hops. We report mean turns for successful solutions only.
The fixed-Solver comparison shows a change in observed solving effort. It does not separate useful reasoning from extra retries or retrieval noise, and does not test whether higher-reward questions produce larger learning gains.
Additional held-out evaluation. We evaluated S0 and S50 on 200 questions generated from held-out seeds. Neither the questions nor their seeds were used in training. Multiple model judges screened the questions for both image dependence and multi-hop requirements.
The 22.5-point gain shows that Solver improvement extends to unseen questions judged to require both image use and multi-hop search.
Additional rollout-count experiment. We also ran co-training with 16 Solver rollouts per question and compared its final results with the eight-rollout run.
With 16 rollouts, mean accuracy is 53.7%, about 1.3 percentage points above the eight-rollout run. This adds evidence about final performance at a larger rollout count. Together, the independent quality checks, observed solution paths, human scoring audit, and matched downstream gains support the reward's usefulness beyond merely producing non-degenerate groups.