- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper studies a failure mode in multi-hop retrieval QA: when a first-hop candidate answer is committed despite insufficient evidence, later queries may continue in the wrong direction and cause the final task to fail. To address this, the authors use LLM-generated simulated retrieval passages to train an evidence verifier. They then train a value model on paired Commit/Repair rollouts to decide whether an uncertain first-hop answer should be repaired before continuing. They evaluate the method on several multi-hop QA datasets using Qwen2.5-3B and Qwen2.5-7B, and report results that exceed many of the baselines in the paper. The paper presents its problem clearly and is well written. However, I find the research area overly saturated, the effectiveness of the proposed training method insufficiently established, and both the motivation and experimental design seriously flawed. I therefore lean toward rejecting the paper.
Critical strengths
-
The paper describes its training and evaluation settings in considerable detail and is generally transparent.
-
The writing is good and easy to follow.
Critical weaknesses
Major Concerns:
-
The only backbone models are Qwen2.5-3B and Qwen2.5-7B. These models are about two years old and are not representative of more recent systems.
-
The method adds a verifier, another round of policy training for repair, and a value model. This makes the training and inference pipeline much more complicated, yet Table 2 shows only a 2.3-point gain in average F1 over the w/o Verifier baseline. I do not think this gain justifies the added complexity.
-
The motivation is intuitive. However, the authors report that 377 of 1,325 sampled trajectories (28.45%) exhibit premature first-hop answer binding without explaining whether the trajectories were assessed by humans or LLMs, or which policy checkpoint generated them. I find this omission puzzling.
-
The number of components substantially increases the complexity of training and evaluation and may complicate real-world deployment. This is especially concerning because the paper does not show whether the verifier and value model generalize to other backbone models on which they were not trained.
-
The policy model is initialized from a ZeroSearch checkpoint. This raises an attribution concern: some of the gains may come from that starting point, so the SOTA comparisons in Table 1 may partly reflect an advantage in initialization rather than the proposed components.
Minor Concerns:
-
The research question feels somewhat dated to me. There is already a large and somewhat redundant body of work on RL-based search agent. The setting also feels toy-like: the evaluation uses only multi-hop QA datasets, and the maximum response length is 2,048 tokens. I do not think the paper offers many new insights. This is a matter of personal taste; I do not expect the AC or the authors to share my view, and I am not asking them to rebut this point.
-
Some evaluation benchmarks, particularly Bamboogle, are small. Averaging results over multiple runs would be preferable.
Questions for authors
-
How do the authors prevent overfitting to simulated retrieval text? How are the simulated search results aligned with real search results?
-
Could a simple prompt asking a frontier LLM (such as GPT-6-Astra, Kimi K3, or Claude Fable 5.1) to review its first-hop candidate against the retrieved evidence address the same problem? More generally, are there simpler ways to address this issue when using frontier models?
-
In my own attempts, I was unable to reproduce stable ZeroSearch training. Independent evaluations also report substantially lower ZeroSearch scores than Search-R1 and R1-Searcher under their respective shared evaluation setups. For example, UR² reports average F1 scores of 33.7 for ZeroSearch, 56.1 for Search-R1, and 54.9 for R1-Searcher (arXiv:2508.06165; see also related evaluations in arXiv:2510.1882, 2505.13258 and 2505.15107). This raises the possibility that ZeroSearch’s simulation-based training has limitations. How did you ensure training stability when continuing from a ZeroSearch checkpoint? Could you specify the exact checkpoint used and report training curves and results?
2 comments
-
Of course, this does not invalidate your existing conclusions. My concern is not whether the gains on older backbones hold, but how prevalent this problem is in current models and how broadly your method applies. To give a simple example, many commercial models fabricated citations in 2023, whereas current models augmented with retrieval rarely exhibit this problem. Similar examples abound in the development of deep learning. Identifying a phenomenon through a preliminary analysis of GlobalRAG alone, and demonstrating improvements only on Qwen2.5-3B and Qwen2.5-7B, provides limited evidence of the broader applicability of this work.
-
Thank you for the clarification.
-
Thank you for providing the additional information.
-
My concern remains unresolved. The analogy to hypothetical future autonomous research is unrelated to my question. Regarding the gap between simulated and real retrieval, you have described several mitigation measures but have not provided direct validation. For example, how well does the verifier assess candidate support in real retrieved documents? The existing results are insufficient to rule out its reliance on features specific to simulated evidence.
-
I have fully reproduced and closely studied ZeroSearch, including using its summarizer, whose outputs contained substantial hallucinations. Its training pipeline also contained implementation errors, such as mixing chat-template conventions for instruction-tuned and base models. If the original ZeroSearch checkpoint cannot even generalize from an online-search corpus to a local corpus, this raises concerns about possible overfitting to the source of retrieval evidence. Since your training starts from their checkpoint, I believe further disclosure or experiments are needed to clarify whether your method exhibits similar problems.
Thank you for the clarifications. I will take your discussions with the other reviewers into account to develop a more comprehensive understanding of the paper before considering whether to revise my score.
Response to Reviewer hfxE (3/3)
Thank you for recognizing our paper’s clear writing and transparent experimental settings, and for your questions on backbones, attribution, and training stability. We respond below.
1. Backbone Selection and Cross-Backbone Transfer
We use Qwen2.5-3B/7B, as in prior work, to reduce the effect of backbone differences on comparisons, and verify our method’s effectiveness at both scales. Their earlier release does not affect the assessment of incremental gains under matched settings. Would the lack of cross-backbone transfer experiments undermine any conclusion of this paper? We hope you will clarify how this requirement relates to our core claims, rather than treating additional generalization as a prerequisite for the method’s validity.
2. Gains, Complexity, and Experimental Scope
We propose a budget-controlled intervention against first-hop error propagation. Verification and value-guided repair are confined to the first hop: each question triggers at most one repair retrieval, independent of reasoning depth. Simulated retrieval and paired rollouts reduce training reliance on real search APIs and manually labeled decisions.
Table 2’s 2.3-point F1 gain is an average across five datasets. Versus w/o Verifier, EvidenceGate improves EM on all five, by 3.2 points on average; 2Wiki EM/F1 rise by 5.7/5.6 points. Average F1 alone does not capture all these gains. Under our scoring rule, unmatched words lower F1 even when the reference answer is present; EM should therefore also be considered (see SPREAD).
Paired rollouts are used only in training; inference executes only the selected action. The value model is an MLP over verifier representations. Cost should be judged against actual operations and gains, not the number of components alone.
We use standard multi-hop QA benchmarks, as does GlobalRAG. The 2,048-token limit applies per generation call, not to the entire search trajectory. Reported Bamboogle EM/F1 are averages over independent runs.
3. Source of the Error Statistics
The 1,325 trajectories were generated by GlobalRAG, initially assessed by GPT-5.6-sol, and reviewed by humans. Appendix B.1 describes the sample and defines an error as an incorrect or insufficiently supported candidate whose later use directs retrieval or reasoning down an erroneous path. The 377/1,325 figure describes this policy on the inspected examples. We added the trajectory source and assessment procedure to Appendix B.1 (page 16).
4. Simulated Retrieval and Prompt-Based Review
ZeroSearch fine-tunes its simulator on real search query–document pairs to reduce the gap with real search results. We train the verifier on evidence of varying support, filter samples, and shuffle documents to reduce format and position cues. Final evaluation uses real Google search.
A frontier model with simple prompting may perform as well or better, but this does not invalidate EvidenceGate’s gains on smaller models. Testing whether prompting can replace our mechanism requires a direct comparison with the backbone, evidence, and retrieval budget fixed.
If a stronger model solving the same problem were enough to dismiss a method, the same logic would dismiss much post-training research, including RLHF. If future AutoResearch could conduct research and peer review autonomously, would today’s research and peer review likewise become meaningless? Our question is specific: given a model and retrieval budget, how much do first-hop verification and repair improve performance?
5. Initialization Attribution and Training Stability
Attribution requires considering Tables 1 and 2 together. Table 1 compares complete systems; Table 2 holds the GRPO policy checkpoint and retrieval configuration fixed and varies only first-hop verification and Commit/Repair. Its average gains of 3.2 EM and 2.3 F1 points measure the intervention’s incremental contribution.
We continue GRPO from ZeroSearch’s public Google-search Qwen2.5-Instruct checkpoints (3B V2; 7B V1). Following ZeroSearch, retrieved documents remain in context but their tokens are masked from the policy loss: they are environment outputs, not policy actions. Only generated reasoning, queries, and answers are optimized, reducing update noise. ZeroSearch (§3.6, Appendix C) reports a substantial reward drop after removing the mask.
Difficulty reproducing ZeroSearch’s original training does not establish that our continuation from its public checkpoint is unstable. The cited lower scores use local Wikipedia retrieval or dataset-provided evidence, whereas ZeroSearch and our paper use live Google search. They therefore do not directly establish performance in our setting. Why ZeroSearch scores lower elsewhere requires controlled analysis of the original method by its authors; the Table 2 matched control tests our method’s incremental contribution.