acceptodds
Under review as a conference paper at ICLR 2027

EvidenceGate: Evidence-Guided Commit-or-Repair Decisions for First-Hop Answers in Multi-Hop Question Answering

Abstract

Reinforcement learning has recently shown promise in improving searchaugmented reasoning. However, existing methods for open-domain multi-hop question answering (QA) may prematurely commit to a first-hop answer based on relevant yet insufficient evidence, causing errors to propagate through subsequent queries. To address this issue, we introduce EvidenceGate, an evidence-guided Commit/Repair framework that formulates first-hop answer commitment as a decision between committing the current candidate and performing additional retrieval for repair. To support this decision, we use a controllable search simulator to generate evidence with varying levels of support and train an evidence verifier to assess whether retrieved evidence supports a candidate answer. The verifier resolves clear evidence states, while a Commit/Repair value model handles ambiguous verifier rejections by predicting the cost-adjusted repair advantage from rollout-level final QA rewards. Extensive experiments on five multi-hop QA benchmarks demonstrate that EvidenceGate achieves state-of-the-art overall performance.

Primary area:
reinforcement learning
Keywords:
Intermediate Answer Binding, Multi-Hop Question Answering, Evidence Verification, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hfxEOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies a failure mode in multi-hop retrieval QA: when a first-hop candidate answer is committed despite insufficient evidence, later queries may continue in the wrong direction and cause the final task to fail. To address this, the authors use LLM-generated simulated retrieval passages to train an evidence verifier. They then train a value model on paired Commit/Repair rollouts to decide whether an uncertain first-hop answer should be repaired before continuing. They evaluate the method on several multi-hop QA datasets using Qwen2.5-3B and Qwen2.5-7B, and report results that exceed many of the baselines in the paper. The paper presents its problem clearly and is well written. However, I find the research area overly saturated, the effectiveness of the proposed training method insufficiently established, and both the motivation and experimental design seriously flawed. I therefore lean toward rejecting the paper.

Critical strengths
  1. The paper describes its training and evaluation settings in considerable detail and is generally transparent.

  2. The writing is good and easy to follow.

Critical weaknesses

Major Concerns:

  1. The only backbone models are Qwen2.5-3B and Qwen2.5-7B. These models are about two years old and are not representative of more recent systems.

  2. The method adds a verifier, another round of policy training for repair, and a value model. This makes the training and inference pipeline much more complicated, yet Table 2 shows only a 2.3-point gain in average F1 over the w/o Verifier baseline. I do not think this gain justifies the added complexity.

  3. The motivation is intuitive. However, the authors report that 377 of 1,325 sampled trajectories (28.45%) exhibit premature first-hop answer binding without explaining whether the trajectories were assessed by humans or LLMs, or which policy checkpoint generated them. I find this omission puzzling.

  4. The number of components substantially increases the complexity of training and evaluation and may complicate real-world deployment. This is especially concerning because the paper does not show whether the verifier and value model generalize to other backbone models on which they were not trained.

  5. The policy model is initialized from a ZeroSearch checkpoint. This raises an attribution concern: some of the gains may come from that starting point, so the SOTA comparisons in Table 1 may partly reflect an advantage in initialization rather than the proposed components.

Minor Concerns:

  1. The research question feels somewhat dated to me. There is already a large and somewhat redundant body of work on RL-based search agent. The setting also feels toy-like: the evaluation uses only multi-hop QA datasets, and the maximum response length is 2,048 tokens. I do not think the paper offers many new insights. This is a matter of personal taste; I do not expect the AC or the authors to share my view, and I am not asking them to rebut this point.

  2. Some evaluation benchmarks, particularly Bamboogle, are small. Averaging results over multiple runs would be preferable.

Questions for authors
  1. How do the authors prevent overfitting to simulated retrieval text? How are the simulated search results aligned with real search results?

  2. Could a simple prompt asking a frontier LLM (such as GPT-6-Astra, Kimi K3, or Claude Fable 5.1) to review its first-hop candidate against the retrieved evidence address the same problem? More generally, are there simpler ways to address this issue when using frontier models?

  3. In my own attempts, I was unable to reproduce stable ZeroSearch training. Independent evaluations also report substantially lower ZeroSearch scores than Search-R1 and R1-Searcher under their respective shared evaluation setups. For example, UR² reports average F1 scores of 33.7 for ZeroSearch, 56.1 for Search-R1, and 54.9 for R1-Searcher (arXiv:2508.06165; see also related evaluations in arXiv:2510.1882, 2505.13258 and 2505.15107). This raises the possibility that ZeroSearch’s simulation-based training has limitations. How did you ensure training stability when continuing from a ZeroSearch checkpoint? Could you specify the exact checkpoint used and report training curves and results?

2 comments
AuthorsOpenReview ↗

Response to Reviewer hfxE (3/3)

Thank you for recognizing our paper’s clear writing and transparent experimental settings, and for your questions on backbones, attribution, and training stability. We respond below.

1. Backbone Selection and Cross-Backbone Transfer

We use Qwen2.5-3B/7B, as in prior work, to reduce the effect of backbone differences on comparisons, and verify our method’s effectiveness at both scales. Their earlier release does not affect the assessment of incremental gains under matched settings. Would the lack of cross-backbone transfer experiments undermine any conclusion of this paper? We hope you will clarify how this requirement relates to our core claims, rather than treating additional generalization as a prerequisite for the method’s validity.

2. Gains, Complexity, and Experimental Scope

We propose a budget-controlled intervention against first-hop error propagation. Verification and value-guided repair are confined to the first hop: each question triggers at most one repair retrieval, independent of reasoning depth. Simulated retrieval and paired rollouts reduce training reliance on real search APIs and manually labeled decisions.

Table 2’s 2.3-point F1 gain is an average across five datasets. Versus w/o Verifier, EvidenceGate improves EM on all five, by 3.2 points on average; 2Wiki EM/F1 rise by 5.7/5.6 points. Average F1 alone does not capture all these gains. Under our scoring rule, unmatched words lower F1 even when the reference answer is present; EM should therefore also be considered (see SPREAD).

Paired rollouts are used only in training; inference executes only the selected action. The value model is an MLP over verifier representations. Cost should be judged against actual operations and gains, not the number of components alone.

We use standard multi-hop QA benchmarks, as does GlobalRAG. The 2,048-token limit applies per generation call, not to the entire search trajectory. Reported Bamboogle EM/F1 are averages over independent runs.

3. Source of the Error Statistics

The 1,325 trajectories were generated by GlobalRAG, initially assessed by GPT-5.6-sol, and reviewed by humans. Appendix B.1 describes the sample and defines an error as an incorrect or insufficiently supported candidate whose later use directs retrieval or reasoning down an erroneous path. The 377/1,325 figure describes this policy on the inspected examples. We added the trajectory source and assessment procedure to Appendix B.1 (page 16).

4. Simulated Retrieval and Prompt-Based Review

ZeroSearch fine-tunes its simulator on real search query–document pairs to reduce the gap with real search results. We train the verifier on evidence of varying support, filter samples, and shuffle documents to reduce format and position cues. Final evaluation uses real Google search.

A frontier model with simple prompting may perform as well or better, but this does not invalidate EvidenceGate’s gains on smaller models. Testing whether prompting can replace our mechanism requires a direct comparison with the backbone, evidence, and retrieval budget fixed.

If a stronger model solving the same problem were enough to dismiss a method, the same logic would dismiss much post-training research, including RLHF. If future AutoResearch could conduct research and peer review autonomously, would today’s research and peer review likewise become meaningless? Our question is specific: given a model and retrieval budget, how much do first-hop verification and repair improve performance?

5. Initialization Attribution and Training Stability

Attribution requires considering Tables 1 and 2 together. Table 1 compares complete systems; Table 2 holds the GRPO policy checkpoint and retrieval configuration fixed and varies only first-hop verification and Commit/Repair. Its average gains of 3.2 EM and 2.3 F1 points measure the intervention’s incremental contribution.

We continue GRPO from ZeroSearch’s public Google-search Qwen2.5-Instruct checkpoints (3B V2; 7B V1). Following ZeroSearch, retrieved documents remain in context but their tokens are masked from the policy loss: they are environment outputs, not policy actions. Only generated reasoning, queries, and answers are optimized, reducing update noise. ZeroSearch (§3.6, Appendix C) reports a substantial reward drop after removing the mask.

Difficulty reproducing ZeroSearch’s original training does not establish that our continuation from its public checkpoint is unstable. The cited lower scores use local Wikipedia retrieval or dataset-provided evidence, whereas ZeroSearch and our paper use live Google search. They therefore do not directly establish performance in our setting. Why ZeroSearch scores lower elsewhere requires controlled analysis of the original method by its authors; the Table 2 matched control tests our method’s incremental contribution.

  1. Of course, this does not invalidate your existing conclusions. My concern is not whether the gains on older backbones hold, but how prevalent this problem is in current models and how broadly your method applies. To give a simple example, many commercial models fabricated citations in 2023, whereas current models augmented with retrieval rarely exhibit this problem. Similar examples abound in the development of deep learning. Identifying a phenomenon through a preliminary analysis of GlobalRAG alone, and demonstrating improvements only on Qwen2.5-3B and Qwen2.5-7B, provides limited evidence of the broader applicability of this work.

  2. Thank you for the clarification.

  3. Thank you for providing the additional information.

  4. My concern remains unresolved. The analogy to hypothetical future autonomous research is unrelated to my question. Regarding the gap between simulated and real retrieval, you have described several mitigation measures but have not provided direct validation. For example, how well does the verifier assess candidate support in real retrieved documents? The existing results are insufficient to rule out its reliance on features specific to simulated evidence.

  5. I have fully reproduced and closely studied ZeroSearch, including using its summarizer, whose outputs contained substantial hallucinations. Its training pipeline also contained implementation errors, such as mixing chat-template conventions for instruction-tuned and base models. If the original ZeroSearch checkpoint cannot even generalize from an online-search corpus to a local corpus, this raises concerns about possible overfitting to the source of retrieval evidence. Since your training starts from their checkpoint, I believe further disclosure or experiments are needed to clarify whether your method exhibits similar problems.

Thank you for the clarifications. I will take your discussions with the other reviewers into account to develop a more comprehensive understanding of the paper before considering whether to revise my score.

Reviewer_E4o7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Summary: The paper proposes a new method, EvidenceGate, which treats the initial answer in a multi-step reasoning chain as tentative. Instead of immediately accepting it, the framework makes an evidence-guided decision to either Commit to the answer or Repair it by performing another, more focused, retrieval for better evidence. This prevents the system from pursuing an incorrect reasoning path. Contributions

Key Contributions:

  1. Identifies and analyzes "premature first-hop answer binding" as a significant error source in multi-hop QA, noting it affects over 28% of reasoning trajectories in their analysis.
  2. Formulates the problem as a decision-making task. The system includes: An Evidence Verifier trained to assess if the retrieved documents sufficiently support a candidate answer.
  3. A Commit/Repair Value Model that predicts the downstream benefit of performing an additional retrieval (repair) for ambiguous cases, based on final QA rewards.
  4. By connecting local evidence assessment with the final answer's correctness, EvidenceGate can selectively trigger a repair action only when it is most likely to be beneficial, balancing accuracy improvements with retrieval costs.

Validation The authors validated EvidenceGate across five challenging multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA, MuSiQue, Bamboogle, and WikiHop).

Critical strengths
  1. Deconstructs "Premature Binding" directly: Instead of treating multi-step reasoning as a black box, the framework identifies and mitigates a highly specific, pervasive failure point. The "first-hop premature binding" issue .
  2. Combining an Evidence Verifier with a Commit/Repair Value Model ensures the system doesn't trigger secondary searches blindly or excessively.
  3. By using selective intervention rather than always-on repair loops, the model optimizes the trade-off between final QA accuracy (SOTA Exact Match scores across five diverse datasets including HotpotQA, MuSiQue, and Bamboogle) and overall computational/retrieval costs.
  4. Achieving results on five structurally different multi-hop benchmarks demonstrates that the framework's logic is robust.
Critical weaknesses
  1. Running two auxiliary modelsalongside the core LLM adds significant complexity and latency compared to standard single-pass or basic iterative RAG setups.
  2. The Commit/Repair Value Model relies on downstream QA rewards to predict utility. Training this model effectively requires diverse, labeled datasets mapping intermediate steps to ultimate outcomes, which may be difficult or expensive to acquire for custom domain applications (and to scale).
  3. While EvidenceGate can initiate a "Repair" step, if the underlying retrieval corpus lacks the correct evidence entirely, the system may get stuck in redundant repair cycles or eventually commit to a wrong answer.
  4. The framework heavily targets first-hop errors. In much longer, highly complex reasoning chains (e.g., 5+ hops), errors can still propagate later in the chain, and scaling this binary commit/repair decision architecture across many steps could compound latency and decision friction.
Questions for authors
  1. Your current evaluation focuses heavily on resolving premature binding at the first hop. How does the complexity, latency, and gating accuracy scale when extending EvidenceGate to deeper, multi-hop chains (e.g., 5+ hops)?"
  2. Since the Commit/Repair Value Model is trained on downstream QA rewards from specific datasets, how well does it generalize to zero-shot, out-of-domain topics without requiring targeted reward-tuning?
  3. In cases where the required evidence is completely absent from the retrieval corpus, how does EvidenceGate prevent infinite repair loops or unnecessary API costs before giving up?"
2 comments
AuthorsOpenReview ↗

Response to Reviewer E4o7 (2/3)

Thank you for recognizing our analysis of first-hop error propagation and the design of selective repair, and for your specific questions about inference overhead, cross-domain generalization, and missing evidence. We respond to each point below.

1. Inference Overhead and First-Hop Intervention

Evidence verification and repair retrieval do add overhead. We therefore concentrate the intervention on the first hop, with at most one additional repair retrieval per question. Because the first-hop answer influences later queries, early correction can reduce retrieval and reasoning along an erroneous path. Table 8 in Appendix C shows that, on Bamboogle, first-hop gating improves EM by 2.4 points with approximately 0.17 additional retrievals per question; gating all hops requires more retrievals and obtains lower EM. This comparison supports our choice to prioritize the first hop.

For tasks requiring five or more hops, EvidenceGate still gates only the first hop: additional hops do not increase the number of gate locations or the first-hop repair budget. First-hop gating accuracy depends more directly on the verifier’s ability and the quality of the candidate answer and evidence.

2. Supervision Cost and Generalization of the Value Model

The value model does not require manual labels for intermediate decisions. We reuse the GRPO-optimized search policy and existing QA data to generate Commit and Repair rollouts from the same first-hop state. Supervision is constructed from the difference between final QA rewards, adjusted for retrieval cost.

The value model estimates repair benefit using the verifier’s representations of the current candidate and evidence. The decision form and training objective do not need to be tailored to each topic, providing a basis for reuse across topics. Bamboogle was excluded from EvidenceGate-specific training, and the full method’s results on that benchmark support cross-benchmark reuse. In specialized domains, reuse primarily depends on whether the verifier can assess domain evidence and the retrieval system can find useful additional information.

3. Repair Limits and Abstention When Evidence Is Missing

The current process cannot repair indefinitely. Inference has a maximum number of interaction turns, and first-hop repair adds at most one retrieval, placing a clear upper bound on additional repair calls. If the required evidence is still not found, the system continues to answer and may be incorrect.

Whether to retrieve, whether to perform an additional search to repair the first hop, and whether to abstain when evidence is insufficient are three distinct decisions. EvidenceGate addresses the second and controls repair cost through the repair limit. For on-demand retrieval, see Self-RAG; for identifying unanswerable questions, see SQuAD 2.0. Explicit abstention is a separate extension that we will consider in future work.

Thank you to the authors for the detailed rebuttal and clarifications. I appreciate the explanation of the bounded repair mechanism, the use of existing QA rewards for training, and the additional results on retrieval overhead.

These responses address some of my concerns, particularly around unnecessary repair loops and supervision costs. However, a few important questions remain. The actual end-to-end latency introduced by the auxiliary models is still unclear, and the evidence for zero-shot generalization is limited. More importantly, the framework remains focused on first-hop errors, without convincing evidence of its effectiveness on longer reasoning chains (5+ hops). The lack of an abstention mechanism when evidence is unavailable also remains a practical limitation.

Overall, while the rebuttal provides useful clarifications, it does not sufficiently resolve my main concerns or provide substantial new experimental evidence. I will therefore maintain my original score of 2 (Weak Reject).

Reviewer_j2ZuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvidenceGate targets premature commitment to first-hop answers in search-augmented multi-hop QA. A verifier trained on role-controlled synthetic evidence first assesses candidate support; ambiguous rejected states are passed to a value model trained on paired Commit/Repair rollout rewards. Repair performs an additional retrieval without including the tentative candidate in the query. The method combines verifier-guided GRPO refinement with selective inference-time repair and is evaluated with Qwen2.5-3B/7B on five benchmarks using real search at evaluation time.

Critical strengths

The paper addresses a concrete source of error propagation and provides useful controlled evidence for intervening before an intermediate answer is reused.

  1. The first-hop focus is well motivated: the trajectory analysis identifies 377 premature binding errors among 1,325 inspected examples. Treating an intermediate answer as tentative provides a clear intervention point (Appendix B.1).
  2. With the same search policy and retrieval configuration, the Full Model improves average EM/F1 from 40.8/45.1 without verification to 44.0/47.4. This matched comparison is more informative about the gate's contribution than the headline comparison with separately trained baselines (Table 2).
  3. Random Repair, Verifier Only, and Always Repair controls help distinguish selective intervention from simply adding retrieval. Evaluation uses real Google search, and Bamboogle is excluded from the EvidenceGate-specific training mixture (Section 4.3; Appendix A.1).
Critical weaknesses

The framework is plausible, but the incremental value-learning evidence, supervision quality, and stated routing mechanism need stronger validation or clarification.

  1. The value model adds only 1.2 average EM and 0.8 F1 over Verifier Only, with F1 decreasing on two datasets. No repeated-seed results or paired uncertainty estimates establish how reliable this increment is. Moreover, error-targeting precision is not a direct test of whether predicted repair advantage identifies states where repair actually improves downstream reward (Tables 2–3).
  2. The synthetic evidence labels have only 66% agreement with the semantic audit, including 76% agreement for support labels. Neutral/refute confusion may preserve some binary sufficiency information, as acknowledged, but the three-way probabilities also influence representations and routing. Performance with cleaner semantic labels or a binary support verifier is not established, and the reported audit covers synthetic rather than real retrieved evidence (Appendix A.4).
  3. Under Appendix A.3's stated rule, conflict-based routing is redundant. Writing s = max support probability, the conflict score satisfies κ ≤ s. For rejected states s < 0.25, κ ≥ 0.20 implies s ≥ 0.20, which already satisfies |s − 0.25| ≤ 0.10. Thus the conflict condition adds no routed states; supported states bypass the value model even when conflicting evidence exists. This limits the claimed conflict-aware interpretation.
Questions for authors
  1. Can you report repeated-run or paired-bootstrap uncertainty for Full Model versus Verifier Only, and compare against verifier-threshold selection at the Full Model's repair budget? How accurately does the predicted repair-advantage sign match downstream benefit on held-out states when Commit/Repair outcomes are estimated with repeated rollouts?

  2. Please clarify the actual routing rule and configuration used for the reported results. With τs = 0.25, δ = 0.10, and τc = 0.20, the conflict condition appears to add no states beyond the margin condition. Is this intentional, and how are strongly supported but conflicting evidence bundles handled?

1 comment
AuthorsOpenReview ↗

Response to Reviewer j2Zu (1/3)

Thank you for recognizing the motivation of our work and the matched controls, and for your specific questions about the value model, supervision labels, and routing rule. We respond to each point below.

1. Value Model Gains and Budget-Matched Comparison

We conducted an additional budget-matched experiment on the same 300 2Wiki questions. We fixed the first-gate candidate answers and retrieved evidence across both methods and matched the number of first-hop repairs at 44. Verifier Only adjusts its support-score threshold to match this budget. EM and F1 are computed over all 300 questions.

MethodFirst-hop repairsEM (%)F1 (%)
Full Model4460.363.3
Verifier Only (budget-matched)4455.759.3

Under the same repair budget, Full Model achieves higher EM and F1, showing that the gains on this subset cannot be attributed solely to the number of repairs. These results support the contribution of the value model to selecting which cases to repair. The setup and results of this single run have been added to Appendix E, Table 10.

After adding the value model, answer-containment EM increases by 0.8 and 1.3 points on Bamboogle and WikiHop, respectively, while token-level F1 decreases by 1.3 and 0.7 points. Under our scoring rule, a prediction receives full EM if it contains the reference answer, while additional unmatched words lower F1. The two metrics can therefore move in different directions. A decline in F1 alone does not establish that the core answer is incorrect. SPREAD discusses the coexistence of correct content and redundant details. The Bamboogle EM and F1 reported in the paper are averages over independent runs.

2. Three-Class Supervision and Label Agreement

The three classes distinguish evidence that supports the candidate, lacks sufficient information to assess it, or explicitly refutes it. Following the document-level verification design of SURE-RAG, we jointly encode the question, candidate answer, and evidence document to produce support, neutral, and refute probabilities. Neutral and refute are both treated as non-support by the first gate, but can provide different signals when estimating the benefit of repair. The first gate therefore uses the support probability, whereas the value model retains the full three-class distribution.

The 66.0% in Appendix A.4 is the agreement between the three-class labels and the semantic audit. Recomputing agreement at the first gate's support/non-support granularity yields 84.7% (127/150) on the same 150 synthetic examples.

3. Routing Rule and Thresholds

Thank you for your careful examination. Appendix A.3 now clarifies the threshold constraint and routing order. The current implementation checks the support score first: a state reaching the support threshold is committed and does not enter the value model, even when a refuting signal is also present. The conflict condition is checked only when the support gate is not passed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.