acceptodds
Under review as a conference paper at ICLR 2027

Evidence-R1: Multi-Turn Search through an Evidence-Commit Protocol

Abstract

Multi-turn search enables language models to progressively acquire the information needed for complex reasoning, but the continual accumulation of retrieved content also increases context and inference costs. Existing methods compress retrieved information through summarization or memory; however, when subsequent actions can still access the original content, the model may continue to rely on the uncompressed context, making it difficult for the explicitly maintained evidence state to truly serve as the medium for information transfer across turns. To address this issue, we propose Evidence-R1, a framework for multi-turn search based on an evidence-commit protocol. The framework explicitly maintains a cross-turn evidence state throughout the search process. After each retrieval step, the model first updates this state through an independent evidence-commit segment, then removes the raw retrieved content and temporary reasoning, and carries the updated evidence state into the next turn. To further improve evidence commitment, Provenance-Routed Segment Credit assigns local rewards to the evidence-commit segments that construct the relevant state versions, based on the provenance of the finally cited evidence and source-support verification results. Across seven open-domain question answering benchmarks, the 3B and 7B variants of Evidence-R1 achieve average accuracies of 44.1% and 49.1%, respectively, both exceeding the strongest same-scale baseline MemSearcher (43.8% and 48.9%). Under a unified efficiency evaluation setting, Evidence-R1 again achieves higher accuracy than MemSearcher while reducing peak context length by 37.5%–49.7% and inference time by 16.9%–21.2%. These results show that a mandatory and traceable evidence state can reduce the cost of multi-turn search while preserving question answering performance.

Primary area:
reinforcement learning
Keywords:
Large Language Models, Multi-Turn Search, Reinforcement Learning, Evidence State Management, Provenance-Aware Credit Assignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XgfeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Evidence-Commit Protocol for solving the issue that searched results are concatenated the context in multi-turn search agent. After retrieval, it renews the evidence state in each evidence commit segment, and removes the raw observation and temporary reasoning, then generation the next action segment. By doing so, newly retrieved information implements commit-before-use that should be recorded in the evidence state to utilize the next action. Furthermore, by leveraging versioned evidence state and provenance store, it records source evidence and version lineage. While training, the final answer keep track on lineage of cited evidence, and it gives the provenance-routed segment credit to corresponding commit segment by evaluating the source support with frozen NLI model.

Critical strengths

S1. Clear problem definition. It is good to separate the memory generation term and the thing that next action should pass through the corresponding memory. STOP-VALIDATE/APPLY-EVICT-REGENERATE protocol constraints information visibility at the environment level, providing clearer structural constraints than simple prompt instructions.

S2. Practical. The most persuasive results in this manuscript is the accuracy-cost trade-off improvement. In 3B, the proposed method reduces the peak context 37.5--49.7%, and inference time 16.9--21.2% while maintaining the accuracy similar to MemSearcher.

Critical weaknesses

W1. Accuracy superiority. The results on this paper are hard to say that "consistent accuracy superiority". It should be toned down into "competitive accuracy" compared to MemSearcher. It is because, in table 1, the proposed method is worse than MemSearcher in 7B under NQ, MuSiQue, and Bamboogle.

W2. Ablation studies. Current ablation shows the effectiveness of adding provenance credit, but it does not demonstrate whether routing that tracks version lineage is necessary beyond a simpler source-support reward.

W3. Source faithfulness directly affects the real answer grounding. The faithfulness evaluation measures whether the source supports evidence statement or not. However, those problems should be separate such as the corresponding evidence is relevant to the query, entails the final answer, or is sufficient for multi-hop reasoning. Although generating source-supported statement and citing, the answer should rely on the parametric knowledge. It should be splitted into Source →\rightarrow Evidence support and Evidence →\rightarrow Answer support, and it is hard to prove the final answer grounding with current result.

W4. Baseline fairness. The baselines of Table 1 are not sufficient to be reevaluated under same retriever, retrieval top-K, and search budget. In efficiency perspective, comparison is needed under the same memory token budget and generation budget. Moreover, we need to dig in where the context reduction comes from: (1) evidence-commit protocol or (2) more rigorous state budget or different generation behavior.

W5. Suspicious of Long-horizon context management. The paper indicates the major motivation of context accumulation as increasing search turn, but the actual average number of searching is about two in efficiency evaluation. The method should be evaluated from the task that requires longer exploration.

W6. Novelty and missing baselines. In each component, there are previous works. For example, context reconfiguration by each turn (MemSearcher, ReSum), structured memory editing (Memory-R1, MemAct), NLI-based rewards for evidence notes (EviNote-RAG), credit assignment for each segment/step (StepSearch, HiPRAG, Memory-R2, MAVEN). In particular, the results of EviNote-RAG, StepSearch, and HiPRAG are missing in Table 1 and 2, even though they utilize the Qwen2.5 backbone, which is the same backbone in the proposed method. Please justify two things: (1) novelty of the proposed method, (2) the reason why those baselines are missing.

Questions for authors

See weaknesses. Here are the minor comments.

1. The information of reproducibility is not sufficient such as hyperparameters of GRPO training. Please report the hyperparameters in Appendix.

2. In Figure 1(a), two queries from search history described "S_1*query". Moreover, oto_t is missing in commit segment context label (q+Ht+Et−1q+H_t + E_{t-1}).

3. In Appendix G.2, it is the second turn, but the tag of retrieval document is [T1D1] and [T1D2]. It should change [T2D1] and [T2D2].

4. In Algorithm 1, to the best of my understanding, Line 3 and 18 are duplicated.

12 comments
AuthorsOpenReview ↗

Response to Reviewer Xgfe - Post (1)

Thank you for your detailed technical review.

W1. Scope of the accuracy claim and updated 3B checkpoint

We agree with the reviewer's assessment of the 7B results on NQ, MuSiQue, and Bamboogle. We will revise the claim to describe Evidence-R1 as achieving competitive accuracy relative to MemSearcher.

For the original 3B result, we followed MemSearcher's setup and trained for one epoch (662 steps). After submission, we continued training and observed further gains. We have now evaluated the 3B checkpoint after 3 epochs. Its unweighted average EM is 44.5%, compared with 44.1% for the original checkpoint and 43.8% for MemSearcher. The table below reports per-dataset EM, the unweighted average, and the average weighted by evaluation question count. We will also continue training the 7B model and report the new per-dataset results once training is complete.

The unweighted average gives each dataset equal weight. The question-weighted average uses the EM and question count for each dataset to estimate overall EM across all 51,713 evaluation questions.

Main 3B comparison (original and later checkpoints)NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleUnweighted Avg.Question-weighted Avg.
Evaluation questions3,61011,31314,2677,40512,5762,417125-51,713
Evidence-R1 (original 3B checkpoint, 1 epoch / 662 steps)47.264.747.044.943.818.243.244.148.5
Evidence-R1 (new 3B checkpoint, 3 epochs)47.564.948.647.444.419.239.244.549.5
MemSearcher47.063.847.943.943.517.942.443.848.3

W2. Ablating version-lineage credit

The B3/B4 comparison shows that adding the full provenance credit mechanism improves source support when the state-shaping reward is held fixed. It leaves open whether assigning credit to the Commit Segments that produced earlier versions is more effective than a simpler source-support reward.

We designed two ablations to test this question.

  • B5 directly computes a source-support reward for the evidence versions added or updated in each successful COMMIT, regardless of whether they are ultimately cited.
  • B6 computes a source-support reward only for the active versions cited in the final answer and assigns it to the COMMIT that created each cited version.

The new ablations follow the 3B main experiment's setup for the global reward, state-shaping reward, evidence-state budget, observation and generation budgets, and evaluation protocol. We use Qwen3-1.7B for all three configurations because of training time constraints. Comparing the full configuration with B6 tests the effect of assigning credit to predecessor versions along UPDATE lineages. Comparing B6 with B5 tests the effect of filtering by final citations.

The table reports EM across seven benchmarks for the three configurations on Qwen3-1.7B after 300 training steps.

ConfigurationCredit assignmentNQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAvg. EM
Full configuration (Qwen3-1.7B)Routes source-support credit from ultimately cited evidence through ADD/UPDATE lineages to the responsible Commit Segments46.9562.9448.2742.1835.4519.1030.8040.81
B5 (Qwen3-1.7B)Applies frozen NLI source-support scores directly to each successful Commit Segment, without filtering by final citation or tracing lineages43.560.744.242.338.914.230.339.15
B6 (Qwen3-1.7B)Rewards source support only for active versions cited in the final answer, without tracing predecessors43.7059.4943.6431.1930.5011.3328.4035.46

With Qwen3-1.7B trained for 300 steps, the full configuration achieves higher EM than B6, which rewards only the ultimately cited versions, on all seven datasets, by 5.35 percentage points on average. Compared with B5, which directly rewards every successful commit, the full configuration scores about 1.66 percentage points higher on average, with lower EM on HotpotQA and 2Wiki. The comparison with B6 shows that crediting the predecessors of ultimately cited versions can improve answer EM in this setting.

AuthorsOpenReview ↗

Response to Reviewer Xgfe - Post (2)

W3. Source–Evidence Support and Evidence–Answer Support

We agree that the two types of support should be evaluated separately. We will rename the original metric Source–Evidence Support to reflect that it measures whether each evidence statement is supported by its declared source. We also evaluate Evidence–Answer Support, which measures whether the evidence cited in the final answer textually supports the model's predicted answer.

We concatenate the active evidence items actually cited in the final answer in ID order. We then use Qwen2.5-3B-Instruct to rewrite the original question and predicted answer as an English statement and use a frozen RoBERTa-large-MNLI model to determine whether the cited evidence entails that statement. We use all 125 Bamboogle questions and the first 2,000 questions in the archive for each of the other six datasets. For each question, we compute

sEA(q,a)=max⁡(0,;pϕent(h(q,a)∣Ecited)−max⁡[pϕneu(h(q,a)∣Ecited),;pϕcon(h(q,a)∣Ecited)])s_{\mathrm{EA}}(q,a)=\max\left(0,\\;p_\phi^{\mathrm{ent}}\bigl(h(q,a)\mid E_{\mathrm{cited}}\bigr)-\max\left[p_\phi^{\mathrm{neu}}\bigl(h(q,a)\mid E_{\mathrm{cited}}\bigr),\\;p_\phi^{\mathrm{con}}\bigl(h(q,a)\mid E_{\mathrm{cited}}\bigr)\right]\right)

where h(q,a)h(q,a) is the statement that Qwen2.5-3B-Instruct generates from the question and predicted answer, the probabilities come from frozen RoBERTa-large-MNLI, and EcitedE_{\mathrm{cited}} is the set of evidence items actually cited in the final answer to that question. We report the NLI entailment rate, the fraction of questions for which entailment is the highest-scoring of the three classes, and mean NLI support, the per-question mean of 100 sEA100\,s_{\mathrm{EA}}.

Evidence–Answer NLI supportNQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleTotal/Avg.
Evaluation questions2,0002,0002,0002,0002,0002,00012512,125
NLI entailment rate (%)68.4052.1591.0545.5543.8038.3052.8056.01
Mean NLI support (%)61.2745.4483.1937.7336.7931.2245.1348.68

The cited evidence provides substantial textual support for the final answers. The NLI entailment rate is 91.05% on PopQA and 68.40% on NQ, with mean NLI support of 83.19% and 61.27%, respectively. The NLI model also judges the cited evidence to entail the final answer in 45.55% of HotpotQA cases, 43.80% of 2Wiki cases, and 38.30% of MuSiQue cases.

W4. Baseline comparability, matched budgets, and the source of context reductions

Thank you for pointing out where the baseline evaluation setup needed clarification.

  1. In Table 1, the results for ZeroSearch and R1-Searcher use Google Web Search. The other baselines use the same retriever or retrieval configuration. For comparability with MemSearcher, we match the generation budget, evaluation questions and splits, decoding, and answer aggregation rules, while allowing retrieval top-K and the search budget to vary.

  2. Retrieval top-K and search budget. These settings depend on how each method handles retrieved context. MemSearcher uses top-5 retrieval and a maximum of 20 searches. Methods that accumulate context, such as Search-R1, use top-3 retrieval and a maximum of 4 searches. Evidence-R1 uses top-20 retrieval and a maximum of 10 searches. To examine these settings, we reproduced Search-R1 results at different values of kk. Increasing kk to 5 reduced performance, consistent with Table 7 of the Search-R1 paper. We hypothesize that a larger kk further lengthens the context as these methods keep appending retrieval results, which can hurt performance. MemSearcher controls context length by updating its memory, which allows it to use more searches. These results suggest choosing top-K and the search budget to suit how each method manages its context.

  3. Matched retrieval, memory, and generation budgets in Table 2. Evidence-R1, MemSearcher, and Search-R1 use the same Qwen2.5-3B-Instruct backbone, top-5 retrieval, and search limits of B=5B=5 or 1010. In the MemSearcher and Evidence-R1 runs, we match the cross-turn memory budget, the visible observation token budget per retrieval, and the total generation budgets per logical search round and per question. MemSearcher's memory and Evidence-R1's evidence state are both capped at 1,024 tokens, and the observation budget is 4,096 tokens for both. MemSearcher uses 512 tokens for Action and 512 for Memory, while Evidence-R1 uses 512 for Action and 512 for Commit.

  4. Source of the context reduction. To test the effect of the commit protocol on context cost, we compared B1 (Soft Commit) and B4 (Hard Commit). Both use the same evidence-state capacity, per-retrieval input limit, and generation token cap per logical search round. Switching to Hard Commit reduces peak context length by 7.2% on NQ and 26.8% on HotpotQA, showing that the protocol reduces peak context length with the same budget limits.

AuthorsOpenReview ↗

Response to Reviewer Xgfe - Post (3)

W5. Search depth, context cost, and evaluation on longer trajectories

Thank you for pointing out this limitation. Evidence-R1 averages 2.07 searches under both search limits in Table 2. The protocol evicts the raw retrieval results after the first retrieval and commit. The next Action then rebuilds its context from the question, query history, and current evidence state. The efficiency conclusions from Table 2 are limited to the search depths observed in that evaluation. Whether the cost advantage and answer accuracy hold over more search rounds requires further evaluation.

We compared Evidence-R1 and MemSearcher on the same questions from the MuSiQue-Ans development split, grouping questions by their annotated reasoning hop count. We used the official hop counts only for grouping. Each method uses its Qwen2.5-3B-Instruct checkpoint from Table 2. We match the retriever and corpus, top-5 retrieval, state and generation budgets, and decoding settings, with B=10B=10 for both methods. To characterize the search depth covered, we report the average number of searches actually made and the proportion of questions for which each method made at least four searches.

Annotated hop countMethodQuestionsAvg. searchesSearches ≥4 (%)EM (%)Peak context (tokens/question)Cumulative tokens (tokens/question)
2-hopMemSearcher1,2523.2532.1125.8827518611
2-hopEvidence-R11,2523.0727.5626.281,293.855,849.85
3-hopMemSearcher7604.3555.7911.45303812492
3-hopEvidence-R17603.8649.4710.921,350.297,643.08
4-hopMemSearcher4055.8580.498.64334718366
4-hopEvidence-R14055.3875.567.901,457.7511,177.67

On the 405 four-hop questions, Evidence-R1 averages 5.38 searches and MemSearcher averages 5.85, with at least four searches on 75.56% and 80.49% of questions, respectively. These trajectories are longer than the overall average of about two searches in Table 2. Evidence-R1 uses less peak context and fewer cumulative tokens than MemSearcher in all three hop groups. Its EM is 0.40 percentage points higher in the two-hop group and 0.53 and 0.74 percentage points lower in the three-hop and four-hop groups, respectively. In the four-hop group, which has the most searches, Evidence-R1 reduces peak context length by 56.4% and cumulative tokens by 39.1%, with 7.90% EM compared with 8.64% for MemSearcher.

Reviewer_6AvqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Evidence-R1, a reinforcement-learning framework for multi-turn search with an explicit, versioned evidence state. After retrieval, a dedicated commit segment updates the state, and the environment removes raw observations and temporary reasoning before generating the next action. Provenance-Routed Segment Credit traces finally cited evidence through its version history and assigns source-support rewards to the corresponding commit segments. Experiments cover seven QA benchmarks with 3B and 7B models, reporting competitive answer accuracy, reduced context and inference costs, and improved evidence source faithfulness. Additional validation includes component ablations and human calibration of an independent evidence judge.

Critical strengths
  1. The paper addresses a relevant and practically important problem: controlling context growth while preserving useful evidence during multi-turn search. The motivation is clear and the proposed workflow is clear.

  2. The explicit evidence state and source-version records provide a useful interface for inspecting evidence updates.

  3. Evaluation across seven benchmarks and two model sizes demonstrates competitive QA performance. The unified 3B comparison reports meaningful reductions in context length and inference time while maintaining similar accuracy.

  4. The empirical analysis includes five-seed component ablations and an independent source-faithfulness judge calibrated against 500 human-labeled pairs, strengthening the evidence that provenance rewards improve source support beyond state-shaping rewards alone.

Critical weaknesses
  1. The incremental contribution over existing memory-based search needs stronger isolation, since MemSearcher already reconstructs subsequent contexts from compact memory.

  2. The average EM improvements over MemSearcher are only 0.2–0.3 percentage points, without uncertainty estimates for the main comparison.

  3. Important experimental details remain underspecified, including evaluation-set sizes, training hyperparameters, inference hardware, and whether Table 1 baselines were reproduced or imported.

  4. In ablation study, the B3/B4 ablation establishes the benefit of adding provenance credit, but does not isolate version-lineage routing from ordinary source-support supervision.

Questions for authors
  1. Regarding baselines, what remains of the advantage over MemSearcher when memory capacity, generation budgets, query-history access, and compactness incentives are matched?

  2. Does the proposed reward outperform the same NLI-based source-support signal applied directly to commits or only to final cited versions, without tracing historical versions?

  3. Could you report per-seed variability for Tables 6–8 and paired uncertainty estimates for the main comparison with MemSearcher?

  4. Which Table 1 results were reproduced, and which were taken from previous work? Please specify evaluation split sizes, training hyperparameters, and the hardware, batching, and baseline memory/generation limits used for Table 2.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 6Avq - Post (1)

Thank you for reading our paper carefully. We address your comments point by point below.

Response to W1 and Q1. Isolating the contribution beyond MemSearcher

We agree that MemSearcher updates a compact, free-form memory after each retrieval and reconstructs the next context from the question and the updated memory. In Evidence-R1, the environment validates and atomically applies a structured evidence update before the model generates its next action. For every successfully committed evidence version, the environment records its source, predecessor version, and associated Commit Segment. These records support auditing and let us assign source-support credit during training to the Commit Segments along the version history of the evidence ultimately cited.

The revised manuscript identifies the contribution of Evidence-R1 as the combination of three mechanisms. Retrieved information must pass through an explicit commit before it can affect subsequent actions. The system tracks evidence sources and version lineages throughout the interaction. Source-support credit can then be assigned to the specific Commit Segments that created or revised the evidence ultimately cited. In addition to keeping the context compact, this design lets the environment enforce the update protocol and trace evidence updates to specific commits.

In the 3B efficiency evaluation in Table 2, both methods use the same backbone, top-5 retrieval, and search limits. Both have a 1,024-token cross-turn state budget and a 512 + 512 token generation budget per turn. The first 512 tokens are for Action, and the second 512 are for Memory in MemSearcher or Commit in Evidence-R1. With search limits of 5 and 10, Evidence-R1 achieves average EM scores of 41.7% and 41.6%, respectively, and MemSearcher achieves 41.1% and 41.4%. In these two settings, Evidence-R1 reduces peak context length by 37.5% and 49.7% and per-question inference time by 21.2% and 16.9%, respectively.

The manuscript reports ablations for both mechanisms. Table 4 defines the configurations, including B1 (Soft Commit) and B4 (Full Model with Hard Commit). The comparison in Table 6, detailed in Appendix F.2, shows that the environment-enforced commit boundary reduces context costs while preserving answer quality. B3 omits provenance credit. The comparison between B3 and B4 in Table 8, detailed in Appendix F.4, shows that adding provenance credit with state-shaping rewards in place improves source support for committed and ultimately cited evidence.

AuthorsOpenReview ↗

Response to Reviewer 6Avq - Post (2)

Response to W2 and Q3. Paired uncertainty in the main comparison and variability across seeds

Thank you for pointing out that the main comparison lacked uncertainty estimates. Using the original per-question predictions for Evidence-R1 and MemSearcher in Table 1, we performed a paired bootstrap analysis of the difference in average EM across the seven datasets, separately for the 3B and 7B models.

We aligned the predictions by dataset and question ID and used the same answer normalization and EM scoring rules for both methods. In each bootstrap replicate, we drew questions with replacement within each dataset, preserving its original sample size and using the same sampled indices for both methods. We computed the EM difference within each dataset and then averaged the seven differences with equal weight, following the Avg. convention in Table 1. The 95% confidence intervals below use the 2.5th and 97.5th percentiles of the bootstrap differences. They quantify uncertainty due to the finite set of evaluation questions for the fixed checkpoints.

Model sizePaired questionsAverage EM difference (Evidence-R1 − MemSearcher, percentage points)95% paired confidence interval (percentage points)
3B51,7130.3[−0.5, +1.1]
7B51,7130.2[−0.6, +1.0]

The point estimates are positive at both model sizes, and both 95% confidence intervals include zero. These results support describing Evidence-R1 as achieving competitive accuracy across the seven benchmarks. We will narrow the claim in the main comparison accordingly.

For Tables 6–8, each configuration was trained independently with the same five random seeds, with initialization matched by seed across configurations. The original manuscript reported the mean of the five runs. Tables 6–8 in the revised manuscript report mean ± SD.

We also evaluated the 3B checkpoint after further training. The original checkpoint was trained for 1 epoch (662 steps), and the new checkpoint for 3 epochs. Their unweighted average EM scores are 44.1% and 44.5%, respectively. Appendix E reports the per-dataset EM scores, unweighted averages, and averages weighted by question count for both checkpoints.

The unweighted average gives each dataset equal weight. The question-weighted average uses each dataset's EM and question count to estimate overall EM across all 51,713 evaluation questions.

Additional evaluation of the 3-epoch checkpointNQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleUnweighted Avg.Question-weighted Avg.
Evaluation questions3,61011,31314,2677,40512,5762,417125-51,713
Evidence-R1 (original 3B checkpoint, 1 epoch / 662 steps)47.264.747.044.943.818.243.244.148.5
Evidence-R1 (new 3B checkpoint, 3 epochs)47.564.948.647.444.419.239.244.549.5
MemSearcher47.063.847.943.943.517.942.443.848.3
AuthorsOpenReview ↗

Response to Reviewer 6Avq - Post (3)

Response to W3 and Q4. Experimental and reproducibility details

Thank you for pointing out the missing experimental and reproducibility details.

  • Results and evaluation splits. The baseline rows in Table 1 are taken from prior work, and the Evidence-R1 rows come from our own evaluation. Table 2 reports efficiency under a common inference setup. Evidence-R1 uses the FlashRAG evaluation splits, totaling 51,713 questions. The test split is used for NQ (3,610), TriviaQA (11,313), PopQA (14,267), and Bamboogle (125). The dev split is used for HotpotQA (7,405), 2WikiMultiHopQA (12,576), and MuSiQue (2,417). Appendix D.2 of the revised manuscript includes the table below.
DatasetNQTriviaQAPopQAHotpotQA2WikiMultiHopQAMuSiQueBamboogleTotal
Evaluation splittesttesttestdevdevdevtest
Questions3,61011,31314,2677,40512,5762,41712551,713
  • Training. Evidence-R1 uses full-parameter GRPO with AdamW, a learning rate of 10−610^{-6}, and zero warmup steps. We sample 5 trajectories per question at temperature 1.0. The policy probability ratio is clipped to [0.8, 1.2], the maximum gradient norm is 1.0, and the KL coefficient in the policy loss is 0.001. For the 3B model, we sample 256 questions per training step and set the policy-update mini-batch size to 128. The 3B checkpoint in Table 1 was trained for 1 epoch (662 steps). Appendix D.3 of the revised manuscript lists the actual training steps, batch sizes, and reward coefficients for the checkpoints in the main table.
  • Efficiency evaluation. In Table 2, Evidence-R1, MemSearcher, and Search-R1 all use the Qwen2.5-3B-Instruct backbone, the E5-base-v2 retriever, the same wiki-18 index, and deterministic decoding on the same questions from the seven datasets above. Each retrieval returns the top 5 documents. Each method is evaluated with limits of 5 and 10 searches per question. We time all methods on one NVIDIA RTX A6000 GPU with the same warmup conditions, processing one question at a time (batch size and concurrency both equal to 1). End-to-end time is measured from the first policy call through completion of the final answer and includes retrieval, all generation calls, and state updates.
  • State and generation budgets. MemSearcher's cross-turn memory and Evidence-R1's cross-turn evidence state are each capped at 1,024 tokens, with up to 4,096 tokens of visible input per retrieval. Action generation is capped at 512 tokens per turn for both methods. When an Action triggers retrieval, MemSearcher generates Memory and Evidence-R1 generates a Commit, each with a separate cap of 512 tokens. Search-R1 retains the retrieval results from previous turns in the interaction context.
Reviewer_ZHwwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Evidence-R1 enforces a strict Evidence-Commit Protocol. After each retrieval step, the model must explicitly commit relevant facts to a dedicated, versioned "Evidence State." Crucially, the raw retrieved documents and temporary reasoning are then discarded before the model takes its next action (either another search or generating a final answer). This forces the model to rely exclusively on the curated Evidence State, preventing it from bypassing the memory and creating a clear, auditable trail of how information is used across turns.

Key Contributions

  1. Establishes a "commit-before-use" boundary by separating evidence selection (Commit Segment) from the next reasoning step (Action Segment). This protocol makes the flow of information explicit and auditable.
  2. It introduces a novel credit assignment method. The system traces the final answer's cited evidence back through its version history to identify which specific "commit" actions were responsible. It then assigns local rewards to those actions based on whether the source document actually supported the committed fact, providing fine-grained, traceable training feedback.
  3. The framework is designed to reduce context length and inference costs while maintaining a verifiable, source-bound record of all evidence used in the reasoning process.

Validation: The authors validated Evidence-R1 through:

  1. Performance: The model was tested on seven open-domain QA benchmarks. It achieved state-of-the-art average Exact Match (EM) scores, outperforming the strongest comparable baseline (MemSearcher) at both 3B and 7B model scales.
  2. Efficiency: In a unified efficiency setting, Evidence-R1 demonstrated significantly lower resource usage. It reduced peak context length by up to 49.7% and inference time by up to 21.2% compared to the baseline, while still achieving higher accuracy.
  3. Ablation Studies: Controlled experiments confirmed the value of the core components. The "hard commit" protocol was shown to improve answer quality while lowering costs. Furthermore, the provenance-based credit assignment was proven to increase the source faithfulness of the committed evidence.
Critical strengths
  1. Enforcing a strict "commit-before-use" protocol compresses the context window by discarding raw documents and temporary reasoning.
  2. Because the model can only generate its final answer using facts preserved in the structured "Evidence State," it creates a verifiable provenance trail from source document to final output.
  3. The Provenance-Routed Segment Credit mechanism traces successful answers back to the specific "commit" actions that enabled them, allowing fine-grained reinforcement learning rewards rather than relying on noisy, end-of-turn global feedback.
  4. Outperformed strong multi-turn baselines (like MemSearcher) in Exact Match (EM) scores across seven open-domain QA benchmarks at both 3B and 7B model sizes.
Critical weaknesses
  1. Information Loss: If the model fails to extract or incorrectly summarizes a critical fact during a "Commit Segment," that information is permanently discarded. There is no fallback mechanism to retrieve it unless the model initiates an expensive re-search.
  2. The entire reasoning chain relies on the model’s initial extraction fidelity. Errors or subtle hallucinations committed to the "Evidence State" cannot easily be debugged or undone in subsequent turns.
  3. The strict boundary separating selection (commit) from reasoning (action) forces the model to structure its thinking into rigid segments, which may restrict organic, fluid reasoning capabilities compared to free-form chain-of-thought.
  4. The provenance-routed credit assignment is highly tailored to RL training setups, making it difficult to implement zero-shot using standard, out-of-the-box foundation models without extensive domain-specific fine-tuning.
Questions for authors
  1. Since raw documents and reasoning steps are discarded after each turn, how does Evidence-R1 handle scenarios where a seemingly irrelevant detail omitted during the 'Commit' phase becomes critical in a later step? Is there a recovery or backtracking mechanism short of re-triggering the search?"
  2. Enforcing a strict 'commit-before-use' boundary structure limits organic, free-form reasoning. How does this rigid segregation affect performance on tasks that require highly implicit reasoning, where the connection between two facts is not obvious until both are synthesized? 3)The Provenance-Routed Segment Credit system relies on structured tracing to assign rewards. How easily does this credit assignment scale or generalize when applied to massive, heterogeneous datasets where source documents have vastly different structures and citation formats?
  3. While you demonstrate a 49.7% reduction in peak context length and a 21.2% speedup in inference time, does the overhead of generating structured 'Commit' and 'Action' steps across multiple turns offset these gains in low-latency production environments?
  4. Because Evidence-R1 relies heavily on custom segment-level RL training, how viable is it for enterprises to implement this protocol using closed-source, API-only models (like GPT-4 or Gemini) without fine-tuning?
5 comments

Thank you for your feedback. We address your comments and questions below.

Response to W1 and Q1. Information loss and recovery

We agree that this is a limitation. If a detail was omitted from the commit and the raw retrieval results have been removed from the model's context, the policy can recover that detail only by retrieving it again. The model can formulate a new query from the question, query history, and existing evidence, then commit the recovered information to the Evidence State. Recovery depends on the remaining search budget and whether the new query returns the relevant passage.

The global reward supervises the entire trajectory. Provenance credit provides local feedback to the Commit Segments that built the ultimately cited evidence, based on whether their declared sources support the committed statements. Under a fixed state budget, the policy must learn which evidence to retain, since it has no rollback mechanism for omitted details.

The environment's provenance store retains committed evidence versions and the source excerpts visible when they were committed. The environment uses this store for auditing and training-time credit assignment, and the store is inaccessible to the policy. Committed facts that are restated in later queries can persist in the query history. We will clarify these two information paths in the Methods section, including why omitted details must be retrieved again.

Response to W2. Diagnosing and correcting errors in committed evidence

Errors or subtle hallucinations in the Evidence State can be traced, removed, or replaced in subsequent turns.

For each successful ADD or UPDATE, the environment records the evidence version, source ID, source excerpt visible at the time, and the Commit Segment that created the version. During auditing or debugging, we can check a suspicious statement against the text the model actually saw and identify the version that introduced the error.

If the model flags an evidence item as suspect in a subsequent turn, it can use DELETE in a Commit Segment to remove the item from the active state. The operation requires only a valid active evidence ID. If the current retrieval provides a source that supports a correction, the model can use UPDATE to replace the active version while retaining its history.

Response to W3 and Q2. Hard segmentation and implicit reasoning

The commit-before-use boundary governs which information carries over between generation segments. Both the Commit Segment and the Action Segment generate nonempty free-form reasoning. The boundary applies between the two generations, determining what the next segment can see. The Commit Segment selects evidence and updates the state using the question, query history, existing Evidence State, and current observation. It can commit inferences or statements about relationships that are supported by the current source text. The subsequent Action Segment can synthesize all active evidence before deciding whether to search or answer. The Evidence State can therefore contain both individual facts and statements about how they relate. If two facts remain in the active state, a later Action Segment can combine them even if their relationship was missed during the commit.

Response to Q3. Heterogeneous sources and citation formats

Credit routing uses the local source IDs supplied by the retriever (e.g., T2D1) and the excerpts actually shown to the model. It follows the version lineages of the ultimately cited evidence and checks whether each relevant version's declared source supports the committed statement. This requires neither parsing the documents' reference formats nor traversing a global citation graph.

For heterogeneous sources, the main limitations are input conversion and source verification. For example, converting a table to text can drop column headers or units, leaving the verifier without the information it needs even when the source ID is correct. Storage cost depends on the number of successfully committed versions and the length of the retained source excerpts. Verification cost depends on the number of versions traced from final citations and the length of the text to be checked. Our current results are primarily based on Wikipedia passages. Assessing source support for tables, complex layouts, and documents from other domains remains future work.

Response to Q4. Inference overhead in low-latency settings

Table 2 measures end-to-end time from the first policy call through completion of the final answer. In the 3B, top-5 setting, the reported time for Evidence-R1 includes retrieval, Action and Commit generation, and state updates. With a search limit of B=5B=5, Evidence-R1 reduces average time from 15.90 seconds per question for MemSearcher to 12.53 seconds, a reduction of 21.2%. At B=10B=10, the time drops from 15.28 to 12.70 seconds per question, a reduction of 16.9%. The 49.7% reduction in peak context length is for the B=10B=10 setting.

Response to Q5. Using closed-source API models without fine-tuning

The interaction protocol and the training credit mechanism have different deployment requirements. Provenance-Routed Segment Credit requires rollouts and policy updates, so realizing its training benefits requires a policy that can be updated or a compatible online adaptation mechanism. At inference time, an API model can implement commit-before-use with two calls per retrieval. The first call includes the current observation and generates the Commit Segment. The second generates the Action Segment after the raw observation has been removed. The environment applies state updates, enforces eviction, and reconstructs the context. A tool schema or strict output parsing can enforce the output structure.

We evaluated three configurations with the same closed-source API model, glm-5.3-flash.

  1. API direct answer. The model generates an answer directly without retrieval.
  2. API + RAG. We use a conventional RAG prompt for retrieval and answer generation.
  3. API + Evidence-R1 protocol (prompt-based). We implement commit-before-use with structured prompts and output parsing. The environment manages state updates, eviction, and context reconstruction. The model parameters are fixed.

The three configurations use the same API model and the same decoding and evaluation protocols. The two configurations with retrieval share the retriever, corpus, top-K, and maximum search budget. API + RAG makes one retrieval per question using the original question as the query. For the prompt-based Evidence-R1 protocol, we also report the actual number of searches and API calls. We randomly sampled 200 questions each from NQ and HotpotQA.

MethodRetrievalCommit-before-useNQ EMHotpotQA EMAvg. EMAvg. F1Source–Evidence Support (commit)Source–Evidence Support (cited)
API direct answerNoNo45.50%41.50%43.50%54.07%——
API + RAGYesNo52.00%43.50%47.75%59.21%——
API + Evidence-R1 protocol (prompt-based)YesYes54.50%47.50%51.00%60.99%87.46%86.85%
MethodActual searches per questionPeak context lengthCumulative tokensAPI calls per question
API direct answer0.00136.71615.171.00
API + RAG1.001004.851253.781.00
API + Evidence-R1 protocol (prompt-based)1.11884.332826.093.23

Across the two datasets, the API implementation of the Evidence-R1 protocol achieved 51.00% average EM and 60.99% average F1, gains of 3.25 and 1.78 percentage points over API + RAG, respectively. EM increased by 2.5 percentage points on NQ and 4.0 on HotpotQA. Source–Evidence Support was 87.46% for committed evidence and 86.85% for cited evidence. These results show that the protocol can be implemented with a closed-source API model without updating its parameters and that it improved accuracy in this evaluation.

Thank you for your comments. We would be happy to clarify any remaining questions.

Dear Reviewer ZHww,

We sincerely thank you for reviewing our paper and for helping us improve our work. With only a short time remaining in the discussion period, we would be very grateful to know whether our responses have addressed your concerns. In particular, we are eager to learn whether our latest response has resolved any remaining issues. We would be happy to continue the discussion if anything remains unclear.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.