acceptodds
Under review as a conference paper at ICLR 2027

Cross-Agent Learning Signals Enable Coordinated Role-Decomposed LLM Training

Abstract

Agentic search systems must coordinate evidence acquisition and response generation, yet existing approaches either couple both roles under a single agent objective or decompose them without disentangling their respective contributions to the final outcome. We introduce DAC (Divide and Cooperate), a role-decomposed training framework that, given task-specific external verification signals, trains a searcher and a generator with role-specific cross-verification rewards. DAC allows the generator to abstain when the retrieved evidence appears insufficient, and uses this decision together with externally evaluated search sufficiency to assign appropriate credit to each role. To prevent degenerate over-abstention, we further introduce hard-positive evidence augmentation, which discourages abstaining on sufficient but hard evidence. Across seven general and multi-hop QA benchmarks and two model backbones, DAC consistently outperforms strong single-agent and multi-agent baselines. Controlled evaluations show that these gains arise from coordinated improvements in both search and generation. Our results highlight the importance of explicitly and properly assigning credit across interacting roles when training agentic search systems.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agentic AI, Large Language Models, Reinforcement Learning, Multi-Agent Training

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_9nD9OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DAC, an RL framework that decomposes agentic search into a searcher and a generator sharing one model. During training, the generator may abstain when evidence seems insufficient. Its decision enters the searcher's reward, so the searcher is rewarded only for evidence that is externally sufficient and accepted by generator. The generator is rewarded for correct answers on sufficient evidence or abstention on insufficient evidence. Hard-positive augmentation adds distractor documents to sufficient evidence to prevent over-abstention, and turn-level difference rewards densify the searcher signal. On seven QA benchmarks with Qwen2.5-7B-Instruct and Qwen3-8B, DAC improves average EM over the strongest baselines by 2.0 and 1.6 points, with consistent trends under LLM-judge evaluation.

Critical strengths
  1. A clear formulation of credit assignment between roles. Coupling the searcher's reward to the generator's abstention avoids penalizing one role for the other's failure. The analysis seperately shows better search accuracy and better answer accuracy given fixed evidence (Figure 2).

  2. Consistent results with supporting analysis. DAC is best on average for both backbones under EM and LLM-judge evaluation (Tables 1 and 3). The ablations, the matched comparison with DART (Table 4), and the analysis of over-abstention support the role of each component.

Critical weaknesses
  1. It is unclear whether the gains come from RL training or from the prompted inference structure. At inference, DAC is a two-role pipeline built by prompting the same backbone as a searcher and a generator. The paper does not report this pipeline with the untrained backbone, so the benefit of training itself is not quantified. Moreover, with the role decomposition kept and an answer-correctness searcher reward, Qwen3-8B reaches 45.6 average EM (Table 2), identical to single-agent Search-R1 (Table 1). This suggests the cross-verification reward contributes about 1.6 points. An untrained-pipeline baseline would clarify how much of the improvement is attributable to training.

  2. Learned abstention is not compared with prompt-only abstention. The generator's abstention is already elicited by its prompt ("answer unknown"; Appendix E). However, the baselines in Figure 5 is never given an abstention option; reaching the turn limit is counted as abstention. Table 8 also lacks a prompted-model reference. A natural control is to replace the co-trained abstention in the searcher reward with a frozen, prompted sufficiency judge. This would test whether co-training abstention, and the hard-positive augmentation needed to counter over-abstention, is necessary.

Questions for authors
  1. What is the performance of the DAC pipeline with the untrained backbone (same prompts, abstention disabled) on the Table 1 benchmarks?

  2. How does a frozen, prompted abstention judge compare with co-trained abstention, both as a searcher reward and in the Figure 5 and Table 8 analyses?

  3. How many runs underlie Tables 1 and 2? Since many margins are within 1–2 points, uncertainty estimates would help. Also, do baseline settings (e.g., top-k, turn limit, answer format) follow the original papers? S3 rises from 14.0 EM to 48.9 under the LLM judge (Tables 1 and 3).

4 comments
AuthorsOpenReview ↗

Response to Reviewer 9nD9 (1/2)

We thank the reviewer for the careful review and the constructive questions. We answer each point below, with new experimental results.

W1, Q1: RL training vs. the prompted inference structure

The decomposed inference structure explains only part of the gain. The table below shows the DAC pipeline with the untrained backbone (same prompts, abstention disabled). The "+ DAC" rows give the gain from RL training. On average, RL training adds 7.5 points on Qwen2.5-7B-Instruct and 10.3 points on Qwen3-8B.

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
Qwen2.5-7B-Instruct No train36.859.336.134.929.013.933.634.8
+ DAC+7.1+4.1+7.7+8.3+10.8+5.1+9.6+7.5
Qwen3-8B No train36.059.938.137.335.515.536.036.9
+ DAC+12.4+7.3+9.8+9.3+13.2+6.8+13.6+10.3

W2: Baselines have no abstention option

This is correct. The baselines have no abstention component, so the natural choice is to count continuing to search until the turn limit as abstention.

That DAC abstains better than these baselines is not, by itself, our main claim. With Figure 5, we wanted to show that (1) the abstention that DAC uses as a training tool is learned reasonably well, so it can provide a meaningful signal during training, and (2) it can also serve as an additional feature at test time. The clear gap in the figure shows that DAC's learned abstention is consistently better than what the other methods may have learned implicitly. This supports both points.

W2, Q2: Frozen, prompted abstention judge vs. co-trained abstention

We ran new experiments for this question. The setup and the tables are in "Abstention at test time" and in 3.3 of "Additional ablation study" in the general response. In short:

  • Abstention at test time. A frozen abstention judge abstains more often than DAC, and more of the questions it abstains on are actually answerable. Enabling abstention therefore costs 6.8 points of answer EM with the frozen judge, compared with 2.1 points with DAC, while abstain accuracy is no higher. So training in DAC improves abstention beyond the frozen judge.
  • Hard-positive augmentation. DAC trained without augmentation also abstains more often than the full DAC, and more of the questions it abstains on are actually answerable. Enabling abstention therefore costs 4.4 points of answer EM without augmentation, compared with 2.1 points with the full DAC, while abstain accuracy is no higher. So augmentation is needed to prevent over-abstention.
  • Searcher reward. When the searcher receives the abstention signal from a frozen judge during training, search accuracy and answer EM are both lower than with DAC. The frozen judge is already slightly better than searcher rewards that use only external signals, and co-training adds a further 1.2 points.
AuthorsOpenReview ↗

Response to Reviewer 9nD9 (2/2)

Q3: Number of runs and uncertainty estimates

Tables 1 and 2 report one training run per method. We have now trained DAC and DART with three training seeds each on Qwen3-8B. The average EM is 47.18 ± 0.05 for DAC and 45.54 ± 0.08 for DART, so the gap of 1.6 points is much larger than the variation between seeds (see "Multi-seed results" in the general response). For Search-R1, the average EM over three runs is 45.52 ± 0.04, 1.7 points below DAC.

Q3: Baseline settings and the S3 scores

We ran all baselines with their public code and evaluated them in the same settings (Appendix C). The only exception is S3's training reward: we used EM, our target evaluation metric, instead of S3's own metric (GenAcc). However, S3 did not improve with training, regardless of this choice. Its paper says that about 15 steps are enough, but in our runs performance did not change between step 0, step 15, and beyond step 100. This held whether we used EM, the LLM judge, or GenAcc as the training reward. For example, on Qwen2.5-7B-Instruct, with GenAcc for both training and evaluation:

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
Step 063.876.855.953.946.119.35.645.9
Step 2064.176.756.154.146.219.64.845.9
Step 10064.477.155.953.847.020.25.646.3
Step 20064.476.855.954.447.220.24.046.1

In our reproduction, RL training therefore does not change S3's performance. S3's results rather come from its inference pipeline (the decomposed structure and a much more detailed prompt than the other baselines), so in practice it behaves like an inference-time method rather than a training method. This also explains the two scores in Tables 1 and 3: EM is low because the frozen generator never learns the required answer format, and the LLM-judge score is reasonable (although below DAC) but practically the same at step 0 as at step 100.


We thank the reviewer again for the valuable feedback. We hope that these answers resolve the reviewer's concerns. If so, we would be very grateful if the reviewer would kindly reconsider the score. We would be glad to answer any further questions.

Thank you for the detailed response and the additional experiments. My concerns have been addressed, and based on the rebuttal I am raising my rating to 3 (weak acceptance).

Dear Reviewer 9nD9,

Thank you for reading our response carefully and for raising your score. Your comments helped us make the paper clearer and more complete. We appreciate your time and your constructive feedback.

Reviewer_LyrZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies how to assign training rewards to search and answer generation in agentic question answering. It proposes DAC, which uses a searcher to collect evidence and a generator to answer or abstain. The searcher is rewarded when the retrieved evidence is judged sufficient and the generator chooses to answer. The generator is rewarded for answering correctly when the evidence is sufficient, or for abstaining when it is insufficient. The authors also add noisy documents to sufficient evidence during training to reduce excessive abstention. They evaluate the method with Qwen2.5-7B-Instruct and Qwen3-8B on seven QA benchmarks and report higher average exact match than the compared methods.

Critical strengths
  1. The paper includes a substantial set of experiments and analyses. The hard-positive evidence augmentation results offer a useful engineering insight into training a generator that can abstain without becoming too reluctant to answer.

  2. The motivation is real. A wrong final answer does not, by itself, tell us whether retrieval failed or the generator failed to use adequate evidence.

  3. The paper is clearly written.

  4. DAC performs favorably against the reported baselines on the evaluated benchmarks. The authors make a serious effort to standardize the backbone models, retrieval corpus, and evaluation setup. They also include an additional comparison with DART under matched SFT and LoRA conditions. These controls make the comparison more informative.

Critical weaknesses
  1. My biggest concern is that the evaluation design does not align with either the paper’s motivation or its training setup.The main evaluation does not give abstention special treatment, and the paper does not provide a convincing explanation beyond comparability with standard QA baselines. The generator can abstain during training, but this option is disabled during the main evaluation. As a result, the main evaluation largely comes down to benchmark answer accuracy. Under this setup, I do not think the improvement over DART delivered by the proposed reward design and training pipeline is enough to justify publication, especially given the additional computation.

  2. The training-time search-sufficiency label relies on substring matching, so a document can be marked sufficient merely because it contains the gold answer, even if it does not provide the evidence needed for the multi-hop inference.

  3. The paper does not disclose enough about evaluation stability. Some evaluation sets, such as Bamboogle, are small. The authors should report the number of runs, decoding temperature, and uncertainty estimates so that readers can judge whether the reported differences are statistically reliable.

Questions for authors
  1. Evaluation allows at most four retrieval turns. DAC appears to produce an answer for every question at test time. How are the other baselines handled when they reach the four-turn limit without producing a final answer? Are these examples scored as wrong or discarded? This is especially important because Section 6.3 treats baselines that continue searching until the turn limit as abstaining in the separate abstention analysis.
  2. Is the string-matching criterion used to label evidence sufficiency during training too coarse, and could its false positives and false negatives affect the reported results?
5 comments
AuthorsOpenReview ↗

Response to Reviewer LyrZ (1/2)

We thank the reviewer for the detailed review. We answer each point below and report new results where they help.

W1: Evaluation design vs. the motivation and the training setup

In DAC, abstention is a tool for better role-decomposed training, while the final goal is still better task performance. This is why the main evaluation measures answer accuracy.

The benefit of training with abstention is visible in this metric. DAC's searcher retrieves sufficient evidence more often than the baselines on both backbones (Table 5), and replacing the cross-verification reward lowers answer EM (Table 2). In a new experiment, using a frozen judge in place of the co-trained generator also lowers answer EM (see 3.3 in "Additional ablation study" in the general response). For the train-test mismatch itself, Section 6.3 shows that a short generator-only stage without abstention improves answer EM further.

Regarding test-time abstention, we have also shown in Section 6.3 that abstention can be used at test time, where DAC's abstention decisions are clearly more accurate than those of the baselines. This shows that (1) the abstention that DAC uses as a training tool is learned reasonably well, so it can provide a meaningful signal during training, and (2) it can also serve as an additional feature at test time.

A new experiment in the general response (see "Abstention at test time") further supports this. A frozen abstention judge and DAC trained without augmentation both abstain more often than DAC and lose at least twice as much answer EM when abstention is enabled, with no gain in abstain accuracy. Taken together, abstention in DAC is effectively learned during training and improves the answer accuracy that the main evaluation measures, so the evaluation is consistent with both the motivation and the training setup.

W1: Improvement over DART and the additional computation

The gain over DART is consistent: DAC is above DART on 13 of the 14 benchmark-backbone pairs in Table 1, and the gain holds under LLM-judge evaluation (Table 3) and under DART's matched conditions (Table 4). For training cost, please see "Training cost" in the general response.

AuthorsOpenReview ↗

Response to Reviewer LyrZ (2/2)

Q1: Baselines that reach the turn limit without a final answer

DAC uses at most 4 search turns and then 1 answer turn. Search-R1 and DART use at most 5 turns in a single loop, and if a baseline tries another search on its last turn and gives no answer, the example is scored as wrong. Therefore, the turn budget is the same for all methods in the main evaluation.

One could worry that the baselines are at a disadvantage in the main evaluation, because no extra prompt tells them that they must answer on the last turn. However, these baselines are trained with the same pipeline and the same turn limit; since their only objective is answer correctness and they get no reward for abstention, they learn to give an answer within 5 turns in most cases (95.6% for Search-R1 and 96.8% for DART on Qwen3-8B evaluation).

We also checked what happens if we allow one more forced generation step for the examples with no answer. The baseline scores rise a little, but DAC still has the higher average.

Answer EM on Qwen3-8B:

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
DAC48.467.247.946.648.722.349.647.2
Search-R148.166.145.345.043.921.748.845.6
+ forced+0.2+0.5+0.3+0.7+0.7+0.7+0.8+0.5
DART46.866.347.345.344.520.948.045.6
+ forced+0.3+0.2+0.1+0.4+1.0+0.4+0.0+0.3

The forced generation step uses the same prompt as DAC.

For the abstention analysis in Section 6.3, the baselines have no abstention component, so the natural choice is to treat "search again on the last turn" as abstaining and "give an answer" as not abstaining. That DAC abstains better than these baselines is not, by itself, our main claim. As in our answer above, we use this analysis to show that DAC's abstention is actually learned effectively and that it can additionally serve as an additional feature at test time. The clear gap in Figure 5 shows that DAC's learned abstention is consistently better than what the baselines may have learned implicitly.

W2, Q2: Substring matching for the search sufficiency label

Please see "Robustness to the search sufficiency verifier" in the general response.

W3: Evaluation stability

The results in the paper come from one training run and one evaluation run per method. Evaluation uses greedy decoding (temperature 0.0), so there is no sampling randomness. We know that vLLM is not fully deterministic, so small differences can still occur. To check this, we ran the main evaluation for DAC on Qwen3-8B 4 more times (5 runs in total). As the table shows, the average score is 47.24 ± 0.06 (mean ± standard deviation), and all five runs are between 47.19 and 47.33. So the variation between evaluation runs is very small.

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
Original48.467.247.946.648.722.349.647.2
5 runs (mean ± std)48.36 ± 0.0567.24 ± 0.1147.84 ± 0.0946.62 ± 0.0848.60 ± 0.1222.26 ± 0.1749.76 ± 0.3647.24 ± 0.06
Range (min - max)48.3 - 48.467.1 - 67.447.7 - 47.946.5 - 46.748.4 - 48.722.1 - 22.549.6 - 50.447.19 - 47.33

We agree that Bamboogle is small (a single question changes the score by 0.8 points), so differences on this benchmark alone should be read with caution. Our conclusions do not rest on any single benchmark. They rest on the average over seven benchmarks and on the consistency of the gain across benchmarks and backbones.

The training seed can also cause variation, so we trained DAC and DART with three training seeds each on Qwen3-8B. The average EM is 47.18 ± 0.05 for DAC and 45.54 ± 0.08 for DART (see "Multi-seed results" in the general response). For Search-R1, the average EM over three runs is 45.52 ± 0.04, 1.7 points below DAC.


We thank the reviewer again for the valuable feedback. We hope that these answers resolve the reviewer's concerns. If so, we would be very grateful if the reviewer would kindly reconsider the score. We would be glad to answer any further questions.

Thank you for the careful responses and the substantial effort during the rebuttal period. This is a very complete paper, and my questions have largely been addressed. I have no further technical concerns.

However, my main reservation remains. I accept the rationale for using abstention as a training-time credit-assignment signal, and I acknowledge that the additional experiments support its effectiveness. I therefore no longer consider this a methodological correctness issue. Nevertheless, disabling abstention in the main evaluation means that the paper primarily demonstrates improved forced-answer accuracy, rather than reliable collaborative decision-making under insufficient evidence. This still limits my assessment of the paper’s contribution and significance. In particular, given the extensive research on RL-based search agents, improvements primarily in benchmark answer accuracy are not sufficiently compelling to me.

I have already raised my soundness score. I remain undecided about whether to raise my overall score, and I am open to reconsidering it in light of the other reviewers’ assessments and the ensuing discussion.

Reviewer_1ME8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DAC trains an agentic RAG system to provide a more accurate retrieval and better answer, with a searcher which issues queries and a geneator which can read and also abstain. The core idea is to credit the searcher using the generator's abstention signal, so the searcher is only rewarded when the evidence is actually sufficient.

Critical strengths
  1. The central mechanism is a good contribution. Using the generator's abstention decision as a gate on the searcher's reward is a concrete cross-role credit-assignment idea, not just splitting roles or turning on turn-level rewards.
  2. The average EM of DAC is 42.3 and 47.2 respectively, which are higher than the strongest baseline DART. This paper also provide rewards and augmented ablation, as well as the cross-evaluation of fixed retrieval results which make the gains more convincing. More comparison experiments with LoRA in Appendix is well organized.
Critical weaknesses
  1. The evidence for a robust advantage is not strong enough yet. The average gain is +1.6 EM on Qwen3, and the paper reports no seed variance, no confidence interval and no significance test. I cannot tell whether this is separable from training noise.
  2. The reward that was actually run differs from the reward written in the paper in a way that affects interpretation that within a group, one sibling answering correctly marks the whole group as retrieval-sufficient. This group-level OR is not stated in the paper, and there is no sensitivity check.
Questions for authors
  1. Baseline insufficient. I wonder why you only compare DAC with prompt engineering methods and single or multi-agent methods? What about RAG, Graph-Based and GNN methods? As I known, these methods have shown very significant results on these datasets, such as hotpot QA and the most difficult one, MuSiQue. The agent-based method group which used to compare in your paper still has many ommisions, like ReAgent, BELLE and THOR.
  2. Narrow novelity. OASES is more directly co-trains search and state evaluation and uses changes in intermediate-state answerability for turn-level credit assignment which is similar with your core method? So what's the different? You mentioned it but not compare, Why?
  3. A slight increase in gain lacks uncertainty and cost comparison. DAC also generates additional answers for intermediate retrieval states and noisy input. As a result, you need more ablation experiments to prove your gain reliability.
4 comments
AuthorsOpenReview ↗

Response to Reviewer 1ME8

We thank the reviewer for the review and the direct questions. We answer each point below.

W2: Group-level OR in the reward and its sensitivity

A verifier based only on a span check can miss some cases, such as yes/no questions. For this reason, our substring EM verifier also counts retrieval as sufficient when the generated answer is correct. This is stated in Section 4. Our training followed this rule. However, because GRPO generates several answers for the same retrieval result, we counted the retrieval as sufficient if at least one of these answers was correct. The paper did not state this clearly. We apologize, and we have fixed the text in Section 4.

As a sensitivity check, we trained DAC again without the answer-based rule. In this run, retrieval is sufficient only when a retrieved document contains the gold answer, so the group-level OR does not apply. The results did not change much, as the table below shows.

[Table 10 of the revised paper] Answer EM on Qwen3-8B:

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
Original48.467.247.946.648.722.349.647.2
Answer-based rule removed49.166.248.246.348.021.849.647.0

We chose this simple verifier for practical reasons. We have also run experiments with an LLM judge. Please see "Robustness to the search sufficiency verifier" in the general response. Both experiments are now in Appendix D.b of the revised paper.

Q1: Choice of baselines

We did not compare with methods that improve results through a more complex inference pipeline. Our work focuses on training, and we see the two directions as orthogonal. We understand the point that training on top of a more complex inference pipeline could give better results, but no single pipeline of this kind is accepted as the standard, and one study cannot cover all of them. Our question is not which pipeline is best for RAG. Rather, many recent works apply RL to the search-and-answer framework, and our claim is that our new training method works better in that framework.

Q2: Difference from OASES

First, turn-level credit assignment is not a main contribution of our method. The core of DAC is role decomposition and cross-verification between the searcher and the generator, and we adopted the temporal-difference idea from prior work only as an additional improvement. However, because the submitted paper did not ablate it, we acknowledge that its role may have been unclear, and we have now added this ablation (see "Additional ablation study" in the general response). Removing turn-level rewards lowers the average EM by only 0.4 points (47.2 to 46.8 on Qwen3-8B), which is the smallest effect of all components, and DAC without turn-level rewards still outperforms the strongest baselines, Search-R1 and DART (both 45.6).

We could not compare with OASES because its code was not public when we submitted, and it is still not public. During the rebuttal period, we looked into reproducing it, but a faithful reproduction was not feasible: OASES computes turn-level rewards from an intermediate answer at each turn, but it has no explicit answer generator, and the paper does not give the exact prompt construction strategy for this step. We instead compared with TIPS, which also uses temporal-difference turn-level credit assignment, and DAC outperforms it on both backbones (Table 1). Together with the new ablation above, this indicates that DAC's gain does not come from the turn-level credit assignment that it shares with OASES.

W1, Q3: Uncertainty and training cost

On uncertainty, we have now trained DAC and DART with three training seeds each on Qwen3-8B. The average EM is 47.18 ± 0.05 for DAC and 45.54 ± 0.08 for DART, so the gap of 1.6 points is well separated from training noise (see "Multi-seed results" in the general response). For Search-R1, the average EM over three runs is 45.52 ± 0.04, 1.7 points below DAC.

On training cost, please see "Training cost" in the general response. In short, DAC makes at most 10 generation calls per question during training, compared with 5 for Search-R1, DART, TIPS and S3, and 17 for M-ASK. Most of the extra calls generate the intermediate answers for turn-level rewards, and DAC without turn-level rewards still outperforms the strongest baselines, Search-R1 and DART (46.8 vs. 45.6 average EM on Qwen3-8B). In addition, we trained every method until its training reward converged or collapsed, so the baselines were not limited by a smaller training budget.


We thank the reviewer again for the valuable feedback. We hope that these answers resolve the reviewer's concerns. If so, we would be very grateful if the reviewer would kindly reconsider the score. We would be glad to answer any further questions.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer 1ME8,

Thank you again for your review. As the discussion period ends soon, we would like to kindly ask whether our response and the revised paper have addressed your concerns.

In short:

  • We trained DAC again without the answer-based rule, so the group-level OR does not apply, and also with an LLM judge as the verifier. Our conclusions did not change in either case (Appendix D.b).
  • We now report three training runs each for DAC, DART and Search-R1. DAC is 1.6 and 1.7 points above the two baselines on average, while the standard deviation across runs is at most 0.08.
  • We added a comparison of training cost, and we explained our choice of baselines and the difference from OASES.

If anything is still unclear, we would be glad to answer in the remaining time.

Reviewer_1ME8OpenReview ↗

Response to Authors

I thank the authors for the new experiments. The variance and group-level OR concerns are now resolved, and these revisions are appreciated.

My remaining reservation is about novelty and contribution level, which the added experiments do not address that the gain is small and the distinction from OASES rests only on indirect evidence without a direct comparison. I am therefore keeping my rating. I will consider the other reviews and remaining discussion before finalizing, and remain open to reconsidering.

Thank you for the reply. We are glad that the concerns about variance and the group-level OR are resolved. We would like to answer the remaining reservation briefly.

1. DAC's novelty

DAC addresses a question that prior work on RL-based search agents has not addressed: in a role-decomposed system, how should the learning signal of one role depend on the other role? Our answer is cross-verification. The searcher receives its search reward only when the co-trained generator accepts the evidence instead of abstaining, so the generator's abstention acts as a gate on the searcher's reward. Existing methods reward all roles with the answer score (e.g., MMOA-RAG, DART, M-ASK), give each role its own external signal (e.g., OPERA), or keep one role frozen (e.g., S3). To our knowledge, DAC is the first method in which the reward of one role depends on a decision that the other role learns during training. We have observed that this signal is effective, as replacing it with simpler searcher rewards lowers the average EM by 1.6 to 2.2 points (Table 2).

2. OASES has a different focus

OASES studies turn-level credit assignment, that is, how much each search turn contributes to the final answer. It has no generator that can accept or reject the evidence, and no reward that depends on such a decision. The only point where OASES and DAC meet is that DAC also uses turn-level rewards. We adopted them from prior work, we do not claim them as a contribution, and removing them lowers DAC's average EM by only 0.4 points. A direct comparison with OASES would therefore test a component that we borrow, not the idea that we propose. For this reason, we see OASES as complementary to DAC rather than as a competing method. Its turn-level credit assignment can be applied on top of the cross-verified reward. The turn-level line of work is also represented in Table 1 by TIPS, which is 4.1 and 6.3 points below DAC on the two backbones.

3. The gain is moderate, but it is consistent

DAC is above DART on 13 of the 14 benchmark-backbone pairs (Table 1), and the gain holds under LLM-judge evaluation (Table 3). Over three training runs, the average EM is 47.18 ± 0.05 for DAC, 45.54 ± 0.08 for DART and 45.52 ± 0.04 for Search-R1, so the gap is much larger than the standard deviation.

We hope this clarifies where the contribution of DAC lies. As you kindly noted that you remain open to reconsidering, we would be very grateful if you could take this clarification into account in your final assessment. Thank you again for the constructive discussion.

Reviewer_wXHFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Divide and Cooperate (DAC), a reinforcement learning framework that separates agentic search into a searcher and a generator. The two roles share a model in the main experiments but receive different training signals: the searcher is rewarded for retrieving evidence that is sufficient and accepted by the generator, while the generator learns to answer or abstain according to evidence sufficiency. Hard-positive evidence augmentation is introduced to mitigate excessive abstention, and turn-level rewards provide intermediate feedback to the searcher. Experiments on two backbones and seven question-answering benchmarks compare DAC with single- and multi-agent baselines and analyze retrieval quality, answer generation, and the contribution of individual training components.

Critical strengths
  1. A concrete treatment of credit assignment between retrieval and generation. DAC recognizes that an incorrect final answer does not identify whether retrieval failed or the generator failed to use adequate evidence. Coupling the searcher's reward to evidence sufficiency and the generator's acceptance provides a simple way to make the two roles responsive to each other. The hard-positive augmentation also addresses a plausible failure mode of this design rather than treating abstention as uniformly beneficial.

  2. Diagnostics that distinguish retrieval gains from generation gains. Figure 2 and Appendix D.3 evaluate retrieved evidence with an independent judge and compare generators while holding the retrieved evidence fixed. These controls are more informative than final-answer accuracy alone and support the claim that the gains are not confined to one stage. Table 2 and Figure 4 additionally connect the reward and augmentation choices to observed training behavior.

  3. Consistent improvements with a useful matched-baseline comparison. DAC obtains the best average exact match on both backbones in Table 1, and the LLM-judge evaluation supports the same overall direction. Appendix D.2 also compares against DART under matched tokenizer, SFT warm-up, and adapter settings. Although the remaining margins are modest, this extra comparison addresses an important source of unfairness in comparing training frameworks.

Critical weaknesses
  1. The implemented evidence-sufficiency signal does not clearly match the method's formulation. Section 3 writes sufficiency as a function of the question and retrieved evidence, but Section 4 states that sufficiency holds when a retrieved document contains the gold answer or the generated answer matches it. Under the latter condition, the searcher could receive credit for an answer produced from parametric knowledge rather than supporting evidence. Substring presence also need not establish sufficient evidence for a multi-hop answer. The paper acknowledges limitations of this proxy, but the answer-dependent definition creates a more specific ambiguity in the central learning signal that should be resolved.

  2. The ablations establish some component benefits, but not the full attribution to cross-verification. Table 2 provides meaningful searcher-reward ablations: replacing the proposed reward reduces average EM by 1.6 or 2.2 points. However, generator-side abstention rewards and turn-level shaping are not independently isolated. Removing augmented samples has an even larger effect of 2.6 points, and also changes the generator's training examples. The results support DAC as a combination of techniques, but a more controlled separation is needed to identify how much of the gain specifically requires cross-agent verification rather than additional supervision or intermediate rewards.

  3. The performance margins need uncertainty estimates. Improvements over the strongest baseline are 2.0 and 1.6 average EM points in Table 1, and 1.5 and 0.9 points in the matched DART comparison in Table 4. Repeated-seed results and confidence intervals are not reported. The consistency across settings is encouraging, but without variability estimates it is difficult to judge whether the remaining advantage is robust to stochastic training and checkpoint choice.

  4. The reported training exposure and computational cost are difficult to reconstruct. Appendix B specifies 500 training steps and a last-stable-checkpoint rule after divergence, while Figures 3 and 6 show the joint-training stage ending around step 260. These need not be contradictory, but the selected checkpoints and stopping reasons should be explicit. Appendix C also describes several Search-R1 variants without clearly identifying the one used in Table 1. Finally, turn-level rewards require generator calls on intermediate evidence states, so parameter sharing alone is insufficient to establish comparable training cost.

Questions for authors
  1. Is evidence labeled sufficient when the generated answer is correct but no retrieved document contains supporting evidence? How are yes/no answers handled by substring matching, and does removing the answer-based sufficiency shortcut materially change the results?

  2. Can you isolate generator-side abstention and turn-level rewards within DAC? For the augmentation comparison, can the number of generator training examples be held fixed so that the effect of hard-positive construction is separated from additional supervision?

  3. Can you provide repeated-seed uncertainty for DAC and the strongest matched baseline on at least one backbone, including the principal searcher-reward ablations?

  4. Which checkpoint and Search-R1 configuration underlie each main result, and what explains the training lengths in Figures 3 and 6 relative to Appendix B? Please also report generator calls or GPU-hours for DAC and the principal baselines.

4 comments
AuthorsOpenReview ↗

Response to Reviewer wXHF (1/2)

We thank the reviewer for the thorough and precise review. The requested controls led to several new experiments, which we report below.

W1, Q1: The sufficiency signal and the method's formulation

As the reviewer notes, a verifier based only on a span check can miss some cases, such as yes/no questions. For this reason, our final substring EM verifier also counts retrieval as sufficient when the generated answer is correct, even if no retrieved document contains the gold answer; this is how yes/no answers are handled. We acknowledge that this verifier is weak. We chose it because it is cheap to compute and still reasonably accurate, which is enough to test the benefit of our idea, and because it keeps the comparison with the baselines fair, as DAC gets no help from an extra external signal. The same check is also used to reward correct retrieval in prior RL work on search agents (Jin et al., 2025, arXiv:2505.15117; Wei et al., 2025, arXiv:2505.11821).

Also, we would like to clarify that the implemented signal is consistent with the formulation. In Section 3, s(Q, E) is meant as the oracle search sufficiency, which depends only on the question and the evidence. Our wording there ("judged by an external verifier") did not make this clear, and we have corrected it in the revised paper. Section 4 does not redefine this quantity; the verifier described there is only a practical approximation of it, since the oracle label is not available. The answer-based rule is part of this approximation, not of the definition. DAC does not prescribe a specific verifier, and any verifier can be substituted without changing the objective.

Still, we understand the concern about relying on a single verifier, so we also trained with two other verifiers. First, as a sensitivity check, we removed the answer-based sufficiency shortcut, which did not change the results much.

[Table 10 of the revised paper] Answer EM on Qwen3-8B:

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
Original48.467.247.946.648.722.349.647.2
Answer-based rule removed49.166.248.246.348.021.849.647.0

Second, we used an LLM judge as the verifier during training; please see "Robustness to the search sufficiency verifier" in the general response. Both experiments are now in Appendix D.b of the revised paper.

W2, Q2: Isolating turn-level rewards, generator-side abstention and augmented samples

We address the three points of this comment in turn.

Turn-level rewards. We have added an ablation that removes only the turn-level rewards (see the "w/o turn-level rewards" row in "Additional ablation study" in the general response). The average answer EM drops by 0.4 points, which is the smallest effect of all components, and this variant still outperforms the baselines. Intermediate rewards therefore explain only a small part of DAC's gain.

Generator-side abstention. The two searcher reward ablations in Table 2 already train without generator abstention, and they are below DAC. To isolate it more directly, we also disabled generator abstention during training and gave the searcher the abstention signal from a frozen judge (see 3.3 in "Additional ablation study" in the general response). This variant (-1.2 from DAC on average) is slightly better than the two searcher reward ablations (-1.6 and -2.2), so a judgment from the generator side already helps a little. Co-training abstention on the generator side then adds the remaining 1.2 points.

Augmented samples. The 2.6-point drop does not mean that augmentation matters more than cross-verification, or that its gain comes from additional supervision. When we add augmentation to the two searcher reward ablations, which do not use cross-verification, the average changes little or even drops (from -1.6 to -1.4, and from -2.2 to -2.6). Augmentation therefore does not improve performance by itself. Its role is to prevent over-abstention, the main failure mode that cross-verification introduces (see "Additional ablation study" in the general response).

AuthorsOpenReview ↗

Response to Reviewer wXHF (2/2)

Q2: Augmentation with a fixed number of generator training examples

We do not yet have results for this exact control, which adds the augmented samples while keeping the minibatch size fixed. The closest evidence is in our answer to W2, Q2 (Part 1/2): adding the same augmented samples to the two searcher reward ablations does not improve the average score, so additional supervision alone does not explain the gain from augmentation in DAC. We agree that a fixed-size control would separate the two effects more cleanly. We have now run this control up to step 120, and we report its training dynamics in a follow-up comment below.

W3, Q3: Repeated-seed uncertainty

We have now trained DAC and DART, the matched baseline, with three training seeds each on Qwen3-8B. The average EM is 47.18 ± 0.05 for DAC and 45.54 ± 0.08 for DART, so the gap of 1.6 points is much larger than the variation between seeds (see "Multi-seed results" in the general response). For Search-R1, the average EM over three runs is 45.52 ± 0.04, 1.7 points below DAC. Due to limited compute, we have not repeated the searcher reward ablations with more seeds. However, these variants use the same training setup as DAC, and their gaps to DAC in Table 2 (1.2 to 2.6 points on average) are much larger than the standard deviation across seeds that we observed (0.05 to 0.08), so we expect these conclusions to hold as well.

W4, Q4: Checkpoints and training length

We trained every run until its training reward started to collapse, and then selected the last checkpoint saved before that point. The 500 steps in Appendix B are the maximum training length. Since DAC has separate rewards for the searcher and the generator, we used the generator reward for this decision. In the DAC run on Qwen3-8B, the collapse started near step 270, and because we save a checkpoint every 20 steps, we selected the checkpoint at step 260.

W4, Q4: Search-R1 variant used in Table 1

We apologize for this omission. We did not select one configuration in advance. We evaluated all four Search-R1 variants (PPO or GRPO, each with the outcome-only or the outcome and format rewards) on both backbones, with the official checkpoints where they are available and our own training runs otherwise (Appendix C). The main results and our analysis use the variant with the highest average EM on each backbone, which is GRPO with the outcome-only reward for Qwen2.5-7B-Instruct and GRPO with the outcome and format rewards for Qwen3-8B. The full results are now in Appendix D.a (Table 9) of the revised paper, and DAC exceeds all four variants in average EM on both backbones.

W4, Q4: Training cost

Please see "Training cost" in the general response for the full table. In short, DAC makes at most 10 generation calls per question during training, compared with 5 for Search-R1, DART, TIPS and S3, and 17 for M-ASK. As the reviewer notes, most of the extra calls generate the intermediate answers for turn-level rewards. DAC without turn-level rewards still outperforms the strongest baselines, Search-R1 and DART (46.8 vs. 45.6 average EM on Qwen3-8B). In addition, we trained every method until its training reward converged or collapsed, so the baselines were not limited by a smaller training budget.


We thank the reviewer again for the valuable feedback. We hope that these answers resolve the reviewer's concerns. If so, we would be very grateful if the reviewer would kindly reconsider the score. We would be glad to answer any further questions.

AuthorsOpenReview ↗

Update on Q2: augmentation with a fixed number of generator training examples

We ran the control that the reviewer asked for. The new run uses augmentation with the same ratio of non-augmented to augmented samples as DAC, but we reduce the total number of samples so that its generator minibatch has the same size as without augmentation. The three runs that we compare are:

RunAugmented samplesGenerator minibatch size
DAC (original)Yes> N
DAC, same batch size (new)YesN
w/o augmented samples (Figure 4)NoN

Here, N is the generator minibatch size without augmentation. Due to limited compute, we could train the new run only up to step 120 (the original run was selected at step 260), so we report the training dynamics up to this point. Both metrics are measured during training, as in Figure 4 of the paper.

Non-abstention recall (%), which measures how often the generator answers when it should answer:

Training step255075100120
DAC (original)85.0384.4585.9088.5190.10
DAC, same batch size85.4385.1487.2389.2091.06
w/o augmented samples (Figure 4)85.0284.2983.8584.9286.19

Training answer accuracy (%):

Training step255075100120
DAC (original)39.2140.3944.4145.7847.98
DAC, same batch size38.7140.5945.8646.5947.10

Up to step 120, the run with the same batch size follows the original run closely on both metrics. Its non-abstention recall reaches 91.1% at step 120, while the run without augmented samples is at 86.2%. The over-abstention that appears without augmentation therefore does not appear here. This indicates that the effect of augmentation comes from the hard-positive samples themselves, and not from a larger number of generator training examples.

We have also added these results, with the full training curves, to Appendix D.c of the revised paper.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer wXHF,

Thank you again for your thorough review. As the discussion period ends soon, we would like to kindly ask whether our response and the revised paper have addressed your concerns.

In short, we added the controls that you asked for:

  • training with two other sufficiency verifiers, including an LLM judge (Appendix D.b),
  • ablations that isolate the turn-level rewards, generator-side abstention, and the augmented samples,
  • the augmentation control with a fixed minibatch size (Appendix D.c, see our update on Q2),
  • three training runs each for DAC, DART and Search-R1.

If anything is still unclear, we would be glad to answer in the remaining time.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response (3/3): Abstention at test time and training cost

4. Abstention at test time

We ran a new experiment to test whether the abstention that DAC learns during training matters at test time. We compared DAC with two variants (on Qwen3-8B):

  • Train DAC as usual, but use a frozen, prompted generator to decide abstention at test time. We call it the frozen abstention judge. It is the initial Qwen3-8B model, before any training.
  • Train DAC without hard-positive evidence augmentation.

The columns are defined as follows (all are macro-averaged over the seven benchmarks):

  • Selective answer accuracy: accuracy when the model does not abstain.
  • Forced answer accuracy: accuracy when the model abstains, when we force it to answer by excluding the abstention prompt. A high value means that the model abstains on questions that it could have answered.
  • Abstain accuracy: agreement between the abstention decision and the search sufficiency label from gpt-oss-120b.
  • Abstain rate: share of examples on which the model abstains.
  • Change in answer EM: difference in answer EM when abstention is enabled, compared with abstention disabled.
Selective answer accuracy (%)Forced answer accuracy (%)Abstain accuracy (%)Abstain rate (%)Change in answer EM (points)
DAC53.810.974.217.1-2.1
Frozen abstention judge at test time55.516.973.424.6-6.8
w/o augmented samples57.114.173.631.2-4.4

Both variants over-abstain. They abstain more often than DAC (abstain rate), and the examples they abstain on are more often answerable (forced answer accuracy). As a result, enabling abstention costs them 6.8 and 4.4 points of answer EM, compared with 2.1 points for DAC. Their selective answer accuracy is slightly higher, but this is expected because they answer fewer and easier questions, and their abstain accuracy is no higher than DAC's. The comparison with "Frozen abstention judge at test time" shows that training in DAC discards far fewer answerable questions than the frozen abstention judge. The comparison with "w/o augmented samples" shows clear over-abstention when augmentation is not used.

5. Training cost

It is true that DAC needs more computation per training step, because it uses turn-level rewards and hard-positive evidence augmentation. The table below shows the number of generation calls per question during training (excluding the count for the repeated sampling for GRPO).

MethodMax Generation CallsDecompositionExtra non-generative LLM passes
DAC104 searcher + 5 generator (for intermediate rewards) + 1 generator (augmented sample)None
Search-R154 search + 1 finalNone
DART5 (called 9 times)Same as Search-R1, but reasoning and tool calling are separate on the search turnsNone
TIPS5Same as Search-R1Up to 5 teacher scoring passes (for intermediate rewards)
S354 searcher + 1 generator1 RAG-baseline call per question, precomputed once
M-ASK171 planning + 4 × (search + summary + update + answer)None

However, for evaluation, we did not compare the methods at the same number of training steps. We trained every method until its training reward converged or collapsed. (Efficiency is not the focus of our paper.) Every baseline was trained to convergence, so DAC's higher final score does not come from a larger training budget.

Also, most of DAC's extra cost comes from the intermediate answer generation for turn-level rewards. As the turn-level ablation shows, DAC without turn-level rewards reaches 46.8 average EM on Qwen3-8B. This is only 0.4 points below DAC (47.2), and it is still above the strongest baselines, Search-R1 and DART (both 45.6).

AuthorsOpenReview ↗

General Response (2/3): Additional ablation study

3. Additional ablation study

The submitted paper has ablations for the two main parts of DAC: the cross-verification searcher reward and hard-positive evidence augmentation. We added three more experiments. All of them are now in Table 2 of the revised paper (Sections 6.1, 6.2 and 6.A).

What each row of Table 2 means. In DAC, the searcher is rewarded when the evidence is sufficient and the generator does not abstain (cross-verification).

  • Answer correctness as searcher reward: the searcher is rewarded when the final answer is correct. There is no abstention.
  • Search sufficiency as searcher reward: the searcher is rewarded when the external verifier marks the evidence as sufficient. There is no abstention.
  • + augmented samples (new): the same variant, trained with the same hard-positive augmented samples as DAC.
  • Frozen abstention judge for searcher reward (new): the reward is the same as in DAC, but the abstention decision comes from a frozen generator (the initial Qwen3-8B before any training), and the generator in training does not abstain.
  • w/o augmented samples and w/o turn-level rewards (the latter is new): DAC without that part.

[Table 2 of the revised paper] Answer EM on Qwen3-8B. The DAC row gives the scores. All other rows are differences from DAC, and the numbers in the text below are these differences in average EM.

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
DAC48.467.247.946.648.722.349.647.2
Answer correctness as searcher reward+0.2-1.2-1.1-1.6-3.4-1.7-2.4-1.6
Answer correctness as searcher reward + augmented samples-0.7-0.9-1.1-0.7-2.0-2.0-2.4-1.4
Search sufficiency as searcher reward-0.3-1.7-1.0-1.0-1.1-1.9-8.8-2.2
Search sufficiency as searcher reward + augmented samples-0.3-1.1-0.2-1.4-3.8-3.8-8.0-2.6
Frozen abstention judge for searcher reward-0.1-0.7-0.2-1.3-2.3-1.4-2.4-1.2
w/o augmented samples-3.4-1.5-2.8-2.4-5.3-1.9-0.8-2.6
w/o turn-level rewards-0.4-1.7-0.7-1.1+1.3+0.8-0.8-0.4

3.1. DAC without turn-level rewards. Turn-level rewards are not a main contribution of DAC. We took the idea from prior work, so we did not ablate it at first. Instead, we compared with TIPS, which adds turn-level rewards directly to single-agent training. DAC outperforms TIPS by 4.1 and 6.3 average EM points on Qwen2.5-7B-Instruct and Qwen3-8B (Table 1), which suggests that turn-level rewards alone are not enough and that they work best together with the other parts of DAC. We have now also trained DAC without turn-level rewards. The average drops by 0.4 points, so turn-level rewards do help, but this is the smallest drop among the ablations.

3.2. Searcher reward ablations with augmentation. The answer correctness and search sufficiency variants train without abstention, so over-abstention, the problem that augmentation is meant to solve, does not occur in them. For this reason we first ran them without augmentation, and for a more complete analysis we have now also run them with it. At first sight, Table 2 could suggest that the extra augmented samples matter more than cross-verification, because removing them from DAC causes the largest drop among DAC's components (2.6 points on average). However, the new results do not support this: adding augmentation to the two ablation variants changes the average little or even lowers it (-1.6 to -1.4 for answer correctness, -2.2 to -2.6 for search sufficiency). This shows that without cross-verification, augmentation does not help, so it does not improve performance by itself. Its main role is to prevent over-abstention, the main failure mode that cross-verification introduces.

3.3. Frozen abstention judge for searcher reward. This variant tests whether it matters that the abstention judge is trained together with the searcher. It is 1.2 points below DAC on average, and below DAC on every benchmark. Its search accuracy (judged by gpt-oss-120b) is also 1.2 points lower on average, indicating that a frozen judge cannot guide the searcher as well as a judge that evolves with the generator. On the other hand, the frozen judge (-1.2) is slightly better than the answer correctness (-1.6) and search sufficiency (-2.2) variants. So a judgment from the generator side already helps the searcher a little, even when it is frozen. Training this judgment together with the generator recovers the remaining 1.2 points.

AuthorsOpenReview ↗

General Response (1/3): Multi-seed results and verifier robustness

We thank all reviewers for their careful and constructive feedback. Several reviewers raised related points, so we answer these in the general response, in five parts: (1) Multi-seed results, (2) Robustness to the search sufficiency verifier, (3) Additional ablation study, (4) Abstention at test time, and (5) Training cost. The individual responses refer to these parts by name. We have also revised the paper accordingly, and the changes are marked in blue.


1. Multi-seed results

We trained DAC, DART and Search-R1 with two more training seeds each on Qwen3-8B, so each method now has three training runs.

Answer EM on Qwen3-8B (mean ± standard deviation over three training runs, including the run reported in the paper):

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
DAC48.50 ± 0.1766.87 ± 0.2948.13 ± 0.3246.40 ± 0.3548.23 ± 0.4222.00 ± 0.3050.13 ± 0.4647.18 ± 0.05
DART47.13 ± 0.5866.23 ± 0.4047.20 ± 0.1044.57 ± 0.6743.90 ± 0.6021.20 ± 0.2648.53 ± 0.4645.54 ± 0.08
Search-R148.00 ± 0.1766.13 ± 0.2545.50 ± 0.1744.90 ± 0.1744.43 ± 0.5021.13 ± 0.9048.53 ± 0.4645.52 ± 0.04

The results are stable across seeds. The standard deviation of the average EM is at most 0.08 for all three methods, while DAC is 1.6 points above DART and 1.7 points above Search-R1. Every DAC run is above every DART and Search-R1 run on the average and on each of the seven benchmarks, with one exception: on MuSiQue, one DAC run and one Search-R1 run are equal (21.7).

2. Robustness to the search sufficiency verifier

We agree that a more accurate verifier should give a better training signal, and we have actually tested this with an LLM judge (see below). However, choosing or designing a better verifier signal is not a main contribution of our work: DAC takes the verifier as given and is agnostic to its choice. We therefore used a simple substring check in the main experiments, because (1) it is cheap to compute and still accurate enough to test the benefit of our idea, and (2) it keeps the comparison with the baselines fair, because DAC gets no help from an extra external signal. The same check is also used to reward correct retrieval in prior RL work on search agents (Jin et al., 2025, arXiv:2505.15117; Wei et al., 2025, arXiv:2505.11821).

In this experiment, we used gpt-oss-20b (reasoning effort: medium) as an LLM judge for search sufficiency during training. Training was stable. The scores are higher than our original results (DAC's average EM rises from 47.2 to 48.1), which is expected because the external signal is richer. The searcher reward ablation also shows the same trend as before, so our interpretation does not change. We left this experiment out of the submission because we thought that a second verifier could confuse readers. We have now added it to Appendix D.b. Please note that we could not rerun every experiment or ablation with the LLM judge.

[Table 11 of the revised paper] Answer EM on Qwen3-8B, with the LLM judge as the search sufficiency verifier during training. The DAC row gives the scores, and the other rows are differences from DAC. The answer correctness variant does not use a sufficiency verifier, so it is the same run as in Table 2.

NQTriviaQAPopQAHotpotQA2WikiMuSiQueBamboogleAverage
DAC50.267.947.348.248.022.852.048.1
Answer correctness as searcher reward-1.6-1.9-0.5-3.2-2.7-2.2-4.8-2.5
Search sufficiency as searcher reward-1.6-1.2-0.3-1.1-0.6-0.6-2.4-1.1

The training dynamics with the LLM judge (Figure 10 in the revised paper) also show the same trend as in Figure 3 of the paper.

Update: Search-R1 multi-seed results are complete

As noted before, Search-R1 had two training runs when we first posted this response, and its third run was still training. That run has now finished. We have updated the table above and the individual responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.