acceptodds
Under review as a conference paper at ICLR 2027

Search-Mem: Learning to Search via Reinforcement Learning with Memory Agent

Abstract

Reinforcement learning (RL) has recently been used to train search agents that iteratively retrieve evidence for large language models (LLMs). However, two problems remain. First, a search agent can stop before gathering enough evidence to produce the correct answer. Once the search stops, the system has no way to recover from insufficient or incorrect evidence. Second, deeper search increases the amount of retrieved information carried across search steps, which can eventually exceed the available context and remove evidence needed for later decisions. We introduce Search-Mem, a memory-augmented multi-agent search system trained with RL. The system consists of five cooperating agents: a planner, compressor, retriever, generator, and memory agent. The planner controls the search by deciding what to query, which retrieved documents to keep, when to use the compressor, and when to stop. The compressor is trained jointly with the planner to condense retrieved evidence into short statements that can be carried into later search steps. Both agents are optimized using the improvement that the gathered context brings to the answer of a frozen generator, while the retriever and generator remain fixed. After RL training, the system constructs the memory agent by replaying multi-hop questions and storing how each question was decomposed without using reference answers. At inference, the trained search agents first gather evidence, and the generator produces several answers. If these answers are inconsistent or empty, the memory agent retrieves similar past decompositions and uses them to guide an additional search. The additional search gives the system another chance to find missing evidence, while the compressor keeps the planner's context short as the search continues. Across six QA benchmarks and three frozen generators, Search-Mem achieves the best average accuracy under both an open and a proprietary judge, with the best result in 34 of 36 settings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Retrieval Augmented Generation, Search Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2T9FOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Search-MEM is a memory-augmented search system with five cooperating agents: a planner, compressor, retriever, generator, and memory agent. The planner controls queries, document selection, compression, and stopping; the compressor shortens retrieved text in the planner’s context; the generator answers from retained documents. The planner and compressor are trained jointly with RL while the retriever and generator remain frozen. The memory is built by replaying multi-hop training questions and storing decompositions that pass reference-free verification checks. At inference, the system performs an initial search and samples multiple answers. If the samples disagree or are empty, the memory retrieves similar decompositions to guide an additional search; newly retrieved documents are appended to the generator’s context, and the generator answers again. The paper evaluates on six QA benchmarks with multiple generator sizes and two judges, reports zero-shot transfer to a news corpus, and ablates the memory, compressor, and cost.

The main contributions are: (1) a memory-gated repair mechanism that triggers additional search when generator samples are unsettled; (2) joint planner–compressor training with a shared trajectory reward, allowing the planner to carry much less retrieved text with approximately unchanged accuracy; and (3) strong average accuracy using about 2.5% of the training questions used by Search-R1, while serving multiple generators including a proprietary one.

Critical strengths

1.The memory-gated repair mechanism addresses a real failure mode of existing search agents. Existing trained search policies decide when to stop before the final answer is produced; if the gathered evidence is insufficient, they have no mechanism to search again. Search-MEM’s use of reference-free decomposition memory to trigger an additional search when generator samples disagree or are empty is a sensible and practical intervention. This is consequential because it improves robustness without retraining or modifying the frozen generator, and the zero-shot news-corpus transfer provides some evidence that the memory carries a decomposition format rather than only dataset-specific content.

2.Joint planner–compressor training with a shared trajectory reward is a clean way to keep search context small. The compressor is optimized by the downstream answer reward rather than by a separate compression objective, and the planner is rewarded for using compression, stopping voluntarily, and improving over naive RAG. This is consequential because it supports deeper search with a much smaller planner context while keeping accuracy approximately unchanged, which is important for context-budget-limited agentic search.

Critical weaknesses

1.The memory’s same-distribution advantage is not sufficiently ruled out. The memory is written from MuSiQue training questions, and MuSiQue is one of the six evaluation datasets. Even with disjoint questions, the memory’s decomposition style and bridge patterns may be unusually well matched to MuSiQue. The largest memory gains are indeed on MuSiQue (e.g., Qwen2.5-7B from 16.2 to 23.3 under Claude judge). This is consequential because the memory is the paper’s central contribution; without a leave-one-dataset-out memory construction or a memory built from a clearly disjoint corpus, the claim that the memory supplies a transferable decomposition format is only partially supported.

2.The gate is blind to confident errors, and the cost–accuracy Pareto advantage is not fully established. The gate fires only when the generator’s samples disagree or are empty; if all samples agree on a wrong answer, the system returns it without repair. The paper does not quantify how often this happens. At the same time, Search-MEM averages about 4.3 retrievals and 10.4 model calls per question, more than the compared search methods. This is consequential because the method’s practical value depends on whether the additional repair cost is concentrated on questions where it is needed and whether the accuracy gain is robust relative to cheaper baselines at matched budgets.

Questions for authors

1.Memory transfer / leave-one-dataset-out memory: Can you report results with the memory built from a dataset other than MuSiQue, or from a disjoint corpus, and evaluated on MuSiQue? How much of the MuSiQue gain remains? This would directly test whether the memory carries a transferable decomposition format rather than a same-distribution advantage.

2.Confident-wrong failures: What fraction of test questions have all k generator samples agree on a wrong answer, so the gate stays silent? Can you characterize these cases and estimate the headroom from a coverage- or uncertainty-based gate extension?

3.Gate and cost sensitivity: How sensitive are the results to the number of samples k, sampling temperatures, and the disagreement/empty gate? Can you provide a cost–accuracy Pareto comparison against s3 and Search-R1 at matched retrievals or matched model calls, rather than only at the deployed operating point?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 2T9F (1/2)

We thank the reviewer for the constructive comments. We ran additional experiments on memory transfer, silent unanimous errors, and inference cost, and address the concerns below.

W1 / Q1. Cross-dataset memory transfer Thank you for the constructive comment. To address this, we additionally built memories from equally sized samples from two other multi-hop datasets, HotpotQA and 2Wiki, and evaluated them on MuSiQue with Qwen2.5-14B as the generator. We note that the contexts, initial answers, gate decisions, and the repair procedure remain fixed; only the supplied memory changes.

Memory source# of examplesGenAcc (%)Gain over no repair (%)Native gain retained (%)
No repair—20.0——
HotpotQA (out-of-distribution)2,50024.2+4.290.1
2Wiki (out-of-distribution)2,50024.1+4.187.4
MuSiQue (in-distribution)2,50024.6+4.6100.0

We call memories built with HotpotQA and 2Wiki out-of-distribution memories, as they differ from the downstream task (MuSiQue). As we observe, they retain most of the in-distribution memory’s gain. We reckon the possible reason could due to the fact that memory stores questions, splits, and bridge/no-bridge flags. This supports transfer of decomposition guidance across datasets.

W2 / Q2. Silent unanimous errors and repair headroom Thank you for raising this. Our deployed gate also checks answer quality, grounding, and question coverage, so agreement alone does not guarantee that repair is skipped. Our repair mechanism fires when any condition below holds:

Gate signalTrigger
Agreement (deployed k=3)Fewer than two nonempty normalized answers agree
Answer qualityThe voted answer is empty or hedged
GroundingThe normalized answer string is absent from the context
CoverageQuestion-content-word coverage is below 60.0%

We will revise the method description and algorithm correspondingly. To quantify the effect, we measured unanimous errors on the full evaluation splits using our planner’s contexts and examined the cases the gate missed. Below, silent and caught both refer to unanimous judged-wrong answers. Error frequencies use all evaluated questions as the denominator; gold-string presence and token overlap use only silent unanimous-wrong cases.

DatasetQuestionsUnanimous wrong (%)Gate silent (%)Gate catches (%)Gold string present (%)Token F1 ≥ 0.5 (%)
MuSiQue2,39513.68.94.826.014.0
HotpotQA7,40013.210.72.548.023.0
2Wiki12,55011.28.42.841.015.0
NQ3,61013.811.32.453.012.0

The additional signals catch some unanimous errors, while the silent column quantifies those that remain undetected. Every silent prediction passes the answer-string grounding check, but appearing in a passage does not establish that it answers the requested relation. Likewise, gold-string presence does not prove sufficient support, and absence does not prove that evidence is missing. These lexical measurements characterize the missed cases without assigning manually verified causes.

We then measured recovery in separate repair analyses. One forces repair on silent unanimous errors and reports both the fraction corrected within that group and the gross corrections among all evaluated questions. The other repairs all silent questions, including non-unanimous cases, and measures the net accuracy gain over gated repair. Both use the evaluation sizes below, but the unanimous-only recovery and all-silent net gain are distinct measurements.

DatasetQuestions per analysisSilent unanimous errors corrected (%)Gross corrections (% of all questions)Net gain from repairing all silent questions (%)Repairs skipped by gating (%)
MuSiQue2,3954.20.4+0.729.0
HotpotQA1,5007.80.8+1.060.0
2Wiki1,5004.70.3+0.741.0
NQ1,5005.10.6+0.861.0

The existing repair corrects part of the missed unanimous-error group. These gross recoveries are not net gains from a new gate or a hard upper bound on better repair methods. The separate all-silent analysis shows that always repairing achieves higher accuracy, while gating avoids many repair invocations. We therefore present gating as an accuracy–computation tradeoff. We also note that skipped invocations do not by themselves establish end-to-end token or latency savings.

AuthorsOpenReview ↗

Response to Reviewer 2T9F (2/2)

Q3. Gate and cost sensitivity. Following the reviewer’s suggestion, we varied k and sampling temperature. The first read is greedy; subsequent reads use the listed temperature. Agreement requires a strict majority of identical non-empty answers, while the other gate signals remain active. Each cell reports the difference between the reported gated-repair and no-repair accuracies, in %.

Datasetk=1, T=0k=3, T=0.3k=3, T=0.7 (Reported setting)k=3, T=1.0k=5, T=0.7
MuSiQue+4.9+4.5+4.5+5.3+4.3
HotpotQA+2.0+2.1+1.8+1.8+2.1

Our deployed setting is k=3, T=0.7. Repair improves accuracy under every tested configuration, including with more initial votes. The gate comparison gives:

GateMuSiQue GenAcc (%)
Disagreement only28.0
Disagreement or empty answer27.1
Deployed combined gate28.4

As shown, the combined gate performs best among these variants. The disagreement-or-empty result also shows that adding a trigger does not automatically improve accuracy.

Cost-accuracy Pareto We compared the methods at the reported call and retrieval budgets to illustrate the cost and accuracy of our method, and s3 and Search-R1. We report Qwen GenAcc, with the highest reported score in each column bolded.

Reported answer-stage model-call budget per question:

Method1.03.05.06.58.0
s319.724.525.025.926.1
Search-R121.924.727.429.428.4
Search-Mem22.724.627.928.429.0

Reported added-retrieval budget per question:

Added-retrieval calls0.01.01.52.02.6
s322.123.824.925.926.1
Search-R123.426.427.229.428.3
Search-Mem24.226.627.828.428.9

Search-Mem exceeds s3 throughout the reported grid and performs better than Search-R1 at most of the budgets. Bold marks the best reported entry in each column; a stronger lower-budget configuration remains feasible at larger budgets.

To test whether generic uses of extra computation reproduce the repair gains, we used Qwen2.5-14B as generator and Qwen-judged GenAcc. Within each comparison, contexts, initial reads, gate decisions, and yes/no retention are shared. Each column below refers to its named baseline; the voting-resource measurements are separate from the call/query-matched accuracy controls.

MuSiQue contextsGain over call-matched sampling (%)Gain over call/query-matched draft retrieval (%)Tokens saved versus seven-sample voting (%)Mean latency saved versus seven-sample voting (%)
s3+3.8+4.76.326.4
Search-R1+0.6+0.717.531.7
Search-Mem+3.9+4.33.825.2

Targeted repair achieves higher accuracy than the generic sampling and retrieval controls, with smaller gains on Search-R1. It also uses fewer tokens and has lower mean latency than the voting baseline in the resource comparisons. A dash indicates an unreported resource comparison. The cost measurements cover the answer stage and exclude upstream search, planning, and compression. We reckon that the results support the stated answer-stage tradeoffs

Based on these findings, we will add the transfer, failure, sensitivity, and budget analyses and clarify the gate. We hope that the reviewer is satisfied with our response and will increase their score accordingly.

Dear Reviewer 2T9F,

With the discussion period ending in approximately 2 days, we would greatly appreciate your feedback on our rebuttal. We believe we have thoroughly addressed your concerns and would welcome your assessment. If you find that we have successfully resolved your major points, we respectfully request that you consider adjusting your score to reflect these improvements.

Thank you for your time and consideration.

Best regards,

The Authors

Reviewer_KdevOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose Search-Mem, which jointly trains a search planner and compressor with answer-based rewards from a frozen generator. It constructs a memory of question decompositions and uses retrieved examples to guide additional search when sampled answers disagree. Evaluation covers six QA benchmarks, three generators, and transfer to a news corpus.

Critical strengths
  • The memory-guided intervention improves accuracy across datasets and generators, with substantial gains on MuSiQue. Random, swapped, and hand-written exemplar controls provide evidence that relevant decomposition examples contribute beyond additional retrieval alone.
  • Compression substantially reduces the planner's context while approximately preserving accuracy as search depth increases. Reusing the planner and memory with different frozen generators makes the system practically relevant.
Critical weaknesses
  • The benefit of joint RL training is unclear. Appendix I states that the planner before any RL step reaches the accuracy of the training phase. The no-memory system also trails s3 on average under the Claude judge for all three generators (Tables 1 and 3). Compression reduces planner context, but its training benefit is unisolated. These results leave the contribution of joint RL training insufficiently established.
  • The overall improvement is not evaluated at matched inference cost. Table 8 reports 10.4 model calls and 4.3 retrievals per question, versus 7.0/3.1 for s3 and 6.5/2.4 for Search-R1. The exemplar controls support a contribution from memory, but do not establish that the trained pipeline outperforms these searchers with the same gate, decomposition, and additional-search budget.
Questions for authors
  1. Can you compare step-0 and step-20 planners, and trained versus frozen compressors, with memory construction and inference held fixed? How much improvement remains attributable to joint training?
  2. How do s3 and Search-R1 perform with the same gate and re-retrieval, with and without memory examples, at a matched token or latency budget?
  3. Section 5.1 reverses the Claude scores for zero examples and four no-bridge examples relative to Table 4 (20.29 and 18.46). Which is correct?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Kdev (1/2)

We thank the reviewer for the constructive comments. We ran additional experiments to clarify the contributions of joint training, repair, and memory examples.

W1 / Q1. Contribution of joint RL training: We independently trained a planner-only variant and a compressor-only variant, keeping the other module frozen throughout each run. We train each module to 20 steps, following our training recipe. All four variants (together with both modules frozen, and Joint training 2 modules) were evaluated on the same 2,395 MuSiQue questions with the same memory bank and inference procedure, using Qwen2.5-14B as the generator.

Training settingGenAcc (Qwen/ Claude)
Both modules frozen24.5 / 22.8
Planner-only RL24.9 / 22.9
Compressor-only RL26.3 / 24.4
Joint RL28.4 / 26.7

Training the compressor alone improves accuracy by 1.8% / 1.6%, while training the planner alone adds 0.4% / 0.1%. Joint training then adds 2.1% / 2.3% over the strongest single-module variant. These results show that compressor learning contributes better to the gain, but neither single-module run reproduces the joint result. Because memory and inference are fixed, the differences come from the training variants rather than different memory banks or inference rules.

W2 / Q2. Additional computation and memory-guided repair: We group W2 and Q2 to address together to have a clearer response, as there are parts of those questions that can overlap.

W2. Do the gains come from additional computation? We compared repair with additional sampling and generic draft-based retrieval. These are new paired experiments on N saved contexts per source, using Qwen2.5-14B and Qwen-judged GenAcc. The sampling control replaces the repair step with additional answer samples and takes the majority over all samples, so it uses the same number of model calls as repair. Here, the draft-retrieval control issues a draft answer as an additional query, matching both the model calls and the retrieval queries of repair.

MuSiQue contextsGain over sampling (%)Gain over draft retrieval (%)
s3+3.8+4.7
Search-R1+0.58+0.67
Search-Mem+3.88+4.34

As shown, repair achieves higher accuracy than the tested generic uses of the additional budget. The smaller gains on Search-R1 show that the observed benefit varies across context sources. These controls evaluate the complete repair strategy.

We additionally report answer-stage accuracy, tokens, and latency on the full MuSiQue split (2,395 questions), comparing memory-guided repair with the voting baseline, which takes the majority answer over K samples on every question and runs no repair. This baseline differs from the call-matched sampling control above, so the gains in the two tables are not expected to coincide

Saved contextsAnswer-stage methodGenAcc (%)Mean total tokensMean latency (s)
s3Voting20.99,42934.1
s3Memory-guided repair23.88,83325.1
Search-MemVoting21.49,22533.7
Search-MemMemory-guided repair28.48,87425.2

Memory-guided repair improves accuracy by 2.9 points on s3 contexts and 7.0 points on our contexts, while using 6.3% / 3.8% fewer answer-stage tokens and 26.4% / 25.2% less time than voting. We reckon one possible reason why repair costs less then voting is because voting draws all K samples on every question, while repair runs only on questions where the initial vote disagrees. The accuracy gain therefore does not come from spending more computation than voting.

AuthorsOpenReview ↗

Response to Reviewer Kdev (2/2)

Q2. What do memory examples contribute under the same repair procedure? Our paper’s table 4 already compared repair with and without memory examples. Repair remains enabled in both cases, as only the examples in the decomposition prompt change.

Decomposition examplesQwen-judged GenAcc (%)Claude-judged GenAcc (%)
None21.5020.29
Deployed mix24.8423.34

Examples improve accuracy under both judges. They guide how the question is split and which additional evidence is retrieved, so their contribution extends beyond enabling repair itself.

We also compare memory-guided repair with repair without examples and repair using a fixed instruction to decompose the question. Repair stays enabled in every version. Each baseline below shows the accuracy gained by using memory and the extra tokens it uses relative to the listed control.

MuSiQue contextsControlAccuracy gain with memory (%)Extra tokens with memory (%)
s3Repair without examples+3.3+5.7
Search-MemRepair without examples+3.0+5.9
s3Repair with forced decomposition+1.0+2.6
Search-MemRepair with forced decomposition+1.0+2.7

On s3 contexts, adding examples improves accuracy by 3.3% while using 5.7% more tokens. On our contexts, the gain is 3.0% for 5.9% more tokens. Memory also improves over forced decomposition by 1.0% on both context sets, with 2.6%–2.7% more tokens. This is consistent with examples helping the model choose useful splits beyond simply asking it to decompose. These comparisons are not token-matched: memory-guided repair uses more tokens.

Q3. Section 5.1 correction Table 4 is correct: Claude-judged GenAcc is 20.29% with zero examples and 18.46% with four no-bridge examples. Section 5.1 swaps these labels. We will correct the prose; the table scores remain unchanged.

Based on these findings, we will add the controlled training and repair analyses, clarify the cost comparison, and correct Section 5.1. We hope that the reviewer is satisfied with our response and will increase their score accordingly.

Dear Reviewer Kdev,

With the discussion period ending in approximately 2 days, we would greatly appreciate your feedback on our rebuttal. We believe we have thoroughly addressed your concerns and would welcome your assessment. If you find that we have successfully resolved your major points, we respectfully request that you consider adjusting your score to reflect these improvements.

Thank you for your time and consideration.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up by the author to Reviewer Kdev

Dear Reviewer Kdev,

As the rebuttal deadline is approaching, we would be grateful if you could take a look at our response. We would appreciate knowing whether it has addressed your concerns, or whether there are any remaining points we could clarify. If you find that we have successfully resolved your major points, we respectfully request that you consider adjusting your score to reflect these improvements. It would mean a lot to our research

If you have further questions, please let us know and we will be happy to respond.

Best regards,

The Authors

Reviewer_yynsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SEARCH-MEM, a memory-augmented search system that jointly trains a planner and compressor with a shared reward from a frozen generator. It builds a memory of question decompositions without reference answers and uses it to guide additional search when generator answers disagree or are empty. Across six QA benchmarks and three frozen generators, SEARCH-MEM achieves the best average accuracy under both judges, while compression reduces the planner's context with little change in accuracy.

Critical strengths
  1. The paper addresses a practical problem: search can stop before enough evidence is collected. Using past question decompositions to guide additional search when answers disagree or are empty is a sensible design.

  2. On MuSiQue, compression reduces the planner's carried text by roughly an order of magnitude with little change in accuracy. The original documents remain available to the generator, preserving its access to detailed evidence.

  3. Results are broadly consistent across six QA benchmarks and three frozen generators. The same planner and memory also transfer to a news corpus, supporting their use beyond the original retrieval setting.

Critical weaknesses
  1. The contribution of joint RL training is not clearly established. The reported ablations show the benefits of memory-guided search and compression, but do not isolate the gains from RL or establish whether jointly training the planner and compressor outperforms keeping them frozen.

  2. The inference description is inconsistent with Algorithm 6. Section 3.5 describes returning a new answer after additional retrieval, whereas Algorithm 6 retains the initial majority answer if it is yes/no, even after repair. This exception is not explained, leaving the rationale for the final answer selection unclear.

  3. Figure 1(a) is unclear, particularly the data flow between the compressor and the generator.

  4. Table 1 does not consistently underline the second-best results as stated in its caption. For example, s3's NQ scores of 58.3/55.1 in the Qwen2.5-7B panel are second-best but unmarked.

Questions for authors
  1. Have you run ablations comparing joint RL training with keeping both the planner and compressor frozen, or training only one of them?

  2. Have you trained the planner and compressor using newer backbones such as Qwen3 or Qwen3.5? Do the gains persist?

  3. Why does Algorithm 6 retain the initial yes/no majority answer after repair, and how does removing this rule affect accuracy?

4 comments
AuthorsOpenReview ↗

Response to Reviewer yyns (1/2)

We thank the reviewer for the constructive comments. We ran additional experiments to examine the contribution of joint training and the yes/no rule, and we address each concern below.

W1 / Q1. Contribution of joint RL training. We independently trained a planner-only variant and a compressor-only variant, keeping the other module frozen throughout each run. We train each module to 20 steps, following our training recipe. All four variants (together with both modules frozen, and Joint training 2 modules) were evaluated on the same 2,395 MuSiQue questions with the same memory bank and inference procedure, using Qwen2.5-14B as the generator.

Training settingGenAcc (Qwen/ Claude)
Both modules frozen24.5 / 22.8
Planner-only RL24.9 / 22.9
Compressor-only RL26.3 / 24.4
Joint RL28.4 / 26.7

Training the compressor alone improves accuracy by 1.8% / 1.6%, while training the planner alone adds 0.4% / 0.1%. Joint training then adds 2.1% / 2.3% over the strongest single-module variant. These results show that compressor learning contributes better to the gain, but neither single-module run reproduces the joint result. This agrees with MMOA-RAG [2], where excluding any single module from joint optimization lowered end-task performance. Because memory and inference are fixed, the differences come from the training variants rather than different memory banks or inference rules.

Q2. Newer planner and compressor backbones To address this concern, we run new experiments, where we trained Qwen3-0.6B planner and compressor for 20 steps and compared Search-Mem with exact-s3 using Claude-Haiku as the generator. Both methods are evaluated on the same questions within each dataset. We kindly note that in our result table below there is no Search-R1 baseline, due to the substantial requirements for computation and time for training Search-R1 from scratch [1].

NQTriviaQAPopQAHotpotQA2WikiMuSiQueDataset average
Questions1,5001,5001,5001,5001,5002,395—
Qwen judge (%)
s367.481.059.548.542.317.652.7
Search-Mem67.882.160.054.647.825.056.2
Claude judge (%)
s364.778.257.943.742.416.850.6
Search-Mem65.179.558.449.548.023.854.1

We observe that Search-Mem achieves higher GenAcc on every evaluated dataset under both judges, with dataset-average gains of 3.5% under Qwen judge and 3.4% under Claude-Haiku judge. The multi-hop gains range from 5.5%–7.3% and 5.6%–7.0%, respectively. The larger gains on multi-hop datasets are consistent with the method’s focus on acquiring evidence across linked sub-questions. These results show that its advantage persists with the tested newer backbone, which supports our method robustness to newer models.

W2 / Q3. Yes/no answer preservation Thank you for identifying this. We used the rule in Algorithm 6 during evaluation but did not describe it in Section 3.5. After repair, we retain the initial majority answer if it begins with the word “yes” or “no,” using a case-insensitive word-boundary check after trimming whitespace. Otherwise, we return the repaired majority answer.

To test the effect of the rule, we ran an additional ablation on Search-Mem, with and without it. We used the full evaluation splits and Qwen2.5-14B as the generator. The rule changes only answer selection; the repair retrievals and model calls have already been incurred. Cells report Qwen / Claude-judged GenAcc.

DatasetWith rule (%)Without rule (%)
HotpotQA55.50 / 50.5055.39 / 50.39
2Wiki43.00 / 43.0042.43 / 42.46

As seen, removing the rule lowers accuracy by 0.11% on HotpotQA and 0.54%–0.57% on 2Wiki. This is why we retain it: replacing an initial binary vote with the repaired vote has a worse net outcome on these datasets. The rule uses no reference answers. We will document it and align Section 3.5, Algorithm 6, and Table 9 with deployment.

AuthorsOpenReview ↗

Response to Reviewer yyns (2/2)

W3. Figure 1(a) is unclear We agree with the reviewer that Figure 1(a) currently does not clearly show the data flow. We would revise it to illustrate the follwoings: Compression updates only the planner’s context; The generator reads selected original documents from marked blocks and all original documents from unmarked blocks, including the initial retrieval, and the compressor therefore affects the answer through later queries and document selection, not by replacing the generator’s evidence with a summary. In our revision, we will redraw the two paths as documents -> planner selection -> generator, and documents -> compressor -> planner context.

W4. Table 1 missing underlines Thank you for catching this formatting issue. We will underline the second-best result in each dataset column within each generator panel, so the marking is consistent throughout Table 1.

Based on these findings, we will add the ablations, results on new backbone and correct the descriptions and figure. We hope that the reviewer is satisfied with our response and will increase their score accordingly.

References

[1] Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning - CoLM

[2] Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning - NeurIPS

Dear Reviewer yyns,

With the discussion period ending in approximately 2 days, we would greatly appreciate your feedback on our rebuttal. We believe we have thoroughly addressed your concerns and would welcome your assessment. If you find that we have successfully resolved your major points, we respectfully request that you consider adjusting your score to reflect these improvements.

Thank you for your time and consideration.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up by the Authors

Dear Reviewer yyns,

As the rebuttal deadline is approaching, we would be grateful if you could take a look at our response. We would appreciate knowing whether it has addressed your concerns, or whether there are any remaining points we could clarify. If you find that we have successfully resolved your major points, we respectfully request that you consider adjusting your score to reflect these improvements. It would mean a lot to our research

If you have further questions, please let us know and we will be happy to respond.

Best regards,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response from the authors

We thank the reviewers for their constructive feedback. In addition to the individual response to each reviewer, we have uploaded a revised paper based on the changes requested, with all changes marked in blue. Please refer to the revised PDF for the newly added rebuttal contents.

We first thank the reviewers for the strengths listed:

  • A practical fix for a real failure mode. Reusing reference-free decompositions to search again when trained search agents stop too early is sensible and practical (Reviewer yyns, Reviewer 2T9F), and it needs no retraining or modification of the frozen generator (Reviewer 2T9F).

  • Memory gains backed by controls. Repair improves accuracy across datasets and generators, most on MuSiQue, and the random, swapped, and hand-written exemplar controls show that relevant examples matter beyond extra retrieval (Reviewer Kdev).

  • A small planner context at near-unchanged accuracy. Compression, trained jointly with the planner on a shared reward (2T9F), cuts the planner's carried text by about an order of magnitude with little change in accuracy (yyns, Kdev, 2T9F), while the generator still reads the original documents (yyns).

  • Consistent, reusable results. Results hold across six benchmarks and three frozen generators (Reviewer yyns). One planner and memory serve every generator, including a proprietary one, with about 2.5% of Search-R1's training questions (Reviewer Kdev, Reviewer 2T9F), and they transfer to a news corpus (Reviewer yyns, Reviewer 2T9F).

Moreover, for each concerns raised, we have run additional experiments, and add explanations to clarify:

Reviewer yyns

  • Joint training (W1, Q1): new Section 5.4 and Table 6 compare training neither agent, only the planner, only the compressor, and both. Joint training is best under both judges.
  • Yes/no rule (W2, Q3): Section 3.5 now states the rule, Algorithm 6 and Table 12 revised, and Appendix C shows that removing it lowers accuracy on HotpotQA and 2Wiki.
  • Figure 1(a) (W3): redrawn so that documents reach the generator through the planner's selection, while compressed statements only update the planner's context.
  • Table 1 (W4): every second-best result is now underlined.
  • Newer backbone (Q2): with the planner and compressor trained from Qwen3-0.6B, Search-Mem outperforms s3 on all six datasets under both judges (Appendix C).

Reviewer Kdev

  • Joint training (W1, Q1): Section 5.4 and Table 6, with memory and inference held fixed.
  • Matched cost (W2, Q2): new Table 7 shows that repair beats equal-budget sampling and draft-retrieval controls on s3, Search-R1, and Search-Mem contexts. It is also more accurate than seven-sample voting on s3 and Search-Mem contexts, and costs less than voting on all three.
  • Section 5.1 (Q3) reports the Table 4 scores correctly.

Reviewer 2T9F

  • Memory transfer (W1, Q1): new Section 5.2 and Table 5 show Search-Mem memory transfer ability, where on MuSiQue, memories written from HotpotQA and 2Wiki keep most of the gain of a memory written from MuSiQue.
  • Gate and silent errors (W2, Q2): Section 3.5 now describes the deployed gate in full. Besides agreement, it checks answer quality, grounding, and coverage, so agreeing samples alone do not skip repair. Appendix C measures the unanimous errors the gate misses and how much repairing them would recover.
  • Sensitivity and cost (W2, Q3): We reported experiment results which varies k, the temperature, and the gate, and compares Search-Mem with s3 and Search-R1 at matched call and retrieval budgets. Table 7 adds the equal-budget controls.

We hope these changes address the reviewers' concerns, and we are happy to answer any further questions from the reviewers in the rebuttal period.

AuthorsOpenReview ↗

Status of our submission at the close of the author discussion

Dear Area Chairs, Senior Area Chairs and Program Chairs,

As the author discussion period is about to close, we would like to briefly summarize the status of our submission.

Our responses were posted on 12 November and the revised paper on 14 November, with all changes marked in blue. We have responded to every weakness and question raised in the three reviews, each with a new experiment or an explanation in the revised paper. No reply to these responses has been posted, and no further concern has been raised during the discussion period. To our knowledge, no concern from the reviews is left unanswered:

  • Reviewer yyns (W1 to W4, Q1 to Q3): the contribution of joint training is isolated in a new ablation where joint training is best under both judges (Table 6), the gains hold with a newer Qwen3 backbone, the yes/no rule is now stated and ablated, and Figure 1(a) and Table 1 are corrected.
  • Reviewer Kdev (W1, W2, Q1 to Q3): the same ablation answers the question on joint training. Repair is more accurate than equal-budget sampling and draft-retrieval controls on s3, Search-R1, and Search-Mem contexts and costs less than seven-sample voting (Table 7). Under the same repair on the full MuSiQue split, Search-Mem contexts reach 28.4 against 23.8 for s3 contexts at nearly the same answer-stage cost. Section 5.1 is corrected.
  • Reviewer 2T9F (W1, W2, Q1 to Q3): memories written from other datasets keep most of the gain (Table 5), the full gate is now described and the errors it misses are measured, and the requested sensitivity and matched-budget comparisons are reported.

A point-by-point account is in our general response.

We would be really grateful if you could invite the reviewers to update their reviews in light of these responses in the AC-reviewer discussion phase, and we kindly ask that the responses and the revised paper be taken into account in the final assessment.

Thank you for your time and effort in processing our submission.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.