acceptodds
Under review as a conference paper at ICLR 2027

MemAgent: Learning to Manage Heterogeneous Memory Providers for LLM Agents

Abstract

Current agents remain largely stateless across tasks, limiting their ability to continually improve from prior interactions and making memory essential for long-horizon agentic behavior. Existing memory methods seek to reuse past experience, but most rely on a single memory representation (, trajectories, reflections, skills, structured knowledge) whose effectiveness varies across task distributions. Rethinking this design space, we evaluate 13 memory methods and find that no single method generalizes across benchmarks, revealing the potential of heterogeneous memory providers. We formulate agent memory as a problem in which a memory agent decides which memory provider to retrieve from, whether to inject short-term memory, and which providers should store the resulting experience. Based on this perspective, we propose , featuring a content-aware routing architecture and a training-data synthesis pipeline. The routing architecture combines content-aware probing before retrieval, short-term memory gating during execution, and selective multi-provider storage, while the training pipeline synthesizes phase-specific supervision for routing decisions. Across GAIA, WebWalkerQA, and xBench-DS, MemAgent improves average accuracy by 10.0% and outperforms the best individual memory method. These gains come with less than 0.3% routing overhead and a 12% reduction in average task steps.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Memory, Adaptive Memory Routing, Heterogeneous Memory Providers

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DQ8MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MemAgent, which formulates LLM agent memory management as a routing problem over heterogeneous memory providers. It manages 13 long-term memory providers and makes decisions at three stages: BEGIN selects a provider for retrieval, IN decides whether to inject short-term working memory, and END selects providers for storing the current trajectory. The router is trained via reward-filtered SFT using data collected through forced exploration, with supervision signals based on task correctness, LLM-as-a-Judge scores, and downstream memory reuse. Experiments on GAIA, WebWalkerQA, and xBench-DS show that MemAgent achieves an average accuracy of 75.7%, a 10.0 percentage-point gain over the no-memory baseline (65.7%), outperforming individual memory providers and several routing and memory-aggregation baselines.

Critical strengths
  1. Practical problem. Different memory representations may suit different tasks, making the selection of appropriate memory systems a meaningful problem beyond designing yet another memory representation.
  2. Comprehensive system design. The BEGIN, IN, and END stages cover retrieval, within-task working memory, and selective storage, providing a more complete approach than methods focused solely on retrieval.
  3. Clear presentation. The paper is generally well written and easy to follow.
Critical weaknesses
  1. The proposed approach is straightforward, but its novelty is limited. It appears to mainly bring existing components together within a unified framework, which does not necessarily constitute a substantial methodological contribution. In addition, the method is not compared against recent approaches to learned memory management and adaptive memory design, such as Memory-R1, MemRL, or other relevant methods.
  2. Potential overlap between memory construction and evaluation. The paper states that the same 53 GAIA Level-1 tasks are used both to build the memory pool and for evaluation. The authors should clarify whether the memory available when evaluating a task contains its own trajectory, answer, or highly similar content. How do they ensure that the reported gains reflect cross-task transfer rather than direct reuse of previously seen tasks? Results excluding Level 1 should also be reported to assess performance under a stricter held-out setting.
  3. Insufficient statistical validation and cost accounting. The main results are reported as single accuracy numbers, without multiple runs, confidence intervals, or significance tests. Given the limited number of benchmark tasks, some gains may depend on a small number of examples. In addition, the reported overhead of approximately 1.2 seconds per task measures routing latency, not necessarily the total cost of the full memory lifecycle. Since all 13 long-term memory providers require LLM calls during storage, the authors should clarify whether the reported overhead of less than 0.3% excludes memory construction, trajectory summarization, provider updates, and training costs.
Questions for authors

See the weaknesses above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer DQ8M (1/2)

We thank the reviewer for recognizing the practical problem, the three-phase design, and the clarity of the paper.

Q1. "The proposed approach is straightforward, but its novelty is limited. It appears to mainly bring existing components together within a unified framework, which does not necessarily constitute a substantial methodological contribution."

A1. Existing work, including adaptive methods such as MemEvolve [1], designs a single memory system. MemAgent addresses an orthogonal problem: learning how to route across a set of heterogeneous memories, so any new memory design can plug in as a provider. Specifically:

  1. Finding. Evaluating 13 memory systems under one interface on three benchmarks shows that the best memory representation changes across tasks and benchmarks.
  2. Formulation. We cast memory management as routing across all three phases of a task: BEGIN selects the long-term provider whose stored content fits the query, IN gates working-memory injection at each step, and END stores the trajectory only into providers where future tasks will reuse it, reducing memory pollution. Prior memory work does not learn where to store experience.
  3. Training. Our label-free training pipeline assigns credit at each phase, including a downstream reuse reward for END storage decisions whose effect appears only on later tasks.

Simple combinations of the 13 providers fall well short of MemAgent (75.7% avg.): concatenating all retrieved memories gets 69.8% (All-Concat) and fusing them with an LLM gets 71.0% (All-Fuse). On the same pool, MemAgent also beats Random Routing (67.2%), the untrained router (68.1%), and a GPT-5-mini router prompted with rules mined from our training data (PromptRoute, 70.9%).


Q2. "In addition, the method is not compared against recent approaches to learned memory management and adaptive memory design, such as Memory-R1, MemRL, or other relevant methods."

A2. Memory-R1 [2] and MemRL [3] manage entries inside a single memory store, which is complementary to routing across multiple stores. Memory-R1 trains an RL policy to apply ADD/UPDATE/DELETE/NOOP to a single dialogue memory bank, and MemRL learns utility scores to re-rank entries inside a single episodic store. MemAgent instead learns which heterogeneous memories to read from and write to, and either method could be added to our pool as a provider. For adaptive memory design, Table 1 already compares against the closest work: LightWeight, the memory system evolved and released by MemEvolve [1]. MemAgent beats LightWeight by +7.1 pp on average (75.7% vs. 68.6%). We have added a discussion of Memory-R1 and MemRL to Related Work.


AuthorsOpenReview ↗

Response to Reviewer DQ8M (2/2)

Q3. "Potential overlap between memory construction and evaluation. ... The authors should clarify whether the memory available when evaluating a task contains its own trajectory, answer, or highly similar content. ... Results excluding Level 1 should also be reported ..."

A3. Following MemEvolve [1], Appendix A.2 separates these two settings: "we therefore treat GAIA L1 as an experience-reuse setting rather than a strict held-out generalization test," while "the remaining 112 GAIA L2/L3 tasks and all WebWalkerQA and xBench-DS tasks are disjoint from training and serve as held-out generalization benchmarks."

To check whether retrieved memories contain same-task content or cross-task know-how, we audited the BEGIN memory injected on every task using an independent judge from a different model family (Gemini-2.5-Flash):

GAIA L1 (seen)GAIA L2/L3WebWalkerQAxBench-DS
Memory contains the current question verbatim11–87%0–2.7%0%0%
Gold answer string appears in memory23–34%1.4–7.2%0–0.6%0–2.7%
Judge: procedural know-how64–79%85–100%98%98–100%

Mostly coincidental matches of common words. Ranges are across runs. Same-task overlap occurs only on L1. On the 382 held-out tasks, retrieved memories are procedural know-how from other tasks. Results on the 382 held-out tasks confirm all conclusions (accuracy %):

MethodGAIA L2+L3WebWalkerQAxBench-DSAvg.
No Memory51.869.469.063.4
ExpeL (best single provider by avg.)56.277.174.069.1
SkillWeaver59.870.077.068.9
Random Routing53.672.471.065.6
MemAgent w/o training59.870.670.066.8
MemAgent63.479.478.073.6

MemAgent raises the held-out average from 63.4% to 73.6% (+10.2 pp) and ranks first on every split. Table 1 already shows this on WebWalkerQA and xBench-DS, which are 100% held out and where MemAgent leads all methods. Within GAIA, MemAgent also gains more on held-out L2/L3 (+11.6 pp, 51.8% → 63.4%) than on L1 (+9.4 pp, 73.6% → 83.0%), showing that the gains come from cross-task transfer rather than L1 replay.


Q4. "Insufficient statistical validation and cost accounting. The main results are reported as single accuracy numbers, without multiple runs, confidence intervals, or significance tests. ..."

A4. The task-level gains across the suite are large and statistically significant:

Tasks solvedAll 435 tasksHeld-out 382 tasks
No Memory284245
Random Routing290254
MemAgent w/o training295257
MemAgent328284

MemAgent solves 44, 38, and 33 more tasks than these baselines on all 435 tasks (p = 0.001, 0.005, 0.013), and 39, 30, and 27 more on the 382 held-out tasks (p = 0.002, 0.017, 0.032). MemAgent also beats the hindsight-best single provider on each benchmark (AWM on GAIA, ExpeL on WebWalkerQA, SkillWeaver on xBench-DS), while every fixed provider trails MemAgent by at least 7.0 pp on some benchmark, a clear and consistent advantage. For single-run reporting, we follow MemEvolve [1] on the same three benchmarks, whose Table 3 likewise reports a single pass@1 accuracy per method. One pass over the 435 tasks already uses over 100M tokens and 10,000 paid search calls.


Q5. "Since all 13 long-term memory providers require LLM calls during storage, the authors should clarify whether the reported overhead of less than 0.3% excludes memory construction, trajectory summarization, provider updates, and training costs."

A5. The ~1.2 s per task (<0.3%) in Table 5 is router decision latency. To account for the full end-to-end cost per task, we tallied all LLM calls across the task lifecycle (task agent, retrieval, END trajectory summary, and storage in the 3–8 selected providers) on our GAIA logs:

Per GAIA taskTokensAPI calls
No Memory0.17M13.6
Random Routing0.27M28.8
PromptRoute0.28M27.8
MemAgent0.30–0.32M30

MemAgent updates only the 3–8 providers chosen at END rather than all 13. Storage runs after the final answer and is shared across later tasks, and router training is a one-time offline cost. The revised version retitles Table 5 "Routing overhead" and adds this table.


We hope these clarifications address the reviewer's concerns and would be grateful if the reviewer would reconsider the score.

[1] MemEvolve: Meta-Evolution of Agent Memory Systems. ICML 2026.
[2] Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning. ACL 2026.
[3] MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory. ArXiv 2026.

AuthorsOpenReview ↗

Polite Follow-up as the Discussion Period Concludes

Dear Reviewer DQ8M,

With the discussion deadline approaching, we wanted to politely check whether our response and revised manuscript have resolved your three core concerns: (1) clarifying our methodological contribution relative to single-store Memory-R1/MemRL and MemEvolve's LightWeight, (2) auditing the BEGIN memories and reporting strict held-out results excluding GAIA L1 across 382 disjoint tasks, and (3) providing full end-to-end storage and LLM cost accounting in Table 5b.

Please let us know if anything remains unclear before the discussion period ends. If our clarifications and revisions have addressed your concerns, we would deeply appreciate it if you could reconsider your rating. Thank you again for your time and helpful feedback.

Thank you for the detailed response. The results excluding GAIA L1 alleviate my concerns. I will keep my score unchanged for now and discuss the paper’s core contribution and applicability with the other reviewers in the next phase. I remain open to revising my score based on that discussion.

Reviewer_av1TOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper argues that no single agent memory design works best across benchmarks, and reframes agent memory as a routing problem over a pool of heterogeneous memory “providers”. The proposed MemAgent uses a small router (Qwen3.5-4B + LoRA) that makes three kinds of decisions. Before execution it probes all 13 long-term providers and picks one to retrieve from. During execution it decides at each step whether to inject a rule-based short-term memory. After the task it picks which subset of providers should store the trajectory. Since there are no ground-truth routing labels, the router is trained with SFT on decisions collected by forcing each task through every provider, then filtered by task correctness, LLM-judge scores, and a downstream-reuse reward for storage. The paper also contributes a graph-based provider (InsightGraph). On GAIA, WebWalkerQA, and xBench-DS with a GPT-5-mini task agent, MemAgent reports a 10.0-point average gain over no memory and beats every single provider, with small routing latency and fewer steps.

Critical strengths

Treating memory heterogeneity as something to route over, rather than designing yet another memory format, is a reasonable angle. The authors put real engineering effort into wrapping 13 methods under one interface, the appendix is detailed, and the code is released.

Critical weaknesses

The motivation is thin. Cross-task experience accumulation is already a crowded direction, and the paper never makes clear what the “experience” is or why it should transfer. On deep-search benchmarks, I’d expect most of what gets distilled from a single task to be task-specific: pages, entities, query paths. Only a little generic know-how transfers, and GPT-5-mini largely knows it already. Case 5 in the appendix even looks like entity-level caching rather than transfer. The 13 providers also feel padded. Many differ mainly in how they summarize or retrieve, several are re-implementations from unrelated settings, and there is no justification for why this set was chosen.

Experimentally, the task agent is only GPT-5-mini, and the only transfer test is GPT-5.6 Luna on GAIA, so generality to other models is unknown. The training has direct leakage: GAIA L1 tasks are used for training, for memory seeding, and as part of the test set. The data is also tiny, 173 tasks in total. How experience accumulates is unclear, and there is no evidence that performance actually improves as experience grows. The hindsight oracle is essentially pass@13 over stochastic runs, so it says little without seed variance, and no confidence intervals are reported at all. Many claims, “content-aware” routing in particular, have no direct supporting experiment.

Questions for authors
  • What do the retrieved memories actually contain? Generic strategies or task- and entity-specific content? How often does a helpful retrieval overlap in entities, URLs, or answers with the current task?

  • What are the results without GAIA L1 in training or memory? Does performance improve as experience accumulates? How does the method do with frozen memory (no online writes)?

  • Does the method transfer to other task agents, both open-source models and other closed models, across all three benchmarks, with multiple seeds and confidence intervals?

7 comments
AuthorsOpenReview ↗

Response to Reviewer av1T (1/2)

We thank the reviewer for the detailed review. We group related weaknesses and questions below.

Q1. "The motivation is thin. Cross-task experience accumulation is already a crowded direction, and the paper never makes clear what the "experience" is or why it should transfer. On deep-search benchmarks, I'd expect most of what gets distilled from a single task to be task-specific: pages, entities, query paths. Only a little generic know-how transfers, and GPT-5-mini largely knows it already."

"What do the retrieved memories actually contain? Generic strategies or task- and entity-specific content? How often does a helpful retrieval overlap in entities, URLs, or answers with the current task?"

A1. Prior work on agent memory shows that LLM agents, including frontier models, improve by reusing experience from other tasks: ExpeL [1] distills insights from past trajectories, Voyager [2] reuses a skill library on new tasks, Agent Workflow Memory [3] induces reusable workflows across websites, and Dynamic Cheatsheet [4] accumulates test-time strategies. In our paper, Table 6 specifies the memory paradigm, storage strategy, and retrieval mechanism of each of the 13 providers. Even when a base model knows a strategy in general, retrieved experience helps it apply the right workflow or check at the right step.

Table 1 tests this directly on GPT-5-mini: across the 382 held-out tasks, all 13 providers beat No Memory (63.4%), and MemAgent reaches 73.6% (+10.2 pp). To check what the retrieved memories actually contain on held-out tasks, we audited the BEGIN memories using an independent judge (Gemini-2.5-Flash): 85–100% are procedural know-how, 0–5.4% are task-specific, and at most 2.7% contain the current question. URL overlap appears mainly on WebWalkerQA (13–15%), where memories capture navigation tips from other questions on the same root site.

The many existing memory designs are our direct motivation. Evaluated under one interface (Sec. 3), the best provider changes with the benchmark (AWM on GAIA, ExpeL on WebWalkerQA, SkillWeaver on xBench-DS). The open question is which memory to read from and write to for each task, which is what MemAgent learns.


Q2. "Case 5 in the appendix even looks like entity-level caching rather than transfer."

A2. Case 5 (Appendix D.3) actually shows a failure case of another similarity-based baseline method, included to illustrate why entity-level matching misleads the agent and why our router avoids it in favor of workflow memory.


Q3. "The 13 providers also feel padded. Many differ mainly in how they summarize or retrieve, several are re-implementations from unrelated settings, and there is no justification for why this set was chosen."

A3. We use the full set of 13 memory systems released in MemEvolve's open-source EvolveLab codebase [5], which adapts all of them to the same deep-research agent and tools under one interface (Tables 6–7). How a memory summarizes and retrieves experience defines the design space of agent memory [5], and we treat each provider as a complete module behind retrieve and store calls. The 13 providers are complementary: the best provider differs by benchmark, the router uses all 13 (Appendix C.3.1), and restricting MemAgent to 5 providers drops average accuracy from 75.7% to 69.8%. With the same 13 providers, MemAgent (75.7%) far exceeds Random Routing (67.2%) and the untrained router (68.1%), showing that learned routing is what turns the pool into accuracy gains.


Q4. "Experimentally, the task agent is only GPT-5-mini, and the only transfer test is GPT-5.6 Luna on GAIA, so generality to other models is unknown."

"Does the method transfer to other task agents, both open-source models and other closed models, across all three benchmarks, with multiple seeds and confidence intervals?"

A4. Our main goal is routing heterogeneous memories for a given task agent, and the trained router also transfers zero-shot to other backbones. Table 4 shows zero-shot transfer to the closed GPT-5.6 Luna on GAIA (+3.6 pp over No Memory, beating the best single provider at every level). We also replaced GPT-5-mini with a locally served open-source Qwen3.5-35B-A3B for both the task agent and provider LLM calls, keeping the 4B router unchanged:

Qwen3.5-35B-A3B task agentGAIAWebWalkerQAxBench-DSAll 435 tasks
No Memory56.450.655.053.8
MemAgent58.264.166.062.3

MemAgent improves Qwen3.5-35B-A3B on all three benchmarks (+8.5 pp overall). A full grid of task agents, benchmarks, and seeds is too costly for deep-research agents, where one pass over the 435 tasks uses over 100M tokens and 10,000 paid search calls. Significance tests are in A8.


AuthorsOpenReview ↗

Response to Reviewer av1T (2/2)

Q5. "The training has direct leakage: GAIA L1 tasks are used for training, for memory seeding, and as part of the test set."

"What are the results without GAIA L1 in training or memory?"

A5. Appendix A.2 states this explicitly, following MemEvolve [5] in seeding memory with GAIA L1 and treating L1 as an experience-reuse split. Training only supervises provider choices, never task answers. For the other 382 tasks (GAIA L2/L3, WebWalkerQA, xBench-DS), stored L1 trajectories are experience from other tasks, just like TaskCraft. Results on these 382 held-out tasks confirm every conclusion (accuracy %):

MethodGAIA L2+L3WebWalkerQAxBench-DSAvg.
No Memory51.869.469.063.4
ExpeL (best single provider by avg.)56.277.174.069.1
Random Routing53.672.471.065.6
MemAgent w/o training59.870.670.066.8
MemAgent63.479.478.073.6

MemAgent raises the held-out average from 63.4% to 73.6% (+10.2 pp) and ranks first on every split. Table 1 already shows this on WebWalkerQA and xBench-DS, which are 100% held out and where MemAgent leads all methods. Within GAIA, MemAgent also gains more on held-out L2/L3 (+11.6 pp, 51.8% → 63.4%) than on L1 (+9.4 pp, 73.6% → 83.0%), showing that the gains come from cross-task transfer rather than L1 replay.


Q6. "The data is also tiny, 173 tasks in total."

A6. Training a router from 173 tasks shows high sample efficiency. From 173 training tasks drawn from the same task sources as MemEvolve [5] (GAIA Level 1 and TaskCraft), forced exploration across the 13 providers produces 8,735 decision-level training examples, which train a 4B router that generalizes to 382 held-out tasks across three benchmarks.


Q7. "How experience accumulates is unclear, and there is no evidence that performance actually improves as experience grows."

A7. Experience accumulates in two stages. Offline, the END router writes training trajectories into selected providers to seed the pool. Online, after each task, the END router writes the new trajectory into chosen providers for later retrieval. Prior work establishes that agent accuracy grows with accumulated experience: ExpeL improves as its experience pool grows [1], Voyager compounds skills over time [2], Dynamic Cheatsheet improves as test-time strategies accumulate [4], and ReasoningBank studies memory accumulation as a test-time scaling dimension [6]. In our setting, the accumulated memory lifts accuracy by +10.2 pp over No Memory on the 382 held-out tasks.


Q8. "The hindsight oracle is essentially pass@13 over stochastic runs, so it says little without seed variance, and no confidence intervals are reported at all."

A8. As in RouterBench [7], the oracle is a hindsight upper bound that shows the headroom across the 13 providers, and all claims in the paper rest on baseline comparisons. Across all 435 tasks, MemAgent solves 328 tasks, beating No Memory (284, p = 0.001), Random Routing (290, p = 0.005), and the untrained router (295, p = 0.013). On the 382 held-out tasks, MemAgent solves 284 tasks versus 245, 254, and 257 (p = 0.002, 0.017, 0.032).


Q9. "Many claims, "content-aware" routing in particular, have no direct supporting experiment."

A9. We ablate each component directly: router training (MemAgent w/o training), pool size (5 providers), IN gating and END selective storage (Table 3), and the reward signals (Table 2). Content-aware probing is needed because a provider's value depends on what it currently stores for the query, which static descriptions miss. Previewing the top retrieved items shows this content using only an embedding lookup, similar to inspecting retrieved passages in Self-RAG [8]. Table 2 and Appendix C.3.2 confirm that the router relies on the preview: keeping raw retrieval scores in the preview causes the router to pick two high-score providers 51.8% of the time and drops GAIA accuracy by 6.7 pp, whereas text-only previews give balanced routing.


We hope these clarifications and new results address the reviewer's concerns and would be grateful if the reviewer would reconsider the assessment.

[1] ExpeL: LLM Agents Are Experiential Learners. AAAI 2024.
[2] Voyager: An Open-Ended Embodied Agent with Large Language Models. TMLR 2024.
[3] Agent Workflow Memory. ICML 2025.
[4] Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory. EACL 2026.
[5] MemEvolve: Meta-Evolution of Agent Memory Systems. ICML 2026.
[6] ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. ICLR 2026.
[7] RouterBench: A Benchmark for Multi-LLM Routing System. ArXiv 2024.
[8] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR 2024.

Thanks for the detailed response. A few of my main concerns are still open:

The 382-task numbers are the held-out columns already in Table 1, and L1 is still in both training and the memory pool there. My question was what happens when L1 is removed entirely, e.g. training and seeding only from TaskCraft. I’m also not convinced by the L2/L3 vs. L1 gain argument. L1 starts at 73.6%, so there’s less room to grow, and the L2/L3 gains could still come from L1 content sitting in memory. Case 5 actually makes me more worried here: the stored Mercedes Sosa trajectory is a near-paraphrase of an eval question. Do you have a sense of how many eval tasks have a near-duplicate like that in the pool?

The cited works show accumulation helps in their settings, but I was asking about yours. A plot of accuracy over the task stream, and one run with online writes turned off, would answer this directly and shouldn’t cost much more than what you’ve already run. As far as I can tell, the frozen-memory question wasn’t addressed.

The “w/o probe score removal” ablation tests whether raw scores should be included in the preview, not whether probing helps at all. The Base row changes several things at once. What I’d like to see is routing from the task description plus static provider descriptions only, with everything else unchanged. Also, the numbers in that Table 2 row (63.0 / 77.4 / 64.0 / 30.8) are identical to the “IN-only, no long-term memory” row in Table 3. Could you double-check whether that’s a copy-paste issue?

In this stage, I would retain my assessment.

Reviewer_muLMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper divides memory management into BEGIN, which selects a long-term provider; IN, which decides whether to inject short-term state; and END, which selects providers for storage. It compares 13 long-term memory approaches through a common interface and constructs SFT data using forced exploration, answer correctness, LLM attribution scores, and downstream reuse rewards. The router is Qwen3.5-4B, and the task agent is Flash-Searcher/GPT-5-mini. Table 1 reports average accuracy across GAIA, WebWalkerQA, and xBench-DS increasing from 65.7 to 75.7: a gain of 10.0 percentage points, not a 10% relative improvement.

Critical strengths
  1. The study of complementary memory representations has practical value. The strongest individual provider varies across benchmarks in Table 1. Comparisons include Random Routing, an untrained router, PromptRoute, All-Concat, and All-Fuse, rather than only a no-memory baseline. These comparisons support selective management beyond simply adding more memory.
  2. The three-phase design includes substantial implementation detail and interventions. Tables 2–3 and Appendix B describe filtering, probing, short-term gating, and selective storage. Training overlap on GAIA does not explain all results: counts approximately reconstructed from Table 1 suggest held-out L2/L3 accuracy improves from 58/112 = 51.8% to 71/112 = 63.4%. These counts are inferred from rounded entries; original item-level results are needed before using them formally.
Critical weaknesses

W1: Evaluation mixes replay, generalization, and online feedback

  • Appendix A.2 (pp. 15–16) confirms that 53 GAIA L1 tasks both seed memory and appear in evaluation. Thus, the 165-task GAIA evaluation is not fully held out, despite the router's task-level split.
  • Router supervision on provider selection does not prevent the agent from retrieving same-task trajectories or answers. The concern is information exposure, not concealed leakage.
  • END uses yτy_\tau, and correctness judging relies on ground truth. It remains unclear whether gold feedback updates memory during evaluation, whether baselines receive equivalent feedback, and how this works in deployment.
  • Held-out L2/L3 and external gains remain valid evidence, but replay and generalization should be reported separately.

W2: Small gains and attribution/cost claims need stronger evidence

  • WebWalkerQA gains 2.3 points (~4/170 cases), while xBench-DS gains 1 point (~1/100). Paired significance tests or repeated runs are needed.
  • The 13-configuration hindsight oracle mixes provider differences with stochastic variation; matched-budget controls or qualification are needed.
  • LLM-based "causal attribution" and reuse success are proxy signals, not causal or counterfactual evidence.
  • GPT-5-mini serves as both actor and judge; independent calls do not eliminate shared bias.
  • Table 5's <0.3% covers router latency, excluding storage LLM calls and forced exploration, so it does not represent total overhead.
Questions for authors

Please refer to weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer muLM (1/2)

We thank the reviewer for the careful review.

Q1. "a gain of 10.0 percentage points, not a 10% relative improvement."

A1. Agreed. All gains in the paper are absolute percentage-point improvements, and the revised version writes them as "pp" throughout.


Q2. "Appendix A.2 (pp. 15–16) confirms that 53 GAIA L1 tasks both seed memory and appear in evaluation. Thus, the 165-task GAIA evaluation is not fully held out, despite the router's task-level split."

"Held-out L2/L3 and external gains remain valid evidence, but replay and generalization should be reported separately."

A2. Our evaluation design follows this exact split. Appendix A.2 states that "we therefore treat GAIA L1 as an experience-reuse setting rather than a strict held-out generalization test," and uses the other 382 tasks to test generalization. Reporting those 382 held-out tasks separately confirms every conclusion (accuracy %):

MethodGAIA L2+L3WebWalkerQAxBench-DSAvg.
No Memory51.869.469.063.4
ExpeL (best single provider by avg.)56.277.174.069.1
SkillWeaver59.870.077.068.9
Random Routing53.672.471.065.6
MemAgent w/o training59.870.670.066.8
PromptRoute59.871.276.069.0
All-Fuse54.575.374.067.9
MemAgent63.479.478.073.6

MemAgent raises the held-out average from 63.4% to 73.6% (+10.2 pp) and ranks first on every split. Table 1 already shows this directly on WebWalkerQA and xBench-DS, which are 100% held out and where MemAgent leads all methods. Within GAIA, MemAgent also gains more on held-out L2/L3 (+11.6 pp, 51.8% → 63.4%) than on L1 (+9.4 pp, 73.6% → 83.0%), showing that the gains come from cross-task transfer rather than L1 replay. The revised Table 1 reports the 382-task held-out average separately from L1.


Q3. "Router supervision on provider selection does not prevent the agent from retrieving same-task trajectories or answers. The concern is information exposure, not concealed leakage."

A3. Same-task exposure occurs only on L1, which is why we treat L1 as an experience-reuse split. To check what the router actually retrieves, we audited the BEGIN memory injected on every evaluated task using an independent judge from a different model family (Gemini-2.5-Flash):

GAIA L1 (seen)GAIA L2/L3WebWalkerQAxBench-DS
Memory contains the current question verbatim11–87%0–2.7%0%0%
Gold answer string appears in memory23–34%1.4–7.2%0–0.6%0–2.7%
Judge: procedural know-how64–79%85–100%98%98–100%

Mostly coincidental matches of common words. Ranges are across runs. On the 382 held-out tasks, retrieved memories are procedural know-how distilled from other tasks.


Q4. "END uses (y_τ), and correctness judging relies on ground truth. It remains unclear whether gold feedback updates memory during evaluation, whether baselines receive equivalent feedback, and how this works in deployment."

A4. Our evaluation protocol follows MemEvolve [1]:

  • Same feedback for all methods. After each task, every baseline and MemAgent receive the same outcome feedback and update memory by their own storage rules, as in ExpeL [2].
  • Post-task updates only. A task's outcome enters memory only after the agent submits its final answer, and retrieval never accesses the gold answer of the current task.
  • Deployment. Outcome feedback comes from the user or an automatic evaluator [3].

All methods therefore run under the same protocol, and the gains come from MemAgent's routing.


AuthorsOpenReview ↗

Response to Reviewer muLM (2/2)

Q5. "WebWalkerQA gains 2.3 points (4/170 cases), while xBench-DS gains 1 point (1/100). Paired significance tests or repeated runs are needed."

A5. Those 2.3 and 1.0 pp margins compare MemAgent against a different top provider on each benchmark (AWM on GAIA, ExpeL on WebWalkerQA, SkillWeaver on xBench-DS), picked from test scores in hindsight. A deployed system must pick its method beforehand, yet MemAgent still beats every hindsight winner. Against deployable baselines, MemAgent gains +10.0 and +9.0 pp over No Memory on WebWalkerQA and xBench-DS, beats the best overall provider (ExpeL) by +7.9, +2.3, and +4.0 pp across the three benchmarks, and beats every fixed provider by at least 7.0 pp on some benchmark, a clear and consistent advantage. Across the suite, the task-level gains over the main baselines are significant:

Tasks solvedAll 435 tasksHeld-out 382 tasks
No Memory284245
Random Routing290254
MemAgent w/o training295257
MemAgent328284

MemAgent solves 44, 38, and 33 more tasks than these baselines on all 435 tasks (p = 0.001, 0.005, 0.013), and 39, 30, and 27 more on the 382 held-out tasks (p = 0.002, 0.017, 0.032). For repeated runs, we follow MemEvolve [1] on the same three benchmarks, whose Table 3 likewise reports a single pass@1 accuracy per method. One pass over the 435 tasks already uses over 100M tokens and 10,000 paid search calls.


Q6. "The 13-configuration hindsight oracle mixes provider differences with stochastic variation; matched-budget controls or qualification are needed."

A6. The oracle counts a task as solved if any of the 13 providers succeeds. As in RouterBench [4], it is a hindsight upper bound that shows the headroom from provider diversity. The pool sets this ceiling, and the router determines how much of it we recover at test time. All claims in the paper rest on measured accuracy against baselines, and the revised version labels the oracle as a loose upper bound.


Q7. "LLM-based "causal attribution" and reuse success are proxy signals, not causal or counterfactual evidence."

A7. Benchmarks only label the final answer, so step-level credit assignment requires proxy signals, as in RLAIF [5], process reward models [6], and LLM-as-a-judge [7]. By "causal attribution" we mean per-decision credit assignment, which Table 2 validates by final routing accuracy. We anchor these signals in ground-truth outcomes: task correctness first filters BEGIN decisions (removing this filter drops accuracy by 7.3 pp, Table 2), and the judge only ranks BEGIN and IN decisions within the same task outcome. For END, a rule-based formula scores each stored item by how often later tasks retrieve it and succeed.


Q8. "GPT-5-mini serves as both actor and judge; independent calls do not eliminate shared bias."

A8. At evaluation, the judge receives only the final short answer and the gold answer, checking whether they match in entity, number, or short phrase. Every baseline and MemAgent also use the same GPT-5-mini task agent, so any grading preference applies equally across methods. On xBench-DS, the judge is Gemini-2.5-Flash from a different model family, and MemAgent's gain there (+9.0 pp) matches GAIA (+10.9 pp) and WebWalkerQA (+10.0 pp). We have clarified this in Appendix A.4.


Q9. "Table 5's <0.3% covers router latency, excluding storage LLM calls and forced exploration, so it does not represent total overhead."

A9. Table 5 reports router decision latency (<0.3% of task time). To account for the full end-to-end cost per task, we tallied all LLM calls across the task lifecycle (task agent, retrieval, END trajectory summary, and storage in the 3–8 selected providers) on our GAIA logs:

Per GAIA taskTokensAPI calls
No Memory0.17M13.6
Random Routing0.27M28.8
PromptRoute0.28M27.8
MemAgent0.30–0.32M30

The extra tokens over No Memory come mainly from updating the 3–8 selected providers at END, which runs after the final answer and is shared across later tasks. Forced exploration is a one-time offline training cost. The revised version retitles Table 5 "Routing overhead" and adds this table.


We hope these clarifications address the reviewer's concerns.

[1] MemEvolve: Meta-Evolution of Agent Memory Systems. ICML 2026.
[2] ExpeL: LLM Agents Are Experiential Learners. AAAI 2024.
[3] Autonomous Evaluation and Refinement of Digital Agents. COLM 2024.
[4] RouterBench: A Benchmark for Multi-LLM Routing System. ArXiv 2024.
[5] RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback. ICML 2024.
[6] Let's Verify Step by Step. ICLR 2024.
[7] Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023.

AuthorsOpenReview ↗

Follow-up on Our Response and Revision

Dear Reviewer muLM,

As the discussion period draws to a close, we wanted to gently check whether our response and revised PDF have addressed your two main concerns: (1) separating the 382 held-out tasks from GAIA L1 in Table 1, auditing the BEGIN memories for information exposure, and clarifying our equal post-task feedback protocol, and (2) reporting full end-to-end token and API cost accounting in Table 5b.

If any questions remain, we would be glad to clarify before the window closes. If these revisions and analyses have resolved your concerns, we would be sincerely grateful if you would consider raising your score. Thank you again for your time and thoughtful review.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

Dear Area Chair, Senior Area Chair, and Reviewers,

As the discussion concludes, we briefly summarize our paper's core contributions and the rebuttal outcome.

Prior work on agent memory searches for one universal memory format. Unifying 13 memory systems across three benchmarks shows instead that different tasks favor different paradigms, while naive pooling and indiscriminate writing degrade reasoning through retrieval pollution. MemAgent shifts the focus from designing another memory format to learning how to manage heterogeneous memories across the task lifecycle. A lightweight router previews retrieved text without scores to choose what to read (BEGIN), gates working memory during execution (IN), and selects where to write completed trajectories (END), the first method to learn cross-provider storage routing. Without routing labels, our pipeline trains this policy via forced exploration and phase-specific credit assignment, including a downstream reuse reward for delayed storage effects. Only this learned policy breaks past the single-memory ceiling, nearly doubling the gain of the best individual system while cutting task steps at negligible routing latency.

Throughout the rebuttal, our revised paper and new experiments resolved every empirical question raised across the reviews. We hope this summary helps you evaluate the paper's scientific insights, contributions, and verified evidence. Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.