- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper build on the premise that including memories in an agentic LLM can help improve the performance. These memories can be kept externally and retrieved when needed, or trained into the LLM. This paper proposes evaluating memories on tasks they were not written from, and using this transfer signal both to train the memory editor and select memories for consolidation.
Critical strengths
- The paper includes many ablations and diagnostic studies examining memory editing, selection, and consolidation.
- Evaluation with memory removed and the controlled comparisons provide useful evidence about the method.
Critical weaknesses
- I had a difficult time going through the paper, mostly due to the density of the information presented. Sections 2.1, 2.2, and 4.4 are particularly terse. It would help if the authors improved the flow and led with explanations before presenting numerical results.
- The comparability of the published results in Table 1 is unclear. Since the authors do not run many of the baselines in Table 1 themselves, it becomes important to clarify whether the experimental and evaluation setups used to obtain those numbers are comparable with this paper's. This is particularly important since the performance gap is relatively small in many cases and differences in training or evaluation setups (e.g., data splits) can have significant impact.
- The evidence for transfer seems to be primarily within task families and fixed environments. The broader claim about when memories should be internalized should be scoped accordingly.
Questions for authors
- Are the training data, evaluation tasks, and Search-QA retrieval setup the same as those underlying the copied results (e.g., SDAR UCOB)?
- Is it possible that a memory is selected for consolidation and later revised or deleted? How often does this happen, and how is guidance already internalized into the model handled if it is subsequently found to be incorrect?
- Could the authors clarify the construction of the cold-start SFT dataset and the exact training protocol used for MECA? Does the dataset include revision and deletion examples, or are these operations learned entirely through online RL?
6 comments
Rebuttal to Reviewer FzgE - Part 2
W2 and Q1. Comparability of the published results in Table 1
Thank you for raising this; the previous version did not state the shared setup clearly. Yes: the training data, evaluation tasks, and Search-QA retrieval setup are the same. MECA and our GRPO and GiGPO runs are built on verl-agent (github.com/langfengQ/verl-agent), the agentic RL framework released by the GiGPO authors (Feng et al., 2025), and use its data-split protocol. SDAR (Lu et al., 2026) builds on the same framework and adopts the same protocol, and the other published baselines follow SDAR's or GiGPO's protocol:
- ALFWorld: the GiGPO training split, with results on the seen evaluation split (
valid_seen). - WebShop: a set of 128 evaluation tasks.
- Search-QA: the Search-R1 protocol: training on NQ and HotpotQA (in-domain), with the other five datasets held out; E5 retriever; evaluation on the seven datasets.
Table F1. Setup of the published baselines in Table 1 (as stated in each paper, in the sections listed under "Stated in"; new Appendix Table 26)
| Method | Protocol | ALFWorld | WebShop evaluation | Search-QA | Stated in |
|---|---|---|---|---|---|
| Ours (MECA, GRPO, GiGPO) | GiGPO | GiGPO split; valid_seen | 128 tasks | Search-R1; NQ + HotpotQA; E5 | Sec. 4.1; App. C, E |
| SDAR (Lu et al., 2026) | GiGPO | GiGPO split; valid_seen | 128 tasks | Search-R1; NQ + HotpotQA; E5 | Sec. 3 |
| OPID (Yang et al., 2026a) | GiGPO | GiGPO split; valid_seen | 128 tasks | Search-R1; NQ + HotpotQA; E5 | Sec. 4.1; App. B.1 |
| SEED (Wu et al., 2026) | GiGPO | GiGPO split; valid_seen | 128 tasks | Search-R1; NQ + HotpotQA; E5 | Sec. 4.1; App. B |
| UCOB (Tu et al., 2026) | GiGPO, via SDAR | GiGPO split; valid_seen | 128 tasks | Search-R1; NQ + HotpotQA; E5 | Sec. 6.1 |
| GRSD (Zheng et al., 2026) | GiGPO, via SDAR | GiGPO split; valid_seen | 128 tasks | Search-R1; NQ + HotpotQA; E5 | Sec. 4.1; App. B, C |
| AgentOPSD (Wang et al., 2026) | GiGPO, via SDAR | GiGPO split; valid_seen | 128 tasks | Search-R1; NQ + HotpotQA; E5 | Sec. 3.1; App. C |
The benchmarks, data splits, evaluation tasks, and Search-QA retrieval setup are therefore shared; each paper keeps only its own method-specific hyperparameters. This is why the numbers in Table 1 are comparable, and why we report the published results rather than re-running each method. We now state this in Section 4.1:
All published baselines in Table 1 use the data splits and evaluation protocol of verl-agent (Feng et al., 2025), on which our pipeline is also built: the GiGPO ALFWorld split, the 128 WebShop evaluation tasks, and the Search-R1 setup for Search-QA (Appendix Table 26).
Our main conclusions still rest on Table 2, where every method, including GRPO and GiGPO, is also trained with the same base models, prompts, context limits, and training schedule as MECA. Table 1 places MECA among published methods; as Section 4.2 notes, MECA is not uniformly best there.
W3. Transfer beyond the training distribution
Thank you for this comment. Our evaluation already measures transfer to data never seen in training, and we have added a new test on unseen ALFWorld environments.
Search-QA already includes held-out datasets. Following the Search-R1 protocol, Search-QA trains only on NQ and HotpotQA, so five of the seven evaluation datasets (TriviaQA, PopQA, 2Wiki, MuSiQue, and Bamboogle) are held out: their questions, sources, and reasoning patterns are never seen in training. Splitting the Search-QA results of Table 1 accordingly:
Table F2. Search-QA exact match on training and held-out datasets (%, size-weighted; computed from Table 1, MECA with memory; new Appendix Table 27)
| Backbone | In-domain: GiGPO | In-domain: MECA | Gain | Held-out: GiGPO | Held-out: MECA | Gain |
|---|---|---|---|---|---|---|
| Qwen2.5-7B | 42.2 | 45.4 | +3.1 | 48.5 | 52.1 | +3.6 |
| Qwen2.5-3B | 39.1 | 43.6 | +4.5 | 45.0 | 48.7 | +3.8 |
| Qwen3-1.7B | 38.4 | 42.5 | +4.1 | 42.8 | 45.4 | +2.6 |
| Average | +3.9 | +3.3 |
MECA's gain over GiGPO on the held-out datasets (+3.3 points on average) is close to its gain on the training datasets (+3.9). The memories written and consolidated from NQ and HotpotQA questions therefore carry over to question sets never seen in training.
Rebuttal to Reviewer FzgE - Part 3
Results on unseen ALFWorld environments [NEW]. ALFWorld provides an unseen split (valid_unseen, 134 tasks) whose environments (rooms, with their layouts and objects) never appear in training. We evaluated the final checkpoints of Table 2 on all 134 tasks (3 seeds, same evaluation protocol, memory frozen), with and without memory:
Table F3 [NEW]. ALFWorld success in seen and unseen environments (%, mean over 3 seeds; seen: Table 2 and Appendix Table 4; new Appendix Table 28)
| Method | 7B seen | 7B unseen | 3B seen | 3B unseen | 1.7B seen | 1.7B unseen |
|---|---|---|---|---|---|---|
| GiGPO | 87.5 | 82.0 | 85.9 | 80.5 | 72.7 | 66.9 |
| GiGPO, 225 iterations | 91.9 | 86.7 | 87.2 | 81.5 | 81.0 | 76.0 |
| GiGPO + unfiltered memory (memory removed) | 88.8 | 82.3 | 83.9 | 78.1 | 78.6 | 72.9 |
| GiGPO + unfiltered memory | 93.5 | 89.1 | 89.1 | 83.9 | 84.9 | 79.9 |
| MECA (memory removed) | 95.8 | 89.8 | 92.7 | 87.0 | 89.8 | 83.9 |
| MECA | 97.7 | 92.9 | 94.5 | 88.8 | 91.4 | 85.2 |
MECA leads every method in unseen environments with every backbone, both with and without memory. Moving to unseen environments lowers success by a similar amount for all methods (5.3–6.0 points on average), so MECA's advantage is largely preserved:
- Without memory, MECA leads GiGPO by 10.4 points on average (seen: 10.7) and the stronger trajectory-matched control by 5.5 points (seen: 6.1).
- With memory, MECA leads GiGPO + unfiltered memory by 4.7 points (seen: 5.4).
- Reliance on memory stays low. Reading memory adds 2.1 points for MECA in unseen environments (seen: 1.8), compared with 6.5 for GiGPO + unfiltered memory (seen: 5.4). MECA's capability in new environments therefore resides mainly in the policy rather than in the memory it reads.
We added both results as Appendix Tables 27 and 28 and cite them in Section 4.4.
MECA's gains are not limited to the tasks it was trained on: they hold on held-out question sets in Search-QA and in unseen ALFWorld environments, and they are carried mainly by the policy rather than by the memory. Extending this evaluation to transfer across different types of tasks is a natural next step. We hope these results address the reviewer's concern about the evidence for transfer.
Q2. Can a consolidated memory later be revised or deleted?
Memories selected for consolidation (review memories) stay in a training-only pool, and during training one of them is retrieved into a task group's context with probability 0.125 (Appendix C, Retrieval). When this happens for a source task, the review memory appears in that task's dossier, and the editor can revise or delete it; such edits are scored by transfer and routed like any other edit. Deletions are never consolidated (Eq. 7), and when a deletion and a review-routed revision of the same memory arrive in the same iteration, the revision is kept (Appendix D, Conflicts between accepted edits).
Frequency. In our training runs, 7.2% of review memories are later revised and 0.8% are deleted, a median of 35 iterations after consolidation.
How incorrect guidance is handled. MECA addresses this mainly by making incorrect consolidation rare, and secondarily by correcting it through continued training.
- Consolidation is reserved for reliably transferable edits. An edit is consolidated only if the lower bound of its transfer exceeds the threshold () and it passes the worst-query veto (); in our runs, 9% of proposed edits meet both conditions, whereas 33% are kept in external memory and 58% are discarded.
- Consolidated guidance is rarely incorrect when re-tested. Appendix G re-scores edits on a fresh panel of query tasks drawn after routing, with new controls (Appendix Table 7). Of the 24 review-routed edits, 23 (96%) still improve success, with a mean transfer of 22.8 points, compared with 9.5 for edits kept in external memory; on an independent probe panel, their read value is 26.3 points. The online estimates used for routing are optimistic (36.7 points), but the selected edits remain strongly positive.
Rebuttal to Reviewer FzgE - Part 1
We thank the reviewer for the careful reading and for recognizing our ablations, diagnostic studies, memory-removed evaluation, and controlled comparisons. We address each weakness and question below and revised the paper accordingly. All changes are highlighted in color in the revised PDF.
Summary of revisions for Reviewer FzgE
W1. Density of Sections 2.1, 2.2, and 4.4
We are sorry that the previous version of these sections did not convey our ideas clearly, and we rewrote them to make the ideas clearer. Below we restate the main insight of each section.
Section 2.1: a memory that helps its own episode need not help other tasks. The most direct way to reward a memory edit is by how much it helps the episode it was written from. However, many such edits record details of that episode, such as where an object happened to be, that do not recur on other tasks:
The two are only weakly related. What matters for a reward, however, is not predicting transfer exactly but ranking edits well, and scoring an edit on other tasks of the same family selects far more transferable edits than scoring it on its own episode:
This is why MECA rewards each edit by its transfer to tasks it was not written from.
Section 2.2: a memory that helps when read need not help once learned. Reading a memory and learning from it are different: a memory can raise success while the agent reads it, yet fail to improve the policy once it is distilled into the parameters (its learning value). Transfer and learning value are only moderately correlated over all edits, but the most transferable edits are clearly enriched for learning value, so transfer is a good criterion for deciding which memories to consolidate, and it comes for free with the editing reward.
Section 4.4: how memory becomes capability in MECA. Training with the transfer reward changes MECA in three connected ways. The agent learns to write memories that transfer better, shifting from episode-specific notes to general procedures; the memories selected by transfer improve the policy once consolidated, even when the memory is removed; and, as a result, the agent depends less and less on reading its memory, whereas a memory baseline without this selection depends on it more.