- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes GenMEM, a generative symbolic memory framework for self-evolving LLM agents. GenMem introduces a generative symbolic memory framework for self-evolving LLM agents to address the challenges of experience sparsity and content-dependent retrieval instability caused by memory updates. Its core innovation is the introduction of hierarchical Symbolic Identifiers (SIDs) constructed via residual quantization to decouple memory addressing from content. The system consists of a MemRetriever, a StackPlanner, and a MemEvolver, trained through SID alignment, supervised mid-training, and GRPO-based policy optimization. Experiments demonstrate that the method achieves good performance across interactive decision-making and QA tasks.
Critical strengths
1: The problem the paper is targeting is important. The paper correctly identifies a practical tension in self-evolving memory systems: retrieved memories can change over time, while the agent must continue accessing them reliably.
2: The experiment results is strong, achieving up to ten points improvement on WebShop Success rate.
Critical weaknesses
1: The paper is hard to read with a lot of theoretical frame. However, after reading the paper, I found the underlying intuition and implementation appear relatively straightforward. For example, the paper claims SID encodes mutually orthogonal categorical axes, however, in equation 3, it is basically a residual quantization.
2: My biggest concern after reading the paper is the over-claim problem. The central contribution of GenMem is presented as generative symbolic addressing for stable memory retrieval and evolution. The paper argues that conventional embedding-based retrieval suffers from address drift when memory content changes. While this is a reasonable motivation, address invariance is not unique to generative retrieval. For example, a memory system could maintain stable identifiers and fixed retrieval representations while allowing memory payloads to evolve. I'd like to see a direct comparison with such a baseline.
3: The data separation remains a concern. Table 2 labels Bamboogle, MuSiQue, NQ, and TriviaQA as out-of-domain benchmarks. However, Table 6 lists many of these rources in the initial memory bank. It also includes 1,664 GPQA-derived entries, while Section 5 evaluates symbolic reasoning on GPQA. I would appreciate more details on data preparation, split definitions, and overlap verification.
4: Unclear training objectives and evaluation configurations. Section 3 introduces separate advantage normalization for proxy and audit channels, but does not clearly define their respective data, rewards, sampling procedures, or combined update.
Questions for authors
1: Are the initial GenMem retrieval results in Table 4 obtained from the same evaluation as the PT-4L results in Table 18? Why does Table 18 show such a large degradation after mid-training, and what are the corresponding post-GRPO retrieval metrics? Please explain why Table 4's initial GenMem results exactly match PT-4L.
2: What are the exact data splits and overlap checks? How are the out-of-domain QA and GPQA evaluations separated from the corresponding source experiences used for memory construction and alignment?
3: Which training configurations and reward specifications were used to produce the reported results? In particular, could the authors clarify the differences between the main-text and appendix descriptions of the GRPO training settings and reward decomposition?
8 comments
Rebuttal for W2
W2. Direct comparison with a Frozen-Key + Evolving-Payload dense baseline
Following your suggestion, we implemented this exact Frozen-Key Embedding + Evolving-Payload baseline and compared it directly against GenMem.
(1) Controlled experimental setup.
Both methods start from the exact same initial memory bank (30,577 entries), planner, updater (MemEvolver), and inference budget. Each entry in the baseline retains its initial embedding as a frozen retrieval key while its textual payload is updated in place during online evolution; at test time, queries match against the frozen keys (cosine Top-5) to fetch the latest payloads. We drive online evolution with a 1,500-query stream (NQ/Bamboogle) and evaluate before and after evolution on a disjoint 200-question probe set (HotpotQA/2WikiMultiHopQA). We also measure Content Relevance Hit Rate (scored 0–3 by a Qwen2.5-7B judge, with score counted as relevant):
| Method (Identical Initial Bank, Updater & Probe Set) | Initial Task Acc (%) | Final Task Acc (%) | Acc (pp) | Content Relevance Hit Rate (Before After) |
|---|---|---|---|---|
| Frozen-Key Embedding + Evolving Payload | 47.0 | 42.0 | -5.0 | 59.0% 52.0% (-7.0 pp) |
| GenMem (Learned SID Read/Write Routing) | 58.0 | 61.0 | +3.0 | 59.0% 67.0% (+8.0 pp) |
(2) Why freezing dense keys alone is insufficient. As payloads are revised over 1,500 evolution steps, the Frozen-Key baseline's content-relevance hit rate drops from 59.0% to 52.0% (-7.0 pp) and downstream accuracy drops from 47.0% to 42.0% (-5.0 pp), whereas GenMem improves from 58.0% to 61.0% (+3.0 pp) (consistent with Table 23, where enabling evolution in GenMem raises accuracy from 46.5% to 49.5% and F1 from 46.8% to 52.1%).
This contrast highlights two distinct failure modes in dense retrieval under evolution:
- Re-indexing keys (Table 4 baseline) causes address drift, shifting which queries reach an entry;
- Freezing initial instance keys (new baseline above) causes key–payload semantic decoupling: an initial embedding encodes surface-level wording of the seed instance rather than an abstract functional role, so uncoordinated updates cause the evolving payload to diverge from what the static vector key represents.
We center GenMem's contribution on learning coordinated read and write routing (MemRetriever and MemEvolver) over a shared, factorized symbolic address space, where each SID represents a coarse-to-fine strategy cluster and both policies are jointly trained via GRPO to read, rewrite, and revise slots coherently.
Rebuttal for W3&Q2
W3 & Q2. Data splits, ID/OOD definitions, and instance-level overlap audit
(1) Meaning of the ID/OOD labels in Table 2.
Following the Search-R1 evaluation convention, the In-Domain (ID) and Out-of-Domain (OOD) labels in Table 2 refer strictly to the training data of the task-execution model (StackPlanner): only 2WikiMultiHopQA and HotpotQA were used to train the task-execution model, whereas Bamboogle, MuSiQue, NQ, and TriviaQA were held out from task-execution model training. However, source questions from these datasets contributed distilled experiences to the offline seed memory bank (Table 6; 4 rollout trajectories per source question). To avoid any ambiguity, we will relabel these columns in Table 2 as "Held-Out from Task-Model Training" and explicitly state their role in offline memory construction. (Note: True zero-shot domain transfer to benchmarks completely excluded from both memory construction and model training—BrowseComp-Plus, DeepResearch Bench, BIRD, and SWE-bench Verified—is evaluated separately in Table 3.)
(2) Exact split definitions and instance-level overlap verification. We performed an instance-level audit across all memory-construction sources (Table 6) and evaluation sets (Table 2 and Section 5). Below is the complete breakdown of source splits, distinct source questions, derived experience entries (4 rollouts per question), evaluation splits, and training-overlap counts:
| Dataset | Memory-Bank Source Split | Source Questions (Distinct IDs) | Retained Experience Entries (Table 6) | Evaluation Split / Selection | Evaluation Questions | Task-Model Training Overlap |
|---|---|---|---|---|---|---|
| MuSiQue | train | 2,328 | 9,312 | Answerable dev | 2,417 | 0 |
| Natural Questions (NQ) | train | 2,020 | 8,080 | validation / dev | 3,610 | 0 |
| TriviaQA | train | 2,473 | 9,892 | Wiki & Web validation | 4,000 | 0 |
| Bamboogle | test (125 Qs) | 125 | 500 | test | 125 | 0 |
| GPQA (Sec. 5) | Seed subset | 416 | 1,664 | Main evaluation / Perturbation diagnostic | 448 | 0 |
For MuSiQue, NQ, and TriviaQA, memory construction strictly uses the train split while evaluation uses the disjoint dev/validation splits (zero instance overlap). For Bamboogle (which only has a 125-question release) and the GPQA diagnostic in Section 5, we will explicitly document the in-bank seed construction protocol and split records in Appendix E.1 so that in-bank conditioning diagnostics are clearly distinguished from held-out generalization benchmarks.
Rebuttal for W1
Thank you for the constructive feedback. Below, we address your concerns by clarifying our terminology and mathematical framing, providing the requested fixed-key baseline, and detailing the exact data splits, retrieval metrics, and GRPO training specifications.
We fully agree with your critique. Equation (3) implements standard multi-level residual quantization (RQ-KMeans), where each level l quantizes the residual ril=ril−1−csil−1l−1 left by the preceding levels. While the (48,16,8,8) codebooks factorize 49,152 possible addresses using 80 level-specific symbols, Equation (3) produces a coarse-to-fine residual hierarchy, not "mutually orthogonal categorical axes."
Our intended point is simply that successive RQ levels capture coarse-to-fine task structure and play distinct functional roles in conditioning the model, which is supported by two diagnostics:
Modifying fine-grained residuals (C/D) barely affects accuracy (-1.0 pp), whereas modifying coarse prefixes (A/B) drops accuracy by 5.0 pp, confirming a coarse-to-fine conditioning hierarchy.
Manuscript revision: We will streamline the theoretical framing in Sections 1–2 and Appendix B, remove the "mutually orthogonal categorical axes" phrasing, and consistently describe SIDs as coarse-to-fine hierarchical residual codes.