acceptodds
Under review as a conference paper at ICLR 2027

Evolving Semantic and Procedural Memory from Shared Experience for LLM Agents

Abstract

LLM agents need persistent memory that keeps factual knowledge current and develops reusable procedures from experience. Factual revision and procedural learning require distinct evidence. Coordinating these updates requires a shared context that connects factual knowledge with the experience underlying reusable procedures. We propose Self-Evolving Experience Memory (SEEM), an architecture that evolves semantic and procedural memory from shared experience scenes that record context, goals, actions, and outcomes while preserving links to source evidence. Semantic updates revise scoped factual states using new observations while retaining history and unresolved conflicts. Procedural updates learn and refine reusable steps from successful traces, while failures inform applicability conditions and reuse eligibility. Retrieved memories and their supporting evidence guide subsequent tasks, whose observations and outcomes drive further updates with model parameters fixed. Compared with the strongest evaluated baselines, SEEM improves accuracy from 67.67% to 79.33% on the MemoryAgentBench variant of LongMemEval and from 75.48% to 92.90% on BFCL V4 Memory. On the -bench family, overall task success increases from 90.65% to 93.17% with DeepSeek-V4-Flash and from 74.46% to 80.22% with Qwen3.8-27B. In a matched Qwen3.8-27B comparison, continued memory updates achieve 81.53% success versus 74.32% with frozen memory, supporting the benefit of ongoing memory evolution.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Agent Memory, Long-Term Memory, Heterogeneous Memory, Memory Evolution, Continual Learning, Self-Evolving Agents, Skill Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_Tw9oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper focuses on factual updates and procedural evolution in persistent memory for LLM agents. The authors formalize a shared experience representation and propose distinct update mechanisms for semantic and procedural memory. Semantic updates revise facts based on entity identity, scope, and observation time while retaining history and unresolved conflicts. Procedural updates derive reusable steps from successful trajectories and use failures to refine applicability conditions and reuse eligibility. Both memory types remain linked to shared experience scenes and source evidence.

Critical strengths
  • Evidence-aware memory management abstraction is clear: preserves provenance, history, conflicts, and skill applicability.
  • Ablation studies and controlled comparisons of this paper is solid.
Critical weaknesses
  • The gain attribution remains unclear: extra calculation, execution repair, and BFCL packing/deletion safeguards may contribute. Reported configurations, baseline parity, and τ-bench feedback permissions remain unspecified.
  • Abstract and introduction list mechanisms without establishing why existing methods fail or shared experience is necessary. No matched unified-versus-distinct update comparison tests the central claim.
  • Only two model is used in evaluation for τ\tau-bench, more model should be included to prove the method can be generalied.
Questions for authors
  • Which τ\tau-bench feedback fields enter memory updates? Are reference actions and hidden evaluator information excluded, with identical feedback access across methods?
  • With calculation, execution checks, and repairs matched across methods, how much improvement remains?
  • Do BFCL baselines receive identical packing, deletion safeguards, and retrieval budgets? What gains remain after matching them?
  • With representation and retrieval fixed, do distinct update criteria outperform a unified outcome-based rule? Also, only use dsv4 as judge is too limited.
2 comments

Thank you for the comments on gain attribution, evaluation conditions, and generalization. We will strengthen the controlled comparisons and clarify the evaluation protocol.

Distinct update criteria and shared experience.

Factual revision and procedural learning require different evidence from the same interaction. An unsuccessful task can establish a factual change, while reusing a procedure requires evidence that its steps were effective under compatible conditions. Shared scenes connect the context and execution evidence supporting these decisions.

With Qwen3.8-27B, full SEEM² achieves 80.22% task success, compared with 74.82% for separate memory organization and 76.98% without updates from failed tasks. We will add a comparison with a unified task-success gate for positive semantic and procedural updates, holding initial memory, representation, retrieval, backbone, and task order fixed. We will report overall and per-domain success.

Auxiliary calculation and continuous updates.

A supplementary fixed-memory evaluation without auxiliary calculation achieves 75.00% on LongMemEval S∗S^{\ast} with Gemma4 and 78.33% with DeepSeek-V4-Flash, using one answer-generation call per question.

The paper’s Qwen3.8-27B Full/Frozen comparison starts from the same initial memory and uses matched task order, backbone, retrieval, and task-execution settings. Full achieves 81.53% success over 222 tasks versus 74.32% for Frozen. Memory updates occur after task scoring, using completed visible trajectories and outcome feedback. We will list the update-feedback fields and document the access available to each baseline in the appendix.

BFCL protocol and token cost.

We follow the official BFCL V4 memory_vector protocol: 37 prerequisite conversations followed by 155 evaluation questions, using the official memory-tool interface, capacity limits, and answer-matching checker.

Our completed cost audit gives:

\text{Method} & \text{Accuracy} & \text{Total LLM tokens} \\\\ \hline \mathrm{SEEM}^{2} & \mathbf{92.90\\%} & 10.22\mathrm{M} \\\\ \text{Mem0} & 75.48\\% & 12.84\mathrm{M} \\\\ \text{Graphiti} & 74.84\\% & 16.01\mathrm{M} \\\\ \text{LightMem} & 74.19\\% & 9.37\mathrm{M} \\\\ \text{Vector Store} & 56.77\\% & 6.53\mathrm{M} \end{array}$$ These totals cover construction and question answering, excluding embeddings. SEEM² achieves higher accuracy with fewer total tokens than Mem0 and Graphiti; LightMem and Vector Store use fewer tokens. To assess the contribution of packing, deletion protection, and retrieval budgets, we will add a comparison between SEEM² and Mem0 with these settings matched and report accuracy and token cost. **Evaluation with another judge.** We have reevaluated the main paper’s saved SEEM² predictions and all eight earlier baseline prediction sets with Gemma4, keeping predictions fixed. The table also includes the newly evaluated original SEEM baseline. $$\begin{array}{lr} \text{Method} & \text{Gemma4 accuracy} \\\\ \hline \mathrm{SEEM}^{2} & \mathbf{75.33\\%} \\\\ \text{Long-context FIFO} & 66.00\\% \\\\ \text{Original SEEM} & 65.67\\% \\\\ \text{Dense RAG, Qwen3-Embedding-8B} & 65.33\\% \\\\ \text{Graphiti} & 64.67\\% \\\\ \text{Dense RAG, KaLM} & 63.00\\% \\\\ \text{Full Context} & 61.00\\% \\\\ \text{LightMem} & 59.67\\% \\\\ \text{A-MEM} & 52.00\\% \\\\ \text{Mem0} & 51.33\\% \end{array}$$ Each row covers all 300 questions. SEEM² remains the highest-scoring method under Gemma4, and the two judges agree on 286 of its 300 predictions. We will include this evaluation in the revision. **Generalization across agent backbones.** SEEM² achieves 93.17% task success with DeepSeek-V4-Flash versus 90.65% for the strongest evaluated baseline, and 80.22% with Qwen3.8-27B versus 74.46%. We will extend the $\tau$-bench evaluation to an additional agent backbone.

Thank you for the detailed response and additional experiments. However, my main concerns remain only partially resolved.

  1. the proposed comparison against a unified success-gated update rule is particularly important, as it directly tests whether the distinct semantic and procedural update criteria are necessary. The existing comparisons do not fully isolate this contribution.

  2. the BFCL comparison with matched packing, deletion protection, and retrieval budgets is still pending. Similarly, the Full vs. Frozen experiment demonstrates the benefit of continued memory updates but does not fully isolate the contribution of auxiliary computation, execution checking, and repair.

  3. I would appreciate explicit clarification of which τ-bench feedback fields are available for memory updates, whether any hidden evaluator information is excluded, and whether all baselines have equivalent feedback access.

The additional judge evaluation is reassuring, although it does not directly address my suggestion of a dsv4-only evaluation. The generalization evidence also remains limited to two agent backbones.

I appreciate the authors' efforts, but I would like to see the promised controlled comparisons before reconsidering my assessment.

Reviewer_8AMjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Self-Evolving Experience Memory (SEEM²), a framework that integrates semantic and procedural memory through shared experience scenes. It employs separate update mechanisms for factual knowledge and reusable skills while retaining supporting evidence. Experimental results on LongMemEval, BFCL V4 Memory, and τ-bench show that SEEM² consistently outperforms existing memory baselines.

Critical strengths

Strong empirical performance. SEEM² achieves substantial improvements across three benchmarks covering both memory retrieval and tool-use tasks.

Useful mechanism analysis. The component ablations and frozen-memory comparison provide evidence for the benefits of shared experience organization and continuous memory updates.

Critical weaknesses
  • The framework extends SEEM's experience representation by integrating existing techniques for semantic memory updates and procedural skill learning. Its novelty over related methods such as MSCE and FluxMem remains unclear.

  • The frozen-memory comparison is limited to a single task split and evaluation run. Further experiments across different task orders and noisy feedback settings are needed to validate continuous evolution.

  • Despite requiring fewer LLM calls, SEEM² consumes more tokens than Mem0 and LightMem in Table E.1. The lack of analysis on memory growth and maintenance costs also raises efficiency concerns.

Questions for authors

Could the authors compare SEEM² directly with the original SEEM to better demonstrate the contributions of the proposed memory evolution mechanisms?

How does continuous memory evolution perform under varying task orders and noisy or inaccurate feedback?

1 comment

Thank you for the comments on novelty, continuous evolution, and efficiency. We have completed an original-SEEM comparison and will expand the robustness and cost analysis.

Contribution beyond SEEM and related methods.

SEEM² extends SEEM’s experience representation with distinct mechanisms for maintaining factual states and reusable procedures. Shared scenes connect observations, actions, and outcomes, while each memory type uses evidence appropriate to its update decisions. We will make the inherited representation explicit and compare SEEM, FluxMem, and MSCE in terms of update targets, supporting evidence, and revision decisions.

We have evaluated original SEEM on all 300 LongMemEval S∗S^{\ast} questions. Without auxiliary calculation, it achieves 71.00% with the DeepSeek-V4-Flash judge and 65.67% with Gemma4. SEEM² achieves 78.33% and 75.00%, respectively, in the supplementary fixed-memory reading evaluation, also without auxiliary calculation. We will add this baseline comparison to the revision.

The paper’s separate-memory ablation retains both memory types and their update mechanisms while separating their experience organization. On Qwen3.8-27B τ\tau-bench, it achieves 74.82% versus 80.22% for full SEEM². We will connect this evidence more directly to the architectural contribution.

Continuous evolution and robustness.

The paper’s Qwen3.8-27B Full/Frozen experiment evaluates 222 tasks from the same initial memory, with matched task order, backbone, retrieval, and task-execution settings. Full succeeds on 181 tasks, or 81.53%, versus 165 tasks, or 74.32%, for Frozen, with improvements in all three domains.

We will extend this comparison with independent repetitions and multiple task orders. We will also add controlled noise to post-task outcome feedback to examine its effect on subsequent memory updates and task success.

Token cost and memory growth.

Table E.1 reports the following LongMemEval S∗S^{\ast} costs:

\text{Method} & \text{Total LLM calls} & \text{Total LLM tokens} \\\\ \hline \mathrm{SEEM}^{2} & \mathbf{1{,}086} & 7.13\mathrm{M} \\\\ \text{A-MEM} & 1{,}780 & 23.29\mathrm{M} \\\\ \text{Mem0} & 1{,}490 & 6.19\mathrm{M} \\\\ \text{LightMem} & 4{,}497 & 5.43\mathrm{M} \end{array}$$ SEEM² uses fewer calls, while consuming more tokens than Mem0 and LightMem. We will state this trade-off explicitly and explain the token cost of maintaining factual context, experience structure, and supporting sources. We also measured the final memory size across the five LongMemEval $S^{\ast}$ histories. Each memory contains 1,135–1,437 factual records and approximately 10MB of graph data, excluding embeddings. We will complement these measurements with analyses of memory growth and maintenance costs over the interaction stream.
Reviewer_o1X7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors tackle the challenging problem of long-term memory management for agents. The authors focus on overcoming the challenges of keeping factual and procedural knowledge current. To do so, they propose a new memory approach, SEEM2, that intelligently evolves semantic and procedural memory from shared experience scenes and source records. The key idea behind SEEM2 is that semantic and procedural updates require different types of evidence and checks. Semantic updates rely on updating entities based on observations and support from multiple events. Procedural updates rely on successful traces, recovery steps, and failed traces for updates. Putting SEEM2 to the test demonstrated promising results with agent performance improvements across several key memory benchmarks.

Critical strengths

-The paper is well organized and generally easy to follow.

-The results of agents equipped via SEEM2 are quite strong. In particular, on LongMemEval SEEM2 shows consistently superior performance. Same for BFCL memory and tau-bench.

-The ablations help to confirm that both the semantic state and memory management and the procedural update management are needed for achieving the superior results of SEEM2.

Critical weaknesses

-Figure 1 could be clearer and more simplified. The Case study portion is excellent. The top portion explaining SEEM2 is hard to follow as is.

Questions for authors

N/A

1 comment

Thank you for the positive assessment of the empirical results and ablations, and for the suggestion to simplify Figure 1.

We will revise the upper portion of Figure 1 around three stages: forming experience scenes from interactions, evolving semantic and procedural memory from those scenes, and retrieving memories with their supporting evidence. The two evolution paths will use distinct visual labels, and the connections to shared scenes and sources will be shown directly.

We will retain the coordinated order-change case study and align its terminology with the method section. The caption will explain how an observed change updates semantic memory and how execution evidence affects procedural reuse. This should make the overview easier to follow while preserving the concrete example that the reviewer found helpful.

Reviewer_rghgOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces SEEM$^2$, a textual memory architecture for LLM agents that maintains two memory types: semantic memory (facts, states, and inferred user profiles) and procedural memory (skills). Both are derived from experience scenes that record the context, goals, actions, and outcomes of an interaction, and link them to source evidence. Facts are updated by newer observations, while skills are refined from successful executions and restricted by failures. SEEM$^2$ outperforms memory baselines on LongMemEval, BFCL V4 Memory, and τ\tau-bench.

Critical strengths

The paper's premise is well-motivated: a memory system that records and updates factual and procedural knowledge, and tracks which interactions they were learned from, addresses a real need for LLM agents.

Critical weaknesses
  • The paper's closest prior works (FluxMem, MSCE, SEEM) are listed in the related work section, but its novelty over them is insufficiently clarified. SEEM$^2$ is not compared to these methods in the results. On τ\tau-bench, removing semantic memory drops performance to AWM's level. This could suggest that performance gains come from adding a semantic memory that other methods lack, instead of a better procedural memory
  • The method is presented as a long list of rules without motivating them or indicating which are core novel contributions and which are implementation details. The ablations only remove whole components, so the importance of individual rules remains unclear
  • The reliability of the results is hard to assess. All results come from a single run, and no variance or error values are reported. LongMemEval and BFCL are evaluated with a single model. Key experimental details such as reasoning effort or which LLMs are used in the baselines are missing. The paper does not analyze how its memory behaves, for example how often facts are replaced or where it still fails.
  • The writing is dense and abstract, which makes Section 3 especially difficult to follow. Many concepts go by several names (e.g. experience, evidence, traces, scenes, source records) instead of using a single term. Section 3 formalizes simple operations but the components that do the actual work (CstateC_{state}, CprocC_{proc}, QQ) are only described informally, and QQ is not fully defined even in Appendix B.1
Questions for authors
  • Can you report results over several runs with standard errors?
  • Can you compare against SEEM and a combined semantic + procedural baseline (e.g., Mem0 + AWM)?
  • How is QQ computed? How sensitive are the results to δQ\delta Q and the thresholds in Table B.3?
2 comments

Thank you for the comments. We will strengthen prior-work comparisons, clarify the update criteria, and improve the presentation.

Novelty and comparison with SEEM.

SEEM² builds on SEEM’s structured experience representation and provenance links. Our contribution concerns how knowledge derived from these experiences evolves. Semantic assertions are revised according to identity, scope, and observation time. Procedural cores require successful execution evidence, while failures refine their applicability and reuse eligibility. Shared scenes connect the observations, actions, and outcomes supporting both decisions. An unsuccessful task can therefore establish a factual change without supporting reuse of the attempted procedure.

We have evaluated original SEEM on all 300 LongMemEval S∗S^{\ast} questions. Both methods use DeepSeek-V4-Flash as the reader without auxiliary calculation. The SEEM² results below are from the supplementary fixed-memory reading evaluation.

\text{Method} & \text{Gemma4 judge} & \text{DeepSeek judge} \\\\ \hline \text{Original SEEM} & 65.67\\% & 71.00\\% \\\\ \mathrm{SEEM}^{2}\text{, without auxiliary calculation} & \mathbf{75.00\\%} & \mathbf{78.33\\%} \end{array}$$ We will include this baseline in the revision and distinguish the inherited representation from our evolution mechanisms. The comparison with FluxMem and MSCE will focus on update targets, supporting evidence, and revision or reuse decisions. **Contribution of semantic and procedural updates.** With Qwen3.8-27B, full SEEM² achieves 80.22% task success on $\tau$-bench. Removing semantic memory gives 73.74%, removing updates from failed tasks gives 76.98%, and separating memory organization gives 74.82%. The last variant retains both memory types and their update mechanisms while separating their experience organization. These results support contributions from semantic memory, failure-informed procedural updates, and their shared organization. We will add a combined Mem0 + AWM baseline and a controlled comparison with a unified task-success gate for positive updates to both memory types. The unified comparison will hold representation, retrieval, initial memory, backbone, and task order fixed. **Reliability and memory behavior.** Gemma4 reevaluation of the main paper’s saved SEEM² predictions and all eight existing baseline prediction sets gives 75.33% for SEEM² versus 66.00% for the strongest baseline. The two judges agree on 286 of 300 SEEM² answers, with identical knowledge-update accuracy of 88.89%. We will add independent repetitions and report means and standard errors, explicitly specifying backbone, reasoning configuration, and judge settings. We will expand the memory-behavior analysis with examples of state revision, retained conflicts, and failures involving temporal interpretation or incomplete evidence across sessions. **Method presentation and definition of Q.** We will organize Section 3 around experience formation, semantic evolution, procedural evolution, and retrieval, using consistent terminology. We will define $C_{\mathrm{state}}$ and $C_{\mathrm{proc}}$ as compatibility checks for factual revision and procedural transfer. For a candidate experience $\epsilon$ with $n > 0$ procedural steps, the fixed execution-evidence score is $$Q(\epsilon)=0.40I_{a}+0.25I_{t}+0.15I_{e}+0.10I_{o} +0.10\min\left(1,\frac{n}{4}\right).$$ Here, $I_{a}=1$ if a nonempty action trace is recorded; $I_{t}=1$ if at least one step is marked terminal; $I_{e}=1$ if every step is attributed to the agent or user/environment; and $I_{o}=1$ if at least one step records success or partial success. Each indicator is zero otherwise. The variable $n$ counts procedural steps, and $Q=0$ for an empty procedure. Overall task success and procedural compatibility are separate replacement requirements. An eligible candidate replaces the stored core when $Q(\epsilon) > Q_{s}+\delta_{Q}$, where $Q_{s}$ is the stored core’s score and $\delta_{Q}=0.05$. We will add this computation and the indicator definitions to Appendix B and connect them explicitly to the core-update rule in Section 3.

I thank the authors for their new experiments and clarifications. However I still have some concerns:

  • Some of the promises made in the review are not present in the current version of the manuscript (e.g. combined Mem0 + AWM baseline, controlled comparison with a combined update rule, examples of how the memory behaves ...). Some others are incomplete, namely I would have liked to see a repeated evaluation on τ\tau-bench and BFCL too, not just on LongMemEval.
  • The style of the paper has not changed and it remains difficult to follow throughout.

I will therefore keep my score for the moment.

The formula defining Q raises a new question: it seems to measure how complete the run record is, rather than how helpful the associated procedure is?

Comments on the paper

1 comment

We thank the reviewers for their constructive feedback. We have extended our evaluation to include SEEM, an additional judge, five repeated runs without auxiliary calculation, and an expanded analysis of LLM usage. All accuracy results below cover the 300 LongMemEval questions.

1. Comparison with SEEM and evaluation with an additional judge.

With DeepSeek-V4-Flash as the reader and DeepSeek-V4-Pro as the judge, SEEM achieves 71.00% accuracy, compared with 79.33% for SEEM².

We also reevaluate the main comparison’s saved predictions with Gemma 4 31B. For SEEM², the same answers achieve 79.33% under DeepSeek-V4-Pro and 75.33% under Gemma 4. The table reports accuracy (%) under both judges.

\text{Method} & \text{DeepSeek-V4-Pro} & \text{Gemma 4} \\\\ \hline \text{Long-context FIFO} & 71.00 & 66.00 \\\\ \text{SEEM} & 71.00 & 65.67 \\\\ \text{Dense RAG (Qwen3)} & 67.67 & 65.33 \\\\ \text{Graphiti} & 64.33 & 64.67 \\\\ \text{Dense RAG (KaLM)} & 66.00 & 63.00 \\\\ \text{Full Context} & 67.33 & 61.00 \\\\ \text{LightMem} & 62.67 & 59.67 \\\\ \text{A-MEM} & 53.33 & 52.00 \\\\ \text{Mem0} & 56.00 & 51.33 \\\\ \hline \mathrm{SEEM}^{2} & \mathbf{79.33} & \mathbf{75.33} \end{array}$$ SEEM² achieves the highest accuracy under both judges, with a margin of 9.33 percentage points over the strongest baseline under Gemma 4. The two judges agree on 286 of the 300 SEEM² correctness judgments. **2. Five repeated evaluations without auxiliary calculation.** We conduct five additional runs of retrieval and answering, holding each method’s memory bank fixed and disabling auxiliary calculation for both SEEM and SEEM². Gemma 4 evaluates the answers from these runs. SEEM² achieves $(77.87 \pm 0.23)\\%$ accuracy, compared with $(65.87 \pm 0.40)\\%$ for SEEM. SEEM² scores higher in all five runs. **3. Expanded LLM cost analysis.** Across the five histories and 300 questions, the main SEEM² configuration uses **1,086 LLM calls and 7.13M tokens**, including **468 calls and 2.58M tokens** during question answering. SEEM uses **2,974 calls and 14.69M tokens** in total. In the five-run evaluation without auxiliary calculation, question-answering usage averages **300 calls and 2.13M tokens** per run. Token counts include model inputs and outputs, excluding embedding and judge usage. For comparison, Mem0 and LightMem use 6.19M and 5.43M tokens in total, respectively. **4. Further experiments and manuscript revisions.** We will add a controlled comparison between distinct update criteria and a unified outcome-based rule, holding memory representation, retrieval, and the memory-context budget fixed. We will also extend the continual-memory evaluation to multiple task orders and add BFCL comparisons with matched packing, deletion safeguards, and retrieval budgets. The revised manuscript incorporates the completed evaluations and expanded cost analysis, with explicit definitions of the key update conditions and procedural quality score. We will further clarify the contribution relative to SEEM, FluxMem, and MSCE, and simplify the overview in Figure 1 while retaining its case study.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.