acceptodds
Under review as a conference paper at ICLR 2027

WeaveMem: Typed Experience Graphs for Applicability Reasoning in LLM Agents

Abstract

Persistent language agents rely on evidence distributed across interaction histories that extend far beyond their context windows to answer queries. Existing memory systems retrieve records through semantic similarity or reconstruct context using associative structures. However, similarity retrieval overlooks indirectly useful evidence, while associative reconstruction may recover connected memories without establishing whether their relations are relevant to the query. We formulate this missing capability as applicability reasoning: determining whether and how retrieved memories jointly support a query through typed and directional relations that form a compatible and complete evidence chain. To operationalize this capability, we introduce WeaveMem, a long-term agent memory system built around a Typed Experience Graph and Applicability Tracing. The Typed Experience Graph grounds typed and directional relations in atomic memories and projects them onto compact experiences with provenance. Applicability Tracing constructs query-specific evidence chains, selects compatible and sufficient traces, and recovers their atomic evidence for answer generation. Across evaluations averaged over five independent runs on LoCoMo, LongMemEval-S, PersonaMem-32K, and BEAM-1M, WeaveMem achieves the highest overall score on each benchmark among the evaluated memory systems. Controlled evaluations on relation-dependent queries further show that WeaveMem more accurately identifies how retrieved memories relate to one another and jointly constitute complete, valid evidence chains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Long-Term Memory, Graph-Based Memory, Applicability Reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CofAOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper tackles long-term memory for LLM agents. The authors argue that semantic similarity and simple associations alone cannot guarantee that retrieved memories jointly form valid evidence for a given query. They propose Applicability Reasoning and implement it in WeaveMem, a system that structures interaction history into source-linked atomic memories and high-level experiences within a typed, directed experience graph (capturing support, causality, update, and contradiction relations). Given a query, WeaveMem retrieves relevant experiences, expands and filters the evidence chain, and feeds the corresponding atomic evidence to the answering model. WeaveMem outperforms existing memory baselines across four benchmarks (LoCoMo, LongMemEval-S, PersonaMem-32K, and BEAM-1M). Ablations and controlled experiments confirm the effectiveness of its core components, and efficiency analyses demonstrate lower input token overhead for the answering model compared to several high-performing baselines.

Critical strengths

Figure 1 clearly explains the three memory reasoning paradigms and makes the motivation easy to follow.

The experiments are extensive and cover a substantial set of analyses.

Organizing the experimental section around three research questions gives it a clear, coherent structure.

Critical weaknesses

My main concern is the potential confound from model choice. The experiments mainly use GPT-4o and GPT-4.1 mini, yet GLM-5.2 is used for graph construction and applicability tracing. If GLM-5.2 provides substantially stronger reasoning at these stages, it could enrich the information passed to a weaker answering model. Without controlling for this, it is hard to tell how much of the improvement comes from the proposed memory mechanism versus the additional model capability.

The baseline evaluation setup also needs clarification. In particular, it is unclear whether GPT-4.1 mini is used for answer generation, judging, or both. If it serves as the judge, how reliable are its assessments, and have they been checked with a stronger evaluator?

Questions for authors

How do you ensure a fair comparison across methods? Do the gains persist when model choice is controlled across comparable stages of the pipeline?

Could you provide a table specifying the model used in each component of every method, including memory/graph construction, retrieval/tracing, answer generation, and evaluation?

Do all baselines have comparable access to GLM-5.2 or other strong models, or is this additional capability available only to your method?

Reviewer_y2VhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes WeaveMem, a memory system that introduces applicability reasoning to memory retrieval. It constructs a typed experience graph to represent logical relations between memories and uses applicability tracing to select relevant evidence chains for answering queries. Experiments on four memory benchmarks show consistent improvements over existing methods.

Critical strengths
  1. The paper introduces applicability reasoning as a new perspective on memory retrieval, focusing on how memories are logically related and jointly support a query.
  2. The paper evaluates WeaveMem on four diverse memory benchmarks, covering different memory tasks and reasoning requirements.
Critical weaknesses
  1. WeaveMem only uses seven predefined relation types and removes other semantic connections. It is unclear whether these relations can cover different memory needs. Some useful memories may not have any of these relations, such as different activities of the same person at different times.
  2. The paper compares semantic retrieval and applicability reasoning under different experimental settings. Existing methods such as A-Mem use LLMs to construct semantic links between memories, while the semantic retrieval baseline in this paper does not use LLMs or graph connections. A comparison under the same settings is needed to better demonstrate the motivation and advantages of applicability reasoning.
  3. The efficiency analysis mainly reports the input tokens of the answer model. It does not report the token costs of graph construction or online applicability tracing, which require additional LLM calls. Therefore, the total computational cost remains unclear.
Questions for authors

Please see the Weaknesses above.

Reviewer_q2sjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce WeaveMem, a framework for handling long-term memory in LLM agents. The proposed method consists of two main components: 1) A Typed Experience Graph, which organizes memories into atomic experiences linked via directional relations and provenance metadata, and 2) Applicability Tracing, which traverses the graph to construct query-specific evidence chains. The framework is evaluated across five benchmarks (LoCoMo, LongMemEval-S, PersonaMem-32k, and Beam-1M) and compared against eight recent long-term memory baselines, all reimplemented using the same answerer model for consistent comparison.

Critical strengths
  • The proposed framework appears to be at the frontier in terms of performance and token efficiency when using gpt-4o-mini as an answerer.
  • The authors standardize the evaluation by implementing the same answerer model across all baselines and provide detailed ablations of their framework's components.

The benchmark comparison is quite broad and well documented. The authors evaluate on four benchmarks that differ a lot in history length and reproduce nine baselines under one shared answer model and judge.

The tracing design is sensible and is well grounded. Letting the controller choose edges from relation type, direction and description while hiding the destination's content forces the expansion decisions to be made on the relation and not on surface similarity.

I also generally appreciate that the appendix reports results against the paper's own story, e.g. Case 2 where the ablation retrieves byte-identical context, the weak BEAM evidence recall, and the explicit caveats on the MQuAKE setup.

Critical weaknesses

While the initial results are promising, the claims regarding hyperparameter sensitivity, efficiency, and overall performance would be substantially strengthened by evaluating across multiple model architectures rather than relying solely on a single model. Additionally, since the answerer and judge models belong to the same family, exploring evaluations with distinct model families could help prevent potential bias or conflation in the results.

I. The main claim, that WeaveMem does "applicability reasoning" and that this explains its gains, is not well supported by the experiments. The MQuAKE-T study doesn't test the Typed Experience Graph at all. The graph is built from the gold chain, the first gold node is given as the seed, and the edges carry native MQuAKE predicates instead of WeaveMem's seven relation types. The main text presents the same numbers as relation and direction accuracy. The gain there also comes entirely from the 95 three-hop cases, since Full and w/o Applicability are essentially tied on the 303 two-hop cases.

II. The LoCoMo diagnostic panel doesn't measure the system that produces the main results. It is a single run on 252 questions under a different configuration entirely, and the w/o Applicability ablation isn't specified beyond one sentence.

III. The evidence labels were produced by AI and then reviewed by humans, but the paper doesn't report how many annotators there were or how well they agreed. The induced relations are also never checked directly, for example by measuring human precision of relation type and direction on a sample of edges.

IV. On the main benchmarks, the increase in scores aren't where relation chains should help most. On LoCoMo multi-hop, both Memora and Full Context score above WeaveMem. On LongMemEval-S, Memora is higher on multi-session and temporal questions. On BEAM-1M, most of the gain comes from Information Extraction, which is not a relational category, while WeaveMem's evidence recall there is among the lowest of all methods. So the conclusion that the gains come from "more complete and logically valid evidence chains" is not clearly shown at all.

V. The margins are quite small, and the cost of getting them isn't reported. The improvements over the best baseline range from 0.47 to 1.80 points, with no paired significance test, and on LoCoMo WeaveMem is only about 2 points above Full Context.

VI. The efficiency analysis counts only answer-model tokens. It leaves out the roughly 21 controller calls per query in the default setting, and the graph-construction cost on BEAM-1M isn't given at all. Without a ‘matched-compute’ baseline, it is hard to tell whether the increase comes from the typed graph or from the extra LLM calls. I would also like to note that defaults, sensitivity and ablations are all tuned on LoCoMo, which is also a test benchmark.

Questions for authors

Clarification Requests:

  • I am unclear on the difference between ded_e and rr (both described as the relation between atomic memories in lines 157 and 189) in defining an atomic edge. Could you please clarify?
  • Why does multihop reasoning seem to lag behind in LoCoMo and LongMemEval-S, but perform significantly better in Beam-1M and PersonaMem-32k? How does this relate to the claim that WeaveMem improves multihop reasoning? (Lines 408–418) Experimental Setup Question:
  • Why is the method implemented and baselined using gpt-4o-mini? How does the underlying model affect performance? Do the claims generalize to other models (for example, open-source models of different sizes or other model families)?

Can you run the MQuAKE (or a similar) with WeaveMem's own induced graph, i.e., relation induction over the fact texts with the seven types, and without supplying the first gold node as the seed? What exactly differs between Full WeaveMem and "w/o Applicability" in both panels? Please give the actual prompt diff. Are the improvements over the best baseline significant under a paired per-question test across runs, in particular +1.24 over Memora on LoCoMo and +0.47 on BEAM-1M? Minor suggestions: The conclusion mentions "complex multi-step interactive decision-making," but all benchmarks are conversational QA. An agentic or task-based evaluation would better match that framing.

Comments on the paper

1 comment
Area_Chair_1yrSOpenReview ↗

Discussion Window Closing in 2 days. Please Respond to the Author's Rebuttal!

Dear Reviewers,

The authors have thoughtfully responded to your reviews and the discussion window is coming to a close on November 18th. Please take a minute to engage with their response and discuss whether it changes your opinions.

Thank you for your time and effort!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.