- Rating
- 3
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper tackles long-term memory for LLM agents. The authors argue that semantic similarity and simple associations alone cannot guarantee that retrieved memories jointly form valid evidence for a given query. They propose Applicability Reasoning and implement it in WeaveMem, a system that structures interaction history into source-linked atomic memories and high-level experiences within a typed, directed experience graph (capturing support, causality, update, and contradiction relations). Given a query, WeaveMem retrieves relevant experiences, expands and filters the evidence chain, and feeds the corresponding atomic evidence to the answering model. WeaveMem outperforms existing memory baselines across four benchmarks (LoCoMo, LongMemEval-S, PersonaMem-32K, and BEAM-1M). Ablations and controlled experiments confirm the effectiveness of its core components, and efficiency analyses demonstrate lower input token overhead for the answering model compared to several high-performing baselines.
Critical strengths
Figure 1 clearly explains the three memory reasoning paradigms and makes the motivation easy to follow.
The experiments are extensive and cover a substantial set of analyses.
Organizing the experimental section around three research questions gives it a clear, coherent structure.
Critical weaknesses
My main concern is the potential confound from model choice. The experiments mainly use GPT-4o and GPT-4.1 mini, yet GLM-5.2 is used for graph construction and applicability tracing. If GLM-5.2 provides substantially stronger reasoning at these stages, it could enrich the information passed to a weaker answering model. Without controlling for this, it is hard to tell how much of the improvement comes from the proposed memory mechanism versus the additional model capability.
The baseline evaluation setup also needs clarification. In particular, it is unclear whether GPT-4.1 mini is used for answer generation, judging, or both. If it serves as the judge, how reliable are its assessments, and have they been checked with a stronger evaluator?
Questions for authors
How do you ensure a fair comparison across methods? Do the gains persist when model choice is controlled across comparable stages of the pipeline?
Could you provide a table specifying the model used in each component of every method, including memory/graph construction, retrieval/tracing, answer generation, and evaluation?
Do all baselines have comparable access to GLM-5.2 or other strong models, or is this additional capability available only to your method?
Discussion Window Closing in 2 days. Please Respond to the Author's Rebuttal!
Dear Reviewers,
The authors have thoughtfully responded to your reviews and the discussion window is coming to a close on November 18th. Please take a minute to engage with their response and discuss whether it changes your opinions.
Thank you for your time and effort!