- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents ENGRAM, a structured long-term memory system that asynchronously extracts and consolidates facts into a bi-temporal knowledge graph, and retrieves a combination of structured facts and raw conversation chunks. The authors evaluate ENGRAM on LongMemEval-S against full-context prompting and an equal-budget RAG baseline. Results show that ENGRAM achieves 78.8% accuracy, compared with 72.4% for RAG and 61.6% for full context, while using substantially fewer context tokens than full history.
Critical strengths
- Comparing structured memory against an equal-budget RAG baseline is important for understanding whether the benefits come from memory organization or simply better context selection.
- Strong reader models and retrievers are considered.
- The empirical analysis is thorough. The authors provide category-level results, component ablations, statistical significance tests, and evaluations across different readers and judges.
Critical weaknesses
- The evaluation is limited to LongMemEval. While the authors additionally report retrieval results on LongMemEval-M, end-to-end QA is only evaluated on LongMemEval-S. It would be helpful to include other memory benchmarks such as LoCoMo [1] and PersonaMem [2].
- The technical contribution is somewhat limited. The proposed system largely combines existing techniques, including fact extraction, temporal knowledge graphs, and hybrid retrieval.
- The comparison with existing memory systems is insufficient. While the equal-budget RAG baseline is useful, the paper does not directly compare ENGRAM with established memory systems discussed in the related work, such as Mem0, Zep/Graphiti, or HippoRAG.
- The efficiency analysis is incomplete. The paper reports context token usage and answer-generation latency, but does not fully account for the cost of memory construction and consolidation.
References
[1] Maharana et al. Evaluating Very Long-Term Conversational Memory of LLM Agents. ACL 2024.
[2] Jiang et al. Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale. COLM 2025.
Questions for authors
Please refer to the weaknesses