LGMem: Lightweight Graph Memory for Long-Term Conversational Agents

Agents engaged in sustained interactions need to accurately recover dispersed and evolving facts from growing dialogue histories. Existing structured memory systems face two challenges: (1) how to construct memory cost-effectively while preserving fine-grained evidence; and (2) how to reliably obtain sufficient evidence from memory. To address these challenges, we introduce Lightweight Graph Memory (LGMem), a memory framework that combines efficient memory construction with reflection-guided iterative retrieval. During construction, LGMem uses lightweight entity and keyword extraction to incrementally connect original sentences through shared anchors, building graph memory that retains raw dialogue evidence without calls to large language models (LLMs). At query time, LGMem uses local graph propagation to retrieve candidate evidence across sessions and stateful reflection to identify evidence gaps and iteratively refine queries to gather missing evidence. Experiments across multiple benchmarks show consistent QA gains over strong baselines. On LongMemEval-S and BEAM-100K, LGMem improves QA scores over the strongest respective baselines by 8.0% and 9.8%, while reducing total token consumption for memory construction and access by 75.8% and 84.5%, demonstrating a better trade-off between QA performance and token cost.