acceptodds
Under review as a conference paper at ICLR 2027

LGMem: Lightweight Graph Memory for Long-Term Conversational Agents

Abstract

Agents engaged in sustained interactions need to accurately recover dispersed and evolving facts from growing dialogue histories. Existing structured memory systems face two challenges: (1) how to construct memory cost-effectively while preserving fine-grained evidence; and (2) how to reliably obtain sufficient evidence from memory. To address these challenges, we introduce Lightweight Graph Memory (LGMem), a memory framework that combines efficient memory construction with reflection-guided iterative retrieval. During construction, LGMem uses lightweight entity and keyword extraction to incrementally connect original sentences through shared anchors, building graph memory that retains raw dialogue evidence without calls to large language models (LLMs). At query time, LGMem uses local graph propagation to retrieve candidate evidence across sessions and stateful reflection to identify evidence gaps and iteratively refine queries to gather missing evidence. Experiments across multiple benchmarks show consistent QA gains over strong baselines. On LongMemEval-S and BEAM-100K, LGMem improves QA scores over the strongest respective baselines by 8.0% and 9.8%, while reducing total token consumption for memory construction and access by 75.8% and 84.5%, demonstrating a better trade-off between QA performance and token cost.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Long-Term Memory, Graph Memory, Graph-Based Retrieval, Memory Management

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_67v6OpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper proposes LGMem, a long-term memory system for conversational question answering. It links original sentences and dialogue segments through extracted entities and keywords, without generative model calls during memory construction. At query time, graph-based retrieval gathers evidence, while a reflection step identifies missing information and guides further retrieval. On LongMemEval-S and BEAM, LGMem improves the primary evaluation scores over the compared methods while using fewer generative tokens for memory construction and query-time reasoning (Tables 1–2).

Critical strengths
  • The design addresses a real difficulty in long-term memory. Keeping the original sentences, rather than only summaries, helps preserve details that may matter to future questions. Checking for evidence gaps before answering is particularly useful when facts are spread across conversations.

  • The evaluation supports the main result. The gains appear across different history lengths, and the ablations show that both graph retrieval and reflection contribute. The paper also reports confidence intervals for its main score comparisons (Section 3.3 and Table 9).

Critical weaknesses
  • The advantage over other iterative memory-retrieval approaches is not yet clear. The paper compares against several memory systems, but not directly against the iterative approaches discussed in its related work. The ablations establish that LGMem’s components help, but do not fully clarify what this particular combination adds relative to those approaches.

  • The efficiency claim has a narrower scope than “lightweight” may suggest. The main token comparison excludes final answer generation and does not capture all non-generative computation or storage. The runtime analysis is helpful, but is limited to one benchmark setting, making the end-to-end trade-off at larger history scales harder to judge.

Questions for authors
  • Clarification: What is the key advantage of LGMem over prior iterative memory-retrieval approaches under a comparable retrieval and reflection budget? Which of the current results best isolate that advantage?

  • Clarification: How should readers interpret the reported token savings in terms of end-to-end cost? In particular, could encoding, retrieval, or graph storage meaningfully change the efficiency comparison at the 500K and 1M history scales?

5 comments

Dear Reviewer 67v6,

Thank you for your careful review of our work and your constructive feedback. Our detailed responses are provided below.

W1 / Q1. Advantage over prior iterative access under comparable settings

“The advantage over other iterative memory-retrieval approaches is not yet clear. […] What is the key advantage of LGMem […] under a comparable retrieval and reflection budget? Which of the current results best isolate that advantage?”

Response. We appreciate your suggestion to clarify the comparison with prior iterative methods. LGMem combines lightweight graph retrieval with explicit task state: the graph connects related original evidence, while reflection uses available information and unresolved gaps to guide subsequent retrieval. We examine this design through comparisons of reflective access methods on the same graph, evaluations against existing iterative memory systems, and fine-grained ablations of the reflection module.

(1) Reflective access on shared graph memory. We compare IRCoT-style and Self-RAG-style access on the same LGMem graph over all 500 LongMemEval-S questions, using GPT-5.6-Luna and the same final-answer settings, and report scores and token consumption.

Table 1. Access controllers on shared graph memory: GPT-5.6-Luna.

Access configurationJudge (%) ↑Access tokens (M)Access callsTotal tokens (M)
Graph + IRCoT (k=4)69.602.5618694.071
Graph + IRCoT (k=6)72.803.4118545.459
Graph + IRCoT (k=8)75.604.0168486.532
Graph + IRCoT (k=12)75.005.0008828.227
Graph + Self-RAG71.005.92311,1559.341
LGMem80.606.3799068.599

LGMem outperforms the best Graph + IRCoT setting (k=8) and Graph + Self-RAG by 5.00 and 9.60 points, respectively. Compared with IRCoT at k=12, total token usage is similar (8.599M versus 8.227M), while LGMem scores 5.60 points higher. Relative to Graph + Self-RAG, LGMem reduces total tokens by 7.9%. These results demonstrate LGMem's QA advantage among the compared access strategies and support organizing follow-up retrieval and task state around unresolved information needs.

(2) Comparisons with other iterative memory systems. We added comparisons with other iterative memory systems, MRAgent (ICML 2026) and REMem-I (ICLR 2026), on LongMemEval-S and BEAM, reporting QA scores and total token consumption.

Table 2. LongMemEval-S performance and token costs (500 questions per setting).

GPT-5.6-Luna

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent74.00127.1895.328132.51749,882
REMem-I78.60305.9445.932311.87749,679
LGMem80.600.0008.5998.5991,406

Qwen3.6-27B-FP8

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent64.80107.22914.726121.95452,144
REMem-I71.80241.8315.995247.82649,570
LGMem77.200.0008.5068.5061,390

On LongMemEval-S with Luna, LGMem improves over REMem-I and MRAgent by 2.00 and 6.60 points while reducing total tokens by 97.2% and 93.5%, respectively. With Qwen3.6, the gains are 5.40 and 12.40 points, with total-token reductions of 96.6% and 93.0%. REMem-I approaches LGMem in Luna accuracy but consumes 36.3 times as many total tokens.

Table 3. BEAM Judge macro scores and token costs with GPT-5.6-Luna.

BEAM-100K (400 questions)

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent56.511.0836.0507.1332,137
REMem-I58.215.3894.78010.1681,718
LGMem67.120.0008.8128.8121,292

BEAM-500K (700 questions)

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent57.746.12512.36618.4914,496
REMem-I59.8138.9198.57047.4893,433
LGMem64.820.00020.18820.1882,988

BEAM-1M (700 questions)

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent57.658.07413.61321.6865,003
REMem-I57.0670.6908.65879.3473,421
LGMem65.340.00018.08318.0832,614

At BEAM scales of 100K, 500K, and 1M, LGMem exceeds the second-best method in this comparison by 8.9, 5.0, and 7.7 points, respectively, with the fewest LLM calls. Relative to REMem-I, total tokens decrease by 13.3%, 57.5%, and 77.2%. These results support the combination of lightweight graph construction and stateful reflection for QA quality and token efficiency.

MRAgent constructs persistent memory from user messages only, which account for approximately 8.5%, 7.0%, and 5.4% of full-history tokens at the three scales. This reduces construction input but leaves information appearing only in assistant messages outside persistent memory.

Together, these comparisons show that lightweight graph construction and stateful reflection give LGMem an advantage in the balance of QA quality and total token consumption over the compared iterative memory systems.

(3) Fine-grained ablations of the reflection module. Following the method-level and system-level comparisons above, we ablate explicit state, temporal processing, and reflection with GPT-5.6-Luna, covering 35 histories and 700 questions.

Table 4. Fine-grained ablations of the reflection module on BEAM-1M.

ConfigurationJudge (%) ↑Δ (pp)
LGMem66.33—
w/o State63.40−2.93
w/o Temporal64.84−1.49
w/o Reflection58.66−7.67

Removing reflection, explicit state, and temporal processing lowers scores by 7.67, 2.93, and 1.49 points, respectively, indicating the largest contribution from iterative retrieval, with state maintenance and temporal processing also aiding evidence integration.

W2 / Q2. Scope of “lightweight” and end-to-end cost at 500K/1M

“The main token comparison excludes final answer generation and does not capture all non-generative computation or storage. […] Could encoding, retrieval, or graph storage meaningfully change the efficiency comparison at the 500K and 1M history scales?”

Response. Thank you for suggesting a more complete efficiency analysis. We added total token consumption including final answers, complete-pipeline timing including all LLM calls, and index-storage and runtime-memory measurements at 500K/1M.

(1) Total tokens including final answers. As shown in Tables 1 and 2 of the submitted manuscript, LGMem offers advantages in both QA quality and token consumption. At BEAM scales of 500K and 1M in Table 2, LGMem reduces memory-processing tokens by 66.57% and 82.16%, respectively, relative to the second-best method by score, SimpleMem. Table 3 in this response compares the iterative memory systems on BEAM, with total tokens including final answering. LGMem scores higher than both MRAgent and REMem-I at 500K and 1M. Relative to REMem-I, it improves Judge scores by 5.0 and 8.3 points while reducing total tokens by 57.5% and 77.2%, respectively. Thus, LGMem retains advantages in both QA quality and token efficiency when final answering is included. Below, we further examine complete-pipeline time and storage costs.

(2) Complete-pipeline timing. Using one L20 and the same preloaded Qwen3.6 service, each method processes the same history and all 20 questions at each scale. We report amortized time per question, including initialization, construction, and final answering.

Table 5. Complete-pipeline amortized time per question (s).

Method100K500K1M
Dense RAG10.828.189.33
LinearRAG9.2411.5616.35
RAPTOR33.9275.6299.48
LightMem42.38122.37165.36
SimpleMem64.45137.88457.26
REMem-I47.3069.03147.83
MRAgent169.20340.70388.70
LGMem35.8445.8254.56

LGMem takes 35.84, 45.82, and 54.56 s per question across the three scales, below LightMem, SimpleMem, REMem-I, and MRAgent at every scale. Dense and LinearRAG remain faster, reflecting the tradeoff between multi-round evidence integration and runtime.

(3) LGMem stage times. We break down the same complete pipelines, counting construction once per history.

Table 6. LGMem complete-pipeline components (s).

ScaleInitializationBuild20-question QATotal
100K7.6117.49691.69716.78
500K7.5061.14847.75916.39
1M7.66129.85953.681091.19

At 500K and 1M, graph construction accounts for 6.7% and 11.9% of total runtime, respectively, and is not the main time cost. Including construction preserves the relative efficiency conclusions above for these long-history settings.

(4) Index storage and runtime memory. We further report index size and peak local-process RAM to assess storage and runtime memory costs for long histories.

Table 7. LGMem index storage and peak RAM.

ScaleIndex (MiB)Peak RAM (GiB)
100K20.631.52
500K83.511.54
1M165.231.76

At 1M, the index size is 165.23 MiB and peak local-process RAM is 1.76 GiB. Together with the complete-pipeline timing results, these measurements show that non-generative processing and storage costs remain manageable and do not offset LGMem's relative efficiency advantages at the tested long-history scales, further supporting its lightweight design.

Reviewer_8G9jOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LGMem, a lightweight graph memory framework for long-term dialogue QA. It incrementally builds an anchor–sentence–segment graph from raw dialogue using spaCy NER and TextRank anchors without LLM calls, preserving evidence, provenance, and timestamps. At query time, it combines anchor activation with local graph propagation and stateful reflection to identify evidence gaps and issue follow-up queries. Experiments on LongMemEval-S and BEAM (100K–1M tokens) show improved Judge scores with substantially lower generative memory-processing token costs.

Critical strengths

The memory construction stage is LLM-free and sufficiently lightweight: extraction relies solely on an NER model and TextRank, without using LLMs. This avoids introducing additional bias while keeping costs low, making it easy to use in practice.

Strong performance: The results show that, with a moderate token overhead, the method achieves a high level of performance among comparable approaches.

The experiments are comprehensive: they validate the contribution of each module and clarify the categorization criteria used for cost statistics.

Critical weaknesses

Graph construction relies on NER and TextRank, and the paper lacks an analysis of graph quality, which may affect overall retrieval performance. Additional analysis of the constructed graphs should be provided.

The absence of LLM-based extraction may lead to insufficient evidence integration, potentially degrading performance on complex multi-hop tasks.

Questions for authors

see above

2 comments

Dear Reviewer 8G9j,

We appreciate your recognition of our work and your thoughtful and constructive review.

W1. Quality of the NER/TextRank graph

“Graph construction relies on NER and TextRank, and the paper lacks an analysis of graph quality, which may affect overall retrieval performance. Additional analysis of the constructed graphs should be provided.”

Response. Thank you for your suggestion to analyze graph quality. We added analyses of anchor coverage, supporting-source access, and graph size to examine how lightweight construction supports evidence retrieval.

(1) Graph quality analysis. We compare anchor configurations on LongMemEval-S with reflection disabled and a shared 4K evidence budget.

Table 1. Effect of NER and TextRank on evidence retrieval (%).

AnchorsSentence coverageStructural reachabilityGraph recallRecall (4K)
Dense———91.41
NER-only29.4066.7855.2192.48
TextRank-only82.5390.4788.5393.49
NER + TextRank83.0092.0690.6593.13

Coverage is the fraction of sentences linked to anchors. Structural reachability and graph recall measure supporting sessions reachable in the graph and actually found by propagation, respectively. Recall (4K) reports hybrid retrieval recall under the shared 4K budget.

Adding TextRank increases sentence coverage by 53.60 percentage points; combined anchors improve recall over Dense by 1.72 points under the 4K budget. Adding NER to TextRank further improves graph recall by 2.12 points, indicating that keywords broaden coverage while entities provide additional routes to supporting sources.

(2) Propagation depth. The table below compares graph recall and recall under the same 4K budget across propagation depths.

Table 2. Effect of propagation depth on supporting-source recall (%).

Propagation depthGraph recallRecall (4K)
183.3794.09
290.6593.13
392.6793.44
493.3093.45

Moving from one hop to two increases graph recall by 7.28 percentage points; deeper propagation provides further coverage with diminishing gains. Recall under the 4K budget remains similar across depths, indicating that evidence ranking and budget allocation still influence which supporting sources are retained after graph expansion.

(3) Graph size across datasets. We report mean node counts per history; each BEAM history shares one graph across its 20 questions.

Table 3. Mean graph nodes per history.

DatasetHistoriesAnchorsSentencesSegmentsTotal
LongMemEval-S5007833.25130.3248.713212.1
BEAM-100K204359.46020.2127.410506.9
BEAM-500K3513546.925982.5543.740073.1
BEAM-1M3521814.450181.4988.172983.9

As BEAM history length grows from 100K to 1M, the mean graph size increases from approximately 10.5K to 73.0K nodes, reaching 6.95 times its original size.

W2. Evidence integration and complex multi-hop tasks

“The absence of LLM-based extraction may lead to insufficient evidence integration, potentially degrading performance on complex multi-hop tasks.”

Response. We appreciate your concern about evidence integration. Lightweight extraction still has limitations in covering implicit relations and long-range semantic associations. LGMem addresses this through graph propagation and gap-driven reflection for cross-session evidence integration.

(1) Graph propagation captures cross-session connections. Entity and keyword anchors connect related sentences across sessions, allowing retrieval to locate intermediate clues and then follow shared anchors to further evidence. This supports a degree of cross-session multi-hop access. In the BEAM-1M ablation, increasing graph propagation from one hop to two raises the Judge score from 64.77% to 66.33%, a gain of 1.56 points.

(2) Reflection completes cross-session evidence. When initial retrieval leaves gaps in the evidence chain, the controller assesses the question and available evidence, identifies unresolved information needs, and generates targeted queries to retrieve additional evidence from other sessions. Explicit state retains known information and unresolved questions across rounds, directing subsequent retrieval toward missing links and supporting final evidence integration. In the BEAM-1M ablation, removing reflection lowers the Judge score from 66.33% to 58.66%, a drop of 7.67 points, showing the contribution of query-time iterative evidence completion to overall performance.

The following cases illustrate how graph propagation connects intermediate clues and how reflection retrieves missing evidence.

Case A: Graph propagation links cross-session clues

Question. At which university did I present a poster on my thesis research?

Retrieval. Session 9 links the thesis poster to “my first research conference.” The shared conference anchor leads to session 13, which mentions “Harvard University” and the same conference.

Answer. Harvard University.

Case B: Reflection retrieves a missing purchase detail

Question. I mentioned an investment for a competition four weeks ago. What did I buy?

Retrieval. Initial retrieval finds the sculpture-competition background. Reflection identifies the missing purchase detail and generates the query "competition" "investment" bought purchase sculpture.

Additional evidence. The next round finds: “I actually got my own set of sculpting tools, including a modeling tool set, a wire cutter, and a sculpting mat.”

Answer. A set of sculpting tools: a modeling tool set, a wire cutter, and a sculpting mat.

Although graph propagation and reflective retrieval strengthen cross-session evidence integration, there remains room to improve LGMem on complex multi-hop questions. At the same time, as shown in Tables 1 and 2 of the submitted manuscript, LGMem achieves higher overall Judge scores than the other compared methods on both LongMemEval-S and BEAM.

Reviewer_pXkYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose the LGMem method for the long-term conversational agent. The proposed LGMem method integrates efficient memory construction with reflection-guided iterative memory retrieval. For the memory construction part, LGMem uses lightweight entity and keyword extraction to incrementally connect original sentences, without using LLMs. For the query part, LGMem uses the local graph propagation to retrieve candidate evidence, and iteratively refines queries to gather missing evidence. The authors have performed extensive experiments on two benchmark datasets to demonstrate the effectiveness of the proposed method. The results on LongMemEval-S and BEAM-100K datasets show that LGMem outperforms baselines by 9.8% in QA scores, and reduces token consumption by 75.8% and 84.5% for memory construction and access, respectively.

Critical strengths
  1. In this paper, the authors propose a lightweight graph memory method, named LGMem, for long-term conversational agents. The proposed LGMem combines efficient memory construction based on entity and keyword extraction, and reflection-guided iterative retrieval.

  2. The authors have performed extensive experiments on benchmarking datasets to demonstrate the effectiveness of the proposed LGMem, comparing with the SOTA memory methods.

  3. The authors have performed comprehensive ablation studies to analyze the contributions of different modules.

  4. The proposed method outperforms SOTA baseline methods in QA scores, and also significantly reduces the token cost.

Critical weaknesses
  1. The technical novelty of the proposed method seems somewhat limited. The proposed method integrates entity and keyword extraction based memory construction and reflection-based iterative memory retrieval. However, the memory construction and iterative memory retrieval methods have been used in previous research studies.

  2. This work lacks comparisons between the proposed LGMem method and existing iterative memory systems, e.g., MRAgent and REMem.

  3. The ablation studies should also be performed across the BEAM benchmark datasets.

Questions for authors
  1. Please perform experiments to compare the proposed method with existing MRAgent and REMem methods.

  2. Please also perform ablation studies on the BEAM benchmark datasets.

3 comments

Dear Reviewer pXkY,

Thank you for recognizing our work and for your detailed and insightful feedback. Your comments and suggestions are valuable for further improving the quality and clarity of the paper.

W1. Technical novelty

“The technical novelty of the proposed method seems somewhat limited. […] the memory construction and iterative memory retrieval methods have been used in previous research studies.”

Response. We understand your concern about LGMem's technical novelty. Our core contribution is the joint design of a lightweight dialogue-memory graph and stateful reflective retrieval, enabling effective cross-session evidence integration without generative LLM calls during construction. The graph incrementally connects entity and keyword anchors to original sentences and dialogue segments, using supporting-sentence aggregation and weighted scoring to rank evidence. Reflective retrieval jointly maintains a raw-evidence pool and explicit task state, using unresolved information gaps to guide cross-session retrieval and answer integration. This design combines low memory-processing token use with strong QA performance. Below, we explain the methodological differences and supporting results in graph design, reflective access, and empirical effectiveness.

(1) Graph structure and ranking for dialogue evidence access. LinearRAG activates entities through entity–sentence links and uses PPR over entity–passage links to rank passages. LGMem constructs a unified anchor–original sentence–dialogue segment graph: shared entity and keyword anchors support local propagation, while sentence–segment membership connects retrieved clues to their dialogue context. Segment ranking uses supporting-sentence aggregation and weighted scoring.

Table 1. Method comparison with LinearRAG.

DesignLinearRAGLGMem
Core linksEntity–sentence; entity–passageAnchor–sentence–segment
AnchorsNER entitiesNER entities and TextRank keyphrases
RankingPPRWeighted scoring

TextRank adds activities, events, and topics beyond named entities: sentence coverage increases from 29.40% to 83.00% across 500 LongMemEval-S histories. With GPT-5.6-Luna, adding TextRank raises LongMemEval-S QA from 79.40% to 80.60%; the corresponding BEAM-1M contrast is 63.30% versus 66.33% (the ablation table in W3).

Under the same Qwen3.6 configuration, our ranking method outperforms PPR (61.40% vs. 60.20%).

(2) Reflective access and differences from prior methods. IRCoT interleaves reasoning sentences with retrieval. Self-RAG uses reflection tokens to control retrieval and critique generated content. MRAgent employs LLM-controlled traversal, selection, and pruning over Cue–Tag–Content memory; REMem uses retrieval and graph-exploration tools over time-aware gist/fact memory.

LGMem instead organizes reflection around joint updates to a raw-evidence pool and an explicit task state. The pool accumulates original evidence across rounds. The state separately records source-supported facts, temporal records, a draft answer, and unresolved gaps. Each reflection reads the current evidence, previous state, and executed queries, generates targeted follow-ups retaining entity, event, and temporal constraints, and updates evidence filtering and stopping decisions. The lightweight graph handles local evidence access, while the LLM identifies information still needed for the task, forming an evidence-completion loop.

We compare IRCoT-style and Self-RAG-style access on the same LGMem graph over all 500 LongMemEval-S questions, using GPT-5.6-Luna and the same final-answer settings.

Table 2. Access controllers on shared graph memory: GPT-5.6-Luna, Judge scores (%).

Access configurationJudge (%) ↑
Graph + IRCoT (k=4)69.60
Graph + IRCoT (k=6)72.80
Graph + IRCoT (k=8)75.60
Graph + IRCoT (k=12)75.00
Graph + Self-RAG71.00
LGMem80.60

LGMem scores 80.60%, improving over the best Graph + IRCoT setting (k=8, 75.60%) and Graph + Self-RAG (71.00%) by 5.00 and 9.60 percentage points, respectively. These results indicate that, on this graph-memory structure, LGMem's reflection strategy better fits the evidence-completion and integration needs of long-term dialogue QA and achieves higher answer quality by organizing follow-up retrieval around information gaps and jointly maintaining a raw-evidence pool and task state.

(3) Quality–token trade-off. As shown in Tables 1 and 2 of the submitted manuscript, LGMem achieves the highest Judge scores on both datasets. With Luna, it reduces memory-processing tokens relative to the strongest respective baselines by 75.80% on LongMemEval-S and 66.57%–84.49% across BEAM scales. These results demonstrate quality–cost advantages over methods such as LightMem, MemGAS, and SimpleMem, supporting the effectiveness of lightweight construction and stateful reflection.

W2 / Q1. Comparisons with MRAgent and REMem

“This work lacks comparisons […] with existing iterative memory systems, e.g., MRAgent and REMem.”

Response. We appreciate your suggestion to include these comparisons. We added comparisons with the iterative memory systems MRAgent and REMem-I on LongMemEval-S and BEAM, reporting QA performance, token consumption by stage, and LLM calls below.

Table 3. LongMemEval-S performance and token costs (500 questions per setting).

GPT-5.6-Luna

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent74.00127.1895.328132.51749,882
REMem-I78.60305.9445.932311.87749,679
LGMem80.600.0008.5998.5991,406

Qwen3.6-27B-FP8

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent64.80107.22914.726121.95452,144
REMem-I71.80241.8315.995247.82649,570
LGMem77.200.0008.5068.5061,390

On LongMemEval-S, LGMem achieves higher accuracy and lower total token consumption with both generators. With Luna, LGMem reaches 80.60% accuracy, improving over REMem-I and MRAgent by 2.00 and 6.60 percentage points while reducing total tokens by 97.2% and 93.5%, respectively. Although REMem-I approaches LGMem in accuracy, it consumes 36.3 times as many total tokens.

With Qwen3.6, LGMem reaches 77.20% accuracy, improving over REMem-I and MRAgent by 5.40 and 12.40 points while reducing total tokens by 96.6% and 93.0%, respectively. These results show that combining non-generative original-text construction with stateful evidence completion improves iterative-memory QA quality and reduces total token consumption with both generators.

We also evaluate the two iterative memory systems on BEAM at 100K, 500K, and 1M history scales using GPT-5.6-Luna, as reported below.

Table 4. BEAM Judge macro scores and token costs with GPT-5.6-Luna.

BEAM-100K (400 questions)

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent56.511.0836.0507.1332,137
REMem-I58.215.3894.78010.1681,718
LGMem67.120.0008.8128.8121,292

BEAM-500K (700 questions)

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent57.746.12512.36618.4914,496
REMem-I59.8138.9198.57047.4893,433
LGMem64.820.00020.18820.1882,988

BEAM-1M (700 questions)

MethodJudge (%) ↑Build Tokens (M) ↓Reasoning + Answer Tokens (M) ↓Total Tokens (M) ↓Calls ↓
MRAgent57.658.07413.61321.6865,003
REMem-I57.0670.6908.65879.3473,421
LGMem65.340.00018.08318.0832,614

Among the iterative memory systems in this BEAM comparison, LGMem achieves Judge scores of 67.12%, 64.82%, and 65.34% at 100K, 500K, and 1M, respectively, exceeding the second-best method at each scale by 8.9, 5.0, and 7.7 percentage points with the fewest LLM calls. REMem-I ranks second at 100K and 500K, while MRAgent ranks second at 1M. Relative to REMem-I, LGMem reduces total tokens by 13.3%, 57.5%, and 77.2%, respectively.

MRAgent constructs persistent memory from user messages only, which account for approximately 8.5%, 7.0%, and 5.4% of full-history tokens at 100K, 500K, and 1M, respectively. This policy can reduce total token consumption but leaves information found only in assistant messages outside persistent memory.

Overall, LGMem offers a stronger balance of QA quality and token efficiency than MRAgent and REMem-I.

W3 / Q2. Ablations on BEAM

“The ablation studies should also be performed across the BEAM benchmark datasets.”

Response. Thank you for suggesting that the ablations be extended to BEAM. We added finer-grained BEAM-1M module ablations using GPT-5.6-Luna on 35 histories and 700 questions.

Table 5. BEAM-1M module ablations.

VariantJudge (%) ↑Δ (pp)
LGMem66.33—
w/o NER64.72−1.61
w/o TextRank63.30−3.03
w/o Multi-hop64.77−1.56
w/o State63.40−2.93
w/o Temporal64.84−1.49
w/o Reflection58.66−7.67

Removing NER, TextRank, and multi-hop expansion lowers the score by 1.61, 3.03, and 1.56 points, supporting the complementary roles of entity/keyword anchors and local graph propagation. Removing explicit state and temporal processing lowers the score by 2.93 and 1.49 points, showing the value of evidence organization and temporal information for final answering.

Removing reflection causes the largest drop, 7.67 points, indicating that iterative retrieval guided by information gaps is central to long-history QA.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.