acceptodds
Under review as a conference paper at ICLR 2027

GraphCard: A Simple Graph Memory Plug-in for Long-Term Conversational Memory Systems

Abstract

Conversational memory in deployed systems is predominantly vector-based: memories are stored as extracted facts or raw dialogue chunks, and a query is answered through fast per-entry similarity search. Production favors this design for its speed, scalability, and strong overall accuracy. However, multi-hop and temporal questions depend on relations and timelines that live between entries, and per-entry retrieval cannot see them. A graph memory captures exactly this missing layer, linking entities through facts. The two designs are natural complements: vector retrieval buys coverage, graph structure buys relations. Their combination, however, remains underexplored: existing graph designs replace the vector stack rather than augment it, and we find they score far below the strongest vector-based systems they aim to improve. In this paper, we take the strengths of both by augmenting rather than replacing: the existing vector memory (the host) stays untouched, and GraphCard, a graph plug-in, runs beside it. At memory ingestion time, a plug-in branch runs parallel to the host's own pipeline, organizing the raw conversation stream into a fact graph through entity deduplication and temporal fact management. At search time, a single query embedding and a bounded graph search, with no LLM call, quickly condense the relevant subgraph into one  150-token graph card prepended on top of the host's retrieved memories. Nothing else in the host memory system changes, yet the missing relational layer is now in the context. Extensive experiments across two memory benchmarks (LoCoMo, LongMemEval), two model groups (GPT, Claude), and four strong vector-based baseline hosts (RAG, Nemori, SimpleMem, Mem0) show that GraphCard improves every host configuration, by up to +12.3 points, with gains concentrated on multi-hop and temporal questions. GraphCard also yields a better accuracy–token trade-off than widening the host's own retrieval: against Mem0, GraphCard is  1.5 points more accurate at a matched memory budget; it matches the same host's accuracy with 79% fewer additional tokens. GraphCard is fast as well, with an LLM-free search that completes in 0.61 s at p50, in the same sub-second range as the vector hosts themselves.

Primary area:
generative models
Keywords:
Long-Term Conversational Memory, Graph Memory, Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_msL5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a key limitation of prevailing long-term conversational memory systems, which rely predominantly on vector-based retrieval and fail to capture relational connections and temporal structure across entries. The authors propose GRAPHCARD, a plug-in graph memory enhancement module. Instead of replacing the existing vector memory host system, GRAPHCARD builds a temporally grounded fact graph in parallel. At query time, it uses a single query embedding combined with bounded graph traversal to generate a structured graph card of roughly 150 tokens, which is prepended to the host’s retrieval result list to supplement the relational layer missing from vector retrieval.

Critical strengths
  1. Unlike existing graph memory systems that replace the existing retrieval pipeline and require rewriting the entire memory stack, GRAPHCARD follows a plug-in design with zero modification to the host system, parallel augmentation, and output attributability. On the input side, it only reads the raw conversation stream. On the output side, it only prepends a standardized summary card. This makes it minimally invasive to existing vector memory systems and easily adaptable to different hosts.
  2. The evaluation spans two long conversation benchmarks, two mainstream model architectures, and four representative vector memory hosts. It systematically verifies multiple dimensions including accuracy improvement, gain distribution across question types, token efficiency, the effect of the temporal module, and retrieval latency.
Critical weaknesses
  1. All underlying modules, including entity extraction, relation extraction, entity deduplication, temporal grounding, and hybrid retrieval, are established techniques from prior work. The paper does not propose new principles of graph memory modeling, novel retrieval mechanisms, or new structured reasoning paradigms. Essentially, it tailors existing graph memory capabilities into an add-on component that integrates with vector systems.
  2. Despite the graph card framing, the actual graph retrieval logic only includes semantic matching and 1-hop neighbor expansion. It does not involve multi-hop reasoning traversal, nor does it leverage any graph topological properties such as community structure, node centrality, path connectivity, or causal chains.
  3. The paper only compares against graph-only memory systems and vector-only hosts. It does not include comparable baselines of vector plus lightweight graph augmentation, nor does it run a fair efficiency comparison against approaches that widen the host’s own retrieval width. Additionally, the paper does not decompose the sources of improvement through component ablation.
  4. The paper only reports query-time latency, 0.61 seconds at p50, but does not evaluate the computational cost of the write path at all. Each incoming message requires two LLM calls for entity and relation extraction, plus additional LLM judgments for entity deduplication and fact consolidation. In high-throughput long conversation scenarios, it remains an open question whether the write latency, cost, and resource consumption are acceptable.
Questions for authors
  1. The write path requires multiple LLM calls per message. Have you evaluated write throughput and latency in long conversation settings? Is this level of write overhead feasible in high-concurrency production environments?
  2. The paper does not include ablation studies of core modules. Among the observed accuracy gains, what is the respective contribution from temporal grounding, entity deduplication, and relation expansion? Which module is the primary driver of the improvement?
  3. Why did you not compare against other lightweight graph augmentation approaches beyond simply widening the host’s retrieval width? For example, entity-level association completion built on top of the host’s top-k results. How would the performance and efficiency of such approaches compare to GRAPHCARD?
  4. The current graph retrieval is limited to 1-hop neighbor expansion. Is this choice driven by performance constraints, or would multi-hop traversal not deliver further accuracy gains? Are there experimental results for 2-hop or higher expansion?
3 comments

We would like to thank the reviewer for the detailed and constructive comments. Unless noted, the new experiments below use the Luna group on all 1,540 LoCoMo questions.

W1. All underlying modules ... are established techniques ... Essentially, it tailors existing graph memory capabilities into an add-on component.

Our work uses established modules, but it is not a repackaging of an existing graph memory. Existing graph memories are built into their own stack (Mem0^g extends Mem0's own memories; Graphiti and A-MEM are complete systems), and in our re-evaluation they score 67.9 to 73.2 on LoCoMo, below the plain vector hosts they aim to improve. None of them is designed to attach to another host. GraphCard is designed differently: a deliberately simple graph memory with three careful design choices (entity deduplication, temporal fact management, and direct query matching with a single query embedding and no LLM call), built only from the raw conversation, independent of the host, and bounded to a card of about 150 tokens. To our knowledge, such a design has not been validated before, and it improves every one of the 16 host configurations in the paper.

W2 / Q4. Only 1-hop neighbor expansion ... Are there experimental results for 2-hop or higher expansion?

The choice of 1-hop is driven by both accuracy and latency. We ran 2-hop (Table 9, Appendix F of the revision):

SettingRAG + cardMem0 + cardSearch p50Search p95
1-hop (paper)89.491.60.64 s0.92 s
2-hop89.091.61.32 s3.79 s

2-hop does not improve accuracy and doubles the p50 latency. Since deeper expansion brings no gain, we keep the simpler design and do not add community or centrality signals. The hop count remains a configurable option: in other settings, for example data with longer relation chains, deeper expansion may bring gains.

W3. No baselines of vector plus lightweight graph augmentation, no fair comparison against widening the host, no component ablation.

Vector plus graph. We tested four such combinations, and none of them reaches GraphCard at a comparable context budget:

  • Existing graph memories (Table 1 of the paper): Mem0^g 67.9, Graphiti 73.2, A-MEM 69.7, against 89.4 to 92.3 for the four hosts with the card.
  • Entity-level completion on top of the host's top-k, as suggested in Q3: 85.6 on RAG against 89.4 for GraphCard (details in Q3).
  • Mem0's own entity linking. Our Mem0 host (mem0ai 2.0.18) links entities at write time and boosts candidates by entity similarity. Switching this boost off does not change Mem0's accuracy (89.2 vs. 89.2), and the card improves Mem0 either way.
  • LightRAG (lightrag-hku 1.5.7, recommended mix mode, same LLM and embeddings; Appendix H of the revision):
SystemContext tokens (median)LLM calls per queryOverallTemporal
LightRAG, default budget28,637194.392.8
LightRAG, 1/5 context budget4,791184.568.2
LightRAG, 1/13 budget1,010169.649.5
LightRAG, 1/19 budget540158.843.0
LightRAG (1/5 budget) + GraphCard~4,940191.391.9
GraphCard alone144083.184.7
Mem0 + GraphCard529091.694.7
Full context (reference)22,890095.096.0

With its default budget LightRAG returns a median of 28,637 tokens, more than the whole conversation (22,890 tokens for the full-context reference), so its 94.3 is effectively a full-context result. Under budgets comparable to ours (we scale its three official budget parameters; same indexes, no truncation by us), it scores 69.6 at about 1,000 tokens and 58.8 at about 540 tokens, far below Mem0 + card at 529 tokens (91.6).

Widening the host. Section 4.4 and Figure 3 of the paper already compare the card with widening Mem0 from top-10 to top-50. To make the comparison stricter, we additionally give each host the same reranker as the card (Cohere Rerank 3.5 over the host's top-50 list) and then widen it (accuracy, with median context tokens in parentheses; Appendix G of the revision):

HostReranked top-10Reranked top-20Reranked top-30Reranked top-10 + card
RAG87.1 (493)87.4 (917)88.6 (1,324)91.2 (642)
Nemori92.1 (2,192)92.7 (3,519)92.4 (4,598)93.4 (2,351)
SimpleMem90.8 (348)91.8 (686)92.1 (1,009)92.6 (497)
Mem090.8 (379)91.2 (746)n/a*92.3 (527)

* Mem0's stored list has up to 20 items.

On every host, the reranked top-10 plus the card is more accurate than the reranked top-20 or top-30, with fewer tokens. Even a reranked and widened host does not match adding the card.

Component ablation. Please see Q2.

Due to the character limitation, we continue to answer the remaining questions here, and this is Part 2/2.

W4 / Q1. Write-path cost; have you evaluated write throughput and latency? Is this feasible in production?

SystemLLM calls / messageInput tokensOutput tokens
GraphCard, per-message writes (paper)9.202.04M0.66M
GraphCard, per-session writes (same prompts)2.980.53M0.20M
Mem01.003.59M0.08M
Nemori1.000.42M0.15M
SimpleMem0.030.03M0.04M

Measured on LoCoMo conversation 0 (419 messages), with the hosts measured on the same conversation (Appendix I of the revision). We tested two write configurations:

  • Per message (paper): 9.2 LLM calls per message (2 for extraction, plus on average 2.7 deduplication and 4.5 consolidation decisions), about 13 s per message. A full LoCoMo build (5,882 messages, ten conversations in parallel) takes 2 h 27 min.
  • Per session: the whole session is given to the same prompts in one call. This needs 2.98 calls per message and about a third of the tokens, and accuracy stays essentially unchanged (per-session vs. per-message: RAG + card 90.9 vs. 89.4; Mem0 + card 92.7 vs. 91.6; Nemori + card 90.9 vs. 91.9; SimpleMem + card 91.4 vs. 92.3).

Writes run in the background and never block the host: the host answers from its own memory as before, and the card catches up asynchronously. On the trade-off against retrieving more from the host: the write cost is paid once per message, while the saving recurs on every query (about 150 added context tokens for the card, against about 720 for widening Mem0 to the same accuracy). Counting tokens only, the two curves cross at about 31 queries per ingested message with per-message writes and about 9 with per-session writes.

Q2. What is the respective contribution from temporal grounding, entity deduplication, and relation expansion?

We added the component ablation to the revision (Section 4.5). Leave-one-out ablation on the RAG host (RAG alone 77.1):

Variant of the cardOverallΔ vs. full card
Full card (graph traversal + semantic channel, reranked)89.4
Graph traversal only (no semantic channel)86.2−3.2
Semantic channel only (no traversal)88.5−0.9
No reranker (cosine ordering)86.4−3.0
No write-time date grounding86.4−3.0
No LLM entity deduplication89.3−0.1

Temporal grounding contributes 3.0 points (6.5 on temporal questions). Entity deduplication is designed to reduce storage and retrieval load (the entity set is about 20% smaller), not to raise accuracy. For relation expansion, the default card pairs traversal with a 50-statement semantic channel; with a smaller semantic channel, relation expansion contributes much more:

Semantic channel sizeWith traversalWithout traversalΔ
top-1088.884.9−3.9
top-2089.486.7−2.7
top-50 (default)89.488.5−0.9

At top-10, the multi-hop accuracy drops from 86.9 to 78.4 without relation expansion. Relation expansion alone, without the semantic channel, lifts RAG from 77.1 to 86.2.

Q3. Why not compare against ... entity-level association completion built on top of the host's top-k results?

We implemented this method and added it to the revision (Appendix H): take the host's top-10 items, match their text against the entity index, keep the 5 best entities, expand one hop from them in the same graph, rerank the pooled statements with the same reranker, and prepend the top 10 as a card.

HostHost alone+ host-anchored completion+ GraphCard
RAG77.185.689.4
Mem089.590.591.6

GraphCard is 3.8 points higher on RAG and 1.1 higher on Mem0.

Thank the author for their efforts. I carefully read the content of the rebuttal. After comprehensive consideration, I think this paper does not meet the standards of ICLR. Therefore, I will maintain the original score.

Reviewer_VtMJOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

GraphCard proposes a framework-agnostic graph memory plug-in that augments existing conversational memory systems without modifying their retrieval pipelines. It independently constructs a temporally grounded fact graph and retrieves a compact set of facts to complement the host's memory. Experiments across multiple memory systems and two benchmarks show consistent overall improvements, particularly on multi-hop and temporal questions.

Critical strengths
  1. Practical and generalizable design. The framework-agnostic approach is a valuable contribution, allowing graph-based memory to complement existing systems without replacing their retrieval mechanisms.
  2. Consistent empirical improvements. Results across several hosts demonstrate the practical effectiveness of the augmentation strategy.
Critical weaknesses
  1. Limited methodological novelty. While the framework-agnostic integration is useful, the underlying components, including fact extraction, entity resolution, graph expansion, reranking and the use of knowledge graphs for RAG in general, are largely established techniques. The paper could better distinguish its architectural contribution from related approaches.
  2. Missing ablation isolating the contribution of graph structure. GraphCard combines two retrieval channels: semantic retrieval over graph-edge statements and graph traversal. The candidates from both channels are pooled, reranked, and the top 10 facts are added to the host's original top 10 retrieved memories. However, the paper does not include an ablation that disables graph traversal while retaining semantic retrieval over the same extracted facts. Consequently, it is unclear whether the reported improvements stem from the graph structure itself or simply from retrieving additional relevant facts using a separate semantic retrieval and reranking pipeline.
  3. Incomplete efficiency evaluation. The reported retrieval latency and token efficiency are encouraging, but the evaluation does not account for the additional LLM costs of constructing and maintaining the graph.
Questions for authors
  1. How much improvement remains when graph expansion is disabled?

  2. What are the ingestion and maintenance costs of the graph memory? Given the additional ingestion and maintenance costs, is maintaining an explicit knowledge graph justified by the observed accuracy gains?

2 comments

We thank the reviewer for the valuable comments. Unless noted, new experiments use the Luna group on all 1,540 LoCoMo questions.

W1. Limited methodological novelty. ... the underlying components ... are largely established techniques.

Our work builds on established components, but this does not make the contribution trivial. The claim of the paper is that GraphCard, a deliberately simple graph memory with three careful design choices (entity deduplication, temporal fact management, and direct query matching with a single query embedding and no LLM call), outperforms the more elaborate graph memories. Under the same protocol GraphCard alone (a 10-statement card) scores 83.1 on LoCoMo, against 67.9 to 73.2 for Mem0^g, Graphiti and A-MEM. In addition, unlike existing graph memories, it is designed as a plug-in: it reads only the raw conversation, leaves the host untouched, and improves four different hosts on two benchmarks with two model groups.

W2 / Q1. Missing ablation isolating the contribution of graph structure. How much improvement remains when graph expansion is disabled?

We conducted the ablation and added it to the revision (Section 4.5, Appendix F). First, the graph card gives more accurate recall at a small size (Table 8). LoCoMo annotates the evidence turns of each question, and every statement in the graph keeps its source turn (n = 1,536):

Retrieved setEvidence recallMulti-hopTemporal
RAG top-1063.834.973.1
RAG top-2071.243.080.2
RAG top-5080.957.885.0
Card alone (10 statements, ~150 tokens)73.454.287.2
RAG top-10 + card82.560.991.4

Ten statements from the graph cover more gold evidence than the host's top-20, and the host's top-10 plus the card covers more than the host's top-50. Organizing memory as entities, relations and dated statements is what makes such a compact card possible.

Second, the leave-one-out ablation of the card (RAG host, 77.1 alone; Table 3):

Variant of the cardOverallΔ vs. full card
Full card (graph traversal + semantic channel, reranked)89.4
Graph traversal only (no semantic channel)86.2−3.2
Semantic channel only (no traversal)88.5−0.9
No reranker (cosine ordering)86.4−3.0
No write-time date grounding86.4−3.0
No LLM entity deduplication89.3−0.1
2-hop traversal89.0−0.4

Entity deduplication is designed to reduce storage and retrieval load (it keeps the entity set about 20% smaller), not to raise accuracy, so a difference of 0.1 is expected.

Third, the effect of graph expansion depends on how wide the semantic channel is. The default card uses a 50-statement semantic channel next to traversal; we varied its size and disabled graph expansion at each size (Table 4):

Semantic channel sizeWith traversalWithout traversalΔ
top-1088.884.9−3.9
top-2089.486.7−2.7
top-50 (default)89.488.5−0.9

With a top-10 semantic channel, disabling graph expansion costs 3.9 points overall and 8.5 points on multi-hop questions (86.9 vs. 78.4). Graph expansion alone, without any semantic channel, already lifts RAG from 77.1 to 86.2.

W3 / Q2. What are the ingestion and maintenance costs? Is maintaining an explicit knowledge graph justified?

We measured every model call and token on LoCoMo conversation 0 (419 messages), for GraphCard and for the hosts (Appendix I of the revision):

SystemLLM calls / messageInput tokensOutput tokens
GraphCard, per-message writes (paper)9.202.04M0.66M
GraphCard, per-session writes (same prompts)2.980.53M0.20M
Mem01.003.59M0.08M
Nemori1.000.42M0.15M
SimpleMem0.030.03M0.04M

Per-message writing makes 9.2 LLM calls per message (2 for extraction, plus on average 2.7 deduplication and 4.5 consolidation decisions) and runs in the background, about 13 s per message; a full LoCoMo build (5,882 messages) takes 2 h 27 min. We also tested a more economical configuration: writing once per session instead of once per message (the whole session is given to the same prompts in one call). It needs about a third of the calls and tokens, and accuracy stays essentially unchanged (per-session vs. per-message: RAG + card 90.9 vs. 89.4; Mem0 + card 92.7 vs. 91.6; Nemori + card 90.9 vs. 91.9; SimpleMem + card 91.4 vs. 92.3).

We feel this cost is reasonable. It is paid once at write time, while every later query benefits: the card adds about 150 context tokens, whereas widening the host to the same accuracy adds about 720 (Section 4.4). Counting tokens only, the write cost is recovered after about 31 queries per ingested message with per-message writes and about 9 with per-session writes. More importantly, the graph buys accuracy that the host does not reach by retrieving more (the recall table above).

Thank you for the additional experiments. However, my main concern remains. The key motivation for using a graph is to exploit its connections, yet your ablation shows only a 0.9 percentage-point improvement from graph traversal under the default setting, with no confidence intervals reported for this gain. I acknowledge that graph traversal provides larger gains with smaller semantic retrieval budgets, particularly on multi-hop questions. However given the increasingly large context windows supported by modern LLMs, it is unclear whether restricting semantic retrieval to such small candidate sets is a practically relevant setting. Evaluating these gains on a benchmark specifically designed for multi-hop reasoning would provide more convincing evidence of the benefits of graph traversal. The default results suggest that most of the improvement can be achieved through semantic retrieval and reranking over temporally grounded, extracted facts, without explicitly exploiting graph connectivity.

I am also still unconvinced by the methodological novelty of the three proposed design choices: entity deduplication, temporal fact management, and direct query matching without LLM calls, as these are largely established techniques that are already employed, in one form or another, in many existing RAG and GraphRAG systems.

Reviewer_wxgxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes GraphCard, a graph memory plug-in that runs alongside an existing vector based conversational memory system. GraphCard reads the raw conversation stream and builds a fact graph on ingestion. It uses two LLM calls to extract entities and candidate edges. At query time, it gathers candidates from a hybrid semantic search over statements and a bounded one-hop traversal from matched entities without any LLM call. A rerank model then selects 10 statements which are prepended to the host’s retrieved list as a 150ish token graph card.

Critical strengths

The proposed solution is simple yet effective. Adding GraphCard is only the change on top of the existing system.

Critical weaknesses
  1. What is the difference of GraphCard from HybridRAG [1] and LightRAG [2]? The proposed method seems a lot similar to HybridRAG.

  2. How much is the cost for ingestion? Since each message requires two extraction calls for graph card, in addition to the host’s own ingestion, I think there should be cost analysis to insist practicality.

  3. It is unclear whether the gains come from the graph structure itself or from the rerank model. GraphCard pipeline combines three components (extracted statements, graph traversal, rerank model) while host lists are not reranked. The traversal fetches up to 500 one-hop edges per matched entity. So the traversal may simply collect a large number of related facts and let the rerank model choose. Ablation studies such as GraphCard using only the semantic channel with rerank model or the host with the same rerank model applied would clarify the contribution of each component.

[1] Sarmah et al, HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction

[2] Guo et al, LightRAG: Simple and Fast Retrieval-Augmented Generation

Questions for authors

Please see section Critical Weaknesses.

3 comments

We would like to thank the reviewer for the helpful comments and references. Unless noted, the new experiments below use the Luna group on all 1,540 LoCoMo questions.

W1. What is the difference of GraphCard from HybridRAG and LightRAG?

Thanks for pointing out these works; we have added a discussion of both to the related work of the revision (Section 2.2). The essential difference is that HybridRAG and LightRAG are RAG systems over a static document corpus, while GraphCard is a memory for an ongoing conversation:

  1. What is modeled. HybridRAG stores short triples from documents without time; LightRAG merges entities by name and does not model when something was said. A conversation requires utterance-relative time ("last week" resolved against the message timestamp), speaker attribution, and facts that are revised later. GraphCard handles all three at write time.
  2. Read path. Both call an LLM per query (a graph QA chain; keyword extraction). GraphCard matches the question directly against the memories, with no LLM call.
  3. Bounded context. HybridRAG concatenates all depth-1 triples without ranking, and reports that context precision drops from 0.84 to 0.79 when the graph context is added. GraphCard emits 10 reranked statements, about 150 tokens.
  4. Plug-in. Both are complete RAG systems that replace the retrieval stack. GraphCard attaches to an existing memory system without changing it, and is validated on four hosts.

We also ran LightRAG (lightrag-hku 1.5.7, recommended mix mode, same LLM and embeddings; Appendix H of the revision). HybridRAG has no public implementation that we could find; the entity-completion baseline in our response to Reviewer msL5 (Q3) is the closest control for "vector context plus depth-1 graph facts", and GraphCard outperforms it.

SystemContext tokens (median)LLM calls per queryOverallTemporal
LightRAG, default budget28,637194.392.8
LightRAG, 1/5 context budget4,791184.568.2
LightRAG, 1/13 budget1,010169.649.5
LightRAG, 1/19 budget540158.843.0
LightRAG (1/5 budget) + GraphCard~4,940191.391.9
GraphCard alone144083.184.7
Mem0 + GraphCard529091.694.7
Full context (reference)22,890095.096.0

With its default budget LightRAG returns a median of 28,637 tokens, more than the whole conversation (22,890 tokens for the full-context reference), so its 94.3 is effectively a full-context result (95.0). Under budgets comparable to ours (we scale its three official budget parameters; same indexes, no truncation by us), it scores 69.6 at about 1,000 tokens and 58.8 at about 540 tokens, far below Mem0 + card at 529 tokens (91.6) and even the card alone at 144 tokens (83.1). The card also complements LightRAG: prepended to the 4.8k-token configuration it adds 6.8 points (84.5 to 91.3), mostly on temporal questions (68.2 to 91.9).

W2. How much is the cost for ingestion?

SystemLLM calls / messageInput tokensOutput tokens
GraphCard, per-message writes (paper)9.202.04M0.66M
GraphCard, per-session writes (same prompts)2.980.53M0.20M
Mem01.003.59M0.08M
Nemori1.000.42M0.15M
SimpleMem0.030.03M0.04M

Measured on one 419-message LoCoMo conversation, counting every model call and token (Appendix I of the revision). Per-message writing makes 9.2 LLM calls per message (2 for extraction, plus on average 2.7 deduplication and 4.5 consolidation decisions). We also tested a more economical configuration: writing once per session instead of once per message (the whole session is given to the same prompts in one call). It needs about a third of the calls and tokens, and accuracy stays essentially unchanged (per-session vs. per-message: RAG + card 90.9 vs. 89.4; Mem0 + card 92.7 vs. 91.6; Nemori + card 90.9 vs. 91.9; SimpleMem + card 91.4 vs. 92.3). We feel this cost is practical: it is paid once at write time and in the background, while every later query benefits. Counting tokens only, the write cost is recovered against widening the host after about 31 queries per ingested message with per-message writes and about 9 with per-session writes.

Due to the character limitation, we continue to answer the remaining questions here, and this is Part 2/2.

W3. It is unclear whether the gains come from the graph structure itself or from the rerank model.

We ran both suggested ablations and added them to the revision (Appendix G); the gain is not explained by the rerank model.

The card without any rerank model (candidates ordered by cosine similarity):

HostHost alone+ card without rerankerΔ+ full card
RAG77.186.4+9.389.4
Nemori86.291.1+4.991.9
SimpleMem89.090.3+1.392.3
Mem089.590.5+1.091.6

Accuracy rises on every host without the rerank model. On RAG, 9.3 of the 12.3 points do not depend on it.

The host with the same rerank model (Cohere Rerank 3.5 over the host's top-50 list, top-10 kept; Mem0's stored list has up to 20 items):

LoCoMoHostHost + rerankerHost + reranker + cardΔ from card
Luna, RAG77.187.191.2+4.1
Luna, Nemori86.292.193.4+1.3
Luna, SimpleMem89.090.892.6+1.8
Luna, Mem089.590.892.3+1.5
Haiku, RAG75.586.590.6+4.1
Haiku, Nemori82.990.092.1+2.1
Haiku, SimpleMem84.587.290.7+3.5
Haiku, Mem086.688.290.9+2.7

With the same rerank model applied to the host, the card still improves every host in both model groups (all eight gains are significant after Holm correction). Widening the reranked host does not close the gap either: reranked RAG reaches 88.6 at top-30 (1,324 tokens), against 91.2 for reranked top-10 plus the card (642 tokens), and the same holds on the other three hosts (table in our response to Reviewer msL5, W3).

GraphCard using only the semantic channel (Section 4.5 of the revision). This depends on the semantic budget:

Semantic channel sizeWith traversalWithout traversalΔ
top-1088.884.9−3.9
top-2089.486.7−2.7
top-50 (default)89.488.5−0.9

With a top-10 semantic channel, the semantic-only card is 3.9 points lower (multi-hop 86.9 vs. 78.4). Traversal alone, without any semantic channel, lifts RAG from 77.1 to 86.2. So traversal does not merely collect a large pool for the rerank model: it reaches statements that a small semantic search misses.

Thank you for your detailed answers and additional experiments. While LightRAG comparison and the reranker experiments clarify that the gains are not merely from the reranker, the new ablations raise doubts about the necessity of the graph structure itself with 0.9% difference as reviewer VtMJ mentioned. Given that substantial ingestion overhead of 9.2 LLM calls per message, it is hard to justify building and maintaining an explicit graph for less than 1% gain.

Comments on the paper

1 comment

Dear reviewers:

We thank all the reviewers for positive feedback and constructive criticism. We greatly appreciate the time and effort to help us improve our work.

Overall, we have provided corresponding responses, and we believe we were able to address the reviewers' concerns. We look forward to engaging with you during the further discussion period.

We have also uploaded a revised version of the paper, in which all changes are marked in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.