- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses a key limitation of prevailing long-term conversational memory systems, which rely predominantly on vector-based retrieval and fail to capture relational connections and temporal structure across entries. The authors propose GRAPHCARD, a plug-in graph memory enhancement module. Instead of replacing the existing vector memory host system, GRAPHCARD builds a temporally grounded fact graph in parallel. At query time, it uses a single query embedding combined with bounded graph traversal to generate a structured graph card of roughly 150 tokens, which is prepended to the host’s retrieval result list to supplement the relational layer missing from vector retrieval.
Critical strengths
- Unlike existing graph memory systems that replace the existing retrieval pipeline and require rewriting the entire memory stack, GRAPHCARD follows a plug-in design with zero modification to the host system, parallel augmentation, and output attributability. On the input side, it only reads the raw conversation stream. On the output side, it only prepends a standardized summary card. This makes it minimally invasive to existing vector memory systems and easily adaptable to different hosts.
- The evaluation spans two long conversation benchmarks, two mainstream model architectures, and four representative vector memory hosts. It systematically verifies multiple dimensions including accuracy improvement, gain distribution across question types, token efficiency, the effect of the temporal module, and retrieval latency.
Critical weaknesses
- All underlying modules, including entity extraction, relation extraction, entity deduplication, temporal grounding, and hybrid retrieval, are established techniques from prior work. The paper does not propose new principles of graph memory modeling, novel retrieval mechanisms, or new structured reasoning paradigms. Essentially, it tailors existing graph memory capabilities into an add-on component that integrates with vector systems.
- Despite the graph card framing, the actual graph retrieval logic only includes semantic matching and 1-hop neighbor expansion. It does not involve multi-hop reasoning traversal, nor does it leverage any graph topological properties such as community structure, node centrality, path connectivity, or causal chains.
- The paper only compares against graph-only memory systems and vector-only hosts. It does not include comparable baselines of vector plus lightweight graph augmentation, nor does it run a fair efficiency comparison against approaches that widen the host’s own retrieval width. Additionally, the paper does not decompose the sources of improvement through component ablation.
- The paper only reports query-time latency, 0.61 seconds at p50, but does not evaluate the computational cost of the write path at all. Each incoming message requires two LLM calls for entity and relation extraction, plus additional LLM judgments for entity deduplication and fact consolidation. In high-throughput long conversation scenarios, it remains an open question whether the write latency, cost, and resource consumption are acceptable.
Questions for authors
- The write path requires multiple LLM calls per message. Have you evaluated write throughput and latency in long conversation settings? Is this level of write overhead feasible in high-concurrency production environments?
- The paper does not include ablation studies of core modules. Among the observed accuracy gains, what is the respective contribution from temporal grounding, entity deduplication, and relation expansion? Which module is the primary driver of the improvement?
- Why did you not compare against other lightweight graph augmentation approaches beyond simply widening the host’s retrieval width? For example, entity-level association completion built on top of the host’s top-k results. How would the performance and efficiency of such approaches compare to GRAPHCARD?
- The current graph retrieval is limited to 1-hop neighbor expansion. Is this choice driven by performance constraints, or would multi-hop traversal not deliver further accuracy gains? Are there experimental results for 2-hop or higher expansion?
3 comments
Part 2/2
Due to the character limitation, we continue to answer the remaining questions here, and this is Part 2/2.
W4 / Q1. Write-path cost; have you evaluated write throughput and latency? Is this feasible in production?
| System | LLM calls / message | Input tokens | Output tokens |
|---|---|---|---|
| GraphCard, per-message writes (paper) | 9.20 | 2.04M | 0.66M |
| GraphCard, per-session writes (same prompts) | 2.98 | 0.53M | 0.20M |
| Mem0 | 1.00 | 3.59M | 0.08M |
| Nemori | 1.00 | 0.42M | 0.15M |
| SimpleMem | 0.03 | 0.03M | 0.04M |
Measured on LoCoMo conversation 0 (419 messages), with the hosts measured on the same conversation (Appendix I of the revision). We tested two write configurations:
- Per message (paper): 9.2 LLM calls per message (2 for extraction, plus on average 2.7 deduplication and 4.5 consolidation decisions), about 13 s per message. A full LoCoMo build (5,882 messages, ten conversations in parallel) takes 2 h 27 min.
- Per session: the whole session is given to the same prompts in one call. This needs 2.98 calls per message and about a third of the tokens, and accuracy stays essentially unchanged (per-session vs. per-message: RAG + card 90.9 vs. 89.4; Mem0 + card 92.7 vs. 91.6; Nemori + card 90.9 vs. 91.9; SimpleMem + card 91.4 vs. 92.3).
Writes run in the background and never block the host: the host answers from its own memory as before, and the card catches up asynchronously. On the trade-off against retrieving more from the host: the write cost is paid once per message, while the saving recurs on every query (about 150 added context tokens for the card, against about 720 for widening Mem0 to the same accuracy). Counting tokens only, the two curves cross at about 31 queries per ingested message with per-message writes and about 9 with per-session writes.
Q2. What is the respective contribution from temporal grounding, entity deduplication, and relation expansion?
We added the component ablation to the revision (Section 4.5). Leave-one-out ablation on the RAG host (RAG alone 77.1):
| Variant of the card | Overall | Δ vs. full card |
|---|---|---|
| Full card (graph traversal + semantic channel, reranked) | 89.4 | |
| Graph traversal only (no semantic channel) | 86.2 | −3.2 |
| Semantic channel only (no traversal) | 88.5 | −0.9 |
| No reranker (cosine ordering) | 86.4 | −3.0 |
| No write-time date grounding | 86.4 | −3.0 |
| No LLM entity deduplication | 89.3 | −0.1 |
Temporal grounding contributes 3.0 points (6.5 on temporal questions). Entity deduplication is designed to reduce storage and retrieval load (the entity set is about 20% smaller), not to raise accuracy. For relation expansion, the default card pairs traversal with a 50-statement semantic channel; with a smaller semantic channel, relation expansion contributes much more:
| Semantic channel size | With traversal | Without traversal | Δ |
|---|---|---|---|
| top-10 | 88.8 | 84.9 | −3.9 |
| top-20 | 89.4 | 86.7 | −2.7 |
| top-50 (default) | 89.4 | 88.5 | −0.9 |
At top-10, the multi-hop accuracy drops from 86.9 to 78.4 without relation expansion. Relation expansion alone, without the semantic channel, lifts RAG from 77.1 to 86.2.
Q3. Why not compare against ... entity-level association completion built on top of the host's top-k results?
We implemented this method and added it to the revision (Appendix H): take the host's top-10 items, match their text against the entity index, keep the 5 best entities, expand one hop from them in the same graph, rerank the pooled statements with the same reranker, and prepend the top 10 as a card.
| Host | Host alone | + host-anchored completion | + GraphCard |
|---|---|---|---|
| RAG | 77.1 | 85.6 | 89.4 |
| Mem0 | 89.5 | 90.5 | 91.6 |
GraphCard is 3.8 points higher on RAG and 1.1 higher on Mem0.
Thank the author for their efforts. I carefully read the content of the rebuttal. After comprehensive consideration, I think this paper does not meet the standards of ICLR. Therefore, I will maintain the original score.
Part 1/2
We would like to thank the reviewer for the detailed and constructive comments. Unless noted, the new experiments below use the Luna group on all 1,540 LoCoMo questions.
Our work uses established modules, but it is not a repackaging of an existing graph memory. Existing graph memories are built into their own stack (Mem0^g extends Mem0's own memories; Graphiti and A-MEM are complete systems), and in our re-evaluation they score 67.9 to 73.2 on LoCoMo, below the plain vector hosts they aim to improve. None of them is designed to attach to another host. GraphCard is designed differently: a deliberately simple graph memory with three careful design choices (entity deduplication, temporal fact management, and direct query matching with a single query embedding and no LLM call), built only from the raw conversation, independent of the host, and bounded to a card of about 150 tokens. To our knowledge, such a design has not been validated before, and it improves every one of the 16 host configurations in the paper.
The choice of 1-hop is driven by both accuracy and latency. We ran 2-hop (Table 9, Appendix F of the revision):
2-hop does not improve accuracy and doubles the p50 latency. Since deeper expansion brings no gain, we keep the simpler design and do not add community or centrality signals. The hop count remains a configurable option: in other settings, for example data with longer relation chains, deeper expansion may bring gains.
Vector plus graph. We tested four such combinations, and none of them reaches GraphCard at a comparable context budget:
mixmode, same LLM and embeddings; Appendix H of the revision):With its default budget LightRAG returns a median of 28,637 tokens, more than the whole conversation (22,890 tokens for the full-context reference), so its 94.3 is effectively a full-context result. Under budgets comparable to ours (we scale its three official budget parameters; same indexes, no truncation by us), it scores 69.6 at about 1,000 tokens and 58.8 at about 540 tokens, far below Mem0 + card at 529 tokens (91.6).
Widening the host. Section 4.4 and Figure 3 of the paper already compare the card with widening Mem0 from top-10 to top-50. To make the comparison stricter, we additionally give each host the same reranker as the card (Cohere Rerank 3.5 over the host's top-50 list) and then widen it (accuracy, with median context tokens in parentheses; Appendix G of the revision):
* Mem0's stored list has up to 20 items.
On every host, the reranked top-10 plus the card is more accurate than the reranked top-20 or top-30, with fewer tokens. Even a reranked and widened host does not match adding the card.
Component ablation. Please see Q2.