- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Lumi-Mem, a hippocampus–neocortex-inspired memory framework for LLM agents that aims to improve retrieval of relevant information across long-term interactions. In detail, this method combines a Sparse Attractor (SA) module for associative topic-level indexing with a Dense Representation (DR) module for fine-grained evidence retrieval. Bidirectional sparse–dense interactions let topic do activation, evidence-based calibration, and adaptive fusion of retrieval scores. Experiments on LongMemEval, 2WikiMultiHopQA, and MuSiQue demonstrate improvements in retrieval and Q-and-A performance over several existing baselines. Ablation studies further suggest that both memory components and their interaction mechanisms contribute to the overall performance.
Critical strengths
1.Memory is an important topic in LLM. The paper focuses on a well motivated problem. It is an important challenge of long-term memory retrieval for LLM agents, particularly when relevant information is distributed across multiple interactions and requires associative reasoning.
-
The authors considered a well motivated method from human brain. The proposed Lumi-Mem framework combines sparse associative indexing with dense evidence retrieval through bidirectional interactions. The overall architecture is intuitive and provides a coherent approach to memory organization and retrieval.
-
Strong empirical results: The method achieves competitive or superior retrieval and QA performance across three benchmarks. The ablation studies also provide useful evidence that the sparse and dense components, as well as their interaction mechanisms, contribute to the overall improvements.
Critical weaknesses
The main weakness I think is in the design and experiment.
-
I think in its current form, the paper makes me not very easy to see what position the proposed method is in among some related/existing framework. Although the paper discusses a number of related methods, I find the positioning of Lumi-Mem relative to existing graph-based, associative, and hierarchical memory approaches insufficiently clear. In particular, it would be helpful to explain more explicitly what distinguishes the proposed sparse–dense interaction mechanism from existing approaches such as HippoRAG-v2 and other memory-enhanced retrieval frameworks.
-
why Lumi-Mem, BKRAG, and GFM-RAG are trained on the same 80/20 query split, while the other baselines are evaluated in their original settings? Consequently, I am not fully convinced that the reported performance gains can be attributed solely to the proposed memory mechanism.
-
Limited evaluation of long-term memory dynamics: Although the paper emphasizes memory self-organization, the memory bank is constructed offline. It doesn't consider continuous memory updates, evolving user information, or long-term interactions in a deployed agent, which is more practical.
-
Insufficient analysis of efficiency and scalability: The proposed framework introduces multiple stages of memory construction, associative propagation, calibration, and retrieval. A more detailed comparison of end-to-end computational costs, memory maintenance overhead, and scalability would help establish whether the performance improvements justify the additional complexity.
Questions for authors
Please refer to the questions in Weakness 1 and 2.
4 comments
W3. Long-term memory dynamics
Thank you for raising this point. To test continuous updates you mentioned, we have added a streaming evaluation, including 78 update cases from LongMemEval and 75 from LoCoMo (a newly added long-term conversational memory benchmark). In each case, a piece of user information changes during the conversation. We replay the conversation in timestamp order, and each query retrieves only from what has already been written. We compare our incremental merging (Appendix D) with an append-only policy, which stores every memory item as a new topic. We report how often Lumi-Mem retrieves the updated information (R@5), and how often its answer still gives the outdated value.
With incremental merging, Lumi-Mem retrieves the updated information in 98.6% of LongMemEval cases and 81.3% of LoCoMo cases (Table C). Compared with appending, merging slightly improves retrieval and roughly halves the number of topics. Evaluating Lumi-Mem over longer interactions in a deployed agent is a direction for future work.
Table C. Retrieval under streaming memory updates.
| Dataset | Write policy | R@5 (%) | Old-value answers (%) | Mean topics | Write time per update (s) |
|---|---|---|---|---|---|
| LongMemEval | Append only | 97.2 | 12.3 | 215.71 | 5.93 |
| Incremental merge | 98.6 | 11.8 | 116.44 | 18.60 | |
| LoCoMo | Append only | 78.7 | 4.2 | 120.88 | 4.57 |
| Incremental merge | 81.3 | 4.2 | 50.67 | 11.97 |
Revised contents: Please see p. 10 (lines 513–514) and pp. 26–27 (lines 1396–1411).
W4. Efficiency and scalability
Thank you for this suggestion. Lumi-Mem's cost is comparable to that of the baselines, while its accuracy is higher. We have compared the end-to-end cost of Lumi-Mem with HippoRAG-v2, BKRAG, and GFM-RAG on all three datasets using the same device (Table D).
- Memory construction (one-time, offline). Lumi-Mem builds its memory bank in about half the time of HippoRAG-v2 and GFM-RAG. Only BKRAG builds faster.
- Training (one-time). Lumi-Mem trains in 19–44 minutes, similar to BKRAG and GFM-RAG. HippoRAG-v2 needs no training.
- Retrieval (per query). Lumi-Mem is faster than BKRAG and GFM-RAG, but slower than HippoRAG-v2. At this speed, it still reaches the highest R@5 of all methods (Table 1).
- Memory maintenance overhead. Lumi-Mem stores a retrieval index of at most 535.2 MiB, similar in size to HippoRAG-v2's. Reducing this maintenance cost is a direction for future work.
Table D. Construction, training, retrieval, and storage cost of Lumi-Mem and baselines.
| LongMemEval | 2WikiMultiHopQA | MuSiQue | |
|---|---|---|---|
| Construction time (h) | |||
| HippoRAG-v2 | 25.1 | 12.1 | 9.0 |
| BKRAG | 8.1 | 0.8 | 1.4 |
| GFM-RAG | 25.1 | 10.1 | 6.4 |
| Lumi-Mem | 12.5 | 5.3 | 3.5 |
| Training time (min) | |||
| HippoRAG-v2 | 0 | 0 | 0 |
| BKRAG | 47.1 | 16.9 | 21.3 |
| GFM-RAG | 21.2 | 21.1 | 26.3 |
| Lumi-Mem | 43.5 | 18.7 | 22.4 |
| Retrieval latency (ms/query) | |||
| HippoRAG-v2 | 41.3 | 41.3 | 82.8 |
| BKRAG | 130.2 | 158.3 | 206.3 |
| GFM-RAG | 63.4 | 105.6 | 155.5 |
| Lumi-Mem | 50.6 | 85.8 | 148.4 |
| Stored retrieval index (MiB) | |||
| HippoRAG-v2 | 565.3 | 268.2 | 388.3 |
| BKRAG | 542.4 | 130.4 | 154.7 |
| GFM-RAG | 236.9 | 381.4 | 713.8 |
| Lumi-Mem | 535.2 | 232.7 | 415.4 |
As for scalability, we have tested how accuracy and latency scale with bank size (Table E). As the memory bank grows three- to fourfold, Lumi-Mem's accuracy declines gradually and its latency grows about 2.5 times. HippoRAG-v2, tested on the same bank sizes, loses more R@5, and its latency grows about fourfold (Table F). Lumi-Mem keeps a higher R@5 at every bank size. Our future work will improve Lumi-Mem's robustness to distractors and its latency on larger memory banks.
Table E. Lumi-Mem retrieval accuracy and latency at four memory-bank sizes.
| Dataset | Memory nodes | Accuracy (%) | Latency (ms/query) |
|---|---|---|---|
| R@5/N@5 | Mean/p95 | ||
| LongMemEval | 15,241 | 95.92/84.26 | 20.8/34.4 |
| 30,483 | 93.42/79.50 | 34.3/53.7 | |
| 45,724 | 90.92/76.19 | 43.4/66.5 | |
| 60,966 | 90.58/74.08 | 50.6/77.7 | |
| 2WikiMultiHopQA | 15,907 | 95.50/95.14 | 34.3/51.8 |
| 34,196 | 94.63/94.38 | 56.3/80.7 | |
| 49,191 | 94.29/93.93 | 72.3/101.8 | |
| 63,559 | 93.38/93.18 | 85.8/121.1 | |
| MuSiQue | 32,993 | 70.67/67.16 | 57.2/82.5 |
| 56,947 | 69.39/65.92 | 85.0/119.6 | |
| 82,378 | 66.93/64.50 | 116.9/160.8 | |
| 106,007 | 65.54/63.90 | 148.4/205.1 |
Table F. HippoRAG-v2 retrieval accuracy and latency at four memory-bank sizes.
| Dataset | Memory nodes | Accuracy (%) | Latency (ms/query) |
|---|---|---|---|
| R@5/N@5 | Mean/p95 | ||
| LongMemEval | 15,241 | 89.50/82.66 | 10.3/20.4 |
| 30,483 | 84.42/75.23 | 20.7/37.4 | |
| 45,724 | 80.50/70.33 | 31.4/54.2 | |
| 60,966 | 80.58/67.99 | 41.3/65.6 | |
| 2WikiMultiHopQA | 15,907 | 90.88/89.57 | 9.8/16.9 |
| 34,196 | 87.33/86.63 | 22.3/33.4 | |
| 49,191 | 86.04/85.68 | 33.6/48.4 | |
| 63,559 | 85.25/85.21 | 41.3/63.7 | |
| MuSiQue | 32,993 | 67.88/67.92 | 23.0/35.9 |
| 56,947 | 65.71/65.60 | 40.1/60.3 | |
| 82,378 | 63.67/64.04 | 61.2/91.9 | |
| 106,007 | 62.63/62.31 | 82.8/118.0 |
Revised contents: Please see pp. 21–22 (lines 1126–1186).
Thank you for the detailed response and additional experiments which mostly address my previous concerns. I thus will raise the score to 3 accordingly.
We are glad that our responses have addressed your concerns, and we greatly appreciate your decision to increase the score.
Thank you for the detailed feedback. Below we clarify how Lumi-Mem relates to prior work and how the baselines were evaluated, and report experiments on streaming updates, computational cost, and scalability. The corresponding changes are highlighted in blue in the revised manuscript.
W1. Positioning relative to related frameworks
Thank you for the comment. Below, we clarify our contributions and how we differ from existing memory-enhanced retrieval frameworks.
Our contribution and position. Inspired by the interaction between hippocampus and neocortex in human memory systems, Lumi-Mem develops bidirectional sparse–dense encoding to enable adaptive memory activation and self-organization for LLM agents. Specifically,
Adaptive Memory Activation. Our sparse level, the Sparse Attractor (SA), is inspired by how hippocampal activity settles into stable patterns. Related topics within SA excite each other, and inhibition limits the spread until activation settles. A learned adapter adjusts these dynamics to each query. In contrast, existing retrievals often apply fixed rules for memory activation. HippoRAG-v2 uses fixed Personalized PageRank, GFM-RAG uses a trained but fixed-depth graph network, and HeLa-Mem spreads activation over Hebbian links using a predefined rule.
Memory Self-Organization. Lumi-Mem introduces a bidirectional sparse–dense interaction. SA's topic activation guides chunk retrieval through a learned gate (sparse-to-dense). During training, chunk scores feed back to refine SA's topic selection (dense-to-sparse). These interactions align topic activation with chunk-level evidence for better memory self-organization. Other methods use one-way retrieval pipelines: dense chunk scores are summarized over graph to activate the memory (HippoRAG-v2), while graph-derived scores are used to retrieve final chunks (GFM-RAG).
Table A summarizes the key differences between Lumi-Mem and existing hierarchical, associative, and graph-based approaches.
Table A. Design features of Lumi-Mem and related memory approaches.
Revised contents: Please see p. 3 (lines 136–139).
W2. Training split and baseline settings
Thanks for the comment.
Why the settings differ. All methods are evaluated on the same test queries. BKRAG, GFM-RAG, and Lumi-Mem are training-based; we therefore train all three on the same 80/20 train/test split for a matched comparison. BM25, HippoRAG-v2, and ComoRAG are training-free, so we directly evaluate them on the same test set.
Whether the gains come from the memory mechanism. Below, R@5 is the fraction of gold evidence found in the top five retrieved results.
The gains therefore come from the memory mechanism, not from training alone.
Table B. Description of ablation variants.
Revised contents: Please see p. 7 (lines 356–360) and p. 23 (lines 1190–1209).