acceptodds
Under review as a conference paper at ICLR 2027

MemorAnDA: Memory-Anchor-Driven Architecture for Long-Term Conversational Memory

Abstract

Answering questions about long conversations can require evidence spread across multiple events, temporal relations, and evolving states. Existing memory systems face a trade-off between reducing query-time computation and preserving historical details. Organizing evidence when each question arrives accommodates its specific information needs, but requires repeated search and selection across questions. Compressing history in advance can reduce query-time work, but must decide what to retain before future questions are known, potentially omitting details needed by later answers. We present Memory-Anchor-Driven Architecture (MemorAnDA; hereafter Memo), which makes history organization reusable without limiting future answers to compressed information. Memory anchors connect organized memories to their original records, allowing the system to use prior organization to find relevant evidence and recover the details needed for each question. This design reduces the need to repeatedly organize history at query time while preserving access to information that compression may omit. Across three matched build/reader model groups, the primary 8K configuration improves answer accuracy over the corresponding LightMem baseline by 1.40–9.80 percentage points on LongMemEval-S and 7.60–10.20 points on LoCoMo.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long-term conversational memory, Large language models, Retrieval-augmented generation, Evidence selection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kNCeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MemorAnDA, a conversational memory architecture, which separates reusable history organization from question-specific access to original evidence. A query-agnostic construction stage produces semantic, temporal, and state-transition views linked to original dialogue records through durable source IDs. At query time, direct and view-based retrieval generate candidates, and a source-aware selector assembles original records under an evidence-token budget without additional generative retrieval calls. Experiments report improvements over LightMem across 2 benchmarks and 3 matched build/reader model groups.

Critical strengths

(S1) Tables 1 and 2 report consistent overall improvements over LightMem across all six benchmark–backbone combinations. At the primary 8K configuration, gains range from 7.60% to 10.20% on LoCoMo and from 1.40 to 9.80 points on LongMemEval-S, demonstrating the practical effectiveness.

(S2) Table 12 provides an important controlled comparison beyond the main baseline results. This supports the usefulness of compiled retrieval cues on LoCoMo with GPT-4o-mini and rules out increased evidence length as the sole explanation in this setting.

(S3) The paper reflects substantial effort in system implementation and experimental evaluation. The presentation is clear and well organized, making the framework and experiments easy to follow.

Critical weaknesses

(W1) Table 3 shows higher accuracy than LightMem, but MemorAnDA also uses 2.76–3.09× the query tokens. The paper is clear about this trade-off. I would like to see both methods tested with similar reader budgets to understand how much of the gain remains.

(W2) It is hard to tell how much the proposed selector helps. Section 4.4 changes the retrieval routes but keeps the selector fixed. Would a simpler approach (scoring sources, removing duplicates, and filling the token budget) work just as well? Since the selector is listed as a main contribution, this comparison matters.

(W3) The expanded atomic control in Section 4.4 only covers GPT-4o-mini on LoCoMo. On LongMemEval, the paper reports better evidence coverage without consistent gains in answer accuracy. I would like to see the same equal-budget comparison across more backbones and both benchmarks to understand when compiled navigation helps.

(W4) What MemorAnDA adds beyond existing source-linked retrieval methods is a concern. Section 2 discusses methods that prepare retrieval cues in advance or preserve the original dialogue, but the experiments do not include a close baseline with both capabilities. Beating raw-only retrieval shows that compiled cues help, but it does not tell us what the proposed architecture adds to this existing approach.

Questions for authors

(Q1) In Section 4.3, Table 3, how does MemorAnDA compare with LightMem when both use similar reader-token budgets?

(Q2) With the same candidates and token budget, does the selector in Section 3.4, Eq (1), outperform simple source ranking, deduplication, and budgeted packing?

(Q3) Does the equal-budget Atomic-480 comparison in Section 4.4 also show gains on LongMemEval and with the other backbones?

(Q4) Section 2 discusses precomputed retrieval cues linked to source evidence. How does MemorAnDA compare with a close baseline using this design under the same reader and evidence budget?

Reviewer_3BkZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MemorAnDA, a long-term conversational memory architecture that organizes conversation histories into reusable semantic, temporal, and state-transition views while preserving the original records. Source anchors link these views back to the original evidence. At query time, MemorAnDA combines direct retrieval with view-based retrieval and selects complementary evidence under a fixed token budget. Experiments on LoCoMo and LongMemEval-S show improvements over several existing memory systems, with ablations suggesting that multi-view retrieval can complement direct retrieval.

Critical strengths
  1. Preserving access to original evidence. Rather than relying solely on compressed summaries, MemorAnDA uses organized representations to locate the original conversation records. This is a sensible design choice for multi-hop and temporal questions where seemingly minor details may be important.

  2. Promising empirical results. The method improves accuracy across three build/reader model configurations on two benchmarks. The expanded atomic-retrieval control also suggests that the gains are not simply due to retrieving more candidates.

  3. A practical budgeted selection mechanism. The method accounts for overlapping source records and the incremental evidence coverage and token cost of selecting them. Retrieval and selection require no additional query-time LLM calls, making the architecture relatively straightforward to integrate into an existing pipeline.

Critical weaknesses
  1. Limited novelty. PREMem [1], SeCom [2], and HippoRAG [3] already explore precomputed reasoning or structured associations for memory retrieval. The paper does not clearly isolate what is novel about its source-anchor mechanism and selection algorithm.

  2. Incomplete ablations. The contribution of temporal and transition views is not statistically established, and the greedy selector is not compared against a simpler budgeted ranking baseline. On LongMemEval-S, improved evidence coverage also does not translate into higher accuracy.

  3. Unclear cost trade-off. MemorAnDA uses 2.76–3.09× more query-time tokens than LightMem on LoCoMo and 7.64–10.02× more on LongMemEval-S, without analyzing when the additional construction cost is amortized.

[1] Pre-Storage Reasoning for Episodic Memory: Shifting Inference Burden to Memory for Personalized Dialogue [2] SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents [3] HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

Questions for authors
  1. How does the proposed selector compare with simple similarity-ranked selection under the same candidate pool and token budget?

  2. Why does higher evidence recall fail to improve accuracy on LongMemEval-S, and which question types account for this gap?

Reviewer_haujOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MemorAnDA (MEMO) builds reusable semantic, temporal, and state-transition views of conversation histories while retaining the original records in RawStore. Memory anchors connect these views to their sources. At query time, MEMO combines atomic and derived candidates, selects original evidence under a token budget, and invokes one reader. Experiments on LoCoMo and LongMemEval-S report accuracy gains over LightMem across three matched build/reader model groups, with additional analyses of retrieval coverage and answer errors.

Critical strengths
  1. The separation between navigation views and original evidence is well motivated. Stable source IDs make the selected evidence traceable and allow the selector to account for overlap across views. Retrieval and selection require no additional LLM calls, and admission checks enforce the budget on the complete rendered evidence packet.
  2. The Atomic-480 control provides useful evidence for mixed retrieval. On LoCoMo with GPT-4o-mini, MEMO improves accuracy by 4.35 percentage points while delivering 8,101.47 evidence tokens per question, compared with Atomic-480's 8,183.62 under the same 8,192-token evidence cap (Table 12). This supports a benefit from combining atomic and derived candidates beyond increasing the amount of text read.
  3. The paper reports answer accuracy, Evidence Recall, and All-Evidence Recall separately. Paired conversation-cluster bootstrap intervals quantify uncertainty in the LoCoMo route ablations. The error analysis also distinguishes missing annotated sources from incorrect answers despite complete annotated-source coverage, which helps identify limitations beyond retrieval.
Critical weaknesses
  1. The gains over LightMem come with substantially different token usage. On LongMemEval-S, MEMO uses 4.04–4.94 times the build tokens and 7.64–10.02 times the reader tokens; the latter ratio is 2.76–3.09 on LoCoMo (Tables 15–16). These counts include prompt and completion tokens at their respective stages. The authors disclose this trade-off, but the comparison does not establish how much advantage remains at matched reader input budgets. This matters especially for GPT-4o-mini on LongMemEval-S, where the accuracy gain is 1.40 percentage points.
  2. The dynamic source selector has not been evaluated against a simpler selection rule. Table 12 holds it fixed, and the other ablations mainly change retrieval routes. A comparison using the same candidate pool and evidence budget, with relevance ordering and source deduplication, would clarify what cluster weighting and dynamic gain recomputation contribute. Algorithm 1 recomputes each selection action's gain and incremental cost after admission, but the computational overhead of this step is not reported separately.
  3. The contribution of temporal and state-transition views remains uncertain. On LongMemEval-S with GPT-4o-mini, Full MEMO reaches 75.20% accuracy versus 75.80% for Atomic-only, despite higher Evidence Recall and All-Evidence Recall (Appendix D.4). On LoCoMo with the same model, Full MEMO exceeds the Temp./Trans. fixed mask variant by 0.84 percentage points, with a 95% confidence interval of [−0.14, 1.95] percentage points (Table 13). This variant masks both view types in a fixed candidate pool without replenishing candidates. These results leave the individual contributions of the two views, and their answer-quality benefits across datasets, unresolved.
Questions for authors
  1. How do MEMO and LightMem compare under matched reader input budgets? Can you show accuracy versus total cost as the number of queries served by each built history varies, including construction costs?
  2. With the candidate pool, reader, and delivered evidence length controlled, how does dynamic source selection compare with relevance ordering plus source deduplication in accuracy, All-Evidence Recall, and selection time?
  3. What are the separate effects of removing temporal views and state-transition views on temporal questions in both benchmarks and knowledge-update questions in LongMemEval-S? Which errors explain the higher Evidence Recall but lower accuracy in the LongMemEval-S route comparison?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.