acceptodds
Under review as a conference paper at ICLR 2027

IMPRINT: Rethinking Retrieval for Long-Term Conversational Agent Memory

Abstract

Long-term memory systems for conversational agents differ in how they organize memory but share one retrieval primitive: dense embeddings ranked by cosine similarity. We argue that this primitive, not the memory structure, is the bottleneck, because personal recall hinges on rare user-specific identifiers whose exact identity mean pooling averages away. We reframe recall as *imprint-following*, where each rare token is a trace pointing back to its source session, and instantiate it in **IMPRINT**, whose single *Token Profile Table* serves as an IDF-weighted inverted index and drives an online consolidation pool, with no retrieval training, no LLM calls at indexing, and no GPU at query time. We prove that rarity-weighted exact matching keeps a separation margin independent of the retrieval-unit length, whereas the dense margin decays as ; the same result explains why session-level sparse retrieval was dismissed by prior work. On LoCoMo (27 sessions) and LongMemEval (500 sessions), IMPRINT beats the strongest prior memory system by up to +15.8 Recall@3 and +3.9 QA-F1, outperforms dense retrieval under an identical pipeline, and produces the best answers on LoCoMo.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent, Long-term Memory, Retrieval

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_1VZrOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3(+1)
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduce IMPRINT, it uses a Token Profile Table to perform exact matching weighted by the IDF and session specificity of rare tokens, combined with online merging to compress context. It requires no training, no LLM at indexing time, and no GPU at query time, substantially improving retrieval and QA on LoCoMo/LongMemEval.

Critical strengths

1.Problem framing is accurate: rare tokens serve as retrieval signals for personal memory, the theory explains dilution from dense pooling, and the method is simple and efficient. 2.Experiments are thorough: it significantly leads on retrieval/QA across two benchmarks, with comprehensive ablations, granularity interaction, scaling, and significance testing.

Critical weaknesses

1.The theoretical model is simplified, ignoring semantic structure and anisotropy; recall drops sharply under alias/paraphrase perturbations, making exact matching fragile. 2.Generation gains are limited, and temporal/open-domain questions remain weak; it relies on English tokenization, and privacy deletion is only discussed but not evaluated.

Questions for authors

1.Will you constructively articulate the similarities and differences between IMPRINT and BM25 (b=0)?

1 comment

given these solid additonal experiments, i will raise my socre to 4

Reviewer_99xKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

IMPRINT is a training-free retrieval method for long-term conversational memory. Instead of relying mainly on dense similarity, IMPRINT treats rare user-specific tokens as identifiers of past sessions. The method builds a Token Profile Table that stores token statistics and uses IDF- and session-specificity-based weighting for retrieval. It also includes an optional dense Contriever component and a statistical consolidation mechanism that provides additional context for generation. The method is evaluated on LoCoMo and LongMemEval and shows strong retrieval results compared to several existing memory systems.

Critical strengths
  • Overall the results are strong, and the method is surprisingly simple compared to many of the memory systems discussed. The sparse retrieval mechanism is easy to understand and does not require additional retrieval-specific training.

  • I liked the granularity analysis. It shows that the choice between session-level and turn-level retrieval has a substantial effect on both dense retrieval and BM25, which helps explain some of the differences between prior retrieval setups.

Critical weaknesses
  • My biggest concern is regarding the claim that dense retrieval itself is the bottleneck. The clean dense comparison mainly uses Contriever, which is relatively old compared to current embedding models. More importantly, the exact setup is difficult to reconstruct as important details are spread across the appendix. For example, the paper states only in Appendix B.4 that Contriever embeddings use 128-token truncation (lines 1088-1090). However, the dataset statistics later show that a LongMemEval turn contains 424 tokens on average (Table 13). This means that a large part of an average LongMemEval turn can be invisible to the dense baseline. The comparison to BM25 + Contriever is important, but on LoCoMo IMPRINT only improves Recall@3 by 0.6 points over this simple hybrid. The larger gain appears on LongMemEval, where the dense component is also affected by the 128-token truncation. This makes the stronger LongMemEval gain somewhat harder to interpret.

  • The HippoRAG 2 setup is also not fully clear to me. The authors state that HippoRAG 2 is re-run by them with the official code (lines 364-365). The detailed HippoRAG description gives the OpenIE, PageRank, and other settings (lines 1202-1211), but does not explicitly state which embedding backbone is used. Since the main claim of the paper is specifically about the retrieval primitive, I think these details should be much easier to find and stated clearly in the main experimental setup. I would also like to see a stronger recent dense retriever evaluated under the same preprocessing and granularity.

  • SPLADE v2 and ColBERTv2 reach higher Recall@3 than IMPRINT but are only reported in the appendix. I understand that IMPRINT does not require additional retrieval training, which is an advantage, but if the main claim is about whether dense or sparse retrieval is the better retrieval primitive, I still think these stronger retrievers are important comparisons.

  • IMPRINT also depends quite strongly on lexical overlap. Under aliases and paraphrases, its performance drops much more than dense retrieval. To me, this suggests that lexical matching and semantic retrieval are probably complementary rather than that dense retrieval should be replaced in general.

Questions for authors
  • Which embedding backbone is used for HippoRAG 2 in your experiments?
2 comments
AuthorsOpenReview ↗

Kind Request for Your Feedback

Dear Reviewer 99xk,

Thank you for your constructive feedback. We believe we have addressed your concerns to a large extent in our response. Following your last comment, we have posted the BM25 + NV-Embed-v2 hybrid comparison under identical turn-level preprocessing, together with a fusion-weight ablation for both methods (Tables F1–F2 in our latest response).

Given that the deadline is approaching, could you please take a moment to review our updates? If you have any further questions, we would be more than happy to provide additional clarification.

Thank you again for your time and effort in reviewing our paper. If our responses have resolved your concerns, we would greatly appreciate it if you would consider reflecting this in your final assessment.

Best regards,

The Authors

Reviewer_99xKOpenReview ↗

Final Response by Reviewer

Thank you for the additional experiments. They resolve my concerns about truncation and the HippoRAG 2 backbone, so I am raising my soundness score to 3. I keep my overall rating, because with NV-Embed-v2 dense retrieval alone beats IMPRINT as submitted (77.10 vs. 74.42 R@3 on LoCoMo), and BM25 gives almost the same gain as IMPRINT on top of it (Table F1: 0.05 and 0.42 R@3 apart), so the paper’s central claim would need substantial reframing.

Reviewer_8mVEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies retrieval for long-term conversational memory, arguing that rare, user-specific identifiers can be weakened in dense representations. IMPRINT maintains token statistics in a Token Profile Table and scores sessions using IDF, session specificity, and logarithmic term frequency. It combines this sparse component with turn-level dense retrieval and an online consolidation pool. The paper provides a dilution analysis under an idealized model and evaluates retrieval, question answering, granularity effects, and robustness on LoCoMo and LongMemEval.

Critical strengths

The method is simple and practically useful. IMPRINT requires neither additional retrieval training nor indexing-time LLM calls, yet achieves competitive results on both benchmarks. In Table 1, Recall@3 improves from 60.52 for the strongest evaluated memory-system baseline to 74.42 on LoCoMo, and from 51.06 to 66.81 on LongMemEval. These results establish simple lexical statistics as an important baseline for conversational memory.

The experimental coverage is substantial. The appendices include turn-level Contriever, BM25, their hybrid, and SPLADEv2/ColBERTv2, together with lexical perturbations, parameter sensitivity, consolidation ablations, and memory-size scaling. The crossed granularity–scoring experiment is particularly informative because it helps distinguish retrieval-unit choices from scoring mechanisms. IMPRINT also achieves the highest LoCoMo QA-F1 among the compared methods, demonstrating downstream utility.

Critical weaknesses

The contribution beyond established lexical weighting is insufficiently established. Session specificity in Eq. (4) is entirely determined by (df(t)/N), so it transforms the same frequency information used by IDF. In Table 5, the IDF-only variant achieves 74.47 Recall@3, compared with 74.42 for the default; the default’s benefit is mainly approximately 0.96 points at Recall@1. This supports rare-term weighting more strongly than the necessity of the proposed modifier. The historical positioning also needs qualification. Although the related-work section cites SeCom, Appendix C.5 broadly suggests that prior work evaluated sparse retrieval only on whole sessions and dismissed it. SeCom already compared BM25 across turn, session, and segment granularities. The paper should identify its new contribution more precisely. SeCom, ICLR 2025

The theory and controlled comparisons do not establish the breadth of the central conclusion. The sparse part of Theorem 5.1 considers a planted exact-match imprint and distractors lacking that imprint; a similar positive margin follows for any positively weighted exact-match scorer. Moreover, the weight depends on realized document frequency, while the expected collision count grows as (M\epsilon L). This is a conditional signal-preservation result, rather than a guarantee of length-independent retrieval accuracy for the complete system. The appendices acknowledge modeling limitations, but the introduction and conclusion still make a broad claim that the retrieval primitive, rather than memory structure, is the bottleneck. Empirically, Appendix C.5 reports LongMemEval gains of only 1.92 Recall@3 and 0.15 QA-F1 over turn-level Contriever, and 0.18 QA-F1 over the BM25–Contriever hybrid. Uncertainty estimates are not provided for these close comparisons. SPLADEv2 and ColBERTv2 also achieve higher retrieval recall on both datasets, which should feature more directly in the central discussion.

Questions for authors

Can the IDF-only comparison in Table 5 be extended to full retrieval and QA metrics on both datasets, alongside BM25 with length normalization disabled? Do paired confidence intervals support reliable gains over the competitive baselines in Appendix C.5?

How are long inputs handled for the session-level dense embeddings in Figure 3? The main text states that the experiment uses no truncation, Appendix C.5 explicitly guarantees this for sparse rules, and Table 2 specifies a 128-token unit limit. Do the dense configurations have equal evidence coverage, allowing the effect to be attributed to pooling dilution rather than truncation?

Equation (11) prunes token sets, but Eq. (15) supplies concatenated raw source text to the generator. What benefit does four-factor consolidation provide over simple evidence reordering or repetition under an equal token budget? Can cross-session merges additionally introduce text from outside the top-three retrieved sessions?

Can the authors report CPU query latency and memory usage for the complete system and clarify its complexity? Explicit co-occurrence sets can require quadratic token-pair updates, and Algorithm 1 refreshes existing pool embeddings after each session. These costs are not fully reflected in the current efficiency argument.

1 comment
AuthorsOpenReview ↗

Kind Request for Your Feedback

Dear Reviewer 8mVE,

Thank you for your constructive feedback. We believe we have addressed your concerns to a large extent in our response.

We added paired bootstrap confidence intervals for the close comparisons (Table R1), the IDF-only and BM25 (b = 0) results on both datasets (Table R2), and CPU latency and memory measurements (Table R3). We also clarified the truncation settings behind Figure 3, the scope of Theorem 5.1, and our positioning relative to SeCom.

Given that the deadline is approaching, could you please take a moment to review our updates? If you have any further questions, we would be more than happy to provide additional clarification.

Thank you again for your time and effort in reviewing our paper. We hope these updates help clarify the manuscript.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.