acceptodds
Withdrawn from ICLR 2027

AttestMem: Turning Long Conversations into Traceable, Revisable Memory

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-horizon language agents require persistent memory. Existing systems reduced dialogue histories to collections of independently embedded chunks: outdated or contradictory information can coexist at retrieval time, and semantic similarity alone decides which memories enter the answering context. We introduce AttestMem, a training-free memory framework that represents conversations as a revisable collection of attested statements and compiles a bounded answering context from recalled candidates. Each statement carries its extraction provenance, utterance time, validity interval, and ingestion order; when deterministic source matching succeeds, the row additionally carries turn-level references and its source-match grade, and ambiguous or unsupported cases are explicitly marked rather than guessed. An append-only event stream incorporates updates so that they supersede rather than overwrite earlier values, while rejected writes and historical versions remain queryable. At inference time, candidates are nominated through complementary routes and compiled under a character budget that prioritizes assertions and recovered source quotations while trimming transcript text. With GPT-4o-mini for memory construction and answering, AttestMem reaches 89.03% accuracy on LoCoMo and 86.00% on LongMemEval-S; with GPT-4.1-mini, 93.44% and 89.10%, using 3.6–3.9k prompt tokens per question. These results match or exceed the strongest reported baseline in each backbone group on both benchmarks. Paired ablations show the largest accuracy decrease when the ledger is removed, and removing the context limit lowers LoCoMo accuracy by 14.25 points while tripling the prompt.

Primary area:
foundation or frontier models, including LLMs
Keywords:
long-term memory, LLM agents, memory admission

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ySTzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper tries to come up with a context management methodology for long conversations. The build a method that converts turns into statements and keep a ledger. They show some improvements over some baselines on LongMemEval-S and LoCoMo benchmarks using fairly older models.

Critical strengths

I think this is an important topic to study, and especially could have useful implications in the for personalization and continued conversations.

Critical weaknesses

This is a very poorly written draft which is very difficult to understand. I suspect most of it is actually written by claude code, some examples from just the intro include: L44: Memory construction and retrieval have both improved, but the step that assembles the context has not. L52: It concerns the decision rather than the annotation, which is why it survives every field the write half adds L55: Coverage does not explain these failures: in our own runs the questions the system fails on every sampling draw are no less covered than the ones it answers

Everything is not motivated, for example: why do you use three routes under nomination? A lof of things are unclear: In figure 1, how is attestation getting info that there are two magazines? Since you bulid a candidate pool if the conversation is long enough won't it run into the same context issue?

The paper attributes the gains to the attestation, but attestation is not novel enough in itself. Mem0, A-Mem and others already use it.

I looked at the prompt and its very specific to the task itself and is overfitting.

ITEMS IN FLIGHT | anything the user still has to pick up, collect, return,
drop off, exchange, repair, collect from a person, lend back, borrow, or is
waiting to receive. Write the ITEM and, when stated, the shop/person and the
date, e.g. "The user still needs to return the boots to Zara (bought 2023-02-05)"
or "The user lent her green sweater to her sister".
This kind exists because counting questions ask how many things are in flight
and the answer is lost when only "I own X" statements are extracted: measured,
22.4% of the gold items of counting questions were absent from the store.
Questions for authors

Please look at the weakness.

Reviewer_6vr7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies memory management for long-horizon language agents. The authors argue that many failures arise not from insufficient evidence coverage, but from how systems manage the available evidence. To address this issue, they introduce a comprehensive pipeline named AttestMem to manage memory across long conversations. The pipeline first applies Attestation to convert conversational events into an attested-statement ledger, then uses Nomination to retrieve candidate statements to support question answering, and finally applies Compilation to construct the answering context. Experiments on LoCoMo and LongMemEval-S show that AttestMem achieves strong performance under both GPT-4o-mini and GPT-4.1-mini backbones. Ablation studies demonstrate the contributions of different components, particularly the statement ledger, and show that removing the context limit can substantially reduce answering accuracy.

Critical strengths
  1. The studied problem is important for LLM agents that need to manage long-term memory during extended conversations. Maintaining temporal consistency, preserving historical information, and retrieving appropriate evidence are important for reliable memory-augmented agents.

  2. The experimental results are positive on the selected benchmarks, particularly on temporal reasoning and knowledge update tasks.

Critical weaknesses
  1. The computational cost of the internal pipeline is not evaluated. The entire pipeline involves multiple LLM calls. As shown in baseline papers such as RecMem and EverMemOS, memory management can introduce substantial computational overhead beyond answer generation. However, the paper only reports the final answering prompt tokens. A comprehensive cost-performance evaluation is critical for assessing such a complex pipeline as AttestMem.

  2. Several important implementation details are missing. The authors do not clearly explain how attestation grades are calculated, which embedding model is used for settlement, or how entities are extracted and matched during retrieval. The setting of the w/o Nomination ablation is also unclear.

  3. The effectiveness of Compilation requires further analysis. Reducing the context budget from 8,000 to 4,000 characters causes noticeable accuracy drops despite only small changes in prompt tokens. The authors do not analyze which information is removed or why performance changes so significantly. The results may reflect the importance of retaining conversational history rather than the proposed compilation strategy itself.

Questions for authors

Please see the weaknesses above for my main questions and concerns.

Additionally, could the authors provide a more representative example than the one in Figure 1 to better illustrate the motivation? The current example appears to demonstrate a retrieval coverage failure, which does not directly support the authors' argument that the main issue lies in evidence utilisation rather than evidence coverage.

Reviewer_YVz5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The author presents a method to manage memory in long context conversation which called AttestMem, a training-free framework that combines a multi-route candidate retrieval and budget-aware context compilation. Experiments on classical LoCoMo and LongMemEval-S report competitive or state-of-the-art performance with GPT-4o-mini and GPT-4.1-mini.

Critical strengths
  1. The paper addresses an important problem in long-term memory systems first in the statement. Retrieving relevant information does not necessarily imply that the answering model can use it correctly which he summarizes it as similarity-sufficiency assumption. It is critical and really exists in most conversations.
  2. The separation of memory construction, candidate nomination, and context compilation provides a useful framework for analyzing memory failures. The results in LoCoMo and LongMem bench demonstrate its accuracy.
Critical weaknesses
  1. Limited novelty and substantial overlap with existing work. The problem addressed in this paper is well recognized in the literature on long-term memory for LLM agents, and many of the proposed challenges, including outdated or conflicting memories, unreliable retrieval, and inefficient context utilization, have already been extensively investigated and addressed by prior studies. More importantly, the proposed contributions substantially overlap with existing approaches. For example, AtomMem and MemTxn have explored temporal memory management, source-supported updates, and historical state tracking, while EverMemOS have investigated context reconstruction and budget-aware memory selection. I did not see the contrast between your work and these works. The proposed Attestation, Nomination, and Compilation components largely combine established techniques, without introducing a sufficiently distinctive methodological innovation. It sounds like a different name for same components.
  2. Unclear sources of performance gains and insufficient prompt ablations. The supplementary code contains extensive task-specific prompting strategies, including temporal reasoning, memory version checking, multi-hop reasoning, and counting instructions, some adapted from prior work such as RecMem and HyperMem. However, the paper does not sufficiently investigate how much of the reported improvement comes from these prompting strategies rather than the proposed memory mechanisms. Additional prompt-only ablations and controlled comparisons are needed to establish the actual sources of the performance gains.

Some additional weaknesses on your paper writing. 1.The presentation language is a serious obstacle to evaluating the paper and really hard to read. The main body is written in an unnatural, AI-like style in sentence fragments, dense nominal chains, overused colons (about nearly over 100) and dashes, and templated phrasing recur across. The authors should conduct a full language and structure rewrite before resubmission, minor copy-editing is unlikely to resolve this. You can use AI to polish your writing but not totally replace. 2.Figures and appendix layout. Figure 3 appears auto-generated that category labels and numbers overlap and occlude the curves. More seriously, the content in Appendix D–E (p.17–18) extends beyond the boundaries and is completely unreadable. If it is in the main body of your paper, it should be desk-reject.

Questions for authors
  1. Inconsistent experimental results between the paper and supplementary materials. Table 2 reports 86.00% overall accuracy for AttestMem on LongMemEval-S with GPT-4o-mini, whereas the supplementary script (make_table_longmemeval.py) reports 82.13% for the same method, dataset, and backbone. Substantial discrepancies also appear across individual evaluation categories. The authors should explain these inconsistencies and clarify which results correspond to the final experimental configuration.
  2. Why are several closely related methods not directly compared that you mentioned in related work? The paper discusses existing approaches addressing similar memory challenges, but does not provide sufficient experimental evidence to demonstrate clear advantages over them.
  3. Missing accuracy–efficiency trade-off analysis. The proposed framework involves multiple LLM-based extraction steps. However, the evaluation mainly reports answer accuracy and final prompt tokens, without considering the overall computational cost, storage overhead, or retrieval latency. Since prior memory systems such as Mem0 and RecMem have explicitly evaluated these trade-offs, it remains unclear whether the reported performance improvements justify the additional complexity of AttestMem.
  4. Prompt ablation experiments are needed as I mentioned in weakness 2, or more datasets to prove your gains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.