acceptodds
Under review as a conference paper at ICLR 2027

Truth in the Trace: Auditable and Lifecycle-Typed Memory for AI Agents

Abstract

A conversational assistant must remember user-provided information and keep it accurate as circumstances change. This requires two capabilities at once: absorbing facts that fit existing knowledge, and restructuring memory when new facts conflict, without discarding the displaced values. Existing memory models satisfy only one demand at a time. Append-only stores keep everything but never restructure, so stale and current values become indistinguishable. Overwriting and re-summarizing stores do restructure, but they erase past values and their provenance. We propose FactTrace, a memory architecture built on knowledge equilibration, the balance between absorbing what fits and reorganizing for what does not, which performs both moves under a single governor and maintains each attribute as an evolving structure. A fitting value is absorbed, a conflicting value supersedes the old one, and the old value is demoted rather than deleted, retaining its provenance and replacement history. Because the current value is always explicit, the assistant conditions on the present state correctly even after many changes. Superseded values stay available, and because stored facts are never re-summarized, fine-grained distinctions survive. We evaluate on two benchmarks, one for personal state tracking and one for truthfulness under pressure, across five underlying models. FactTrace outperforms strong memory baselines on both axes, significantly so in the controlled single-backbone comparison in which we test significance, while across the remaining backbones the same ordering holds as a consistent point-estimate margin that we do not test individually. Staying truthful costs raw goal completion (6.55 to 4.68 on a 0–10 scale), and our margin on the benchmark's truthfulness-gated composite is directional but not significant. A per-period decomposition, a read-time causal ablation, and a head-to-head retrieval analysis each tie a gain to the corresponding non-destructive design choice. Our code is available at: https://anonymous.4open.science/r/FactTrace-92CB/

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Memory, Knowledge Equilibration, Large Language Models, State Tracking, Truthfulness, Non-destructive Update, Provenance, Conversational Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_urRgOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This project proposes FactTrace, a memory framework for LLM agents that maintains current and historical facts while preserving their provenance. It uses LLM-based comprehension to resolve conflicting information and update memory without deleting previous facts. Experiments on AMemGym and AI-LieDar show improvements in state tracking and truthfulness.

Critical strengths

Meaningful problem: Maintaining accurate and trustworthy memory as user information changes is an important problem for long-term LLM agents.

Practical design: Separating current facts, historical facts, and inferred information while preserving provenance is a reasonable and useful approach.

Positive results: The method achieves improvements on both benchmarks. The authors also provide several ablation studies and statistical analyses.

Critical weaknesses

Limited methodological novelty: Dynamic state updates, conflict resolution, historical retention, and provenance tracking have already been explored in works such as Zep/Graphiti, Hindsight, Memanto, and CUPMem. The proposed framework appears to combine several existing ideas without clearly demonstrating what is fundamentally new. These closely related works are also missing from the discussion and experimental comparisons.

Gaps between benchmarks and claimed contributions: AMemGym mainly evaluates current-state tracking, while AI-LieDar focuses on truthfulness under social pressure rather than memory evolution. Neither directly evaluates historical rollback, provenance auditing, or recovery from incorrect updates. Therefore, the experiments only partially support the claimed capabilities of FactTrace.

Insufficient mechanism isolation: The truthfulness improvement may partly come from explicit response instructions requiring the assistant to disclose verified facts, rather than the proposed memory architecture. Although the authors provide some ablations, they do not sufficiently separate the effects of provenance labeling, memory updates, and response prompting. A simpler versioned memory baseline with the same response policy would be particularly useful.

Experimental reporting and robustness: The paper should clarify whether the 60 AI-LieDar scenarios include the original scenarios or their paraphrased variants. Additionally, the reported 50.9% truthfulness for Mem0-G does not directly correspond to an integer number of successful cases out of 60 under binary evaluation. The single-seed experiments and potential dependence among questions from the same user also limit the statistical conclusions.

Questions for authors

What are the key differences between FactTrace and existing temporal or versioned memory frameworks such as Zep and Memanto? Could the authors provide direct comparisons?

How much of the truthfulness improvement comes from memory management rather than the response prompt?

Can the authors provide evaluations of historical recovery, provenance correctness, and rollback after incorrect updates?

Could the authors clarify the AI-LieDar scenario selection and the Mem0-G result?

Reviewer_ifChOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes FactTrace, a conversational memory that groups everything known about a user attribute into one "axis" holding a single current value, a list of superseded values linked by supersedes pointers, open user concerns, and assistant inferences. Each fact is stamped with a provenance label at write time, and a trust rule stops lower-provenance claims from displacing higher-provenance ones. Writes are done by an LLM "surgery" prompt (on alternate turns), reads retrieve a top-3 typed subgraph, and stored facts are never re-summarized. The design is framed through Piaget's assimilation and accommodation. Experiments on AMemGym (state tracking) and AI-LieDar (truthfulness under social pressure) with gpt-5.4-mini as the main backbone show 0.456 vs 0.370 Overall and 91.7% vs at most 53.3% truthful against RAG, A-MEM, AWI, Mem0-G and AWE, with a cross-backbone comparison against Pure LLM and AWE only.

Critical strengths

The AMemGym result at gpt-5.4-mini is the most convincing part of the paper. The per-period breakdown (Table 6, Figure 2) shows FactTrace tied with the best baseline at P00 and pulling ahead once state starts changing, which is exactly what a current/history split should do, and the retrieval analysis in Table 7 and the error attribution in Table 10 make a reasonable case that the typed current value is easier to find than raw dialogue (RAG) and keeps more detail than summaries (AWE). They are also open about the statistical limits (single seed, clustering, TUS margin over RAG not significant), which I appreciate.

Critical weaknesses

My main problem is the truthfulness result, which is the headline number and, as far as I can tell, is confounded by the response prompt. App. H says the FactTrace response policy requires that "a verified unfavorable fact bearing on the user's current question must be stated plainly and first," and that user claims may only be acknowledged as claims. That is an explicit honesty instruction. Do the baselines get anything like it? Nothing in Section 4 suggests they do. In AI-LieDar the private verified fact sits in the scenario brief and dialogues are at most 20 turns, so the Pure LLM baseline already has the inconvenient fact in context and still lies 58% of the time. That makes it hard to accept the claim in Section 6.2 that truthfulness is "an information-availability" problem. The read-time ablation actually points the other way: with the verified fact stripped, FactTrace is still 78.3% truthful, far above every baseline. The authors credit "the typed structure itself," but the far simpler explanation is that the honesty-gated prompt is doing most of the work. The obvious control (Pure LLM and RAG with the same response policy text) is missing, and without it I cannot attribute the 91.7% vs 43 to 53% gap to memory at all.

The state-tracking claims are weaker than the abstract makes them sound. Only Pure LLM and AWE are run across backbones, and Mem0-G (which ties Pure LLM at gpt-5.4-mini) is not. On Qwen3.5-9B and GPT-5.4 the margins are +0.014 and +0.008 from one seed, which the authors admit is noise. The argument that the benchmarks therefore give "a lower bound on the practical gain" in long-horizon settings is speculation with no experiment behind it. The ablations in App. E are run on gemini in a non-canonical configuration. On novelty, a per-attribute versioned store with provenance and supersession is close to existing temporal knowledge-graph memories that invalidate facts instead of deleting them (Zep/Graphiti for example). The paper sidesteps this by saying systems it did not run "are not positioned as competitors," which I don't find acceptable for a related work section.

A couple of other things. The prose is repetitive and full of rhetorical claims ("deception harder by construction, not by instruction"), and the paper reports no wall-clock or total token cost for the LLM write step compared to the baselines.

Questions for authors
  1. Do any of the baselines on AI-LieDar get the provenance-gated disclosure instruction from App. H, or an equivalent one? Please report Pure LLM and RAG with that exact response policy added.

  2. How does FactTrace do on AI-LieDar with a neutral response prompt (same memory, no disclosure rule)? That would separate the memory from the instruction.

  3. Can you run Mem0-G (or another graph memory with invalidation) across all five backbones on AMemGym, and add at least a second seed for the small-margin backbones?

  4. How does FactTrace differ, in mechanism and not in framing, from temporal knowledge-graph memories that mark edges invalid with timestamps instead of deleting them?

Reviewer_jB3BOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how conversational AI agents can maintain an accurate current state as user information changes while retaining historical facts and their provenance. The authors propose FactTrace, which organizes memory by attribute and explicitly distinguishes current facts, superseded historical facts, user concerns, and assistant inferences. When conflicts arise, displaced values are retained, while provenance-based trust rules govern updates and responses. The system bounds the retrieved context through selection rather than repeated re-summarization of stored facts. Evaluations on AMemGym and AI-LieDar assess state tracking and truthfulness under social pressure, respectively. The paper reports improvements over several memory baselines and uses per-period analysis, retrieval analysis, and ablations to investigate the roles of historical retention and fact exposure.

Critical strengths
  1. The paper explicitly distinguishes current facts, historical facts, and assistant inferences, while preserving provenance and supersession relationships during updates. This makes the memory state clearer and facilitates tracing changes and diagnosing errors.
  2. The paper evaluates both dynamic state tracking and truthfulness under user pressure. It combines per-period results, retrieval analysis, and ablations to examine how memory organization affects answer accuracy and reliability.
Critical weaknesses
  1. The paper’s core novelty is unclear, and its distinctions from prior work are not clearly established. For example, A-MEM [1] already supports updates to memory representations, and the graph-based variant of Mem0 [2] retains invalidated relationships. Existing methods therefore cannot simply be characterized as “append-only” or as “restructuring without retention.” The motivation for designing FactTrace is insufficiently established.

  2. The paper does not sufficiently clarify the respective roles of the memory structure, provenance labels, and response policy, and some details remain unclear. The truthfulness improvement may jointly depend on provenance-based trust constraints and explicit response instructions. The existing ablation establishes that fact exposure matters, but does not demonstrate that the gains primarily arise from lifecycle typing and non-destructive accommodation.

  3. The experimental results lack generalizability and have limited reproducibility. The full baseline comparison is conducted on only one answer backbone, while comparisons across the remaining backbones primarily involve Pure LLM and AWE. Some gains are small and lack validation through multiple independent generation runs. The claim in the main text that the advantage would widen with longer histories also lacks direct experimental support.

[1] A-MEM: Agentic Memory for LLM Agents. NeurIPS, 2025.

[2] Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. ECAI, 2025.

Questions for authors

see weaknesses

Comments on the paper

1 comment
AuthorsOpenReview ↗

Reminder: the discussion period closes today

Dear Reviewers jB3B, ifCh and urRg,

As the discussion period closes today (Nov 18 AoE), we would be grateful to know whether our responses and the revised PDF address your concerns. The main points, in brief:

  • Prompt confound (all). With no disclosure rule in any method's prompt, FactTrace is truthful in 85.0 / 91.7% of dialogues (H / L), against at most 65.0 / 66.7% for the seven external baselines (p<10−4p<10^{-4} vs. Pure LLM). Adding the rule to the baselines raises their truthfulness but lowers TUS; FactTrace without the rule keeps the highest TUS (§7).
  • The 78.3% residual (ifCh). Without the rule, hiding only the typed verified node lowers truthfulness to 51.1 / 63.3% (p<10−4p<10^{-4}) (§7).
  • Novelty (all). Retention is shared ground, and the framing that suggested otherwise is removed. §2 and Table 1 compare Zep/Graphiti, Mem0-G, A-MEM, Memanto, Hindsight and CUPMem. A versioned store without the trust order ties FactTrace on AMemGym and trails it on AI-LieDar, where the trust rule fired (§7, App. C).
  • Scope (jB3B, ifCh). The full baseline field runs on two further backbones, Mem0-G on four, and the small-margin backbones have a second seed; we describe those margins as small (§5.2, §7).
  • Untested capabilities (urRg). Provenance-label accuracy (186 / 184 of 200) and recovery from incorrect updates (37 / 36 of 40) are now measured (§7, App. J). Mem0-G is corrected to 29/60 = 48.3% (App. B).

We are happy to clarify anything further before the discussion closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.