- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies a problem regarding long-term memory in LLM agents: whether delayed feedback about stale memories should influence only the queried fact or memories use across all facts. In order to address this problem, this paper formalizes a sequential decision problem and demonstrates that the strategies adopted in previous work are not sufficient enough. Besides, this paper further proposes a RegimeCouncil method that combines a globally pooled advisor and a per-key advisor. As validation, both theoretical and empirical results are provided in the paper.
Critical strengths
-
This paper introduces local and global feedback, and the distinction between the two types of feedback is clear and relevant.
-
The router proposed in the paper is easy to implement, interpretable, and computationally lightweight.
Critical weaknesses
-
The formulation of the research problem in this paper is poor. In the Introduction section, this paper fails to state the problem well and clearly formulate their research problem. Too many examples are mentioned, which makes me confused and makes them feel trivial about their work.
-
The statements of previous work and the definitions in the paper are quite confusing. After reading the content, I can hardly understand the core problem and technical insights here.
-
The writing of this paper is quite poor. The sentences and structure of this paper do not fit the norms of a research paper.
Questions for authors
-
Could you please provide a clear summary regarding your research problem in this paper, including the memory conflict governance and the concrete problem regarding this paper?
-
Why did you mention "We index each stored fact by a key ..." (cf. Lines 52-73)? Is this a method proposed in your work? This part is quite confusing.
-
What are insights regarding the previous work mentioned in Lines 75-80? Are these terms defined by you or common sense in the community?
7 comments
Response to Reviewer Xtwa (Part2, Q1 & W1 and Q2 & W2)
Q1 & W1. Clear summary of the research problem and memory conflict governance (and removing distracting examples)
- Why Section 1 felt cluttered (Addressing W1): In the submission, lines 45–60 mixed the main
Beijing → Shanghaiexample with several briefly mentioned attributes (volleyball_record,job,hobby) before stating the formal problem. In the revision, we removed those extra examples and immediately follow the singleBeijing → Shanghaiexample with the formal problem definition below: - Concrete problem formulation:
- Setting: An LLM agent maintains a long-term memory store of tracked facts/attributes indexed by (e.g.,
user_city). At step , a query arrives for fact with prompt context . The agent chooses a binary action : inject the stored memory record for () or answer without it using the base model, which we call the no-memory anchor (). - Why it is non-trivial: Because user facts change without explicit deletion, the stored record for may be valid or stale, and the prompt alone does not reveal which is true (decision-time unobservability, Assumption 1). Only after a delay of up to steps—when a user correction, complaint, or task outcome arrives—can the agent evaluate whether using memory helped or hurt on episode , yielding the paired utility difference (Assumptions 2–3).
- The research problem (memory conflict governance): Once delayed outcomes arrive, at what scope should they govern future memory-use decisions ?
- A key-local policy uses only past outcomes from the same fact ().
- A globally pooled policy pools recent outcomes across all facts ().
- Setting: An LLM agent maintains a long-term memory store of tracked facts/attributes indexed by (e.g.,
- Core technical insights:
- Proposition 1: Any stationary policy using only the current prompt suffers a constant regret floor , whereas conditioning on delayed outcome history escapes this floor with regret .
- Proposition 2: Neither fixed scope is uniformly sufficient—global pooling suffers regret when facts drift asynchronously (majority keys overwrite minority keys), while key-local isolation suffers regret on unseen or sparse keys (zero local history while pooled evidence is already informative).
- RegimeCouncil & Proposition 3: RegimeCouncil maintains both the Global Advisor () and the Local Advisor (), and uses a Meta Governor that defaults to Local and switches to Global only when Global accumulates a strictly positive recent counterfactual surplus over the last revealed episodes (guaranteeing finite-time switching within steps).
Q2 & W2. Why we wrote "We index each stored fact by a key …" (Lines 52–73), and whether it is a proposed method
- Direct answer: It is not a proposed method; it is standard key–value notation so that each tracked attribute has an identifier , allowing us to distinguish "feedback on the same fact" (, Local) from "feedback pooled across all facts" (, Global).
- Why Lines 52–73 were confusing and how we fixed them: In the submission, we placed the sentence "We index each stored fact by a key..." in Paragraph 2 of the Introduction right after critiquing prior work, which followed the rhetorical pattern of a methodological claim and made a simple notation look like a proposed contribution. In the revision, we deleted this sentence from the Introduction and introduce once in Section 3.1 inside the formal problem setup block.
Response to Reviewer Xtwa (Part3, Q3 & W2 and W3)
Q3 & W2. Insights on previous work (Lines 75–80), and whether "global policy" / "local policy" are our terms or community terms
- Are these terms defined by us or community terms? Global policy, local policy, and memory conflict governance are terms defined by us in this paper to formalize two feedback-sharing scopes that prior systems adopt implicitly. In the submission, introducing them in parentheses in Lines 75–80 without an explicit definition made it unclear whether they were established community terms. In Section 1 (Paragraph 2) of the revised PDF, we now state explicitly: "When adapting memory use from past outcomes, however, prior designs implicitly fix one of two opposite sharing scopes, which we define as a local policy and a global policy."
- Concrete insights on previous work (Lines 75–80): Rather than stating abstractly that prior systems fix one policy at design time, the revised Introduction and Section 2 explicitly map prior families to these two scopes and explain their failure modes:
- Implicit global-scope mechanisms (e.g., global retrieval-confidence gates or shared recency/trust windows in RAG and memory pipelines) tune or gate memory injection across all queries jointly—which fails when a few volatile attributes (e.g., current travel city) go stale and cause the gate to suppress valid stable memories (e.g., dietary restrictions).
- Implicit key-local mechanisms (e.g., per-item utility/Q-value updates in MemRL [Zhang et al., 2026], or per-fact overwrite in Mem0 [Chhikara et al., 2025] and Zep [Rasmussen et al., 2025]) update only the accessed memory record in isolation—which cannot transfer staleness warnings across keys when a batch of facts goes stale together (e.g., during batched consolidation after a relocation or job change) or when querying a newly added key with no prior history.
W3. Rewriting sentences and structure to fit research-paper norms
To address W3 ("The sentences and structure of this paper do not fit the norms of a research paper"), we rewrote the Abstract and Sections 1–5 in the uploaded PDF:
- Section 1 (Introduction): Rewritten into four standard paragraphs: (i) Problem motivation via the single
Beijing → Shanghaiexample, (ii) Explicit definitions of local policy vs. global policy and how prior memory systems map onto them, leading to memory conflict governance, (iii) Clear overview of RegimeCouncil and Propositions 1–3, and (iv) Three-point summary of contributions. - Section 2 (Related Work): Reorganized into three thematic paragraphs, each ending with an explicit contrast between prior memory/bandit/expert-tracking methods and our online feedback-scope arbitration.
- Section 3 (Method): Section 3.1 now defines all problem notation () and Assumptions 1–3 in one self-contained block before Proposition 1, followed cleanly by paired evidence (Section 3.2), the Global and Local Advisors (Sections 3.3–3.4, Proposition 2), and the Meta Governor (Sections 3.5–3.6, Proposition 3).
Response to Reviewer Xtwa (Part1, Core Contribution)
Thank you very much for the constructive review and for the helpful suggestions. We address all points in three comments. Before answering each of your questions and weaknesses point by point, we want to highlight why the underlying contribution of this work is both practically important and new. This work's core contribution is solving an unaddressed practical problem in agent memory with strong empirical gains.
knowledge-updatequeries in LongMemEval [Wu et al., 2025]). Existing memory frameworks (MemGPT, Mem0, Zep, Mem1, MemRL) focus on what to write, retrieve, or compress, or how to overwrite a single item once its replacement value is given. However, they do not address how an agent should govern memory use before the memory store is updated (e.g., under asynchronous batched updates or when negative feedback flags an error without supplying the new fact): once delayed feedback reveals that using memory hurt on one fact, should that signal suppress memory use only on that specific fact (local scope), or across the memory store (global scope)?