acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Memory Index Evolution for Agents

Abstract

Long-horizon agents continually accumulate experience, yet retaining a memory does not guarantee that it will be retrieved when needed. Existing retrieval-based memory systems primarily address this problem by modifying what is stored, while the retrieval index itself is typically treated as fixed. We introduce Memory Index Evolution, a complementary adaptation axis that learns how retained experience should be reached from deployment interactions, and instantiate it with Adaptive Memory Index Evolution (AMIE). AMIE maintains a bounded, persistent state for each memory and uses it to reshape future retrieval without modifying stored content or retraining the host memory system. Specifically, AMIE (i) derives a persistent interference state from deployment retrieval traffic, (ii) uses suppression dynamics to reduce repeated slot competition on subsequent queries, and (iii) performs index updates in the retrieval flow as a lightweight wrapper over existing memory systems. Across all evaluated memory hosts, AMIE consistently improves retrieval of fresh evidence, showing that index-side adaptation transfers across heterogeneous memory systems. Downstream evaluations further translate these retrieval gains into improved answer quality. Our results establish that how memories are reached is a powerful optimization axis distinct from what is stored.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Memory, Memory Index Evolution, Lifelong Agents, Adaptive Retrieval

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YVRcOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Previous comment

AMIE adapts the index structure of an agent's memory online as the query distribution evolves (merging, splitting, and reorganizing entries) instead of using a static index. The method comes with a clear algorithmic description and theoretical analysis (complete proofs in the appendix). Experiments across dialogue-memory benchmarks and backbones show consistent but small gains (mostly <2pp, roughly 0.7–3.4pp), supported by significance tests and ablations.

Updated:

AMIE maintains persistent per-memory suppression state to adjust future retrieval scores without changing stored content. It is evaluated across five state-of-=the-art memory systems and three benchmarks. Therefore, my original description of merging and splitting entries was incorrect.

Critical strengths

Separating memory content from retrieval adaptation is very useful. The motivation is sound. The added cost measurements and future-query evaluations protocol make the practical benefits clearer, particularly for the limited LLM budget.

Critical weaknesses

Small effect sizes:Most gains are under 2pp (≈0.7–3.4pp); the paper does not sufficiently argue when the evolution overhead is worth paying. Overhead vs. assumptions:Maintenance and extra LLM-call costs are not systematically reported; the stationarity assumption behind the theory is not validated on real agent workloads.

Update:

However, the benefits remain dependent on the retrieval setting. The decay reduces but does not eliminate this count loss (1.71 Recall@10pp). Besiodes, the semantic policies provide the better selectivity at additional inference cost.

Questions for authors
  1. How do gains and overhead scale when the memory store grows 10×?
  2. What is the full cost budget of index evolution (LLM calls, latency, storage)?
  3. Do the guarantees and gains hold under non-stationary query distributions (bursty topics)?
  4. Are there settings with larger effect sizes (long-horizon personalization, cross-session tasks)?
2 comments
AuthorsOpenReview ↗

Follow-up on Rebuttal

We thank the reviewer again for the thoughtful feedback. As the discussion phase is approaching its end, we would like to briefly follow up. We have provided additional experiments addressing your concerns about the practical significance of AMIE's gains, computational and memory overhead, robustness under non-stationary query distributions, and benefits in longer-horizon and larger-scale memory settings. Specifically, we report detailed latency, token, and storage measurements; evaluate memory stores expanded up to 21× and multiple non-stationary query schedules; and demonstrate larger retrieval gains on held-out future queries, cross-session tasks, and previously unseen evidence.

We hope these additional results and clarifications satisfactorily address your concerns. If any questions remain, we would be very happy to provide further clarification or discussion before the discussion phase concludes.

Thanks authors for the very detailed responses. The updated experiments about scaling, cost, and future-query results address my main questions and concerns. I also correct mistakes in my review, like AMIE does not merge or split memories. However, the related near-domain failures and semantic-policy costs still limite the performance. Therefore, I will keep my score at 3.

Reviewer_MQ2ZOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies persistent changes to memory access while keeping stored content fixed. AMIE distils teacher decisions into a query-conditioned policy that increments per-memory suppression counters after the current answer context has been fixed. A bounded transformation of these counters rescales host retrieval scores on subsequent queries, with prompted and unconditional-count variants using the same state mechanism. Across five hosts and three benchmarks, the paper reports downstream gains in all 15 comparisons and uses controlled replays to examine when suppressing recurring candidates helps or harms retrieval.

Critical strengths
  1. Separating content from persistent access state gives a concrete intervention for studying retrieval adaptation without rebuilding the memory store.
  2. The matched BM25 replay reports a Recall@10 gain of 1.48 points with a conversation-clustered 95% interval of [0.70, 2.21], and the frozen-state experiment evaluates benefits on later queries.
  3. The count baseline and suppression-risk analyses contribute useful evidence that retrieval exposure can improve access while repeated suppression can also damage useful memories.
Critical weaknesses
  1. Appendix A.2 omits the source-evidence mappings for Mem0 and Memory-R2, preventing reconstruction of four of the ten retrieval comparisons in Table 1 from the reported protocol.
  2. GroupMem and MemBench provide the independent transfer tests because LoCoMo was used for fitting and selection (Appendix A.5). Their utility gains lack reported uncertainty estimates; Table B.1 provides intervals only for two LoCoMo hosts.
  3. MemBench has one terminal question per trace, but prior retrieval events are documented only for BM25 (Appendix A.1). The other four hosts' protocols do not sufficiently explain how state accumulates before the sole evaluated answer.
  4. The disabled-state checks do not establish a control on the same expanded candidate pool (Appendix A.4). For hosts that expand the pool, attribution to persistent reweighting therefore remains insufficiently verified.
  5. Keep decisions cannot reverse accumulated penalties (Equations 9–11). Section 4.5 reports benefits from decay in count replays, but the main policies omit this mechanism for reducing stale penalties.
Questions for authors
  1. What entry-to-source mapping was used to compute the reported Mem0 and Memory-R2 recall scores in Table 1?
  2. In the reported Mem0–MemBench experiments, which retrieval events generated the state before the terminal question, and did the baseline use the same candidate pools as AMIE?
3 comments
AuthorsOpenReview ↗

Response to Reviewer MQ2Z (Part 3 of 3)

W5: ... Keep decisions cannot reverse accumulated penalties ... decay improves count replays but is absent from the main policies.

Response: We thank the reviewer for highlighting an important insight from our decay analysis. To clarify, we intentionally evaluate decay separately from the main policies to isolate persistent index evolution and policy selectivity without introducing host-specific lifetime hyperparameters. Appropriate decay rates depend on memory granularity, retrieval frequency, and state lifetime.

Existing experiments already demonstrate the value of decay. Section 4.5 and Appendix C report that a half-life of 64 improves Count Recall@10 by +0.23 pp on BM25 and +0.44 pp on MemoryOS relative to non-decaying accumulation. We further evaluate decay sensitivity under near-domain corpus expansion, where each LoCoMo memory store is enlarged to 3×, 10×, or 21× its original size using real messages from other LoCoMo conversations and datasets' facts, while keeping the evaluation queries, gold evidence, and BM25 retrieval protocol fixed. We also test decay on held-out future queries following a 75/25 history–evaluation split.

\hline \text{Setting} & \text{No Decay} & h=32 & h=64 & h=128 \\\\ \hline \text{Near }3\times & \color{green}{+0.49} & \color{green}{+0.67} & \color{green}{+0.79} & \color{green}{+0.70} \\\\ \text{Near }10\times & \color{red}{-0.59} & \color{green}{+0.35} & \color{red}{-0.02} & \color{red}{-0.16} \\\\ \text{Near }21\times & \color{red}{-1.84} & \color{red}{-0.50} & \color{red}{-0.63} & \color{red}{-1.20} \\\\ \text{Future }1\times & \color{green}{+3.24} & \color{green}{+2.98} & \color{green}{+3.37} & \color{green}{+3.24} \\\\ \text{Future }3\times & \color{green}{+1.04} & \color{green}{+0.91} & \color{green}{+2.20} & \color{green}{+2.33} \\\\ \text{Future }10\times & \color{green}{+0.39} & \color{green}{+2.72} & \color{green}{+1.81} & \color{green}{+1.68} \\\\ \hline \end{array}$$ *ΔRecall@10 (pp) versus Host only, BM25. Near-domain rows average four corpus seeds; future-set rows use the fixed prospective replay. Half-lives are evaluated as a sensitivity analysis, not selected separately for each test condition.* The results demonstrate a clear persistence–forgetting trade-off: - **Shorter decay mitigates stale penalties under strong interference.** At 10× near-domain mixing, $h=32$ improves Count from −0.59 to +0.35 pp; at 21×, it reduces the loss from −1.84 to −0.50 pp. - **Longer retention benefits prospective retrieval.** On the clean held-out future set, $h=64$ achieves **+3.37 pp [1.41, 5.09]**; at 3×, $h=128$ reaches +2.33 pp. - **Decay protects useful evidence from accumulated suppression.** In the 10× future-set high-exposure stratum, the Count loss decreases from **−11.11 to −5.56 pp** with decay. In the separate no-headroom stratum, the loss decreases from **−4.76 to −2.38 pp**. These findings highlight the complementary roles of semantic selectivity and decay: **Frozen outperforms Count by +1.93 pp [1.33, 2.60]** at 10× near-domain mixing by avoiding harmful suppression, while decay mitigates stale penalties. Since no single half-life dominates across regimes, we retain the main policies to isolate persistent index evolution and policy selectivity, treating decay as a complementary state-lifetime mechanism. We have clarified this distinction and added the sensitivity analysis in the revision. --- We sincerely thank the reviewer for the detailed and constructive feedback, which has helped us substantially strengthen the reproducibility and experimental rigor of our work. We hope our revisions and additional evidence have addressed your main concerns, and would greatly appreciate your consideration of a higher rating. We would be happy to discuss any remaining questions.

The clarifications and additional experiments address several concerns, and I am raising my rating from 1 to 2. However, the equivalence between the expanded-pool no-op and the “Host only” baseline still requires verification of native top-k prefix consistency in the actual retrieval calls. Although evaluating decay separately is reasonable, the inability of keep decisions to reverse accumulated penalties remains a limitation of the main policies.

AuthorsOpenReview ↗

Additional Verification of Native Retrieval Equivalence and Penalty Recovery

We thank the reviewer for reconsidering our work and raising the rating. We appreciate the two remaining concerns and have conducted an additional verification experiment and theoretical analysis to address them.


[F1] "... equivalence between the expanded-pool no-op and the 'Host only' baseline still requires verification of native top-k prefix consistency in the actual retrieval calls."

Response: We now verify this using actual native retrieval calls, rather than relying solely on algebraic equivalence. On the matched BM25 replay covering 1,540 LoCoMo questions, we independently retrieve top-10 and top-100 candidates.

  • Native prefix consistency: The two calls agree on 1,459/1,540 questions (94.7%). The remaining 81 mismatches arise entirely from exact-score ties, including 12 cases where a tied memory changes the returned set. Importantly, no gold evidence changes, and per-question Recall@10 remains identical across all 1,527 scored questions.

  • Wrapper no-op verification: AMIE explicitly anchors the native top-10 results before appending expanded candidates, eliminating dependence on the host's raw prefix consistency. We verify exact ID-and-score agreement in 1,540/1,540 calls, both with fresh state and accumulated state at λp=0\lambda_p=0. At nonzero strength, the wrapper's output also matches an independent implementation of the documented re-ranking procedure in all 1,540 calls.

Thus, the expanded-pool no-op reproduces the actual Host-only retrieval output, even when independently issued native top-k and top-K calls differ due to ties. We will include the complete verification in Appendix A.4.


[F2] "... the inability of keep decisions to reverse accumulated penalties remains a limitation of the main policies."

Response: AMIE can recover previously suppressed useful memories without explicitly reversing their penalties. This follows from its relative-ranking mechanism: even if a useful memory retains an early penalty, it can regain its position as competing irrelevant memories accumulate greater suppression. We formally establish the conditions under which relative rankings converge despite early suppression errors. Moreover, when relative recovery is insufficient, we have already investigated and experimentally validated decay as a mechanism for explicitly reversing stale penalties.

1. Relative ranking convergence.

Let ui,t=ni,t/(ni,t+κ)u_{i,t}=n_{i,t}/(n_{i,t}+\kappa) and st(q,i)=σ^(q,i)(1−λpui,t)s_t(q,i)=\hat\sigma(q,i)(1-\lambda_p u_{i,t}). Consider a useful memory ii mistakenly suppressed finitely many times and an irrelevant competitor jj that continues receiving suppression. Under a stationary query distribution and sufficiently selective updates,

ui,t→ui∗<1,uj,t→1.u_{i,t}\to u_i^*<1,\qquad u_{j,t}\to1.

Consequently, ii eventually and permanently outranks jj whenever

\hat\sigma(q,i)(1-\lambda_p u_i^*)>\hat\sigma(q,j)(1-\lambda_p) }$$ If this strict separation holds for all relevant–irrelevant pairs competing for the top-$k$ positions, **the retrieved set converges to the correct top-$k$ despite finite early suppression errors**. Hence, **irreversible penalties do not imply irreversible rankings**. This guarantee is conditional on sufficient separation and selective updates, not stationarity alone. **2. Decay explicitly reverses stale penalties.** When relative recovery is insufficient, our evaluated decay variant provides an additional solution: $$n_{i,t+1}=\gamma n_{i,t}+a_{i,t}, \qquad \gamma=2^{-1/h}.$$ Once suppression stops, $$n_{i,t+\Delta t}=\gamma^{\Delta t}n_{i,t}\to0, \qquad s_{t+\Delta t}(q,i)\to\hat\sigma(q,i).$$ Thus, **decay restores the original retrieval score**. Over one half-life, the remaining penalty changes by $$\frac{D_i(n/2)}{D_i(n)} =\frac{n+\kappa}{n+2\kappa}.$$ Recovery is slower for heavily penalized memories; continued suppression can sustain a nonzero penalty. **3. Experimental validation and design choice.** Our replay experiments confirm decay's effectiveness: it improves Recall@10 from **−0.59 to +0.35 points** under severe near-domain interference and reduces high-exposure damage from **−11.11 to −5.56 points**. At $h=64$, its additional future-retrieval gain increases from **+0.13 to +1.16 to +1.42 points** as store size grows from 1× to 3× to 10×. We therefore **have investigated and validated a practical recovery mechanism**, rather than leaving this issue unaddressed. Decay is not our universal default because it also weakens useful suppression history, and no single half-life dominates across environments. Our main policies retain persistent state, while the separately evaluated Count+decay variant provides recovery when stale penalties matter. **In summary, relative rankings can converge despite early errors, while decay explicitly reverses stale penalties.** --- We appreciate the reviewer’s thoughtful feedback and would be happy to discuss any remaining questions or concerns further.
Reviewer_FtxTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Adaptive Memory Index Evolution (AMIE), a plug-in layer that adapts memory retrieval by maintaining persistent access weights without modifying stored content or retraining the host system. Experiments across five memory systems and three benchmarks show consistent improvements in retrieval and downstream task performance. The paper also finds that a simple retrieval-counting policy can be surprisingly competitive with learned policies. However, the novelty of the proposed adaptation mechanism and the reliability of some reported comparisons remain open concerns.

Critical strengths
  1. The paper treats persistent access state as an independent adaptation target rather than modifying stored memories. This provides a clean way to study how retrieval behavior changes over time while keeping the underlying memory store fixed.
  2. The evaluation spans five heterogeneous retrieval systems, and the controlled studies examine both when suppressing frequently retrieved memories exposes missing evidence and when it harms useful retrieval. Together, these results suggest that AMIE is not tied to a single architecture and provide a more nuanced account of its trade-offs.
Critical weaknesses
  1. Persistent access adaptation is not entirely new. MemoryBank, Generative Agents, and ExpGraph already explore persistent memory strength, access recency, or node-level utility updates (Zhong et al., 2024; Park et al., 2023; Feng et al., 2026). AMIE's main distinction appears to be its independently controlled plug-in interface and retrieval-conditioned updates. The experiments demonstrate that AMIE is useful across different hosts, but the methodological advance over these existing mechanisms remains less clear.
  2. The retrieval evaluation has some reproducibility and fairness concerns. Appendix A.2 notes that source-evidence mappings for some generated memories are not fully specified, making the corresponding retrieval scores difficult to reproduce independently. In addition, the expanded candidate pool lacks a matched no-op control, leaving some uncertainty about how much of the gain comes from persistent index updates rather than additional candidate acquisition. Zhong et al. (2024). MemoryBank: Enhancing Large Language Models with Long-Term Memory. Park et al. (2023). Generative Agents: Interactive Simulacra of Human Behavior. Feng et al. (2026). ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents. arXiv:2605.30712.
Questions for authors
  1. A more direct comparison with MemoryBank, Generative Agents, and ExpGraph would help establish which aspects of AMIE go beyond existing persistent access-update mechanisms, particularly in terms of the update signal, persistence rule, and prospective evaluation.
  2. The retrieval evaluation would be more convincing with explicit source-to-memory mappings for all compared hosts and a matched candidate-pool control that separates the effect of index evolution from additional candidate acquisition.
  3. A more systematic comparison between trained, frozen, and count policies across all host–benchmark pairs would clarify when semantic selection is necessary and when a zero-call counting rule provides a comparable benefit.
1 comment
AuthorsOpenReview ↗

Follow-up on Rebuttal

We thank the reviewer again for the thoughtful and constructive feedback. As the discussion phase is approaching its end, we would like to briefly follow up. We have provided additional experiments and clarifications addressing your specific concerns regarding AMIE's methodological novelty relative to existing persistent memory mechanisms, source-to-memory provenance mappings, matched candidate-pool and no-op verification, and the comparative strengths of Trained, Frozen, and Count policies. In particular, we have further verified native retrieval consistency through actual retrieval calls and clarified how AMIE preserves Host-only behavior when reweighting is disabled.

We hope these additions satisfactorily address your concerns. If any questions remain, we would be very happy to provide further clarification or discussion before the discussion phase concludes.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Discussion Summary

We sincerely thank the reviewers and the Area Chair for their careful evaluations, thoughtful suggestions, and time throughout the discussion. Below we summarize the strengths recognized by the reviewers, the additional evidence we provided, and the reviewers' subsequent feedback.

1. Strengths Recognized by the Reviewers

  • FtxT: Recognized persistent access state as an independent adaptation target that preserves memory content, as well as the breadth of evaluation across five heterogeneous memory hosts and analysis of suppression benefits and risks.
  • MQ2Z: Highlighted the controlled, fixed-store intervention, memory system gains with confidence intervals, frozen-state evaluation, and Count/suppression-risk analysis.
  • YVRc: Emphasized the value and sound motivation of separating memory content from retrieval adaptation, and valued our new cost and future-query evidence.

2. Concerns Addressed with Additional Evidence

  • Methodological novelty (FtxT): We added direct comparisons with MemoryBank, Generative Agents, and ExpGraph, distinguishing AMIE's query-conditioned, per-entry suppression, bounded persistent index state, reward-free deployment updates, host-independent intervention, and prospective fixed-store evaluation.

  • Reproducibility and retrieval protocols (FtxT, MQ2Z): We specified source-to-entry provenance for Mem0 and Memory-R2, evidence-ID mapping for LoCoMo/MemBench, and historical retrieval replay and state accumulation before terminal MemBench questions across five hosts.

  • Expanded-pool fairness and no-op verification (FtxT, MQ2Z): Beyond the algebraic argument, we checked 1,540 actual BM25 retrieval calls. With reweighting disabled, the anchored wrapper exactly reproduces native top-10 IDs and scores in all 1,540 accumulated-state calls, even though separately requested native top-10/top-100 lists can differ under score ties. Score ties do not affect gold-evidence Recall@10.

  • Policy selectivity and operating regimes (FtxT, YVRc): We clarified when Count, Frozen, and Trained policies are preferable. Under 10× near-domain interference, Frozen's semantic selectivity exceeds Count by +1.93 Recall@10 points (95% CI [1.33, 2.60]); Count remains a zero-policy-call alternative when interference is repetitive.

  • Uncertainty and state persistence (MQ2Z): We supplemented LoCoMo confidence intervals for A-MEM and Memory-R2 and explained the cost of independent host-level reruns on transfer benchmarks. We also established conditions for relative-ranking recovery despite finite early suppression errors, and evaluated Count+decay as an explicit stale-penalty recovery mechanism. We distinguish this variant from the non-decaying main policies.

  • Scale, cost, and non-stationarity (YVRc): We tested memory expansion up to 21×, measured Count's 0.054–0.060 ms added retrieval latency and sub-10 KB state on Mem0, and evaluated bursty, abruptly shifting, and recurring query schedules.

  • Larger prospective benefits (YVRc): Held-out future queries gain +3.24–3.37 Recall@10 points, unseen later-session queries gain +3.32 points, and previously unexposed gold evidence yields +5.05–6.78 points in the examined settings.

3. Reviewer Feedback After the Rebuttal

Importantly, the discussion has received positive acknowledgment from both responding reviewers.

  • MQ2Z explicitly stated that our clarifications and additional experiments addressed concerns and raised their rating. They identified two residual questions about native prefix consistency and irreversible penalties; we supplied a further actual-call verification and conditional recovery/decay analysis in direct response. Those follow-up results have not yet been endorsed.

  • YVRc explicitly confirmed that the scaling, cost, and future-query experiments addressed their questions and concerns, corrected an earlier misunderstanding about AMIE's mechanism, and maintained the positive rating.

  • FtxT has not yet responded. Importantly, their reproducibility and candidate-pool concerns overlap with MQ2Z's review, while their policy-trade-off questions overlap with YVRc's. The same experimental evidence has therefore already received positive acknowledgment on closely related issues from the other reviewers; we additionally addressed FtxT's distinct novelty question through a direct prior-work comparison.

We appreciate the Area Chair's consideration of these substantive additions and the documented reviewer feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.