acceptodds
Under review as a conference paper at ICLR 2027

EMIR: Evolution-Aware Memory with Intent-Guided Multi-Round Retrieval

Abstract

Long-term memory enables large language model (LLM) agents to leverage historical interactions for future tasks. However, existing memory systems struggle to utilize continuously evolving historical information, as they often rely on static memory representations and single-round retrieval strategies, failing to track factual changes or integrate distributed evidence across long-term interactions. To address these challenges, we propose EMIR, an **E**volution-Aware **M**emory framework with **I**ntent-Guided Multi-**R**ound **R**etrieval, enabling LLM agents to maintain evolving historical knowledge and adaptively retrieve relevant evidence. Specifically, EMIR constructs a State-Evolving Memory Graph (SEMG) that represents long-term memory as evolving knowledge states supported by temporal event trajectories and evidential associations. By maintaining semantic states through evidence-based updates, SEMG preserves historical evolution and enables evidence tracing under complex and conflicting scenarios. Building upon this, we introduce an intent-guided multi-round retrieval mechanism that iteratively identifies missing evidence and expands retrieval based on accumulated information. Experiments on LoCoMo and MemConflict demonstrate that EMIR improves long-term memory utilization, dynamic and static conflict handling, and complex retrieval performance, achieving relative improvements of more than 12% in certain categories. These results highlight the effectiveness of jointly modeling memory evolution and adaptive evidence acquisition for long-term agent interactions. The code is available at: https://anonymous.4open.science/r/EMIR-2-9027.

Primary area:
generative models
Keywords:
Agent Memory, Memory Management, Memory Retrieval

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4swNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses tracking factual changes and integrating distributed evidence across long-term interactions. The proposed method consists of two core components:

  1. a State-Evolving Memory Graph, which organizes events into temporal chains and maintains semantic states with historical versions and supporting evidence
  2. Intent-Guided Multi-Round Retrieval, which decomposes queries when needed, assesses retrieved evidence, and iteratively expands retrieval to fill evidence gaps before reranking the final candidates.

Experiments on LoCoMo with two LLM backbones show modest overall gains over MRAgent. On MemConflict, the method improves dynamic-conflict accuracy and several retrieval metrics, but underperforms the strongest baseline in overall answer accuracy, particularly on conditional conflicts.

Critical strengths
  • State-Evolving Memory Graph makes sense: using a graph to make evidence traceable
  • The evaluation is sufficienty: Beyond answer accuracy, the paper examines evidence retrieval, controller capacity, and graph-edge contributions, helping identify the method’s strengths and limitations.
Critical weaknesses
  • The evaluation does not isolate the proposed memory mechanisms. Removing semantic nodes also removes summaries and retrieval targets, so the ablation cannot attribute gains specifically to historical versioning. Evidence coverage metrics also conflate construction omissions with retrieval failures, leaving the source of gains unclear.
  • The practical conflict-handling advantage is limited. Average answer accuracy is 47.59%, below the strongest baseline’s 55.39%. Static-conflict accuracy improves by only 0.14 percentage points, while conditional-conflict performance drops substantially. Despite improved retrieval metrics, the answer-level benefits remain uneven.
Questions for authors
  • With semantic content, retrieval, and budgets held fixed, what improvement comes specifically from historical versioning and state evolution?
  • How often is supporting evidence missing from the constructed memory versus present but not retrieved? Have extraction omissions or incorrect entity merging been measured?
  • Does memory construction respect each query’s history prefix? Is the detected question type inferred from the query or supplied by benchmark annotations?
  • Were Table 1 baselines rerun or taken from MRAgent? How does Figure 3’s Full multi-hop score of 86.3 correspond to Table 1’s 84.40 ±1.23 across three runs, and are all ablations reported using the same statistical procedure?
10 comments
AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 1

Response to Reviewer Feedback:

We appreciate the reviewer’s time and effort in evaluating our submission and thank them for the valuable feedback. In the following, we provide a detailed response to the reviewer’s comment and clarify the relevant issues.

Response to Weakness 1 and Question 1:

We address the reviewer’s concerns about Weakness 1 in two parts. Here, we discuss the contribution of historical versioning and state evolution; the concerns regarding evidence coverage and construction versus retrieval errors are addressed in our response to Question 2 below.

We acknowledge that the original ablation variant without semantic components removes too many components simultaneously, making it difficult to isolate their specific contributions. To address this, we have introduced a new ablation variant without state transition, which removes the explicit semantic evolution edges while keeping all other components and settings unchanged, in order to analyze the contribution of historical state evolution. The experimental results show that, compared to the full model, the performance of this variant decreases by 3.2 pp and 15.6 pp on the Overall and Multi-hop metrics, respectively. In particular, the significant performance drop on the Multi-hop category indicates that, even with the complete semantic state content retained, relying solely on the retrieval mechanism with multiple rounds is insufficient to fully capture the evolution of different historical states. Explicit state transition relations can establish structured connections among historical states, thereby helping the model better acquire the multi-hop evidence required for complex questions. This result further verifies the independent value of state evolution relations in evolutionary memory modeling. The relevant experimental settings and result analysis have been added to Section 5.3 and Figure 3 of the manuscript.

AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 2 [ Weakness 1, Question 2 — Part 1 of 2 ]

The final evidence coverage is affected by both the memory construction and the retrieval processes. However, all configurations in Section 5.5 share the same memory construction snapshot. Therefore, the annotated sources not retained during the construction phase remain consistent across all configurations. The differences in coverage across different retrieval rounds in Figure 4 reflect the changes in the number of retrieved annotated evidence items under fixed memory conditions.

We agree that the final evidence coverage itself cannot distinguish between the evidence not retained during the construction phase and the evidence stored in the memory but not retrieved. To address this, we further conducted a phased analysis of the constructed memory and the final retrieval results. We analyzed the logs of three runs for each of the two backbones in the main experiment, with each run containing 2360 valid question and evidence pairs. In our statistics, we directly used the gold evidence annotated in the LoCoMo dataset [1] to calculate the coverage. This means that if a specific annotated source is not retained, it does not imply that equivalent supporting facts expressed by other sources are absent from the memory. In the following table, we report five metrics separately: extraction omissions, extracted but not retained, absent from memory, present in memory but not retrieved, and retrieved. The values represent the mean plus or minus the sample standard deviation across three runs. The first two items are subdivisions of absent from memory. The standard deviation of the total item is calculated from the total proportions of each run.

Table 1: Analysis of evidence source retention and retrieval coverage.

BackboneSource not cited in extraction outputSource cited but not retained in final memoryAbsent from memory (sum of the first two categories)Present in memory but not retrievedRetrieved
Gemini 2.5 Flash3.63 ± 0.13%0.18 ± 0.09%3.81 ± 0.18%23.38 ± 0.67%72.81 ± 0.63%
Claude Sonnet 4.52.94 ± 0.09%0.34 ± 0.07%3.28 ± 0.16%19.42 ± 0.25%77.30 ± 0.09%

Table 1 further distinguishes the stages where construction omissions occur. The proportions of extraction outputs from Gemini and Claude that did not cite the annotated sources are 3.63% and 2.94%, respectively. Meanwhile, the proportions of sources that were cited in the extraction outputs but ultimately not retained in the final memory are 0.18% and 0.34%, respectively. This indicates that, when tracked by annotated sources, the construction omissions primarily occur during the extraction output stage. After further aggregating the unretrieved evidence pairs across the three runs, we found that 85.97% for Gemini and 85.56% for Claude of the annotated evidence sources were already present in the memory. Therefore, in terms of the availability of annotated sources, most of the unretrieved evidence still exists in the memory, indicating that there is still room for improvement in the retrieval phase. We will consider this issue in our future work to improve the retrieval coverage of existing evidence and reduce source omissions during the extraction phase.

[1] Maharana, Adyasha, et al. "Evaluating very long-term conversational memory of llm agents." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 3 [ Weakness 1, Question 2 — Part 2 of 2 ]

Regarding entity merging errors, referring to the research by Ravideshik and Kejriwal [2] on judgment bias in the evaluation of large language model entity alignment, we employed GPT-4o-mini to conduct a blind automated audit of the entity merging results. The judge determined whether the entities were identical based on the original dialogue context of the associated parties. The original model, the merging decision, and its rationale were hidden in the audit input. The audit covered a total of 2452 alias associations across three runs for each of the two backbones. The results are as follows.

Table 2: Automated audit results of entity alias associations. The proportions represent the mean plus or minus the sample standard deviation across three runs, with each run using all confirmed and effective model determined alias associations as the denominator.

BackboneAlias associations across three runsFlagged as potentially incorrectInsufficient evidenceOutput validation failure
Gemini 2.5 Flash425 / 465 / 4459.93 ± 1.27%25.89 ± 2.07%2.47 ± 0.97%
Claude Sonnet 4.5363 / 370 / 38415.83 ± 1.31%34.05 ± 1.52%2.32 ± 0.39%

The above analysis supplements the source coverage analysis from the perspective of entity associations. Among the results, the suspected error label indicates that the automated judge determined the association might not hold based on the provided context. The insufficient evidence label indicates that the current audit input cannot support a definitive judgment. Therefore, we report these as automated diagnostic results rather than the true merging error rate verified by humans. Future work will incorporate more complete context and human calibration to improve the reliability of the association audit and further evaluate the actual impact of suspected erroneous associations on downstream retrieval and answering.

[2] Ravideshik, Vaibhava Lakshmi, and Mayank Kejriwal. “Reliability of LLM Judges for Evaluating Entity Alignment.” In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, 39175–39200. Budapest, Hungary: Association for Computational Linguistics. 2026.

Reviewer_M58pOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes EMIR², a training-free memory framework for long-term conversational agents. It builds a State-Evolving Memory Graph in which LLM-extracted events form topic-specific temporal chains, and per-topic semantic states are updated through reinforce, extend, revise, and uncertain decisions, keeping earlier states rather than overwriting them. A lightweight controller (Qwen3.5-9B) then performs intent-guided multi-round retrieval over this graph. Experiments on LoCoMo and MemConflict show improvements over several memory baselines, especially on multi-hop and temporal questions and on dynamic and static conflicts.

Critical strengths
  • The motivation is clear. Tracking how a fact changes over time, and not just its latest value, is an important and under-explored aspect of agent memory.
  • Separating state revision from event arrival, with explicit supersede and contradict links, is a clean design that supports questions about past states.
  • Results under the Gemini backbone are strong relative to the chosen baselines, and the white-box retrieval results on MemConflict static conflicts are notably better.
Critical weaknesses
  • Missing full-context baseline: LoCoMo conversations average roughly 16–18K tokens, which fits easily in the context windows of Gemini 2.5 Flash and Claude Sonnet 4.5. Recent work reports about 93% on LoCoMo by simply placing the whole conversation in context (e.g., GPT-5-mini, arxiv 2603.04814), above EMIR²'s 86.7 and 89.1. Without this baseline under the same backbone, it is hard to tell how much the memory pipeline actually helps on this benchmark.
  • Recent audits report that about 6% of LoCoMo gold answers are incorrect, and that a GPT-4o-mini judge with the standard prompt accepts a large share of answers that are on-topic but wrong. Given this, small differences such as +0.75 over MRAgent with Claude are difficult to interpret. A stricter judge, multiple judges, or a small human check would make the results more convincing. Evaluation on a less saturated benchmark such as LongMemEval would also help.
  • EMIR² uses an additional controller model and a multi-stage LLM construction pipeline, but there is no comparison of tokens or latency against the baselines. Under the Claude backbone, EMIR² only surpasses MRAgent on multi-hop and temporal questions when the controller is also upgraded to Claude.
Questions for authors
  • What is the full-context performance for each backbone, and how do EMIR²'s token cost and latency compare to those of the baselines?
  • Can you unify the numerical formats in Table 1 and Table 2?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 1

Response to Reviewer Feedback:

We thank the reviewer for the careful assessment and helpful feedback. We appreciate the reviewer’s engagement with our work and the insightful questions, which provide an opportunity to further clarify several aspects of our approach. We address the raised points below.

Response to Weakness 1 and Question 1:

Following the reviewer's suggestion, we evaluated the Full Context approach using Gemini 2.5 Flash and Claude Sonnet 4.5 as the backbone models on all 1,540 LoCoMo questions, with GPT-4o-mini consistently used as the judge model.

Table 1: Accuracy and answer-generation input token comparison between EMIR² and Full Context on LoCoMo.

BackboneMethodAccuracy (%)Input Tokens / Question
Gemini 2.5 FlashEMIR²86.693.46K
Gemini 2.5 FlashFull Context89.5527.68K
Claude Sonnet 4.5EMIR²89.075.32K
Claude Sonnet 4.5Full Context90.2627.82K

We acknowledge that Full Context achieves higher accuracy on LoCoMo. However, EMIR2EMIR^{2} substantially reduces the number of input tokens during answer generation, by approximately 87.5% and 80.9% under the two respective backbones. We also acknowledge that prompt caching significantly reduces the effective cost of Full Context in our dense evaluation setting, with cache-read ratios of 99.63% for Gemini and 95.27% for Claude. Nevertheless, the benefits of prompt caching depend heavily on query frequency, cache reuse, and history updates. In long-running agent scenarios, where interaction histories continuously grow and queries may occur intermittently, repeatedly processing the entire conversation history can still pose scalability challenges. In contrast, EMIR2EMIR^{2} maintains dynamically evolving structured memories, enabling more efficient management, organization, and access to continuously growing historical information.

Regarding the cost and latency comparison between EMIR² and the baselines, the baseline results in Table 1 of the manuscript are adopted from [1], which does not provide the cost and latency data required for a unified comparison. Reproducing all baselines under a unified experimental setting would require substantial additional implementation and evaluation efforts. In future work, we will further consider a detailed evaluation of the cost and inference latency of EMIR² and the baselines under a unified setting.

[1] Shuo Ji, Yibo Li, and Bryan Hooi. Memory is reconstructed, not retrieved: Graph memory for LLM agents. In Forty-third International Conference on Machine Learning, 2026.

AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 2

Response to Weakness 2:

We fully agree with your insightful observation regarding the presence of incorrect answers in the LoCoMo dataset and the limited reliability of GPT-4o-mini evaluations. During our experiments, we also noticed instances where correct answers were incorrectly penalized and incorrect answers were mistakenly accepted. These issues stem from the inherent limitations of both the dataset and the paradigm of using LLMs as judges. Admittedly, it is not feasible for us to comprehensively correct the LoCoMo dataset manually or conduct large scale human verification to completely eliminate such evaluation noise within the short timeframe of the rebuttal. However, we ensured that all models were evaluated under the exact same configuration in our experiments, meaning that all compared models are subject to the same evaluation noise. Therefore, the relative performance improvements observed under these controlled conditions can still objectively reflect the effectiveness of our proposed method compared to the baselines. We highly appreciate your suggestion to evaluate our method on less saturated benchmarks. This would effectively mitigate the limitations of the current evaluation and further validate the performance of EMIR2EMIR^{2}. Due to time and resource constraints, it is challenging to complete all these additional evaluations during the current rebuttal phase. We will certainly incorporate this highly constructive suggestion into our future work. We thank you once again for your professional and constructive feedback!

Response to Weakness 3:

Following your suggestion, we organized the logs of three main experiment runs for each of the two backbones, and additionally report the input and output token costs of the memory construction, retrieval, and final answering stages. The values in the table are the averages over the three runs, and each run contains 10 complete dialogues and 1,540 questions. Memory construction counts the cumulative tokens of multi stage LLM calls such as event extraction and semantic state updates; retrieval counts calls such as the controller, query decomposition, semantic compression, and reranking. The retrieval stage in both backbone configurations uses the local Qwen3.5-9B.

Table 2: Stage-wise LLM token usage of EMIR², averaged over three runs per backbone.

StageAccounting unitGemini: input / output tokensClaude: input / output tokens
Memory constructionCumulative usage per complete conversation417,503 / 167,827634,234 / 187,798
RetrievalCumulative usage per question26,157 / 1,31047,741 / 2,520
Answer generationCumulative usage per question3,478 / 1555,314 / 207

Memory construction and multi-round retrieval introduce additional overhead, while also filtering and integrating evidence relevant to the current question, reducing the historical information that the final answering model needs to process. The constructed memory can be reused by multiple subsequent queries, so the construction cost can be amortized across multiple queries. Under the above configurations, the final answering model receives on average approximately 3.48k and 5.31k input tokens per question; combined with the aforementioned Full Context experiment, the corresponding input tokens in the answer generation stage are reduced by approximately 87.5% and 80.9%, respectively. This reflects the design tradeoff of EMIR2EMIR^{2}: through upfront memory organization and evidence retrieval at query time, it provides the answering model with a more compact relevant context.

Regarding whether a lightweight controller is sufficient, we agree with the phenomenon pointed out by the reviewer: on questions involving multiple entities, events, or fine grained temporal constraints, a stronger controller can further improve performance through more accurate question decomposition and evidence completion. Appendix C provides a question by question analysis: among the 603 aligned multi hop and temporal questions, both controllers answer 508 questions correctly; after replacing with Claude, 31 questions change from incorrect to correct, while 16 questions change from correct to incorrect. This indicates that the gains from a stronger controller are concentrated on a subset of questions with more complex retrieval decisions. In the main experiment, EMIR² with the local Qwen3.5-9B controller already achieves an overall accuracy of 89.07% under the Claude backbone, higher than the 88.32% of MRAgent. These results support that a lightweight controller can handle the retrieval needs of most questions, while also revealing room for further improving controller capability on complex questions.

AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 3 [ Question 2 ]

We thank the reviewer for pointing out the inconsistency in numerical formatting. We have revised Table 2 in the main text and Tables 9 and 10 in the Appendix to report the numerical results as percentages (%), following the format of Table 1. This ensures consistent numerical presentation across the main text and Appendix.

Reviewer_Cb2EOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EMIR², a long-term memory framework for LLM agents that organizes conversation history into a graph of temporally connected events and evolving factual states that can be updated over time. It introduces an iterative retrieval mechanism that identifies missing evidence and performs additional searches over the graph. Experiments show improved performance on LoCoMo [1] and better handling of dynamic and static memory conflicts on MemConflict, although overall performance on MemConflict remains below the strongest baseline.

[1] Maharana, Adyasha, et al. "Evaluating very long-term conversational memory of llm agents." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Critical strengths

Critical Strengths

  1. Important and practically relevant problem. I like this paper because it addresses a fundamental challenge in long-term memory for LLM agents: maintaining evolving information and retrieving relevant evidence from long interaction histories. This is an important problem for building reliable agents that operate over extended periods, and the proposed combination of temporal memory organization and adaptive retrieval is a promising direction.

  2. Promising empirical results. EMIR² achieves strong results on LoCoMo [1], outperforming existing memory systems, particularly on temporal and multi-hop questions. It also achieves the highest accuracy on dynamic conflicts in MemConflict [2]. These findings suggest that the framework can improve long-term memory utilization, although further experiments are needed to isolate the contributions of its individual components.

[2] Tao, Zhen, et al. "MemConflict: Evaluating Long-Term Memory Systems Under Memory Conflicts." arXiv preprint arXiv:2605.20926 (2026).

Critical weaknesses

Critical Weaknesses

Overall assessment: The paper addresses an important problem and achieves promising results, particularly on LoCoMo. However, the evaluation does not convincingly establish that evolution-aware graph structure is responsible for these gains rather than the multi-round retrieval mechanism. Moreover, the weak performance on conditional conflicts raises concerns about the proposed semantic-state representation. Together with insufficient methodological detail and incomplete cost comparisons, these issues prevent me from confidently assessing the contribution and effectiveness of the proposed architecture.

  1. Insufficient evidence isolating the contributions of memory structure and iterative retrieval. The paper simultaneously introduces a new memory representation and retrieval mechanism, but does not adequately isolate their contributions. In Figure 3, removing multi-round retrieval decreases accuracy by 9.1 pp, while removing cross-chain edges has almost no effect (−0.2 pp) and removing semantic nodes has a smaller effect (−2.7 pp). These results suggest that iterative retrieval may account for most of the improvements, but do not establish the added value of the full graph representation over simpler structured or flat memory. Furthermore, the underlying retrieval engine is insufficiently specified, including how different search signals are combined, how the controller selects between graph expansion and query-based retrieval, and how intermediate candidates are ranked. A controlled comparison of flat versus graph-based memory under identical retrieval controllers and budgets, including query-only versus graph-enabled iterative retrieval, is needed to substantiate the claimed benefits of the proposed architecture.

  2. Insufficient validation of evolution-aware memory modeling. Although tracking evolving factual states is a central contribution, the evaluation does not convincingly demonstrate that explicit state evolution is responsible for the improvements. LoCoMo [1] primarily evaluates general long-term memory, while MemConflict [2] shows mixed results: EMIR² underperforms the strongest baseline overall (47.59% vs. 55.39%) and performs particularly poorly on conditional conflicts (43.24% vs. 84.49% for MemOS). Even on static conflicts, the improvement over MemOS is negligible (43.89% vs. 43.75%). These results raise questions about whether the proposed semantic-state representation can handle facts that are simultaneously valid under different conditions. Moreover, important design choices remain unclear, such as how partially updated states are represented. Controlled experiments isolating semantic-state evolution, together with evaluation on evolution-specific benchmarks such as MEME [3], are needed to substantiate this central contribution.

[3] Jung, Seokwon, et al. "MEME: Multi-entity & Evolving Memory Evaluation." arXiv preprint arXiv:2605.12477 (2026).

Questions for authors

Questions for Authors

  1. Contribution of the memory graph. The ablations suggest that multi-round retrieval drives most of the improvement (−9.1 pp without it), while removing cross-chain edges has almost no effect (−0.2 pp) and removing semantic nodes has a smaller effect (−2.7 pp). What evidence establishes that the proposed evolution-aware graph is necessary? Could the authors provide controlled comparisons of flat memory, temporal chains, and full SEMG under identical single-round and multi-round retrieval settings?

  2. Evaluation of evolving memory. The paper's main contribution is explicitly modeling factual evolution, yet LoCoMo [1] primarily evaluates general long-term memory and temporal reasoning. Could the authors evaluate on benchmarks specifically designed for evolving memory, such as MEME [3], and provide an analysis isolating questions that require tracking factual updates? In particular, how much improvement comes from semantic-state evolution rather than better retrieval?

  3. Computational efficiency. The paper evaluates retrieval cost across different round budgets but does not provide systematic end-to-end cost comparisons with baselines. Could the authors report memory construction cost, per-query inference cost, and latency, including a performance–cost comparison against simpler memory representations with iterative retrieval?

  4. Conditional conflict handling. Why does EMIR² substantially underperform MemOS on conditional conflicts (43.24% vs. 84.49%) despite its explicit semantic-state representation? How does the system represent multiple facts that are simultaneously valid under different conditions, and what are the main failure modes in these cases?

  5. Ablation fairness. Figure 3 states that the best result from the main experiments is used for the full model. Were all ablation variants evaluated with the same configurations, random seeds, and result-selection procedure? Otherwise, the reported performance gaps may overestimate the contributions of individual components.

  6. Retrieval implementation: Could the authors clarify the exact retrieval operations available to the controller? Specifically, what content is indexed (episodic nodes, semantic states, summaries, facts), which retrieval methods are used (embedding similarity, BM25, exact matching, etc.), and how are generated subqueries and structured constraints executed? It would also be useful to show an actual multi-round retrieval trace, including queries, selected graph-expansion anchors, retrieved nodes, and evidence-sufficiency decisions.

  7. Retrieval action selection: How does the controller decide between graph expansion and query-based retrieval? Could the authors report the frequency of each action and their respective contributions to retrieving relevant evidence? In particular, a controlled comparison between query-only multi-round retrieval and graph-enabled multi-round retrieval, using the same controller and retrieval budget, would help establish whether graph traversal provides benefits beyond iterative query reformulation.

  8. Reranker score clarification. Could the authors provide the exact implementation of the ranking function in Equation (4), including the weights and normalization of retrieval signals, the value of λ\lambda, and how the LLM relevance score is elicited? Additionally, is the same scoring mechanism used for intermediate candidate selection (Select_B), or only for final reranking?

Suggestions for Future Work

  1. Qualitative analysis. Could the authors provide a few concrete benchmark examples illustrating episodic chains, evolving semantic states, and multi-round retrieval trajectories? In particular, examples where the full SEMG succeeds but flat memory or iterative retrieval without graph structure fails would help clarify the contribution of the proposed representation.

  2. Evaluation reliability. Could the authors validate the LLM-based evaluation through human judgments, particularly on examples where EMIR² and the strongest baseline receive different correctness scores? Reporting human–LLM agreement and inter-annotator agreement would help establish the reliability of the reported performance differences.

  3. Modern memory-construction models. How does the effectiveness of the proposed memory structure change when using more capable memory-construction models (e.g., Claude Sonnet 5.5, Claude Opus 5.5, or GPT-6)? Could the authors evaluate whether the proposed graph structure remains beneficial compared with simpler memory representations when stronger models are used?

  4. Retrieval controller generalization. Appendix C provides some analysis of controller capacity, but could the authors systematically compare different retrieval controllers beyond Qwen3.5-9B? How sensitive are the results to controller choice, and does the graph structure provide consistent benefits across controllers?

  5. Controlled answer generation and oracle evaluation. Could the authors provide a controlled comparison where all methods use identical answer-generation models and retrieval budgets, and report answer accuracy with gold supporting evidence? This would help separate improvements in memory representation and retrieval from differences in answer generation, while establishing an oracle reference for answer accuracy.

21 comments
AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 1

Response to Reviewer Feedback:

We sincerely thank you for the thoughtful evaluation. The detailed and constructive comments have helped us improve our manuscript. We provide detailed responses and additional experimental analyses below.

Response to Weakness 1 and Question 1:

We thank the reviewer for pointing out the limitations in our experimental analysis, as the original experiments did not sufficiently distinguish the individual contributions of the retrieval mechanism with multiple rounds and the memory graph structure. To address this issue, we have added two ablation studies. First, the variant w/o graph-expansion retains the query rewriting with multiple rounds and the retrieval mechanisms based on queries while disabling the retrieval expansion based on graph structures, thereby isolating the effect of graph expansion. The results show that, compared to the full model, this variant decreases by 3.6 pp and 19.5 pp on the Overall and Multi-hop metrics, respectively. In comparison, the variant w/o multi-round, which disables the retrieval with multiple rounds, decreases by 9.1 pp and 18.4 pp on these two metrics, respectively. Both variants exhibit performance drops, especially on Multi-hop questions that require integrating multiple pieces of historical evidence. This indicates that the graph expansion provides complementary retrieval capabilities beyond iterative query rewriting, demonstrating that the performance improvement of EMIR2EMIR^{2} does not solely come from the retrieval mechanism with multiple rounds. Second, we introduced the variant w/o state-transition, which retains the semantic state nodes and other memory components while only removing the explicit state evolution relations. The Multi-hop accuracy of this variant drops by 15.6 pp, further indicating that explicit state transition relations help correlate different historical states and support complex multi hop evidence retrieval. The experimental results have been updated in Section 5.3 of the paper.

Regarding the structural comparison suggested by the reviewer, we have further organized and added a comparison among Temporal Chains, Flat Memory, and the full SEMG under single round and multi-round retrieval, with the results shown in Table 1. Specifically, Temporal Chains retains the episodic nodes and chain head nodes, removes the semantic state nodes, and only allows navigation along forward temporal relations. Flat Memory retains the node content of the full SEMG but disables graph expansion and virtual navigation. The full SEMG retains all node types and navigation mechanisms. The single round setting uses the original question to perform one full text retrieval, and after rule based ranking, the top 16 results are selected as the answering context, without query rewriting or graph expansion. Therefore, Flat Memory and the full SEMG follow the same retrieval process under this setting, and the two share the single round evaluation results.

Table 1: Supplementary comparison of memory and retrieval configurations on multi-hop questions. Scores are reported as percentages.

Memory StructureSingle-roundMulti-round
Temporal Chains59.665.6
Flat Memory67.466.3
Full SEMG67.485.8

As shown in the table, after removing the graph structure, Flat Memory does not achieve performance improvements even with the introduction of multi-round retrieval, indicating that under this configuration, increasing the number of retrieval rounds is not sufficient to guarantee better multi hop answering performance. The multi round performance of Temporal Chains is also lower than that of the full SEMG, suggesting that relying solely on episodic memory and forward temporal navigation still has limitations. The full SEMG performs best under the multi round setting, and this result pattern is consistent with the explanation that the memory graph structure and multi-round retrieval play complementary roles: query rewriting provides new retrieval entry points, while explicit graph relations support evidence association across historical states. Combined with the aforementioned ablations on graph expansion and state transition, these results further support the contribution of structured memory relations to multi hop evidence retrieval.

For specific details on the retrieval implementation, action selection, and candidate scoring, please refer to our responses to the corresponding Questions for Authors.

AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 2 [ Weakness 2 ]

We thank the reviewer for the attention to the effectiveness of evolutionary memory modeling and the details of state representation. Regarding the independent contribution and evaluation of explicit state evolution, we have added the ablation variant without state transition, which retains the semantic state nodes, other memory components, and the original multi-round retrieval mechanism, while only removing the explicit state evolution relations. Compared to the full model, the Overall and Multi-hop accuracies of this variant decrease by 3.2 and 15.6 percentage points, respectively. This shows that even with multi-round retrieval retained, removing the state evolution relations still causes a clear performance drop, providing direct evidence for the role of explicit evolution relations in supporting evidence association across historical states. This experiment and the discussion on the MEME benchmark are detailed in our response to Question 2.

Second, regarding the state representation for partial updates. As described in Section 4.1.2, a semantic node stores a set of valid facts, and updates are performed at the granularity of individual fact entries. The following table illustrates this mechanism with an example from an actual state transition process: the same node records two independent facts, “being interested in the Antarctic ice sheet” and “having never been to Boston”. When a new dialogue indicates that the user has visited Boston, the system only replaces the travel experience, while the interest fact is preserved unchanged in the successor state. The old node is retained as a historical version and connected to the successor node through a state transition relation. Therefore, the end of the validity interval of the old node indicates that the complete state version has been succeeded, rather than all facts within it becoming invalid at the same time; facts unaffected by the update are inherited, while changed facts can be traced through the historical versions.

Table 2: Illustration of a partial semantic-state update: the revised fact is updated while an independent, unaffected fact is preserved in the successor state.

Earlier dialoguePrevious semantic state s(k)s^{(k)}New dialogueSuccessor semantic state s(k+1)s^{(k+1)}
“I am fascinated by the Antarctic ice sheets.”   “I have never been to Boston.”Interest: Antarctic ice sheets.   Boston visit status: Never visited.“I had a wonderful experience during my time in Boston.”Interest: Antarctic ice sheets. [Preserved]   Boston visit status: Has visited. [Updated]

Finally, regarding conditional conflicts and overall answering performance. The current framework mainly targets updates of facts that change over time and the maintenance of historical versions, and the improvements of our model on dynamic and static conflicts demonstrate the effectiveness of this evolutionary memory; conditional conflicts, however, require distinguishing facts that are simultaneously valid under different conditions, and the two have different modeling requirements. Therefore, the effectiveness of explicit state evolution relations does not imply that the problem of conditional coexistence has been fully addressed. Further details are provided in our response to Question 4.

AuthorsOpenReview ↗

Response to Reviewer Feedback – Part 3 [ Question 2 ]

To address the concern that the original experiments did not fully demonstrate the role of explicit state evolution relations, we have introduced a new ablation variant without state transition. This variant retains the semantic state nodes and other memory components while only removing the explicit state transition relations. The experimental results show that, compared to the full model, the performance of this variant drops by 3.2 pp and 15.6 pp on Overall and Multi-hop, respectively. It is worth noting that this experiment still retains the original retrieval mechanism with multiple rounds. However, the Multi-hop performance still shows a significant decrease after removing the state transition relations. This indicates that explicit state evolution relations can help the model correlate different historical states and provide additional support for multi hop evidence retrieval. Therefore, semantic state evolution and the retrieval mechanism with multiple rounds are complementary, and both jointly enhance the capability of EMIR2EMIR^{2} in processing complex historical information. The relevant experimental results and analysis have been added to Section 5.3 and Figure 3 of the manuscript.

We greatly appreciate the reviewer's suggestion regarding the MEME benchmark [1] and fully recognize its value in evaluating evolutionary memory capabilities. Our paper primarily focuses on fact state updates based on historical interaction events, the maintenance of historical versions, and the modeling of their evolution relations, utilizing this information to support historical fact tracing and multi hop retrieval. In contrast, the Cascade and Absence tasks in MEME also involve cross entity dependency propagation, along with the resulting state cascade updates and uncertainty judgments. These tasks go beyond the current research and evaluation scope of our paper. We will certainly consider these forms of memory evolution in our future work.

[1] Jung, Seokwon, et al. "MEME: Multi-entity & Evolving Memory Evaluation." arXiv preprint arXiv:2605.12477 (2026).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.