acceptodds
Under review as a conference paper at ICLR 2027

G-Evolver: Learning to Evolve Graph Memory with Continual Structural Refinement

Abstract

Memory is central to how large language models (LLMs) sustain long-term, multi-session dialogue. As interactions accumulate, organizing ever-growing memories to support efficient retrieval becomes a critical challenge. Graph structures, which can explicitly model the relationships between memories, have emerged as a promising approach. Existing graph-based memory methods typically extract entity–relation triples from the corpus and link them via shared entities, organizing scattered knowledge into a unified graph. However, such paradigms suffer two fundamental drawbacks. Firstly, relationships based on shared entities fail to capture implicit causal and temporal relationships across sections, resulting in inferior retrieval results. Secondly, queries arrive continuously in real-world deployments while the constructed graph remains static, where structural flaws that are revealed during retrieval cannot be corrected, and rebuilding the entire graph is excessively costly. To this end, we propose G-Evolver, which turns the graph from a frozen topology into a structure that evolves continuously with the query stream. Concretely, G-Evolver decomposes evolution into two coupled decisions: an extractor first localizes the query-relevant subgraph to be evolved, and an evolver then applies repair operations to it before retrieval; operations verified by retrieval feedback are consolidated into the persistent graph, so that subsequent queries no longer re-encounter the same defects. To train the two modules jointly, we propose Graph Evolution Policy Optimization (GEPO), which turns the verification signal into hierarchical credit over the extraction and the evolution decisions. We further prove that continual evolution converges geometrically to the optimum. Across 4 memory QA benchmarks, G-Evolver consistently outperforms 16 competitive baselines

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Memory, Graph Memory, Agent Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_SiQXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

S1: Using retrieval feedback to repair graph memory, rather than keeping the graph fixed after construction, is a well-motivated idea. S2: The method outperforms 16 baselines on four benchmarks, and the learned policies transfer to GraphRAG and LightRAG graphs and to other backbones without retraining.

Critical strengths

S1: Using retrieval feedback to repair graph memory, rather than keeping the graph fixed after construction, is a well-motivated idea. S2: The method outperforms 16 baselines on four benchmarks, and the learned policies transfer to GraphRAG and LightRAG graphs and to other backbones without retraining.

Critical weaknesses

W1: The novelty is limited. Repairing graph links from feedback has been explored by FluxMem [1], which the paper identifies as the closest work but does not compare against. The three edit operations also resemble the add and update operations already learned by RL-based memory managers such as Memory-R1[2]. W2: The main results do not test the claimed continual evolution. W3: The extractor already identifies the evidence passages among the top 25 results, so the gain may come from this selection rather than from editing the graph.

[1] FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding [2] Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning

Questions for authors

Refer to Weakness.

5 comments
AuthorsOpenReview ↗

Response to Reviewer SiQX [Part 1]

Dear Reviewer SiQX:

Thank you so much for your thoughtful review and valuable feedback. We deeply appreciate the time and effort you devoted to evaluating our work.

We also sincerely apologize for the delay in our response. To address your comments as thoroughly as possible, we conducted a substantial number of additional experiments, along with a dedicated round of human annotation. Some of these experiments were completed only very recently, and we truly appreciate your patience. Below, we have provided detailed responses to your comments.

1. The Novelty of G-Evolver

Your Concern (W1): The novelty is limited. Repairing graph links from feedback has been explored by FluxMem [1], which the paper identifies as the closest work but does not compare against. The three edit operations also resemble the add and update operations already learned by RL-based memory managers such as Memory-R1 [2].

Thanks for this comment, and we apologize for any confusion. We first clarify the FluxMem reference, and then discuss FluxMem and Memory-R1 separately.

(1) Clarification of the FluxMem Reference. The FluxMem that we identify as the closest work in Appendix E is Fang et al. (2026), "Rethinking Memory as Continuously Evolving Connectivity" (arXiv:2605.28773). The reference [1] listed in the review, "FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding" (Xie et al., 2026, arXiv:2603.02096), is a different paper that shares the same name. It is a training-free visual token compression method for streaming video understanding with multimodal LLMs: it removes redundant visual tokens across adjacent frames and merges spatially redundant regions within each frame. It involves neither a memory graph nor retrieval feedback, and therefore does not repair graph links. Below, we discuss Fang et al. (2026), the work cited in our paper.

(2) Difference from FluxMem (Fang et al., 2026). First, FluxMem is a training-free method: when execution fails, an LLM is prompted to attribute the failure and choose an edit. G-Evolver instead learns where to edit (the extractor) and how to edit (the evolver) with GEPO, where each module and each operation is credited by the retrieval gain it brings to the query and its neighboring queries. Second, the two methods address different scenarios. FluxMem targets agentic task execution, such as web navigation on Mind2Web and general assistant tasks on GAIA, organizing memory into semantic, episodic, and procedural layers, and its feedback is the success signal of task execution or self-verification at each step. G-Evolver targets graph memory built over long-term interaction histories, such as HippoRAG2-style entity–passage graphs, and its feedback is whether a query retrieves the passages it requires. Third, the two methods edit different objects. FluxMem's link expansion and pruning act on the subgraph activated for the current step, connecting memory nodes to or disconnecting them from the current step anchor to refine the context of that step, and its long-term stage distills successful trajectories into procedural skills. G-Evolver edits the structure among stored memories themselves, namely the triples extracted from each passage and the edges between passages, and only edits that improve the current query without degrading any neighboring query are consolidated into the persistent graph to serve later queries. Regarding the comparison, FluxMem was released on arXiv on May 27, 2026, shortly before our submission, and its code had not been released at that time, so we could not faithfully compare against it under our setting.

(3) Difference from Memory-R1. The operations of G-Evolver and Memory-R1 act on fundamentally different objects. Memory-R1 manages a flat bank of textual memory entries: when a new dialogue turn arrives, its Memory Manager decides whether to ADD, UPDATE, or DELETE a stored entry or leave it unchanged (NOOP), and it is rewarded by whether the downstream Answer Agent answers correctly. It maintains no graph and no links between entries. G-Evolver never modifies the stored memory text. It edits the graph built over the memory: inserting a triple that a passage states but its extraction misses, linking two related passages with a semantic edge, and rewriting a triple that misrepresents its source passage, which itself stays unchanged. These edits are triggered at query time by the retrieval defects a query exposes, rather than at write time by incoming information, and are rewarded by the retrieval gain on the query and its neighboring queries rather than by answer correctness. In short, Memory-R1 decides what is stored, whereas G-Evolver decides how the stored memories are connected for retrieval.

We have revised Section 5 and added a detailed comparison with FluxMem and Memory-R1 to Appendix E.3 (Page 30), highlighted in dark red.

AuthorsOpenReview ↗

Response to Reviewer SiQX [Part 2]

2. The Continual Evolution Experiment

Your Concern (W2): The main results do not test the claimed continual evolution.

Thanks for raising this point. We would like to clarify that the main results adopt the per-query setting by design, while the continual setting is evaluated separately in Appendix C.3 (Table 7, Page 22). The reason is fairness: deciding whether a repair should stay in the graph requires retrieval feedback, and if this feedback came from the labels of the evaluation set, G-Evolver would receive supervision that none of the baselines has. The per-query setting therefore measures what a single repair brings without any such feedback, and Table 7 tests the full continual mode. We summarize this experiment below and extend it to show how performance develops along the query stream.

(1) Continual Evolution in Table 7. The queries of each conversation in LoCoMo and PersonaMem are randomly split into two halves. The first half simulates past interactions: its queries arrive in sequence, and the repair produced for each of them is written into the graph only if it passes the consolidation rule, using either the first-half labels with Eq. (5) or its label-free form judged by GPT-5-mini (Appendix C.3), where the neighbors checked are only the previously served queries. The second half simulates future interactions: it is evaluated on the graph accumulated from the first half, still with its own per-query repairs, and its labels are never used. The numbers in parentheses are gains over the per-query setting of the main results.

Per-query (main results)Continual (GPT-5-mini verification)Continual (gold-label verification)
LoCoMo F137.4838.43 (+0.95)39.89 (+2.41)
LoCoMo Judge58.0259.49 (+1.47)62.32 (+4.30)
PersonaMem-32K ACC58.7459.59 (+0.85)62.31 (+3.57)
PersonaMem-128K ACC49.1749.91 (+0.74)52.00 (+2.83)
Avg. Δ over HippoRAG2+5.80+6.67+8.78

Since the second half never contributes labels, these gains come from repairs consolidated for earlier queries that carry over to later ones, which is exactly what continual evolution claims. The gains hold on all four metrics even when consolidation is verified by GPT-5-mini instead of gold labels, so continual evolution does not depend on annotated feedback. This also means the main results are a conservative estimate of G-Evolver.

(2) Performance along the Query Stream. Table 7 reports only the final state after consolidation. To examine how performance develops as repairs accumulate, we replay the same protocol conversation by conversation on LoCoMo (10 conversations) and PersonaMem-32K (37 conversations) in their original order, and evaluate the full query set after every 10% of the conversations are processed. Conversations not yet processed are answered by plain retrieval on their initial graphs, so 0% corresponds to the unrepaired graphs and 100% coincides with the continual setting in Table 7. QA uses the same Qwen3-8B reader as the main experiments.

Processed Conversations0%10%20%30%40%50%60%70%80%90%100%Δ
LoCoMo F1 (GPT-5-mini)31.9432.4932.7533.6533.5034.1835.2835.8036.7537.1538.43+6.49
LoCoMo F1 (gold labels)31.9432.6633.0034.0834.6935.1236.3837.0638.1838.6939.89+7.95
PersonaMem-32K ACC (GPT-5-mini)53.9954.5054.8455.5255.8656.7157.0556.8857.7258.5759.59+5.60
PersonaMem-32K ACC (gold labels)53.9954.8455.1856.0356.5457.8958.4059.5960.2761.2962.31+8.32
  • Gains keep accumulating rather than saturating. With gold-label verification, performance increases at every checkpoint, reaching +7.95 F1 on LoCoMo and +8.32 ACC on PersonaMem-32K. At the query level, 650 LoCoMo queries improve versus 230 that degrade (p < 1e-40), and 65 versus 16 on PersonaMem-32K (p < 1e-7).
  • The same trend holds without gold labels. With GPT-5-mini verification, the gains reach +6.49 and +5.60. The only dips (40% on LoCoMo and 70% on PersonaMem-32K) are small and come from occasional false positives of the LLM verifier, and the curves recover at the next checkpoint.

Together, Table 7 and the stream analysis show that continual evolution is tested and supported: repairs consolidated from past interactions benefit future queries, and the benefit grows as more interactions are processed. The stream analysis has been added to Appendix C.4 (Page 23), highlighted in dark red.

AuthorsOpenReview ↗

Response to Reviewer SiQX [Part 3]

3. The Direct-Use Ablation of the Extractor

Your Concern (W3): The extractor already identifies the evidence passages among the top 25 results, so the gain may come from this selection rather than from editing the graph.

Thanks for this valuable point. If the gain came from selection alone, then giving the reader the passages selected by the extractor, without any graph edit, should perform as well as G-Evolver. We test this directly, and further compare with a reranker trained for the same task on the same candidates, which is a stronger form of selection than the extractor itself.

Setup. Both variants start from the same top-25 candidates as G-Evolver, and the reader always receives three passages, as in the main experiments.

  • Extractor only. The same trained extractor checkpoint selects evidence from the top-25 candidates, but no graph edit is performed. The selected passages are placed first in the reader's context, and the remaining slots are filled in the original retrieval order.
  • Task-trained reranker. A Qwen3-8B reranker is trained with GRPO on the same 500 training queries, using the retrieval score in Eq. (6) as the reward and the same training budget as the other training-based baselines (150 steps, 8 queries per step, 4 rollouts per query). Its top-3 passages among the same top-25 candidates are passed to the reader.
Extractor onlyTask-trained rerankerG-Evolver
LoCoMo F1 / Judge34.78 / 54.4636.72 / 56.8937.48 / 58.02
LongMemEval-S F1 / Judge34.02 / 45.1835.87 / 47.8036.44 / 48.57
PersonaMem-32K ACC / MRR55.18 / 73.0857.56 / 75.2258.74 / 75.89
PersonaMem-128K ACC / MRR46.10 / 66.5548.22 / 68.3149.17 / 68.87
PrefEval-Explicit ACC / MRR74.50 / 83.4577.10 / 85.5378.00 / 86.08
PrefEval-Implicit ACC / MRR55.30 / 70.8257.50 / 72.0858.70 / 72.81

Analysis.

  • Selection alone does not explain the gain. With the same extractor and the same candidates, removing graph editing lowers performance by 2.0–3.6 points on every metric of all six benchmarks, so this part of the gain can only come from the edits. The extractor's output is also not meant to be a final answer context: it selects the passages that the evolver may edit, and its reward includes the evolution gain RevoR_{\mathsf{evo}} (Eq. (8)), so it may keep passages that are useful for linking to the evidence but do not contain the answer themselves.
  • Even a selector trained for this task falls short. The task-trained reranker is stronger than the extractor, but G-Evolver still leads on every metric, by 0.55–1.20 points. Any selection method can only reorder the fixed top-25 candidates. G-Evolver instead changes the graph and reruns Personalized PageRank on the modified structure (Eq. (4)), so its edits change how relevance propagates and can bring in evidence that no reordering of the original candidates would reach.
  • Edits also benefit later queries, which selection cannot do. A selection is made for one query and leaves nothing behind. Graph edits persist: as shown in our response to W2, edits consolidated from earlier queries further improve later ones (Table 7), a benefit that selection by construction cannot provide.

Therefore, selecting evidence among the top-25 candidates, even with task-specific training, cannot account for the improvement of G-Evolver, and the remaining gain comes from repairing the graph. This ablation and the task-trained reranker comparison have been added to Appendix C.7 (Page 25), highlighted in dark red.

Thanks again for the time and effort you have invested, and we apologize once again for our delayed response. We would be sincerely grateful if you could find some time to go over our rebuttal!

We hope these responses address your concerns comprehensively. Feel free to let us know if you have any additional questions, and we are more than happy to provide further clarification on any aspect of our work.

Respectfully and sincerely,

The Authors

Reviewer_2DXmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes G-Evolver improving graph-based conversational memory using two learned modules. An extractor selects a query-relevant subgraph, and an evolver repairs it by adding missing triples, correcting incorrect triples, or connecting related passages. GEPO trains these modules with retrieval-based rewards and separate credit assignment. The proposed framework also verifies useful edits before retaining them for future queries. Experiments cover four memory benchmarks, and a conditional theorem establishes geometric convergence of a retrieval score.

Critical strengths
  • The problem is well motivated and the graph local editing is a reasonable response that avoids rebuilding the entire graph.

  • Extensive experimentation across four long-term conversational memory benchmarks demonstrates that G-Evolver consistently outperforms a comprehensive suite of 16 baselines.

Critical weaknesses
  • The paper motivates G-Evolver through missing implicit causal/temporal relationships and frozen graph topology. However, improved retrieval and QA do not directly demonstrate that the method correctly recovers those relationships.

  • All main experiments discard edits after each query, leaving limited evidence that persistent graph evolution reliably benefits subsequent queries.

  • Neither answer judging nor edit verification is validated against human judgments, and the editing policies are tested only with Qwen3-8B despite additional reader-backbone experiments.

Questions for authors
  • Did you evaluate if generated edits genuinely recover implicit causal and temporal relationships, or are benefits inferred solely from end-to-end QA metrics?

  • How does performance change under a fully persistent continual evolution deployment across multiple interactions, and what prevents error accumulation?

  • Was any human evaluation conducted to measure the precision and false-positive rate of the automated retrieval feedback and LLM judgment gate?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 2DXm [Part 1]

Dear Reviewer 2DXm:

Thank you so much for your thoughtful review and valuable feedback. We deeply appreciate the time and effort you devoted to evaluating our work.

We also sincerely apologize for the delay in our response. To address your comments as thoroughly as possible, we conducted a substantial number of additional experiments, together with the human validation you suggested, which required a dedicated round of manual annotation. Some of these experiments were completed only very recently, and we truly appreciate your patience. Below, we have provided detailed responses to your comments.

1. Evidence of Recovered Implicit Relationships

Your Concern (W1): The paper motivates G-Evolver through missing implicit causal/temporal relationships and frozen graph topology. However, improved retrieval and QA do not directly demonstrate that the method correctly recovers those relationships.

Your Concern (Q1): Did you evaluate if generated edits genuinely recover implicit causal and temporal relationships, or are benefits inferred solely from end-to-end QA metrics?

Thanks for this insightful comment, and we apologize for any confusion caused. Below, we demonstrate from three perspectives that G-Evolver genuinely recovers the implicit temporal and causal relationships missing from the memory graph: (1) real cases, (2) per-category performance, and (3) counterfactual attribution.

(1) Real Cases. To verify that G-Evolver recovers temporal and causal relations, we release 30 real cases from LoCoMo, LongMemEval, and PersonaMem [link]. Below we show the three most representative ones, one from each benchmark. For each case, we quote the relevant memory, the edit produced by G-Evolver, and the resulting retrieval change. Recall@3 is computed against the annotated evidence sessions.

Case 1 (LoCoMo, temporal relation).

  • Query: "How long did James and Samantha date for before deciding to move in together?" (Answer: nearly three months)
  • Memory: The duration is never stated. It can only be inferred from the order of events across three sessions:
    • session_19 (10 Aug 2022): "I even met one beautiful girl. I'm thinking of asking her out on a date! ... She is Samantha."
    • session_23 (4 Sep 2022): "Yesterday, when we were at the theater, ... I asked her to become my girlfriend, and she agreed."
    • session_29 (31 Oct 2022): "Samantha and I have decided to move in together!"
  • Edit: The original graph has no path to session_29, so retrieval returns session_19 and session_23 but misses the move-in decision. G-Evolver adds a document-to-document edge session_23 → session_29, described as "session_23 covers asking Samantha to become his girlfriend, and session_29 covers their later decision to move in together."
  • Effect: Recall@3 rises from 0.5 to 1.0 as session_29 enters the top-3.
  • What it shows: G-Evolver recovers the before/after relation between two events recorded in different sessions, which is exactly the implicit temporal relation needed to derive the duration.

Case 2 (LongMemEval, temporal relation).

  • Query: "How much faster did I finish the 5K run compared to my previous year's time?" (Answer: 10 minutes)
  • Memory: The two required facts appear in two sessions on unrelated topics:
    • 872e8da2_1_chunk_0 (a 5K training plan): "I've done a 5K run last year, but it took me 45 minutes to complete."
    • 872e8da2_2_chunk_0 (a charity bake sale): "By the way, I just got back into running and recently finished a 5K in 35 minutes."
  • Edit: Since the second session is about a bake sale and mentions the recent run only in passing, it is not retrieved. G-Evolver adds an edge 872e8da2_1_chunk_0 → 872e8da2_2_chunk_0, described as "872e8da2_2_chunk_0 provides the user's recent 5K finish time (35 minutes) that can be compared to the previously stated 45-minute time in 872e8da2_1_chunk_0."
  • Effect: Recall@3 rises from 0.5 to 1.0.
  • What it shows: G-Evolver links an earlier and a later state of the same activity across sessions, recovering the temporal comparison that neither session states on its own.
AuthorsOpenReview ↗

Response to Reviewer 2DXm [Part 2]

Case 3 (PersonaMem, causal relation).

  • Query: "I recently put together some music based on a concert I attended." (Expected response: recall that the user enjoys creating personal playlists blending classics and newer tracks)
  • Memory: session_11_chunk_2: "Just the other day, I created a playlist inspired by a recent live concert experience I had. ... I included some of the classics along with some newer tracks."
  • Edit: Although the passage states this relation, the originally extracted triples miss it, so the session ranks only 10th. G-Evolver adds the triple (playlist | is inspired by | recent live concert experience), together with two triples recording that the playlist includes classics and newer tracks.
  • Effect: The evidence session rises from 10th to 1st, and Recall@3 rises from 0 to 1.0. Keeping only the causal triple still ranks it 1st, whereas removing it leaves the session at 7th.
  • What it shows: G-Evolver recovers a causal relation that the initial extraction missed, and this relation alone accounts for the retrieval gain.

More detailed versions of these three cases are provided in Appendix D (Pages 25–29).

(2) Per-Category Performance. The per-category results in Appendix C.1 (Table 6) further show that G-Evolver improves on the question categories that require temporal or cross-session reasoning. We summarize below the Judge scores on the multi-hop and temporal reasoning categories of LoCoMo and the temporal reasoning and multi-session categories of LongMemEval.

LoCoMo Multi-hopLoCoMo Temporal ReasoningLongMemEval Temporal ReasoningLongMemEval Multi-session
HippoRAG242.5539.2516.5423.14
Strongest Baseline54.9641.7421.2635.54
G-Evolver57.0951.0922.8340.50
Δ vs. HippoRAG2+14.54+11.84+6.29+17.36

Since G-Evolver shares its retriever and initial graph with HippoRAG2, the gap between the two directly reflects the effect of the edits. On all four categories, G-Evolver improves over HippoRAG2 by 6.29 to 17.36 points and also outperforms the strongest baseline, most notably on LoCoMo temporal reasoning (+9.35) and LongMemEval multi-session (+4.96). These results provide supporting evidence, but they remain end-to-end metrics. We therefore further isolate the contribution of temporal/causal edits below.

(3) Counterfactual Attribution. To verify that the recovered relations indeed drive the gains, we perform a counterfactual analysis on 132 cases from the four benchmarks in which G-Evolver's edits encode temporal or causal relations (identified by GPT-5) and improve retrieval. On the same graph state, we compare Recall@3 (%) before the edits, with all edits, and with the temporal/causal edits removed.

LoCoMoLongMemEvalPersonaMemPrefEvalAll
Before Edits1.241.00.01.85.2
With Edits97.593.6100.080.090.3
w/o Temporal/Causal Edits27.559.054.219.131.9

Removing only the temporal/causal edits loses about 69% of the gain overall, and the drop is consistent across all four benchmarks, confirming that the recovered temporal and causal relations are the main source of the improvement.

These results confirm that the benefits of G-Evolver are not inferred solely from end-to-end metrics: the edits recover concrete temporal and causal relations grounded in the conversation, and in the cases where they occur, these edits account for most of the retrieval gains.

AuthorsOpenReview ↗

Response to Reviewer 2DXm [Part 3]

2. Continual Graph Evolution across Multiple Interactions

Your Concern (W2): All main experiments discard edits after each query, leaving limited evidence that persistent graph evolution reliably benefits subsequent queries.

Your Concern (Q2): How does performance change under a fully persistent continual evolution deployment across multiple interactions, and what prevents error accumulation?

Thanks for this important question, and we apologize for any confusion caused by the description in the main text. Discarding edits after each query in the main experiments is intentional. In general, consolidating an edit requires verifying it with feedback from retrieval metrics, and if this feedback relied on the labels of the evaluation set, the comparison with the baselines, none of which has access to such labels, would be unfair. The persistent setting has already been evaluated in Appendix C.3 (Table 7, Page 22), and we further track how its performance changes along the query stream.

(1) Existing Continual Evolution Experiment. As reported in Table 7, for each conversation in LoCoMo and PersonaMem, the queries are randomly shuffled and split into an earlier and a later half. The earlier half is processed in sequence: each query receives a provisional repair, which is consolidated into the graph only if it passes the consolidation rule, using either the labels of the earlier half with Eq. (5) or its label-free form judged by GPT-5-mini (Appendix C.3), where the earlier queries play the role of past interactions. The later half is then served on the consolidated graph, where each query still receives its own provisional repair, and its labels are never used.

LoCoMo F1LoCoMo JudgePersonaMem-32K ACCPersonaMem-128K ACCAvg. Δ over HippoRAG2
G-Evolver (per-query)37.4858.0258.7449.17+5.80
G-Evolver (continual, verified by GPT-5-mini)38.4359.4959.5949.91+6.67
G-Evolver (continual, verified by gold labels)39.8962.3262.3152.00+8.78

Consolidating verified repairs improves every metric, even when verification is performed by GPT-5-mini rather than annotated labels. Later queries thus benefit from repairs made for earlier ones instead of re-encountering the same defects.

(2) Evolution over the Entire Query Stream. To show how performance changes as repairs accumulate, we further track the continual setting of Table 7 along the query stream on LoCoMo (10 conversations) and PersonaMem-32K (37 conversations), with feedback from either GPT-5-mini or the earlier-half labels. Since each conversation has its own graph, we process the conversations one after another in their original order, each following the protocol in (1), and evaluate all queries of the dataset after every 10% of the conversations, i.e., after every conversation on LoCoMo and about every four on PersonaMem-32K. At each point, queries of the processed conversations are served as in (1), while those of the remaining conversations are served by plain retrieval on their initial graphs without any repair, so the results move from the unrepaired initial graphs (0%) to the continual setting applied to all conversations (100%). QA uses the same Qwen3-8B reader as the main experiments.

Results.

Processed Conversations0%10%20%30%40%50%60%70%80%90%100%Δ
LoCoMo F1 (GPT-5-mini)31.9432.4932.7533.6533.5034.1835.2835.8036.7537.1538.43+6.49
LoCoMo F1 (gold labels)31.9432.6633.0034.0834.6935.1236.3837.0638.1838.6939.89+7.95
PersonaMem-32K ACC (GPT-5-mini)53.9954.5054.8455.5255.8656.7157.0556.8857.7258.5759.59+5.60
PersonaMem-32K ACC (gold labels)53.9954.8455.1856.0356.5457.8958.4059.5960.2761.2962.31+8.32

Analysis.

  • QA improves steadily as the graphs evolve. Under gold-label verification, LoCoMo F1 rises monotonically by 7.95 points (650 queries improve vs. 230 degrade, p < 1e-40), and PersonaMem-32K ACC rises monotonically by 8.32 points (65 vs. 16 queries, p < 1e-7). Under GPT-5-mini verification, LoCoMo F1 rises by 6.49 points and PersonaMem-32K ACC by 5.60 points, with minor dips at isolated conversations due to occasional false positives from the judgment gate.

Comments on the paper

6 comments
AuthorsOpenReview ↗

Response to Reviewer GkG6 [Part 1]

Dear Reviewer GkG6:

Thank you so much for your thoughtful review and valuable feedback. We deeply appreciate the time and effort you devoted to evaluating our work, and we also sincerely apologize for the delay in our response. To address your comments as thoroughly as possible, we conducted a substantial number of additional experiments. Some of these experiments were completed only very recently, and we truly appreciate your patience. Below, we have provided detailed responses to your comments.

1. The Protocol Details of the Continual Refinement Experiment

Your Concern (W1, Part 1): ... the referenced two-batch protocol is not actually specified: the partition, query order, available feedback, and construction of the verifier's neighboring-query set are unclear. In particular, Eq. (6) requires the required-passage set P*, but the paper does not explain how the non-oracle verifier obtains an equivalent signal without gold annotations.

Your Concern (Q1, Part 1): For the GPT-5-mini continual setting in Table 7, what exactly are the two batches, what information is available when an edit is accepted, and how are P* and the neighboring queries approximated?

Thanks for this careful comment, and we apologize for any confusion caused by the unclear description of the continual setting in the paper. Below, following the order of your questions, we clarify it from four aspects: (1) the two-batch protocol, (2) the feedback available when an edit is accepted, (3) the neighboring queries, and (4) the approximation of P⋆\mathcal{P}^\star.

(1) Two-Batch Protocol. For each conversation, we randomly shuffle its queries and split them into two halves, which form the two batches. The queries in the first batch are processed sequentially in the shuffled order: for each query, G-Evolver repairs the graph provisionally as in the per-query setting, the verifier then decides which edits are consolidated into the graph, and later queries in the first batch are served on the updated graph. After the first batch is processed, the consolidated graph is used to serve the queries in the second batch. Each of these queries still receives its own provisional repair, which is discarded afterward, so no edit is consolidated from the second batch.

The continual results in Table 7 are computed over all queries, where each query in the first batch is evaluated on the answer produced when it was served during consolidation. Neither the queries nor the labels of the second batch take part in deciding which edits are consolidated.

(2) Available Feedback. The information available when an edit is accepted differs between the two continual settings in Table 7.

  • w/ gold labels (†). The verifier uses the annotated evidence passages of the current query and of its neighboring queries defined in (3), and applies the consolidation rule in Eq. (5) with the retrieval score in Eq. (6).
  • w/ GPT-5-mini judge. The verifier only has access to the texts of the current query and its neighboring queries, together with their top-3 retrieved passages before and after the edit. It uses neither gold evidence passages nor gold answers, and has no access to the second batch.

(3) Neighboring Queries. The neighboring queries are defined differently in training and inference. When constructing the training set, we sample groups of related queries from the same conversation, and the neighbors of a training query are the queries in its group whose top-3 retrieved passages overlap with its own. In the main experiments, edits are discarded after each query, so no verification is performed and no neighbor needs to be computed. In both continual settings, the neighbors of a query are the queries in the first batch that have already been served before it and whose top-3 retrieved passages are changed by the edit. Queries that have not yet been served are never included.

AuthorsOpenReview ↗

Response to Reviewer GkG6 [Part 2]

(4) Approximation of P⋆\mathcal{P}^\star. P⋆\mathcal{P}^\star is used only during training, where it is available from the annotations of the training queries and is used to compute the retrieval score in Eq. (6). Inference in the main experiments does not require it. In the GPT-5-mini continual setting, we replace the score comparisons in Eq. (5) with judgments on the retrieved passages, and verify the edits of each round as a whole.

  • Source query. If a round leaves the top-3 retrieved passages of the source query unchanged, it brings no retrieval gain and is not consolidated. Otherwise, GPT-5-mini receives the query together with its top-3 passages before and after the edit, and judges which set better supports answering the query. The round can be consolidated only if the post-edit passages are judged better, which replaces the strict-improvement condition in Eq. (5).
  • Neighboring queries. For each neighbor, the same judge compares its top-3 passages before and after the edit. The round is consolidated only if the neighbors judged improved are no fewer than those judged degraded, which replaces the no-regression condition in Eq. (5).

Therefore, at inference time, P⋆\mathcal{P}^\star is used only in the "w/ gold labels" row of Table 7, which serves as an oracle reference for consolidation under an exact signal.

Once again, we apologize for the confusion caused by the unclear description of this protocol in the paper. We have added the complete protocol of the continual setting to Appendix C.3 (Pages 22–23) and the construction of the neighboring queries and P⋆\mathcal{P}^\star during training to Appendix B.3 (Page 20), and pointed to both from the consolidation rule in Section 2.2 (Page 3), all highlighted in dark red.

2. The Label-Free Sequential Evaluation

Your Concern (W1, Part 2): The theorem does not resolve this gap: Section 3 checks every query in the finite support and assumes uniform positive κ-progress, whereas the practical rule checks only overlapping neighbors. These are legitimate assumptions for a conditional result, but they do not establish non-regression of the deployed procedure. A fully specified, label-free sequential evaluation with variation across query orders and measured regressions would materially strengthen the claim.

Your Concern (Q1, Part 2): Can you report held-out subsequent-query gains and regressions over multiple query orders, with no gold evidence or future-query labels used for consolidation?

Thanks for this constructive suggestion. To verify that the deployed procedure benefits subsequent queries without any gold labels, we evaluate it directly under multiple query orders and track its performance along the query stream.

Before presenting the experiments, we apologize for any confusion caused by the paper and would like to clarify: consolidation in Table 7 never uses the labels of future queries (Section 1). The GPT-5-mini setting uses no gold evidence or gold answers at all, and its verifier only compares the retrieved passages before and after each edit. Even the gold-label setting only uses the retrieval feedback of the first-batch queries that have already been served, which play the role of past interactions, while the queries and labels of the second batch never take part in consolidation.

Setup. We follow the GPT-5-mini continual setting of Table 7 and keep the checkpoint, the verifier, the two-batch protocol, and the reader unchanged, varying only the order of the queries within each conversation.

  • Random orders. We shuffle the queries with three random seeds, where Seed 1 is the order used in Table 7.
  • Evidence-sorted order. To examine how sensitive graph evolution is to the overlap between earlier and later queries, we sort the queries of each conversation by the session ID of their annotated evidence before splitting them into two batches. The first batch then mainly concerns earlier sessions and the second batch later ones, so the repairs consolidated from the first batch overlap less with the evidence required by the second batch.
AuthorsOpenReview ↗

Response to Reviewer GkG6 [Part 3]

In the table below, the last five rows are G-Evolver under continual evolution with the corresponding query order.

LoCoMo F1LoCoMo JudgePersonaMem-32K ACCPersonaMem-32K MRRPersonaMem-128K ACCPersonaMem-128K MRRAvg. Δ over HippoRAG2
HippoRAG229.7845.3154.6772.6844.9665.95–
G-Evolver (per-query)37.4858.0258.7475.8949.1768.87+5.80
Random order (Seed 1)38.4359.4959.5976.5649.9169.37+6.67
Random order (Seed 2)38.9860.3460.2777.0950.6169.85+7.30
Random order (Seed 3)38.5859.7259.7676.7050.0269.45+6.81
Random order (Mean ± Std)38.66 ± 0.2359.85 ± 0.3659.87 ± 0.2976.78 ± 0.2250.18 ± 0.3169.56 ± 0.21+6.93 ± 0.27
Evidence-sorted order37.9658.7659.2576.2949.6969.22+6.30

Analysis.

  • The gains are robust to the query order. Under all three random orders, continual evolution improves over the per-query setting on every metric, with an average gain of +6.93 ± 0.27 over HippoRAG2 compared with +5.80 for the per-query setting. The small variance across seeds shows that the benefit of consolidation does not depend on a favorable order.
  • The gains come from later queries reusing earlier repairs. The evidence-sorted order yields the smallest gain (+6.30), since the repairs consolidated for earlier sessions are rarely needed by the queries on later sessions. It still remains above the per-query setting, indicating that consolidation does not harm later queries even when few repairs can be transferred. This confirms that the benefit of continual evolution stems from later queries reusing the repairs made for earlier ones.

Performance along the Query Stream. We further track the GPT-5-mini continual setting along the query stream (Appendix C.4). Since each conversation has its own graph, we process the conversations one after another and evaluate all queries of the dataset after every 10% of the conversations, where the queries of the remaining conversations are served by plain retrieval on their initial graphs.

Processed Conversations0%10%20%30%40%50%60%70%80%90%100%Δ
LoCoMo F131.9432.4932.7533.6533.5034.1835.2835.8036.7537.1538.43+6.49
PersonaMem-32K ACC53.9954.5054.8455.5255.8656.7157.0556.8857.7258.5759.59+5.60

Without any gold labels, LoCoMo F1 rises by 6.49 points and PersonaMem-32K ACC by 5.60 points as the graphs evolve, with only minor fluctuations at isolated points and no sustained decline, indicating that the occasional errors of the verifier do not accumulate along the stream.

We have added this experiment and the corresponding analysis to Appendix C.4 (Page 23), highlighted in dark red.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.