- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies GraphRAG as the underlying triple store keeps growing. It identifies a phenomenon it calls path dilution: when new triples and edges are inserted, the Personalized-PageRank mass that previously flowed from query seeds to supporting documents is split among the newly added neighbors, so the same query at a later step no longer surfaces the document it used to. The proposed framework, EviWeaver, (i) decomposes the query into atomic sub-questions and builds bridge-aware seed distributions over both query-mentioned and intermediate bridge entities; (ii) after retrieval, runs a backward PPR from the retrieved documents to score each traversed edge by its contribution, retaining the top edges as a per-query path memory; and (iii) at later steps either strengthens the stored edges at retrieval time (a query-specific weight bonus) or refreshes the memory when a staleness statistic s_new crosses a threshold. All adaptation happens at retrieval time — no retraining, no structural graph edit — and the method is evaluated on MuSiQue, 2WikiMultiHopQA, and HotpotQA under fixed-, query-expanding, and delayed-evidence protocols.
Critical strengths
- The observation that prior incremental / continual GraphRAG work updates the graph or the retriever without explicitly preserving which path made an old query succeed is a reasonable one, and the "path dilution" vocabulary is useful.
- Backward attribution of PPR flow to individual edges (rather than to nodes or documents) is a clean and implementable idea.
Critical weaknesses
-
The central problem is scoped to a special case, not characterized as such. The paper treats "corpus expansion dilutes previously correct evidence" as a general law of continual GraphRAG. It is not. It holds only under an implicit assumption that the triples/documents added during expansion are irrelevant distractors. When the newly added content is correct, on-topic, and supplementary, a retriever that changes its evidence set is not failing — it is behaving correctly. The paper never separates three qualitatively different expansion regimes: (a) noise/distractor insertion, (b) complementary correct evidence, (c) corrections or superseding facts. The phenomenon EviWeaver defends against is regime (a). As written, the abstract and introduction overclaim beyond that.
-
The "good evidence" and "memory is stale" signals are not independently validated. Refresh is triggered by s_new, the share of query–triple similarity mass assigned to triples outside the stored memory. The paper treats this as evidence that new evidence is better. But: (i) there is no ground-truth comparison at refresh time of new-vs-old quality — the system cannot actually know whether the new triples are better; (ii) in the worst case the system must pay for both a memory-augmented pass over the old rails and an unweighted exploration of the new content, and the paper does not quantify this dual-search cost; (iii) "supporting document" is operationalized as the benchmark gold passage, which does not transfer to open-ended settings where relevance is multi-faceted. The decision boundary that drives the whole method is therefore heuristic and under-justified.
-
Edge re-weighting is the wrong primitive for a sizable class of queries. The method systematically biases importance toward previously traversed edges. This is defensible when the task is "find the one gold passage that makes this multi-hop fact true" — exactly the HotpotQA / MuSiQue / 2WikiMultiHopQA family. It breaks for aggregative queries — "how many documents mention X?", "which topics co-occur with X?", coverage/trend questions. On such queries every newly added document should count, and there is no principled notion of edges worth preserving versus edges worth diluting; the method would, by construction, under-count new evidence. The paper does not discuss this boundary at all.
-
Backward attribution is noisy and can over-commit to incidental paths. STPPR assigns nonzero credit to essentially every edge on the flow paths; selecting a top-k and then up-weighting them (with an explicit bias toward low-degree entities) encodes a strong prior that the first successful traversal was the right one. Under a noisy seed, a spurious bridge, or a one-off good hit, this incident is baked into memory and then strengthened on every future query. The ablation in Table 1 removes modules, but does not measure the false-memory rate — how often the stored path was, on a later oracle check, wrong, and how much strengthening amplifies that error.
Questions for authors
Q1. Can you report a regime where newly added documents are correct and supplementary (e.g., a new edition of the same topic) rather than distractors? Does EviWeaver still help, or does path memory now fight against legitimate new evidence?
Q2. When s_new triggers a refresh, how often does the system actually rebuild, and what is the retrieval-time cost of running both the memory-augmented PPR and the fresh bridge-aware construction?
7 comments
Response to Reviewer MjRm (Part 2/6)
W1 & Q1. Expansion with Complementary Correct Evidence
The central problem is scoped to a special case, not characterized as such. The paper treats "corpus expansion dilutes previously correct evidence" as a general law of continual GraphRAG. It is not. It holds only under an implicit assumption that the triples/documents added during expansion are irrelevant distractors. When the newly added content is correct, on-topic, and supplementary, a retriever that changes its evidence set is not failing — it is behaving correctly. The paper never separates three qualitatively different expansion regimes: (a) noise/distractor insertion, (b) complementary correct evidence, (c) corrections or superseding facts. The phenomenon EviWeaver defends against is regime (a). As written, the abstract and introduction overclaim beyond that.
Can you report a regime where newly added documents are correct and supplementary (e.g., a new edition of the same topic) rather than distractors? Does EviWeaver still help, or does path memory now fight against legitimate new evidence?
Response to W1 and Q1.
Your distinction among these three expansion regimes helped us clarify the scope of our evaluation. Our evaluation covers both regimes (a) and (b), including complementary evidence arriving later in the delayed-evidence setting.
- (a) Noise/distractor insertion. The fixed-query protocol of HippoRAG2 and our query-expanding setting evaluate retrieval as additional documents arrive.
- (b) Complementary correct evidence. Our delayed-evidence setting releases some supporting documents after step 250, removing the assumption that all required evidence is available from the beginning.
The structural mechanism of path dilution does not depend on the added content being incorrect or irrelevant. When a new document connects to an entity on an existing path, that entity gains neighbors. PPR importance is then distributed across more transitions, weakening the contribution of the existing path and potentially pushing previously retrieved gold documents out of the top-k. For the multi-hop queries in our setting, newly arriving gold documents complement existing gold documents rather than replace them. Both are needed. Losing still-required evidence is therefore a retrieval failure, even when the newly added evidence is correct.
We have clarified in Section 3 that existing supporting evidence remains valid and that new documents may provide complementary evidence. Extensions to regime (c), including fact corrections and contradictory information, are identified as future work in Section 6.
Does path memory obstruct new evidence, and does EviWeaver still help?
We compare three methods in the delayed-evidence setting at the highest delayed-query ratio (100%) used in our evaluation.
- HippoRAG2 performs PPR-based retrieval without path memory.
- w/o Refresh continues strengthening the initial memory without rebuilding it.
- EviWeaver uses the full method, including adaptive refresh.
New gold recall measures retrieval of supporting documents at their first arrival, averaged across queries. It directly examines whether memory obstructs newly available evidence. The all-gold joint retrieval rate measures the proportion of queries for which all supporting documents appear together in the top-5 at step 1000.
All values below are percentages.
| Dataset | Method | New gold recall, step 500 | New gold recall, step 750 | All-gold joint retrieval, step 1000 |
|---|---|---|---|---|
| MuSiQue | HippoRAG2 | 86.2 | 95.2 | 47.4 |
| w/o Refresh | 86.2 | 90.5 | 46.6 | |
| EviWeaver | 89.0 | 100.0 | 49.1 | |
| 2WikiMultiHopQA | HippoRAG2 | 88.8 | 88.5 | 62.4 |
| w/o Refresh | 89.2 | 86.9 | 56.3 | |
| EviWeaver | 87.6 | 90.2 | 66.0 |
-
EviWeaver achieves the highest result in five of the six comparisons. In particular, it achieves the highest all-gold joint retrieval rate on both datasets. On 2WikiMultiHopQA, this rate reaches 66.0%, compared with 56.3% for w/o Refresh. This shows that EviWeaver incorporates new evidence while retaining access to existing evidence.
-
Memory without refresh can hinder retrieval when new evidence first arrives. At step 750, w/o Refresh has lower new gold recall than HippoRAG2 on both MuSiQue (90.5% vs. 95.2%) and 2WikiMultiHopQA (86.9% vs. 88.5%). With adaptive refresh, EviWeaver achieves higher new gold recall than both HippoRAG2 and w/o Refresh on both datasets.
Response to Reviewer MjRm (Part 3/6)
W2 & Q2. Meaning of the Refresh Signal, Effectiveness, and Cost
The "good evidence" and "memory is stale" signals are not independently validated. Refresh is triggered by s_new, the share of query–triple similarity mass assigned to triples outside the stored memory. The paper treats this as evidence that new evidence is better. But: (i) there is no ground-truth comparison at refresh time of new-vs-old quality — the system cannot actually know whether the new triples are better; (ii) in the worst case the system must pay for both a memory-augmented pass over the old rails and an unweighted exploration of the new content, and the paper does not quantify this dual-search cost; (iii) "supporting document" is operationalized as the benchmark gold passage, which does not transfer to open-ended settings where relevance is multi-faceted. The decision boundary that drives the whole method is therefore heuristic and under-justified.
When s_new triggers a refresh, how often does the system actually rebuild, and what is the retrieval-time cost of running both the memory-augmented PPR and the fresh bridge-aware construction?
Response to W2 and Q2.
The refresh signal measures how retrieval has shifted toward information added since memory construction. It does not judge whether newly added evidence is better. We have clarified this distinction in Section 4.4.
1. Meaning of the refresh signal
is the share of similarity mass in the current top- retrieved triples assigned to triples added to the triple store after the latest memory construction time . A triple is considered new based on its absence from the triple store at , not its absence from the stored memory edges.
A larger share means that triples unavailable when memory was constructed have become prominent retrieval candidates for the current query. These triples can affect seed selection and subsequent PPR exploration. The signal therefore informs whether to rebuild memory for the current graph. EviWeaver aggregates query-level values using RMS to choose between strengthening and refresh.
When refresh is selected, EviWeaver performs bridge-aware seed construction over the current triple store, including both existing and newly added triples. It retrieves documents on the unmodified current graph and reselects the transitions that contributed to retrieval. Newly added triples receive no special priority, and transitions leading to existing evidence can remain in the rebuilt memory. Refresh therefore does not assume that new evidence is better.
2. Effectiveness and sensitivity of the decision boundary
The delayed-evidence comparison in our response to W1 and Q1 shows that strengthening memory without refresh can hinder new evidence retrieval. EviWeaver mitigates this effect and improves the joint retrieval of existing and newly available evidence.
The refresh criterion is a heuristic based on retrieval shift, but its gains do not depend on a narrowly tuned threshold. In Appendix D.4, Table 11, average Recall@5 ranges from 81.00% to 81.29% for , a variation of only 0.29 percentage points. Every evaluated threshold outperforms HippoRAG2's 79.55%. A lower threshold triggers refresh more readily and incurs additional QD cost, while retrieval quality remains stable across the evaluated range.
Response to Reviewer MjRm (Part 1/6)
We thank you for your constructive review and detailed questions about our evaluation scope and memory maintenance mechanism. We appreciate your recognition of the motivation for preserving paths that supported earlier retrieval and your assessment that the “path dilution” vocabulary is “useful.” We also thank you for describing backward attribution of PPR flow to individual edges as “a clean and implementable idea.”
Your distinction among the three expansion regimes helped us clarify the evaluation scope. Your questions also motivated additional analyses of complementary evidence uptake, refresh costs, and the effects of strengthening memory constructed from imperfect retrieval.
As a brief summary, we first provide our main responses and the corresponding revisions to the manuscript:
Sections 3 and 5.2, andAppendix D.3).Section 4.4andAppendix E.1,Table 19).Section 6).Section 5.5andAppendix D.6,Tables 15–16).We provide detailed responses below. If you have any additional questions or comments, we would be happy to discuss further.