acceptodds
Under review as a conference paper at ICLR 2027

EchoRAG: Evolving Communities and Harmonizing Intent for Retrieval-Augmented Generation

Abstract

Graph-based retrieval-augmented generation (RAG) organizes external corpora into reusable knowledge communities, but typically overlooks the complex query process that determines knowledge utility. This creates two coupled gaps: an *intent-blind knowledge utilization gap*, where underspecified queries yield evidence misaligned with latent needs, and a *query-blind knowledge organization gap*, where structurally coherent communities ignore downstream query distributions. To bridge them, we propose **E**volving **C**ommunities and **H**arm**o**nizing Intent for **R**etrieval-**A**ugmented **G**eneration (EchoRAG), a self-evolving framework that approximates query-specific knowledge organization without per-query reconstruction. EchoRAG begins with a structural initialization to establish a baseline graph partition and then introduces query-guided collaborative community evolution: accumulated query feedback determines where and when to update the knowledge base, while structural-semantic signals guide community evolution. EchoRAG further incorporates an intent-harmonized knowledge utilization mechanism: it infers latent intent to retrieve and transform retrieved knowledge into grounded, intent-aligned evidence anchored to source spans. Utilization traces provide delayed feedback for future evolution, closing the loop between organization and utilization. Extensive experiments on three benchmark datasets demonstrate that EchoRAG significantly outperforms existing state-of-the-art baseline approaches. Our code is available at <https://anonymous.4open.science/r/EchoRAG>.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Retrieval-Augmented Generation, Question Answering, Knowledge Organization, Community Detection, Intent-Aware Retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HTHbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies two fundamental flaws in existing graph-based RAG systems: an intent-blind knowledge utilization gap where under-specified queries retrieve evidence misaligned with latent user needs, and a query-blind knowledge organization gap where static structurally-defined communities ignore downstream query distributions. To address these, the authors propose EchoRAG, a self-evolving graph RAG framework with two core mechanisms. First, query-guided collaborative community evolution starts from an initial Leiden-based community partition, continuously tracks query demand distribution, and triggers selective community updates (refresh or evolve) when demand shifts exceed thresholds, using a joint structural-semantic objective with LLM calls only when signals conflict. Second, intent-harmonized knowledge utilization infers latent user intent from each query, generates an intent blueprint and hypothetical answer template, combines it with the explicit query, and retrieves and refines evidence into grounded, intent-aligned evidence cards anchored to source spans.

Critical strengths
  1. Formalization and theoretical grounding of two distinct problems (query-blind organization, intent-blind utilization) with regret bounds, providing rare theoretical rigor in the RAG field.
  2. Closed-loop adaptive framework that unifies knowledge organization and utilization into a self-reinforcing system, representing an important break from the static one-time indexing paradigm.
  3. Cost-aware practical design that minimizes expensive LLM calls, using lightweight signals for routine decisions and invoking LLM only for conflict resolution, with tiered maintenance policies well-suited for real deployment.
Critical weaknesses
  1. Evolution trigger thresholds are key parameters but not deeply analyzed. How they should be tuned across different datasets, query distributions, and update frequencies, and how sensitive performance is to suboptimal settings, remain unclear.
  2. Limited long-term evolution analysis: self-evolution analysis shows only a few rounds; performance over longer time horizons with continuously shifting query distributions is not established.
  3. LLM dependency of intent inference: the intent harmonization step adds per-query LLM calls, increasing latency and cost. The paper shows cost advantages over some baselines but does not quantify per-query latency overhead.
  4. The paper ablates the two main modules but does not compare against simpler baselines such as query expansion for intent or demand-based community reweighting without structural changes.
Questions for authors
  1. How sensitive is the community evolution mechanism to the threshold parameters τ_D and τ_H? Were these tuned per dataset, and can you comment on their generalizability across different corpora and query workloads?
  2. The self-evolution analysis shows results for only a few rounds. How does EchoRAG behave over longer time horizons with continuously shifting query distributions? Is there convergence or a steady state?
  3. How much latency does the intent induction step add per query compared to standard graph RAG retrieval? Has the latency-performance tradeoff been evaluated across different query types?
  4. Have you considered simpler baselines for each mechanism, such as query expansion for intent or demand-based community reweighting without structural changes? How much of the total gain comes from the full evolution mechanism versus simpler adaptations?
3 comments
  1. Performance remains stable across a broad threshold range on MultiHop-RAG, and the same intermediate default (8,25)×10−3(8,25)\times10^{-3} is used throughout the main experiments rather than tuned to individual queries. We would like to clarify that Appendix Fig. 6(c) already analyzes the coupled thresholds (τD,τH)(\tau_D,\tau_H) on MultiHop-RAG over a broad range. For completeness, we reproduce the results below:
(τD,τH)×10−3(\tau_D,\tau_H)\times10^{-3}AccuracyRecallComprehensivenessOverall
(16,40)(16,40)0.820.700.660.66
(12,35)(12,35)0.740.660.630.60
(8,25)(8,25) (default)0.790.690.660.64
(6,20)(6,20)0.810.670.640.61
(4,15)(4,15)0.760.670.650.62

The results remain stable without performance collapse even under substantial deviations from the default. These thresholds jointly control maintenance frequency: lower values trigger more frequent updates, whereas higher values update more conservatively. We therefore use the intermediate (8,25)×10−3(8,25)\times10^{-3} setting across the main experiments to balance adaptation and maintenance, rather than tuning thresholds to individual test queries.

The long-term evolution shows stable, selective long-horizon behavior rather than uncontrolled reorganization: only 13 of 20 checkpoints trigger structural changes, totaling 36 node movements. We agree that examining longer sequential execution is valuable. Beyond Section 4.5, we tracked 1,000 sequential queries in their original order, recording community adjustments and cumulative QA performance at every 50-query checkpoint.

QueriesAcc.F1Answer RecallHitComp.Div.Emp.Overall
5074.0062.0066.0062.0058.4062.8060.0052.80
10075.0068.6770.5068.0062.6066.8064.8058.00
15074.0070.4471.3370.6763.6067.8766.0059.33
20075.0071.0772.2570.5063.7068.0066.1059.10
25074.4070.1971.0069.2063.1267.6065.6858.64
30073.6769.1669.8368.3362.0766.8065.0057.73
35072.2968.3269.1467.7160.9166.2364.1756.86
40073.0069.5370.2568.7562.0067.2565.2058.05
45073.7870.1871.4469.5662.0967.5665.4258.36
50074.4069.6971.0069.4062.1267.6465.4858.20
55074.5570.3671.6470.0062.9868.1866.0759.09
60074.8370.1171.4269.5062.8768.0365.8759.03
65074.6270.0571.2369.6962.9268.2866.1559.08
70074.4369.9571.0069.2962.5768.0365.9758.60
75073.8769.6470.5368.9362.2467.7965.7158.27
80073.8869.3070.1968.3862.2867.8065.7858.25
85073.0669.0770.0068.1262.1967.7665.7458.12
90073.2268.9769.8368.0062.0067.6065.6057.87
95072.9568.6169.8467.5861.8767.4965.4957.75
100073.7069.1470.5068.2062.2067.7665.7258.04

Across 20 checkpoints, only 13 trigger structural changes, totaling 36 node movements, while global modularity remains close to its initial value (0.9187 →\rightarrow 0.9171) and cumulative QA performance remains stable. The alternating active and inactive checkpoints show that EchoRAG continues to respond selectively as demand changes without repeatedly reorganizing the knowledge base. We will add this longer-horizon analysis and clarify that it demonstrates empirical stability on the observed stream rather than formal convergence under arbitrary future shifts.

  1. We quantify both complete-pipeline and isolated intent-induction overhead: EchoRAG takes 33.12 seconds per MultiHop-RAG query, while intent induction averages 2.60 seconds over 500 queries. We would like to clarify that Appendix Table 8 already reports per-query latency. For a directly aligned comparison, we combine the MultiHop-RAG timing results with the token-consumption results from main-text Fig. 4 (time in seconds):
MethodIndex TimeQuery Latency (s)Prompt TokensCompletion TokensTotal Tokens
RAPTOR11,1183.75696,37385,888782,261
ArchRAG39,6695.161,945,896100,4402,046,336
LinearRAG6,99711.793,452,010212,1703,664,180
HippoRAG 226,90911.535,042,912226,1145,269,026
LightRAG32,30417.9728,042,361184,10428,226,465
DA-RAG3,075162.61159,901,691371,799160,273,490
HyperRAG25,292364.9441,515,09011,320,99852,836,088
CogRAG10,57431.5134,275,8951,084,60335,360,498
GraphRAG516,41930.1312,763,383374,96413,138,347
EchoRAG33,14533.1239,589,0931,533,24441,122,337

To isolate the overhead specifically attributable to intent induction, we additionally profiled 500 queries and report both average and tail latency across question types:

Question Type# QueriesIntent-Induction Latency (s)P95 (s)
Temporal1132.553.75
Inference1522.623.74
Comparison1682.573.55
Null672.704.10
All5002.603.75

The mean intent-induction latency remains within 2.55--2.70 seconds across question types, with an overall mean of 2.60 seconds and P95 of 3.75 seconds. EchoRAG is not the lowest-overhead system: it incurs higher query latency and token consumption than lightweight baselines, yet uses fewer total tokens and substantially less query time than DA-RAG and HyperRAG, while delivering stronger performance. We therefore view EchoRAG as a performance-cost trade-off rather than claiming uniform efficiency. We will add the intent-induction profiling protocol and these results to the revised paper. Complete timing results for HotpotQA and NarrativeQA remain available in Appendix Table 8.

  1. Under the same 1,000-question protocol, EchoRAG consistently outperforms both Query Expansion and non-structural Demand Reweighting. Our original ablation focused on isolating the two proposed modules and therefore did not include alternative, non-structural adaptation strategies. Following the reviewer's suggestion, we added two controlled baselines: Query Expansion (QE) uses an LLM to generate semantically equivalent queries, while Demand Reweighting (DR) accumulates community demand and periodically reweights community retrieval without modifying the knowledge structure. All methods use the same 1,000-question subsets, initial index, generator, retrieval budget, prompts, and evaluator.
DatasetMethodAcc.F1BLEU-1METEORDiv.Overall
MultiHop-RAGEchoRAG73.769.1----67.858.0
QE68.065.3----62.448.2
DR66.062.3----60.645.5
HotpotQAEchoRAG85.191.6----78.369.8
QE76.081.7----67.656.8
DR77.078.6----68.062.6
NarrativeQAEchoRAG51.2--10.217.347.934.6
QE42.0--8.513.339.331.0
DR45.0--9.114.743.229.6

EchoRAG consistently outperforms both simpler alternatives, showing that query reformulation or demand-based retrieval reweighting alone does not reproduce the gains from structural evolution and intent-aware evidence utilization. We will report these baselines, their implementation details, and the above results in the revised paper.

Thank you for the additional experiments and clarifications. The intent-latency profiling and simpler baselines substantially solve my concerns. However, threshold sensitivity is evaluated only on MultiHop-RAG, and cumulative performance on a single 1,000-query stream does not yet establish robustness under sustained distribution shifts.I also noticed discrepancies between the QE/DR results in the rebuttal and Table 10 of the revised paper,please also clarify the corresponding configurations.

Reviewer_usazOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Existing GraphRAG methods construct static knowledge communities without adequately considering users' latent intent, potentially leading to irrelevant evidence retrieval or fragmented information across communities in multi-hop QA. To address this, EchoRAG introduces Query-guided Community Evolution, which selectively updates communities based on shifts in accumulated query distributions while jointly considering structural cohesion and semantic consistency. Through Intent-harmonized Knowledge Utilization, EchoRAG infers latent query intent and refines retrieved knowledge into source-grounded Evidence Cards for answer generation, improving accuracy and response quality over existing GraphRAG methods on multi-hop and long-document QA tasks.

Critical strengths
  • Meaningful Problem Formulation and Motivation

The paper identifies the insufficient consideration of query distributions in existing GraphRAG knowledge organization. It also addresses the mismatch between explicit queries and latent user intent, providing a meaningful analysis of GraphRAG limitations from both knowledge organization and utilization perspectives.

  • Integrated Knowledge Organization and Utilization Framework

EchoRAG combines Query-guided Community Evolution (QCE) and Intent-Harmonized Utilization (IHU) to incorporate accumulated query feedback into knowledge organization. Selectively updating relevant communities rather than reconstructing the entire graph offers a practical design advantage.

  • Strong Performance Across Benchmarks

The paper reports competitive results against multiple GraphRAG baselines on MultiHop-RAG, HotpotQA, and NarrativeQA. Strong performance on key QA metrics, including Accuracy and F1, demonstrates the practical potential of the proposed framework.

  • Comprehensive Experimental Analysis

The authors provide module-level ablations, component analyses, hyperparameter sensitivity studies, LLM backbone comparisons, and case studies. These experiments help clarify the roles of individual components and provide a broader understanding of the framework.

  • Evidence-Grounded Generation and Selective LLM Utilization

EchoRAG refines retrieved evidence according to query intent while preserving citations to original text spans, improving answer traceability. Additionally, it selectively invokes LLM judgments when structural and semantic assessments conflict during community evolution, aiming to reduce unnecessary computational overhead.

Critical weaknesses
  • Limited Methodological Novelty and Justification

Although the paper presents query-adaptive community organization as a key contribution, the query distribution only determines when and where communities are updated, without directly influencing the node movement objective. Thus, its methodological distinction from existing structural-semantic community refinement remains unclear.

  • Insufficient Validation of Query-Adaptive Community Evolution

The paper lacks direct experiments validating its core self-evolving community mechanism. In particular, it does not evaluate adaptation to query distribution shifts or compare against query-independent periodic refinement. Therefore, it remains unclear whether the reported improvements genuinely result from query-driven community evolution.

  • Questionable Suitability of LLM-Based Evaluation

Table 2 uses Comprehensiveness, Diversity, Empowerment, and Overall scores, which are primarily suited to global understanding tasks such as UltraDomain. However, MultiHop-RAG and HotpotQA focus on factual answer correctness and supporting evidence. The suitability of these metrics for such benchmarks, as well as their evaluation procedures, is insufficiently justified.

  • Insufficient Analysis of Retrieval and Answer Performance

The Recall metric in Table 6 measures answer-level recall rather than retrieval recall. Although EchoRAG improves Answer Recall over HippoRAG by only 0.5 and 0.4 percentage points on MultiHop-RAG and HotpotQA, respectively, the Accuracy and F1 improvements are substantially larger. These discrepancies are not adequately analyzed. Moreover, retrieval evaluation relies mainly on Hit metrics, lacking comprehensive comparisons of gold-evidence Recall@k and supporting-fact retrieval performance.

  • Statistical Reliability and Reporting Inconsistencies

Despite marginal improvements on some metrics, the paper provides no standard deviations, confidence intervals, or statistical significance tests. Furthermore, EchoRAG's NarrativeQA results differ between Tables 2 and 3 (Diversity: 47.9 vs. 48.4; Overall: 34.6 vs. 36.8). These unexplained inconsistencies raise concerns about experimental reliability and reproducibility.

  • Limited Improvements and Potentially Misleading Presentation in Figure 3

Figure 3(a) uses a narrow y-axis range that visually exaggerates relatively small improvements, with Joint Quality increasing by only approximately 1% and no statistical uncertainty reported. In Figure 3(b), QCE produces modest Accuracy and F1 gains but substantially larger improvements in LLM-judged Overall and Diversity. The reasons for this discrepancy and its implications for downstream QA performance remain insufficiently explained.

  • Missing Strong Baselines and Efficiency Concerns

The evaluation omits HippoRAG 2, a strong successor to HippoRAG. Furthermore, EchoRAG exhibits higher query latency than HippoRAG across all datasets, with increased index construction time in some cases. A comprehensive performance–cost analysis accounting for the additional maintenance overhead of community evolution is needed.

Questions for authors

See Weakness.

8 comments
  1. We respectfully believe this concern stems from a misunderstanding of QCE, as query distribution does not merely determine when and where communities are updated. Actually, QCE is query-adaptive beyond deciding when and where to update: accumulated demand also determines the maintenance action, local optimization domain, candidate movements, and resulting evolved knowledge base. Specifically, Accumulated query demand determines (i) which maintenance action is taken (keep, intent-conditioned summary refresh, or structural evolution), (ii) which communities and induced local subgraphs enter the evolution process, and therefore (iii) the candidate nodes, destination communities, and sequence of movements that are actually optimized. This distinguishes QCE from existing structural-semantic community refinement, which optimizes a fixed knowledge organization independently of the observed query workload. Our contribution is not merely a new static node-movement score, but a closed-loop, workload-conditioned evolution mechanism that converts accumulated query demand into selective changes to the reusable knowledge organization while avoiding repeated global reconstruction.

  2. We supplement the 1,000-query controlled experiment, which confirms adaptation to a natural query shift and shows that demand-guided evolution improves Accuracy/F1/Overall over query-independent periodic refinement by 5.90/5.45/3.08 points. We agree that the original submission did not include a direct query-shift analysis or a query-independent periodic-refinement baseline. We initially focused on module-level and self-evolution analyses, while this controlled 1,000-query comparison was omitted because of its additional execution and evaluation cost. We have now added both evaluations on MultiHop-RAG.

Using a fixed topic taxonomy, the original-order query stream exhibits a clear natural shift between its last two 50-query windows:

TopicQueries 901--950Queries 951--1000
Technology26%38%
Sports30%16%
Consumer devices and e-commerce0%10%

We further compare EchoRAG with a query-independent periodic baseline. Both start from the same fully refined knowledge base and use identical queries, models, retrieval/generation pipeline, 50-query update interval, and local refinement engine. The periodic baseline performs community maintenance every 50 queries without using query-distribution feedback; thus, the query-feedback mechanism is the only difference.

MethodAcc.F1Answer RecallHitComp.Div.Emp.Overall
EchoRAG72.6072.7472.4369.3063.9266.5866.9458.98
Periodic refinement66.7067.2968.1868.0063.9865.4863.6255.90

EchoRAG improves seven of eight metrics, including Accuracy/F1/Overall by 5.90/5.45/3.08 points, while Comprehensiveness is effectively tied (-0.06). This controlled result shows that the gains are not explained by periodic maintenance alone: using accumulated demand to determine the scope and content of local evolution materially improves the resulting knowledge organization.

The EchoRAG trajectory also confirms that this adaptation remains selective. We tracked community states over 1,000 sequential queries, checking maintenance every 50 queries; selected checkpoints are shown below.

Queries ProcessedNode Moves in WindowCumulative MovesModularity QQSemantic Cohesion SSJoint Quality JJ
0000.9187200.5862570.772809
50000.9187200.5862570.772809
100330.9182420.5862720.772696
2006120.9175320.5862930.772529
4003190.9172520.5862940.772460
4500190.9172520.5862940.772460
6002240.9167650.5863170.772350
7502270.9166810.5863320.772336
8000270.9166810.5863320.772336
9008360.9170850.5863800.772461
10000360.9170850.5863800.772461

Across 20 checkpoints, only 13 trigger structural changes, totaling 36 node movements. Global modularity remains close to its initial value, while semantic cohesion slightly increases, showing that query adaptation produces sparse local changes without destabilizing the global organization. We will add the controlled comparison, query-shift case, and long-horizon trace to the revised version.

  1. EchoRAG does not rely solely on LLM judgments; task-specific non-LLM metrics provide the primary factual evaluation, while LLM-judged dimensions are supplementary. We would like to clarify that EchoRAG does not rely solely on LLM-based evaluation. Following established QA evaluation practices, we report Accuracy, F1, Answer Recall, and Hit for MultiHop-RAG and HotpotQA, while additionally adopting BLEU-1, BLEU-4, METEOR, and ROUGE-L for NarrativeQA. These task-specific metrics provide the primary evidence for factual correctness and answer quality. The four LLM-judged dimensions (Comprehensiveness, Diversity, Empowerment, and Overall), adopted from prior studies (e.g., ArchRAG, LightRAG), are included as supplementary evaluations to provide a broader perspective on generated responses.

  2. We add strict Gold-Evidence Recall, which confirms stronger supporting-evidence retrieval. We thank the reviewer for identifying the ambiguity in Table 6. Its Recall metric measures reference-answer coverage in the generated response, not retrieval recall; we will rename it Answer Recall throughout the paper.

Answer Recall measures reference-answer coverage in the generated response, while Hit only checks whether any answer-bearing context is retrieved; neither captures evidence completeness or distracting content. EchoRAG's QCE and IHU instead improve evidence organization, filtering, and utilization, thereby yielding larger gains in Accuracy and HotpotQA F1.

Following the reviewer's suggestion, we additionally report strict Gold-Evidence Recall on all 1,000 questions under identical 25k/50k-character budgets. We match dataset-annotated source passages/supporting facts without an LLM judge and macro-average per-question recall across the shared evaluation set.

MethodMultiHop 25k (%)MultiHop 50k (%)HotpotQA 25k (%)HotpotQA 50k (%)
BM2523.4529.6265.3468.73
VanillaRAG28.2332.4381.7881.83
HippoRAG 243.8249.4084.1288.62
CogRAG12.4616.0113.2626.52
DA-RAG17.2618.2415.3574.52
GraphRAG28.1230.7462.9876.77
HyperRAG11.7317.0852.2063.58
LinearRAG44.9152.8378.2882.94
LightRAG (hybrid)37.0144.1767.9083.55
RAPTOR28.6530.2223.0229.41
ArchRAG40.0248.6166.9271.00
EchoRAG47.6257.6692.4392.95

Compared with HippoRAG 2, EchoRAG improves Gold-Evidence Recall by 3.80/8.26 points on MultiHop-RAG and 8.31/4.33 points on HotpotQA at the 25k/50k budgets, respectively. This confirms that the downstream gains reflect both stronger supporting-evidence retrieval and better evidence utilization, rather than answer generation alone. NarrativeQA is excluded because it does not provide annotated gold supporting evidence. We will add this table, the full protocol, and the corrected Answer Recall terminology to the revised paper.

  1. We further clarify that the results in the original submission already represent averages over five runs with different random seeds. To provide a more complete assessment of statistical variability, we now additionally report the corresponding standard deviations (mean ± standard deviation): | Dataset | Comp | Div | Emp | Overall | |---|---:|---:|---:|---:| | MultiHop | 62.2 ± 1.55 | 67.8 ± 2.29 | 65.7 ± 2.18 | 58.0 ± 1.68 | | HotPotQA | 76.8 ± 2.49 | 78.3 ± 2.67 | 75.9 ± 2.67 | 69.8 ± 2.14 | | NarrativeQA | 38.1 ± 1.76 | 47.9 ± 2.16 | 43.4 ± 3.28 | 34.6 ± 2.13 |
DatasetAccF1Answer RecallHitBLEU-1BLEU-4METEORROUGE-L
MultiHop73.7 ± 1.5369.1 ± 2.0970.5 ± 1.5768.2 ± 1.45————
HotPotQA85.1 ± 2.4691.6 ± 3.0491.4 ± 1.0696.0 ± 2.19————
NarrativeQA51.2 ± 1.89———10.2 ± 1.465.4 ± 0.1317.3 ± 1.9419.16 ± 1.02

We thank the reviewer for identifying the NarrativeQA discrepancy. Tables 2, 3, and 7 all use the same 1,000-question evaluation subsets, so the full EchoRAG row should be identical across them. The Diversity/Overall values in Tables 3 and 7 were transcription errors: 48.4/36.8 should be 47.9/34.6, as reported correctly in Table 2. We will correct both tables. The corrected Table 3 still shows clear gains over w/o QCE and w/o IHU. In Table 7, EchoRAG remains higher in NarrativeQA Diversity than every fine-grained ablation, while w/o ea obtains 35.0 versus 34.6 in Overall.

  1. We agree that the changes in Fig. 3(a) are modest. This is expected because offline refinement already establishes a strong global partition, while online QCE performs only selective local updates. We will report the absolute changes and their variability explicitly to avoid overstating the magnitude.

In Fig. 3(b), although QCE introduces only limited structural changes, these adjustments can improve the organization of query-relevant evidence, which is subsequently refined by IHU to reduce irrelevant information and better align generated responses with query intent. Thus, modest gains in Accuracy/F1 alongside larger improvements in LLM-judged response quality are consistent with the complementary roles of QCE and IHU.

  1. The evaluated baseline is the official HippoRAG 2 implementation, and we will provide its pinned commit/configuration/commands while explicitly presenting EchoRAG's higher latency as a performance-cost trade-off. We clarify that the baseline labeled “HippoRAG” in our submission was run using the latest HippoRAG 2 implementation from its official repository (https://github.com/OSU-NLP-Group/HippoRAG). The original label and citation were incomplete, and we will rename the baseline to “HippoRAG 2”, add the corresponding paper citation, and revise the Appendix description. To make this implementation choice directly verifiable and reproducible, our anonymous code repository will provide (i) the pinned commit hash of the official HippoRAG 2 repository used in our experiments, (ii) the original configuration files, and (iii) the complete execution commands.

To further quantify the recurring cost of community evolution, we analyzed the complete 1,000-query execution logs:

Maintenance MetricValue
Total maintenance LLM tokens598,646
Share of recorded LLM tokens1.46%
Total maintenance time922.60 s
Amortized maintenance time per query0.923 s

We acknowledge that HippoRAG 2 has lower per-query latency on all three datasets. EchoRAG trades this additional online computation for stronger answer accuracy, response quality, and gold-evidence retrieval, while also requiring less index-construction time on HotpotQA and NarrativeQA. Its recurring maintenance remains selective, accounting for 1.46% of recorded LLM tokens and an amortized 0.923 seconds per query in the measured 1,000-query stream. We therefore view EchoRAG as offering a favorable performance--cost trade-off, rather than claiming uniformly lower cost than HippoRAG 2. We will present the quality, indexing, query-time, and maintenance results together in the revision for a transparent comparison.

Reviewer_nfYtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors proposes EchoRAG, which combines persistent graph-community maintenance with intent-conditioned evidence utilization. It initializes Leiden communities, refines them using structural and semantic node-movement signals, induces query intent, and extracts source-cited evidence cards for generation. Accumulated routing demand triggers keep/refresh/evolve decisions. Theoretical results characterize fixed-loss organization regret and linear-score sensitivity to intent error. Experiments use 1,000-question subsets of Multihop-RAG, HotpotQA and NarrativeQA, thirteen baselines, module/component ablations, a 2×2 organization/utilization study, diagnostics, costs and backbone substitutions. LLM-judged Accuracy improves on all three subsets, with particularly strong HotpotQA Best-Span F1 results.

Critical strengths
  1. A coherent integration of maintenance and grounded utilization. The selective update policy avoids rebuilding the graph for every query. These choices offer a sensible way to adapt a reusable index while preserving provenance, although provenance alone does not guarantee entailment or truth.

  2. Substantial empirical support beyond the main leaderboard. Accuracy exceeds the strongest baseline by 4.5/3.1/1.3 percentage points on the three datasets; HotpotQA Best-Span F1 is 91.6 versus the strongest baseline's 84.7.

Critical weaknesses
  1. The defining persistent-adaptation benefit is not separated from offline refinement under a reconstructible protocol. The existing ablations and 2×2 experiment support organization and query-feedback utility. However, sec 3.1.2 already refines communities globally before serving queries, and the paper does not establish whether the static S-I branch retains this complete initialization, what replaces query feedback in w/o-qdf, or whether w/o-QCE removes both initialization refinement and later updates. The method specifies delayed windowed updates; the experiments do not report the actual window size, query order, adaptation versus scored questions, number of passes, or index state used to answer each scored query. Hence it remains unclear how much benefit reflects historical demand improving later queries rather than initial semantic refinement or exposure to the evaluation workload. This is a protocol ambiguity, not evidence of test-answer use. Vector gain Delta S_0, normalization, verifier and certain update/bookkeeping rules also need executable definitions. The idealized theory does not resolve this empirical distinction.

  2. The strength of some conclusions is difficult to calibrate from the reported measurements. The main evaluator prompt, scoring aggregation and uncertainty are not provided. Only seed 42 is specified, while most supplementary experiments default to 100 questions. This is especially relevant to the 1.3-point NarrativeQA gain and .010 interaction, rather than grounds to disregard the larger ablation effects. NarrativeQA Diversity/Overall are 47.9/34.6 in Table 2 but 48.4/36.8 in Table 3&7; different subsets or judge runs may explain this, but their provenance must be stated. Fig. 4 and Table 8 provide genuine cost evidence, yet recurring maintenance, adaptation exposure and their amortization are not clearly charged. Extra computation is a legitimate design trade-off, not automatically an unfair comparison; its accounting is necessary to assess the advantage claimed for persistent maintenance.

Questions for authors

see above

2 comments

(1) w/o QCE removes both QCE-based initialization refinement and subsequent online evolution. By contrast, S-I and w/o qdf are two names used in different experimental sections for the same static control: both retain the fully refined offline index and disable query-distribution-driven online maintenance. EchoRAG distinguishes two stages: offline structural-semantic community refinement, which establishes the initial knowledge organization, and query-driven evolution, which selectively maintains and updates communities according to accumulated query demand. We clarify the configurations as follows:

VariantIndexOnline maintenanceQuery utilization
w/o QCEInitializationDisabledFull IHU
w/o qdfK(0)K^{(0)}DisabledFull IHU
S-IK(0)K^{(0)}DisabledFull IHU

(2) In our implementation, each query constitutes one feedback window, and queries are processed sequentially in their original test-set order, without prior access to future queries. After every 50 query windows, EchoRAG checks the evolution conditions governed by the thresholds τD\tau_D and τH\tau_H (reported in the Appendix). Community maintenance is triggered only when the corresponding conditions are satisfied. Importantly, query-demand feedback is constructed exclusively from the query content, without using ground-truth answers, generated responses, or evaluation scores. When maintenance is triggered, the updated knowledge base becomes available for subsequent retrieval and generation operations.

(3) We clarify the semantic gain, normalization, verification mechanism, and update/bookkeeping rules as follows. For a candidate movement of node vv from community AA to BB, the semantic gain is defined as: ΔS0=cos⁡(ev,zB)−cos⁡(ev,zA),\Delta S_0 = \cos(e_v,z_B)-\cos(e_v,z_A), where eve_v denotes the node embedding and zA,zBz_A,z_B are the corresponding community representations. To place structural and semantic gains on comparable scales while preserving their signs, we use maximum-absolute-value normalization over the current candidate movement set D\mathcal D. For X∈Q,SX\in\\{Q,S\\}, define sX=max⁡δ′∈D∣ΔX(δ′)∣,s_X=\max_{\delta'\in\mathcal D}\left|\Delta X(\delta')\right|, and

\begin{cases} \operatorname{clip}\\!\left(\dfrac{\Delta X(\delta)}{s_X},-1,1\right),&s_X>0 \\\\ 0,&s_X=0. \end{cases}$$ Both normalized gains lie in $[-1,1]$, with positive, negative, and zero gains retaining their original signs. Movements with positive structural and vector-semantic gains are directly accepted, whereas movements with both gains non-positive are rejected. When their signs conflict, the LLM provides a refined semantic score. The IB-based verifier uses historical delayed movement utility to decide whether to retain this score or set its semantic contribution to zero, thereby falling back to the non-LLM structural decision; the movement is then accepted iff the resulting joint gain is positive. Each accepted movement synchronously updates the node-community assignment, community membership list, community representation, and movement log. After maintenance, EchoRAG updates the demand baseline, persists the new state, refreshes affected summaries and the retrieval matrix, and deduplicates shared local-evolution records when counting maintenance events, moved nodes, and LLM calls. (4) **Under the same refined index and query exposure, demand-guided EchoRAG outperforms query-independent periodic refinement by 5.90/5.45/3.08 points in Accuracy/F1/Overall, isolating the benefit of persistent adaptation.** To directly examine whether the gain arises from accumulated historical demand rather than initial semantic refinement or generic exposure to the same query stream, we further compared EchoRAG with a query-independent periodic-refinement baseline. Results from the same controlled 1,000-query MultiHop-RAG run are: | Method | Acc. | F1 | Answer Recall | Hit | Comp. | Div. | Emp. | Overall | |---|---:|---:|---:|---:|---:|---:|---:|---:| | EchoRAG | **72.60** | **72.74** | **72.43** | **69.30** | 63.92 | **66.58** | **66.94** | **58.98** | | Periodic refinement | 66.70 | 67.29 | 68.18 | 68.00 | **63.98** | 65.48 | 63.62 | 55.90 | EchoRAG improves almost all metrics, including Accuracy/F1/Overall by 5.90/5.45/3.08 points, while Comprehensiveness is effectively tied (-0.06). Because the initial refined index and query-stream exposure are controlled, these results show that using historical query demand provides additional gains beyond offline refinement and generic periodic maintenance. Together with the existing S-I versus E-I comparison in Fig. 2(c), this controlled experiment validates the contribution of persistent query-adaptive evolution. We will supplement these implementation details in the revision.

(1) We would like to clarify that our evaluation follows the established Comprehensiveness, Diversity, Empowerment, and Overall dimensions adopted in prior work such as LightRAG and ArchRAG. We will provide a simplified version of the prompts in the appendix. The complete evaluator prompt is provided in our anonymous code repository. We use 1,000 questions from each dataset for the main experiments and ablation study. We additionally conducted an evaluation-size sensitivity analysis in Fig. 6(d), showing that 100-question subsets preserve the broad performance trends and method comparisons. Therefore, supplementary analyses use 100-question subsets to balance computational costs and evaluation representativeness. The main results in the original submission are averages over five complete independent runs with different random seeds. The seed-42 statement in Appendix C.4 applies only to the supplementary diagnostic experiments. We will clarify these detailed settings. The corresponding mean ±\pm standard deviation results are:

DatasetComp.Div.Emp.Overall
MultiHop-RAG62.2 ±\pm 1.5567.8 ±\pm 2.2965.7 ±\pm 2.1858.0 ±\pm 1.68
HotpotQA76.8 ±\pm 2.4978.3 ±\pm 2.6775.9 ±\pm 2.6769.8 ±\pm 2.14
NarrativeQA38.1 ±\pm 1.7647.9 ±\pm 2.1643.4 ±\pm 3.2834.6 ±\pm 2.13
DatasetAcc.F1Answer RecallHitBLEU-1BLEU-4METEORROUGE-L
MultiHop-RAG73.7 ±\pm 1.5369.1 ±\pm 2.0970.5 ±\pm 1.5768.2 ±\pm 1.45--------
HotpotQA85.1 ±\pm 2.4691.6 ±\pm 3.0491.4 ±\pm 1.0696.0 ±\pm 2.19--------
NarrativeQA51.2 ±\pm 1.89------10.2 ±\pm 1.465.4 ±\pm 0.1317.3 ±\pm 1.9419.16 ±\pm 1.02

These deviations provide the requested calibration: we will distinguish small mean differences from the larger ablation and controlled-comparison effects, and will treat the 0.010 interaction in Fig. 2(c) as descriptive rather than standalone evidence.

We also thank the reviewer for identifying the NarrativeQA inconsistency. Tables 2, 3, and 7 use the same 1,000-question evaluation subsets, so their full EchoRAG rows should agree. The Diversity/Overall values 48.4/36.8 in Tables 3 and 7 are transcription errors; the correct values are 47.9/34.6, as reported in Table 2. We will correct both tables and the associated statements. The correction does not change the module-level conclusion in Table 3. In Table 7, EchoRAG still has higher Diversity than every fine-grained ablation, while w/o ea obtains 35.0 versus 34.6 in Overall; we will therefore modify the original claim.

(2) We analyze the complete process, which accounts for 1.46% of recorded LLM tokens, corresponding to 598.65 amortized tokens per query. Its total measured time is 922.60 seconds, or 0.923 seconds per query. This demonstrates that EchoRAG is a performance-cost trade-off. We thank the reviewer for pointing out that the recurring maintenance cost was not separately charged. The original submission focused on aggregate indexing and query costs in Fig. 4 and Table 8; because maintenance is conditionally triggered and interleaved with online serving, we did not separately aggregate its checkpoint-level exposure and amortized overhead. We have now analyzed the complete 1,000-query execution, including demand checks, summary refreshes, and local structural updates:

StepMaintenance RecordsSummary RefreshesMoved NodesLLM TokensTime (s)
5000000.05
10076338,54953.20
15086343,83254.26
20073649,17769.02
25043120,96334.44
30098245,73168.16
35065129,36945.61
400108355,23679.05
45033012,17018.63
50086345,56062.04
55066021,38938.92
60043224,01835.44
65033012,70318.86
70098143,84164.58
75065231,69048.42
8002209,97115.67
85076134,34861.01
90087844,91489.42
95033015,08929.26
100044020,09636.56

Across the 20 checkpoints, the log contains 114 maintenance records, 95 summary refreshes, and 36 node movements; a checkpoint therefore does not necessarily produce a structural change. The corresponding LLM-token consumption is:

ComponentLLM Tokens
Answer pipeline40,523,691
Feedback-driven maintenance598,646
Total recorded41,122,337

We will add these results and analysis to the revised paper.

Comments on the paper

1 comment

Dear Chairs and Reviewers,

We sincerely thank all reviewers for their time throughout the discussion. We have carefully addressed their concerns and conducted substantial additional experiments specifically in response to their requests.

We would like to reiterate the core contributions of EchoRAG. Unlike existing graph-based RAG methods that rely on static knowledge organization, EchoRAG introduces a persistent query-adaptive framework that integrates Query-guided Community Evolution (QCE) and Intent-Harmonized Utilization (IHU). By combining accumulated query-demand feedback with selective community evolution and intent-aware evidence refinement, EchoRAG addresses both knowledge organization and utilization, while avoiding costly global reconstruction.

In direct response to the reviewers' concerns, we have provided substantial additional evidence and clarifications:

(1) Reproducibility: Clarified the online adaptation protocol, controlled ablation configurations, community evolution rules, and implementation details.

(2) Evaluation Reliability: Supplemented statistical variability, clarified evaluation procedures, and provided additional objective retrieval evidence through Gold-Evidence Recall comparisons.

(3) Community Evolution Validation: Extended the evolution analysis to query sequential streams, reporting community changes, structural and semantic properties, and representative cases.

(4) Effectiveness and Efficiency: Added comparisons including query expansion and demand reweighting, alongside detailed latency profiling and amortized maintenance costs.

We also clarify that LLM-judged metrics serve as supplementary assessments of response quality, while our conclusions regarding factual QA effectiveness are primarily supported by objective QA and retrieval metrics.

We acknowledge that further investigations could provide valuable insights. However, we respectfully hope that the remaining opportunities for further analysis can be distinguished from the contributions and experimental evidence already established. We sincerely hope that the chairs and reviewers will consider the full body of evidence when assessing the novelty, effectiveness, and overall contributions of EchoRAG.

Thank you again for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.