acceptodds
Under review as a conference paper at ICLR 2027

ConRAG: Lightweight Inference of Multi-Hop Relations

Abstract

Understanding how two entities are connected often requires tracing multi-hop relations across documents to identify intermediate entities and supporting evidence that explain a connection. This is a task that appears frequently in scientific research and other knowledge-intensive analyses. We formalise this setting as multi-hop relation inference: given two known endpoint entities, we aim to recover the bridge entities and evidence-grounded reasoning chains that connect them across a document corpus, and to generate an explanation grounded in the retrieved evidence. Existing multi-hop RAG systems typically seek an unknown answer entity rather than explicitly recovering the connection between two known endpoints and graph-based approaches often rely on costly LLM-extracted knowledge graphs that limit scalability to large document collections. We introduce ConRAG, which builds a lightweight entity–document graph from corpus co-occurrence and LLM-based entity filtering. Its connective retrieval infers and semantically ranks paths between two endpoints. On MuSiQue and 2WikiMultiHopQA, ConRAG consistently improves bridge entity and reasoning chain recovery over strong RAG baselines, while reducing graph-indexing token cost by up to roughly 1.5 orders of magnitude. Our results show that endpoint-constrained path retrieval provides an effective and index-efficient approach to evidence-grounded relation discovery.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Retrieval Augmented Generation, Multi-hop Relations, Graph Based Retrieval, Entity-Centric Retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xtksOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies multi-hop relation inference: given two known endpoint entities, recover the bridge entities and the chain of links that connect them in a document corpus, and explain the connection. The proposed ConRAG mines word and phrase candidates from the corpus, filters them into an entity vocabulary with an LLM, and builds a relation-free entity-document graph. At query time it retrieves seed documents that mention the endpoints, enumerates document paths between anchor-side and target-side seeds, and ranks them by query-document similarity before generation. The authors also re-purpose MuSiQue and 2WikiMultiHopQA into relation inference benchmarks with gold bridges and chains. Validation compares ConRAG with four RAG baselines on bridge and chain recovery, document recall, LLM-judged explanation quality, and graph-indexing token cost.

Critical strengths
  1. Recovering the chain between two known entities is a useful framing, and the re-purposed MuSiQue and 2WikiMultiHopQA give it a benchmark.
  2. The method is simple, the code is released, and the limitations and the false-negative analysis are stated openly.
Critical weaknesses
  1. The experiments do not support the central claim

Neither the path step nor evidence grounding is tested, and the comparison is not matched.

  1. Novelty is limited

The retrieval core follows MR.COD, and the closest published methods are not compared.

Questions for authors
  1. Seeds are restricted to documents that string-match the anchor or the target (lines 237 to 244), and 2-document paths are excluded by the target lengths {3, 4, 5} (Table 5). In 2-hop questions (all of 2WikiMultiHopQA, 1,252 of the 2,066 MuSiQue questions), both gold documents mention an endpoint and can enter as seeds without any path. What are BCM, CR, CP, and document recall with seeds only, and for Naïve RAG with the same endpoint filter?
  2. At the default p_hub = 0.9, about 34% of MuSiQue gold bridges stay traversable, down from about 72% (Figure 5), and 4-hop recall is flat across p_hub (Figure 4). What share of the recovered gold bridges comes from path documents rather than seeds?
  3. The motivation is that existing RAG misses bridge documents with weak query similarity (lines 088 to 090), yet Eq. 3 ranks paths by query similarity. What is the recall of gold bridge documents stratified by their dense rank, for ConRAG and each baseline?
  4. What do ablations of the LLM vocabulary filter (document frequency only), canonicalization, and hub suppression show?
  5. "Evidence-grounded" (lines 175 to 176) is never measured, and the endpoints are Wikipedia entities that GPT-4o largely knows. Beyond string matching, the only checks are an LLM judge that does not state whether answer order is swapped (lines 1385 to 1386) and a "blind" validation (Table 9) that does not say who judged. What does closed-book GPT-4o with the same prompt achieve, how faithful are the chain and the explanation to R(q), and who judged Table 9 with what agreement?
  6. BCM counts a gold bridge whenever its string appears in the response (lines 300 to 301), and the shared prompt asks every system to "Be comprehensive" (Appendix E.3), so longer answers score higher. What is the response length of each method?
  7. Baselines get exactly 10 documents while ConRAG averages 10.8 (lines 321 to 323), and on 2WikiMultiHopQA ConRAG's prompt has 3,301 ± 1,707 tokens vs 2,542 ± 699 for Naïve RAG (Table 3). What is baseline recall at k = 15 and 20, or under a token-matched context?
  8. The only p_hub sweep (Figure 4) runs on the evaluation questions themselves, and no development split is described (lines 291 to 292). At p_hub = 0.5, MuSiQue document recall is about 64%, close to HippoRAG 2's 64.2 (Table 4). How were the values in Table 5 chosen?
  9. The "1.5 orders of magnitude" saving (lines 025 to 026) compares GPT-4o tokens with GPT-4o-mini tokens (lines 318 to 319), although GPT-4o is priced far higher per token, so the token ratio does not show that ConRAG's indexing is cheaper, and "CONRAG also clearly undercuts the comparatively lightweight HippoRAG 2" (lines 421 to 422) is not established. What are the dollar cost, indexing time, and query latency of each method, and how does ConRAG do with a GPT-4o-mini filter?
  10. Table 6 "full" MuSiQue (71.7, 28.4, 34.5) differs from Table 1 (71.2, 29.5, 34.3) for the same setting. Which run is Table 1? Line 106 says ConRAG "outperforms all evaluated RAG baselines", yet HippoRAG 2 has the higher MuSiQue CP (36.4 vs 34.3, Table 1).
  11. The retrieval core is MR.COD [1]: a passage graph with shared-entity edges, depth-first search from head to tail passages, and paths ranked by average query-passage similarity, which lines 246 to 256 follow. PathRAG, HiRAG [2], and GNN-RAG [3] already retrieve paths between entities as LLM context, so "the first graph-based RAG framework designed to recover connections between known entities" (lines 104 to 105) is not earned. What is new beyond MR.COD?
  12. LinearRAG [4], named as closest in lines 145 to 146, reports zero LLM tokens for indexing, and MR.COD [1] is the method's own retrieval core; neither is compared. How do they perform on both benchmarks under the same document budget?
  13. All four baselines are answer-oriented, and no published multi-hop retriever evaluated on MuSiQue and 2WikiMultiHopQA is compared, such as PropRAG [5], HopRAG [6], GFM-RAG [7], or IRCoT [8] (cited at lines 154 to 155). How does ConRAG compare with them?

[1] Keming Lu, I-Hung Hsu, Wenxuan Zhou, Mingyu Derek Ma, and Muhao Chen. Multi-hop evidence retrieval for cross-document relation extraction. In Findings of ACL, 2023.

[2] Haoyu Huang, Yongfeng Huang, Junjie Yang, Zhenyu Pan, Yongqiang Chen, Kaili Ma, Hongzhi Chen, and James Cheng. Retrieval-augmented generation with hierarchical knowledge. In Findings of EMNLP, 2025.

[3] Costas Mavromatis and George Karypis. GNN-RAG: Graph neural retrieval for efficient large language model reasoning on knowledge graphs. In Findings of ACL, 2025.

[4] Luyao Zhuang, Shengyuan Chen, Yilin Xiao, Huachi Zhou, Yujing Zhang, Hao Chen, Qinggang Zhang, and Xiao Huang. LinearRAG: Linear graph retrieval augmented generation on large-scale corpora. In ICLR, 2026.

[5] Jingjin Wang and Jiawei Han. PropRAG: Guiding retrieval with beam search over proposition paths. In EMNLP, 2025.

[6] Hao Liu, Zhengren Wang, Xi Chen, Zhiyu Li, Feiyu Xiong, Qinhan Yu, and Wentao Zhang. HopRAG: Multi-hop reasoning for logic-aware retrieval-augmented generation. In Findings of ACL, 2025.

[7] Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Dinh Phung, Chen Gong, and Shirui Pan. GFM-RAG: Graph foundation model for retrieval augmented generation. In NeurIPS, 2025.

[8] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In ACL, 2023.

8 comments
AuthorsOpenReview ↗

Answer to Reviewer xtks [1/N]

[S1] Useful framing of multi-hop relation inference and benchmark adaptation; [S2] Simple method, open-source code, and transparent discussion of limitations.

Thank you for recognizing these strengths and for your detailed and constructive feedback! We have carefully considered your concerns and conducted additional experiments to address them. We provide a point-by-point response below and would be very happy to discuss any remaining questions or concerns.

[W1] The experiments do not support the central claim. The path step is not tested. [Q1] Seeds are restricted to documents that string-match the anchor or the target (lines 237 to 244). In 2-hop questions, both gold documents mention an endpoint and can enter as seeds without any path. What are BCM, CR, CP, and document recall with seeds only, and for Naïve RAG with the same endpoint filter?

Thank you for highlighting this concern. We have added a task-adapted seeds-only baseline that retains ConRAG's endpoint-constrained dense retrieval but removes graph-based path traversal. This isolates the contribution of path retrieval and provides a comparison with endpoint-filtered dense RAG.

  1. Path retrieval contribution (updated Tables 1 and 4):
DatasetMethodBCMCRCPDoc Recall
MuSiQueSeeds only68.5±0.726.3±1.035.7±1.966.2±0.4
ConRAG71.2±0.729.5±0.534.3±1.170.2±0.5
2WikiMultiHopQASeeds only76.6±1.446.2±1.842.6±1.374.5±0.4
ConRAG81.0±1.449.4±1.848.1±0.984.1±0.6
  1. Hop-stratified analysis (updated Table 7). To further investigate whether the improvements arise only from endpoint coverage in 2-hop questions, we additionally compare the seeds-only baseline against ConRAG across different hop counts on MuSiQue: 2-hop: CR improves from 40.7% to 43.7%. 3-hop: CR improves from 4.9% to 9.4%. 4-hop: CR improves from 3.0% to 3.8%. These results demonstrate that path retrieval provides measurable benefits beyond endpoint-constrained seeding, including on longer reasoning chains.

[W1] Evidence grounding is not tested. [Q5] "Evidence-grounded" (lines 175 to 176) is never measured, and the endpoints are Wikipedia entities that GPT-4o largely knows. Beyond string matching, the only checks are an LLM judge that does not state whether answer order is swapped (lines 1385 to 1386) and a "blind" validation (Table 9) that does not say who judged. What does closed-book GPT-4o with the same prompt achieve, how faithful are the chain and the explanation to R(q), and who judged Table 9 with what agreement?

Thank you for raising these concerns. We clarify that BCM is intentionally a bridge-identification metric, rather than a measure of evidence grounding. To address your questions, we have expanded our evaluation as follows:

  1. Evidence grounding and closed-book evaluation (new Tables 12 and 13). We evaluate GPT-4o on the same 100 MuSiQue questions, with and without ConRAG's retrieved context, using the same generation prompt.
MethodDoc RecallBCMCRCP
ConRAG68.168.826.731.7
No retrieval0.037.516.324.1

ConRAG substantially improves all metrics over closed-book GPT-4o. This demonstrates that retrieval contributes substantially beyond the model's parametric knowledge alone. Moreover, since all methods use the same GPT-4o generator, the potential influence of prior knowledge applies across baselines.

To directly measure evidence grounding, we additionally evaluate generated responses against their retrieved passages using GPT-4o-mini (new Table 12, 100 examples per dataset). For ConRAG, 71% of responses on 2WikiMultiHopQA are fully grounded (6% partially grounded), while 40% on MuSiQue are fully grounded (12% partially grounded); we further verify that 71.9 ± 11.4% of correctly recovered chain links on the MuSiQue sample are supported by retrieved evidence (Appendix C.4).

  1. Blind validation (Table 9). The validation of alternative paths was performed by GPT-4o-mini, the same model used for our original qualitative evaluation. The judge evaluates each alternative path against its respective retrieved evidence. We have clarified this protocol in the manuscript. As this was a single-judge evaluation, inter-judge agreement was not measured.

  2. Evaluation robustness. We have clarified that answer presentation order was randomized to mitigate position bias in the pairwise evaluation (Table 2). Additionally, we repeated the qualitative evaluation with Qwen3-32B, a judge from a different model family (new Table 14). ConRAG maintains majority overall win rates against all evaluated baselines on both datasets with both judges, supporting the robustness of our qualitative findings.

AuthorsOpenReview ↗

Answer to Reviewer xtks [2/N]

[W1] The comparison is not matched. [Q7] Baselines get exactly 10 documents while ConRAG averages 10.8 (lines 321 to 323), and on 2WikiMultiHopQA ConRAG's prompt has 3,301 ± 1,707 tokens vs 2,542 ± 699 for Naïve RAG (Table 3). What is baseline recall at higher k?

We agree that matching the number of retrieved documents does not necessarily imply matching the token budget, as document lengths vary across methods. To strengthen the comparison, we extended Naïve RAG to k=15 documents, exceeding ConRAG's average of 10.8 documents, and evaluated it on both benchmarks (updated Tables 1 and 4).

DatasetMethodAvg. DocsDoc Recall (%)Avg. Prompt Tokens
MuSiQueNaïve RAG (k=10)1061.1 ± 0.62,242 ± 344
Naïve RAG (k=15)1564.5 ± 0.4Not measured
ConRAG10.870.2 ± 0.52,261 ± 983
2WikiMultiHopQANaïve RAG (k=10)1067.0 ± 0.32,542 ± 699
Naïve RAG (k=15)1569.4 ± 0.3Not measured
ConRAG10.884.1 ± 0.63,301 ± 1,707

Increasing Naïve RAG's retrieval budget from 10 to 15 documents improves recall by 3.4 percentage points on MuSiQue and 2.4 points on 2WikiMultiHopQA. Nevertheless, ConRAG retains a substantial advantage (+5.7 and +14.7 points, respectively), despite retrieving fewer documents on average. The extended baseline also achieves lower BCM and chain recall than ConRAG on both benchmarks (updated Table 1). This experiment demonstrates that increasing the dense retrieval baseline's document budget does not close the performance gap.

[W2] Novelty is limited, The retrieval core follows MR.COD, and the closest published methods are not compared. [Q11] The retrieval core is MR.COD [1]: a passage graph with shared-entity edges, depth-first search from head to tail passages, and paths ranked by average query-passage similarity, which lines 246 to 256 follow. PathRAG, HiRAG [2], and GNN-RAG [3] already retrieve paths between entities as LLM context, so "the first graph-based RAG framework designed to recover connections between known entities" (lines 104 to 105) is not earned. What is new beyond MR.COD?

Thank you for the opportunity to clarify our contributions. Our novelty claim has never been about introducing graph traversal or path retrieval, which we explicitly acknowledge and build upon from prior work, including MR.COD. Rather, our contribution lies in formulating and evaluating multi-hop relation inference within RAG, together with an efficient, automatically constructed relation-free graph and an end-to-end framework specifically designed to recover intermediate entities, reasoning chains, and supporting explanations. We have refined the wording of our contributions to make this distinction clearer and avoid any ambiguity regarding prior path-based retrieval methods.

  1. Differences from MR.COD. We acknowledge that our path retrieval builds on MR.COD, which we explicitly cite. However, ConRAG differs in three important respects: Multi-document paths: MR.COD restricts candidate paths to passages within two endpoint-associated documents, whereas ConRAG permits paths spanning multiple documents across the corpus. Automatic indexing: MR.COD relies on annotated entity mentions, whereas ConRAG automatically mines, filters, and canonicalizes entities from raw documents without requiring relation extraction or task-specific retriever training. Different objective: MR.COD retrieves evidence to predict predefined relation labels, whereas ConRAG directly generates intermediate entities, reasoning chains, and explanations.

  2. Differences from other graph-based RAG methods. We agree that PathRAG, HiRAG, and GNN-RAG already use paths as retrieval context. However, PathRAG uses LLM-extracted relation graphs for general QA, HiRAG focuses on hierarchical knowledge retrieval, and GNN-RAG uses a trained GNN over an existing structured KG to retrieve paths toward answer candidates. Unlike these methods, ConRAG explicitly targets reconstructing connections between two known endpoints directly from unstructured documents, without relation triples or a trained retriever.

To expand our baseline suite, we have added new results comparing with LinearRAG, a closely related relation-free graph RAG method, and component ablations that clarify all the components of ConRAG (Tables 1, 19–20). We have clarified these distinctions and narrowed the original "first graph-based RAG …" claim in the revised manuscript. Our novelty lies in the combination of a dedicated relation-inference formulation, lightweight automatic indexing, and endpoint-constrained evidence retrieval, with explicit evaluation of bridge and chain recovery.

AuthorsOpenReview ↗

Answer to Reviewer xtks [3/N]

[Q2] At the default p_hub = 0.9, about 34% of MuSiQue gold bridges stay traversable, down from about 72% (Figure 5), and 4-hop recall is flat across p_hub (Figure 4). What share of the recovered gold bridges comes from path documents rather than seeds?

We now explicitly quantify the provenance of recovered gold bridge entities in new Table 17, distinguishing whether they occur in seed documents, path-selected documents, or both. While most recovered bridges are already mentioned in seed documents, identifying a bridge entity is not equivalent to recovering the evidence needed to reconstruct its relationships. Importantly, 43.97% of recovered bridges on MuSiQue and 45.62% on 2WikiMultiHopQA also appear in path-selected documents. Our controlled seeds-only ablation further demonstrates that adding path retrieval consistently improves bridge recovery, document recall, and chain recall. These results show that path traversal contributes meaningfully beyond endpoint-constrained seeding, even when bridge entities are already present in the seed documents.

AnalysisMetricMuSiQue (%)2WikiMultiHopQA (%)
Bridge provenanceSeed documents only55.37 ± 1.3654.38 ± 2.11
Path documents only0.49 ± 0.060.16 ± 0.07
Both seeds and paths43.48 ± 1.1945.46 ± 2.08
Neither0.66 ± 0.120.00 ± 0.00
Path contributionBCM (seeds → full)68.5 → 71.2 (+2.7)76.6 → 81.0 (+4.4)
CR (seeds → full)26.3 → 29.5 (+3.2)46.2 → 49.4 (+3.2)
Doc Recall (seeds → full)66.2 → 70.2 (+4.0)74.5 → 84.1 (+9.6)

[Q3] The motivation is that existing RAG misses bridge documents with weak query similarity (lines 088 to 090), yet Eq. 3 ranks paths by query similarity. What is the recall of gold bridge documents stratified by their dense rank, for ConRAG and each baseline?

We clarify that ConRAG does not rely solely on query similarity to retrieve documents. Instead, graph connectivity first identifies candidate paths between the two endpoints, and Equation 3 subsequently ranks these paths by their average semantic similarity. This allows to surface supporting documents that would otherwise rank poorly under standard dense retrieval. To investigate this, we added a dense-rank analysis (new Table 18) comparing the original dense-retrieval ranks of gold documents with their positions in ConRAG's final context.

Gold documentsDense Rank (Mean / Median)Context Rank (Mean / Median)Rank Uplift (Mean / Median)
On retrieved paths181.66 / 21.99 / 2+179.67 / 0
Path-only696 / 324.20 / 4+691.80 / 26

This shows that graph connectivity can promote relevant evidence that would otherwise be missed by top-k dense retrieval, despite using semantic similarity for path ranking. Moreover, our seeds-only ablation (Tables 1 and 4) confirms that path traversal improves document recall and chain recovery beyond endpoint-constrained dense retrieval.

[Q4] What do ablations of the LLM vocabulary filter (document frequency only), canonicalization, and hub suppression show?

We have expanded our ablation study (new Tables 19–20), which complement our existing analyses of hub suppression (Figure 4) and entity vocabulary filtering (Figure 5):

  1. We attempted to remove the LLM filter while retaining document-frequency filtering. However, even on a reduced corpus, retrieval exceeded 5 minutes per query, compared with approximately 2–3 seconds with filtering enabled. This prevented completing the recall evaluation, but demonstrates the importance of LLM vocabulary filtering for computational tractability. As already shown in Figure 5, this filtering substantially reduces the entity vocabulary while largely preserving gold bridge coverage.

  2. Canonicalization, endpoint filtering, and hub suppression (new Tables 19–20). We systematically vary these components and measure document recall and retrieval time on MuSiQue. Results show that endpoint-aware filtering improves document recall, with modest additional retrieval time; hub suppression provides a clear efficiency–recall trade-off, and canonicalization has a relatively small effect on recall, but helps reduce graph size and redundant entity noise by merging equivalent entity nodes.

Hub ThresholdEndpoint FilterCanonicalizationDoc Recall (%)Retrieval Time (s)
80%OnOn75.67 ± 29.392.44 ± 0.76
80%OnOff75.92 ± 29.282.43 ± 0.60
90%OnOn79.33 ± 27.943.03 ± 1.63
90%OnOff79.08 ± 28.492.79 ± 1.08
80%OffOn69.75 ± 30.282.35 ± 0.65
80%OffOff70.33 ± 30.002.17 ± 0.49
90%OffOn72.83 ± 29.602.90 ± 1.70
90%OffOff73.08 ± 30.102.60 ± 1.00
Reviewer_ukzbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on multi-hop relation inference in RAG, a task of recovering the relationship between two known endpoint entities that existing RAG systems are not specifically designed to address. The authors propose ConRAG, which constructs a lightweight entity–document graph using entity co-occurrence and LLM-based entity filtering, and retrieves paths between the endpoints. They repurpose two commonly used multi-hop QA datasets, MuSiQue and 2WikiMultiHopQA, as benchmarks for this task. Their experiments demonstrate improved relation-chain recovery and reduced indexing token costs compared with existing RAG baselines.

Critical strengths
  1. This paper provides a new perspective on RAG by shifting the focus from multi-hop question answering to multi-hop relation inference, and presents a well-designed method with experiments showing improved relation-chain recovery and reduced indexing token costs.

  2. The paper further explores this task by repurposing existing benchmarks and proposing corresponding evaluation metrics.

Critical weaknesses
  1. The task of Multi-hop Relation Inference itself is a valid research task and is worth exploring. The paper discusses potential applications, but it would benefit from more evaluation in real-world relation inference scenarios to further highlight its practical significance.

  2. The baseline comparisons could be more comprehensive, especially with additional task-adapted baselines, to demonstrate the effectiveness of the proposed approach.

Questions for authors
  1. In Section 5.2, the authors use GPT-4o-mini as the LLM judge. Are there any experiments or analyses to address potential evaluation biases, such as using a judge from a different model family, randomly swapping the answer order, or conducting human evaluation?

  2. Could you consider including additional task-adapted baselines to further validate the effectiveness of ConRAG?

3 comments
AuthorsOpenReview ↗

Answer to Reviewer ukzb [1/2]

[S1] New perspective on RAG and well-designed method; [S2] Benchmark adaptation and evaluation metrics

Thank you for recognizing these contributions! We appreciate your constructive feedback and provide a point-by-point response to your concerns below.

[W1] The task of Multi-hop Relation Inference itself is a valid research task and is worth exploring. The paper discusses potential applications, but it would benefit from more evaluation in real-world relation inference scenarios.

Thanks for appreciating the motivation behind the research problem! We have now added experiments on DWIE [1], a real-world document-level relation extraction dataset built from real-world news articles (Table 15 in the updated manuscript). We construct anchor–target queries from annotated entity relations and use the original news articles as the retrieval corpus.

MethodDoc RecallBCMCRCP
ConRAG51.9 ± 1.274.1 ± 1.943.5 ± 2.129.8 ± 1.4
Naïve RAG32.8 ± 1.553.7 ± 2.232.3 ± 1.928.5 ± 1.7

These results provide additional evidence that ConRAG's benefits extend to real-world scenarios. Given the computational cost of running the full baseline suite, we present the Naïve RAG comparison here and will extend the evaluation to the remaining baselines in the revised manuscript as their results become available.

[W2] The baseline comparisons could be more comprehensive, especially with additional task-adapted baselines. [Q2 ]Could you consider including additional task-adapted baselines to further validate the effectiveness of ConRAG?

Thank you for this suggestion. We have expanded our experimental evaluation with additional baselines and ablations to better isolate ConRAG's contributions and updated the manuscript accordingly:

  1. Additional baselines. We now compare against LinearRAG [1], a particularly relevant relation-free graph RAG approach. We also introduce a task-adapted dense RAG baseline (ConRAG with seeds only), which extends standard dense retrieval with endpoint constraints, without requiring graph construction or path traversal.

The new results (added to Table 1) are summarized below:

MethodMuSiQue BCMCRCP2Wiki BCMCRCP
LinearRAG70.7±0.926.7±0.331.8±1.579.2±0.946.1±2.049.2±0.8
ConRAG (seeds only)68.5±0.726.3±1.035.7±1.976.6±1.446.2±1.842.6±1.3
ConRAG71.2±0.729.5±0.534.3±1.181.0±1.449.4±1.848.1±0.9

ConRAG achieves the highest bridge recovery and chain recall on both datasets, while maintaining comparable chain precision (CP). This demonstrates that path traversal contributes meaningfully beyond endpoint-constrained retrieval, and that ConRAG's advantages extend to comparisons with recent, closely related state-of-the-art relation-free graph RAG methods.

  1. We additionally evaluate ConRAG's individual components (new Tables 19–20) as task-adapted baselines to assess their individual contributions:
  • Endpoint-aware retrieval filtering: We ablate the seed filtering and injection mechanism that ensures coverage of both query entities. Enabling it improves document recall by approximately 6 pp., demonstrating the importance of endpoint constraints.

  • Node canonicalization: We ablate the merging of equivalent entity nodes to assess its contribution to graph connectivity and retrieval. Results show a relatively small effect on recall, but it helps reduce graph size and redundant entity noise by merging equivalent entity nodes.

Together, these experiments provide a more comprehensive comparison with closely related methods and help disentangle the contributions of ConRAG's individual components.

AuthorsOpenReview ↗

Answer to Reviewer ukzb [2/2]

[Q1] In Section 5.2, the authors use GPT-4o-mini as the LLM judge. Are there any experiments or analyses to address potential evaluation biases, such as using a judge from a different model family, randomly swapping the answer order, or conducting human evaluation?

Thank you for these suggestions! We have strengthened the robustness of our LLM-based evaluation as follows:

1. Judge from a different model family. We repeated the qualitative evaluation using Qwen3-32B, an independent model family, and compared the results with our original GPT-4o-mini judge. The overall ConRAG win rates (%) are summarized below (new Table 14). ConRAG maintains a higher overall win rate against every baseline on both datasets with both judges, demonstrating that the qualitative ranking is robust across model families.

BaselineMuSiQue GPT-4o-miniMuSiQue Qwen3-32B2Wiki GPT-4o-mini2Wiki Qwen3-32B
Naïve RAG53.5 ± 3.152.9 ± 3.157.2 ± 3.157.6 ± 3.1
LightRAG69.6 ± 2.972.3 ± 2.859.8 ± 3.163.7 ± 3.0
PathRAG62.0 ± 3.063.9 ± 3.061.8 ± 3.065.5 ± 2.9
HippoRAG 265.7 ± 3.065.8 ± 3.061.1 ± 3.063.2 ± 3.0
LinearRAG54.8 ± 3.153.1 ± 3.155.3 ± 3.156.7 ± 3.1

2. Answer-order randomization. We already randomized the presentation order of ConRAG and baseline responses to mitigate position bias. We have explicitly clarified this procedure in the revised manuscript.

3. Human evaluation. Large-scale human evaluation is not feasible with our current resources, but we believe the two complementary checks above provide sufficient evidence of robustness against judge-specific and position biases.

References list:

[1] Zhuang, L., Chen, S., Xiao, Y., Zhou, H., Zhang, Y., Chen, H., ... & Huang, X. (2026, April). Linearrag: Linear graph retrieval augmented generation on large-scale corpora. In International Conference on Learning Representations (Vol. 2026, pp. 147053-147075).

AuthorsOpenReview ↗

Follow-up before the discussion period ends

Thank you again for your feedback!

With the discussion period ending in the next day, we’d appreciate hearing your thoughts on our responses and revised manuscript. We’ve responded to your concerns point by point and added experiments and clarifications. We’d be very happy to discuss any remaining questions.

Note: Following a further manuscript update addressing another reviewer’s follow-up, the component ablations cited as Tables 19–20 in our response are now Tables 20–21. The numerical results remain unchanged.

Reviewer_kt9XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ConRAG uses a lightweight entity–document graph to explain connections between two known entities. It improves multi-hop evidence retrieval while reducing indexing costs compared with existing graph-based RAG methods.

Critical strengths

ConRAG generates explicit relationship chains grounded in documents and recovers more intermediate entities and supporting evidence.

Critical weaknesses
  1. Path search becomes costly in dense graphs, and performance declines on longer chains.
  2. Experiments rely on adapted QA datasets rather than real-world discovery tasks.
Questions for authors
  1. In ConRAG, could valid alternative paths be marked as incorrect?
  2. Is the benchmark derived from 2WikiMultiHopQA included in the dev.parquet file in the codebase?
2 comments
AuthorsOpenReview ↗

Answer to Reviewer kt9X

Strengths

Thank you for recognizing ConRAG's strengths and for your constructive feedback that helped improve the manuscript! We have answered the questions and weaknesses point-by-point below.

[W1] Path search becomes costly in dense graphs,

We acknowledge that dense graphs increase search complexity, which is the case across all graph-based methods. In particular, to reduce the cost of path search, ConRAG includes hub suppression, a maximum path length of 5 documents, and endpoint-constrained traversal. Figure 4 illustrates the trade-off: increasing the hub threshold beyond 0.8 substantially increases retrieval time, with only modest improvements in document recall.

To assess scalability quantitatively, we conducted an additional experiment on the same 100 MuSiQue questions using different corpus sizes. Average retrieval time increased from 0.87 s with 500 documents to 2.57 s with the full ~26.3k-document corpus. We have added these new results and discussion in the updated manuscript (Table 21). Increasing the corpus size by ~53× increases retrieval time by only ~3×, suggesting that ConRAG's path-based retrieval remains computationally manageable as the corpus grows.

[W1] and performance declines on longer chains.

As the reviewer points out, performance declines on longer chains, a challenge shared by all evaluated baselines. Nevertheless, ConRAG achieves the highest 3-hop chain recall (9.4%), compared to 8.1% for the strongest baseline, LinearRAG (Table 7). Four-hop recovery remains challenging across all methods. However, our new hop-stratified false-negative analysis (Appendix C.3.1, Tables 10–11) shows that many alternative paths penalized by single-reference evaluation are actually valid, strengthening ConRAG's success. Specifically, 87.5% of distinct alternative paths for 2-hop, 87.5% for 3-hop, and 87.5% for 4-hop were validated against the retrieved evidence. These findings highlight a side-effect of single-reference evaluation, which we now discuss in the revised manuscript.

[W2] Experiments rely on adapted QA datasets rather than real-world discovery tasks.

We agree that evaluation beyond adapted QA benchmarks is important. To address this, we added experiments on DWIE [1], a document-level relation extraction dataset built from real-world news articles (Table 15 and below). We construct anchor–target queries from annotated entity relations and use the original news articles as the retrieval corpus.

MethodDoc RecallBCMCRCP
ConRAG51.9 ± 1.274.1 ± 1.943.5 ± 2.129.8 ± 1.4
Naïve RAG32.8 ± 1.553.7 ± 2.232.3 ± 1.928.5 ± 1.7

These results provide additional evidence that ConRAG's benefits extend beyond adapted QA benchmarks to real-world text, and we have added them to the updated manuscript. Given the computational cost of running the full baseline suite, we present the Naïve RAG comparison here and will extend the evaluation to the remaining baselines in the revised manuscript as their results become available.

[Q1] In ConRAG, could valid alternative paths be marked as incorrect?

Yes, this is a limitation of single-reference evaluation: valid alternative paths can be marked as incorrect when they differ from the annotated gold chain. We investigate this in Appendix C.3 (Tables 8–9) through a false-negative analysis across all methods, using LLM-based blind validation against each method's retrieved evidence. For ConRAG, 87.5% of distinct alternative paths on MuSiQue and 95.1% on 2WikiMultiHopQA were validated as correct. Importantly, including these validated alternatives preserves ConRAG's leading position among the methods evaluated in this analysis. We have further expanded this analysis with new hop-stratified results (Tables 10–11), showing that 87.5% of distinct alternative paths are valid across 2-, 3-, and 4-hop MuSiQue questions. This limitation can be addressed by either expanding the benchmark to include more complete sets of valid reference paths or complementing exact-match evaluation with evidence-grounded LLM validation, as demonstrated in our analysis. We have clarified the validation protocol in the revised manuscript.

[Q2] Is the benchmark derived from 2WikiMultiHopQA included in the dev.parquet file in the codebase?

Yes, the benchmark derived from 2WikiMultiHopQA is included in the dev.parquet file provided in our code repository.

References list:

[1] Zaporojets, K., Deleu, J., Develder, C., & Demeester, T. (2021). DWIE: An entity-centric dataset for multi-task document-level information extraction. Information Processing & Management, 58(4), 102563.

AuthorsOpenReview ↗

Follow-up before the discussion period ends

Thank you again for your feedback!

With the discussion period ending in the next day, we’d appreciate hearing your thoughts on our responses and revised manuscript. We’ve responded to your concerns point by point and added experiments and clarifications. We’d be very happy to discuss any remaining questions.

Note: Following a further manuscript update addressing another reviewer’s follow-up, the retrieval-latency results cited as Table 21 in our response are now in Table 22. The numerical results remain unchanged.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to all Reviewers

Dear Reviewers,

Thank you all for your constructive feedback, which has helped us substantially strengthen our work. We have updated the manuscript, with changes highlighted in blue. Below, we summarize the main additions:

  • Real-world evaluation (kt9X, ukzb): We extended our experiments to DWIE, a dataset of real-world news articles.

  • Additional baselines (ukzb, xtks): We included LinearRAG and a task-adapted, endpoint-constrained seeds-only baseline, allowing us to better evaluate the contribution of path retrieval.

  • Extensive component ablations (ukzb, xtks): We investigated the effects of entity vocabulary filtering, endpoint-aware retrieval, canonicalization, and hub suppression, including their recall–efficiency trade-offs.

  • Evaluation robustness (ukzb, xtks): We repeated our qualitative evaluation with Qwen3-32B, a judge from a different model family, and clarified our answer-order randomization procedure to address potential evaluation biases.

  • Evidence grounding (xtks): We added evaluations of response and chain-link groundedness against retrieved evidence, and a closed-book comparison to assess the contribution of retrieval beyond parametric knowledge.

  • Scalability and efficiency (kt9X, xtks): We evaluated retrieval latency across different corpus sizes, measured response lengths, and assessed the impact of using GPT-4o-mini for indexing.

  • Clarifications and limitations (kt9X, xtks): We strengthened our discussion of related work, clarified our contributions relative to existing multi-hop retrieval methods, and expanded the analysis of alternative valid reasoning paths.

Overall, these additional experiments reinforce our main findings: ConRAG achieves strong bridge and reasoning-chain recovery across benchmarks, while the new baselines, ablations, and robustness analyses provide a more comprehensive understanding of its strengths and limitations.

We have added an answer to every raised question in our point-by-point responses below and expanded on the contributions.

Thank you again for your valuable feedback!

We would be very happy to hear your thoughts and address any additional questions or comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.