- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes SCOPE, a multi-hop retrieval method based on entity-guided passage expansion. It first retrieves passages using dense retrieval, expands the candidates through shared entities, and then uses rank fusion and LLM-based selection to retrieve supporting evidence.
Critical strengths
-
The problem of multi-hop evidence retrieval is important for RAG.
-
The paper provides detailed ablation studies and evaluates different embedding models.
Critical weaknesses
-
The novelty is very limited. The main idea is to expand retrieved passages through shared entities, followed by rank fusion and LLM-based reranking. These are well-established techniques in RAG and GraphRAG. In particular, KG²RAG [1] already uses initially retrieved passages as seeds and performs knowledge graph-guided chunk expansion. Other related works [2] have also explored graph-based evidence retrieval and multi-hop retrieval. I do not see a clear technical contribution beyond combining existing components.
-
The improvements over HippoRAG 2 are not convincing. In Appendix C.4 (Table 23), when HippoRAG 2 is equipped with the same LLM-based evidence selector, SCOPE does not achieve statistically significant improvements in retrieval recall or QA F1 across the four tested settings. This suggests that the improvements in the main tables may largely come from the LLM-based selection rather than the proposed entity expansion.
-
The evaluation is limited to traditional benchmarks. Most experiments use MuSiQue, 2Wiki, and HotpotQA, which have been widely used in previous studies. On LoCoMo, the proposed method performs worse than graph-free hybrid retrieval with selection. It is unclear whether the approach can generalize to other domains and retrieval tasks.
-
The efficiency advantage is unclear. Although the paper emphasizes the simplicity of SCOPE, it still requires LLM-based indexing and evidence selection. According to Table 7, SCOPE uses more indexing and online LLM tokens than HippoRAG 2 on 2Wiki. Therefore, the proposed method does not demonstrate a clear efficiency advantage.
[1] Zhu et al. Knowledge Graph-Guided Retrieval Augmented Generation (KG²RAG).
[2] Asai et al. Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering.
Questions for authors
Please refer to the weaknesses.
Thank you for the detailed review.
W1 — novelty vs KG²RAG and Asai et al. KG²RAG expands seed chunks through an LLM-extracted triple KG and organizes a subgraph as context. Asai et al. learn a recurrent retriever over Wikipedia hyperlinks. SCOPE uses a passage–entity bipartite index built from entity mentions, takes a single hop, and does no graph-side reasoning or learning at query time. We agree the components are known. The paper's contribution is the controlled dissection (see our reply to WDWC W1). In the revision we will cite both works in §2 and state the differences.
W2 — are the gains from the selector rather than the expansion? Table 15 was designed to answer this question; see the general response. With the selector held fixed, removing the expansion lowers R@10 significantly in 10/10 (corpus × selector) cells on 2Wiki and HotpotQA. On HotpotQA the loss is about the same whichever selector is used (+4.4 to +5.1 pp). On MuSiQue the effect is smaller: positive in 9/10 cells, significant under the two LLM selectors in R@10 and in 7/10 cells in F1 (Table 15d). So the selector does not create the candidate gain; it determines how much of it reaches the reader. We agree with the second part of your reading: with the same selector, SCOPE does not beat HippoRAG 2. Pooled over the four Table-23 settings plus HotpotQA, the two are equivalent within ±1 pp on R@10 and F1, and HippoRAG 2 is better at R@5. We will remove the "outperforms HippoRAG 2" claim in the revision.
W3 — benchmarks, LoCoMo, generalization. See our reply to TbzC W3. In the revision we will move the LoCoMo analysis (entity sparsity, within-session coreference) from the appendix into §6, framed as a failure condition of entity-based expansion, together with the comparison-question result above.
W4 — efficiency. We agree that Table 7 does not support an indexing-cost advantage, and we will remove that wording. The data support an online-cost advantage. Under a shared selector and equal quality, SCOPE uses 4,175 online tokens per HotpotQA question against 7,132 for HippoRAG 2 + selector (Table 15a). Measured sequentially on 200 HotpotQA questions, it answers in about 2.7 s per question against about 3.9 s. Selector and reader times are identical, and the gap comes from HippoRAG 2's query-time retrieval (Table 15e). The revised efficiency paragraph will read "comparable indexing cost, lower online cost and latency".