- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes BubbleRAG, a training-free GraphRAG framework for multi-hop question answering. Prior to resolving queries, the corpus is segmented to chunks for triple extraction with an LLM, which the triples form a KG. Given a query, Semantic Anchoring Grouping first retrieves relevant text chunks as contextual clues, using them to answer questions if they are sufficient or to guide the LLM to refine query concepts via anchor specialization and schema relaxation. These refined phrases are then used to retrieve candidate nodes and edges within the KG, which the LLM further cluster them into weighted semantic anchor groups. Since finding an optimal informative (minimal sufficient) subgraph that covers all semantic anchor groups remains NP-hard, BubbleRAG adopts an alternative approach to find discover candidate evidence graphs, which contain two phases: CEG discovery and CEG ranking. The CEG discovery stage is implemented via bubble expansion, which starts from each anchor and gradually explores nearby nodes, visiting the most query-relevant ones first. When searches from different anchors meet, their paths are combined into a small connected subgraph that links the key concepts in the query. The CEG ranking phase scores the quality of each subgraph, and prioritize those that are close to the original query and miss less important concepts. The top-ranked CEGs will be provided to the LLM for answer generation. BubbleRAG, as well as GraphRAG baselines, are evaluated on the three widely adopted benchmarks, HotpotQA, MuSiQue, and 2wiki.
Critical strengths
-
The paper contains a detailed related work section, which summarizes the pros and cons of different types of GraphRAG methods in detail.
-
Different from linking to isolated entities, BubbleRAG explores and prunes evidence at the subgraph level, yielding evidence with global coverage of the query’s relevant concepts and logical relationships.
-
The proposed evidence subgraph retrieval method (w/ CEG discovery and CEG ranking steps) is able to bypass the need of resolving the NP-hard graph-join problem that requires minimal-sufficiency.
Critical weaknesses
-
The manuscript contains many technical terms that readers from other domains cannot easily understand. The relationships between different terms, namely “semantic anchor group”, “alternative concept match”, “candidate evidence graph” and “optimal informative subgraph retrieval” are less intuitive. It is hard for readers to understand that the purpose of optimal informative subgraph retrieval is to combine different semantic anchor groups with a minimal-sufficient subgraph within the KG. This reviewer suggests the authors refrain from the excessive use of technical terms without establishing the necessary connections between them.
-
The proposed method is not compared with the latest GraphRAG solutions, namely GFM-RAG [1] and G-Reasoner [2].
-
The experimental section lacks a detailed analysis of the performance improvements. Section 4.2 merely restates results that can be read directly from Table 1. It would be more informative to explain in depth why BubbleRAG outperforms the baselines.
[1] Luo et al., GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation (Neurips 2025)
[2] Luo et al., G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge (ICLR 2026)
Questions for authors
-
What if there are no direct connections between different semantic anchor groups, in this case, how can they be connected into a candidate evidence graph?
-
Given that questions in HotpotQA, MuSique, and 2Wiki datasets are constructed based on 1 single path only. It is possible that some of the questions may have multiple solution paths, but whether it is necessary to consider all of them remains questionable.
3 comments
Rebuttal by Authors (2/2)
Q1. Connecting semantic anchor groups when no direct connections exist.
Thank you for raising this point. In BubbleRAG, we consider two cases where semantic anchor groups cannot be connected.
In the first case, some groups cannot be connected to each other, but useful connections can still be found within individual groups or between some groups. This often occurs in comparison or aggregation questions, where the required evidence may lie in separate subgraphs. For example, to compare which of two directors was born earlier, we can form a subgraph containing birth information for each director without connecting the two subgraphs. BubbleRAG keeps these locally connected subgraphs, ranks them, and selects multiple subgraphs up to the specified limit. These are then passed to RAE for further retrieval, as shown in Figure 3 of the main paper.
In the second case, the search cannot form any useful connected subgraph: both within-group and between-group connections fail. Among queries that entered graph retrieval, this occurs in only 6.18% and 5.11% of questions on 2Wiki and HotpotQA, respectively. Algorithm 1 handles this case by keeping the initial anchors and allowing RAE to continue retrieval from them. This prevents the process from stopping when no connected subgraph can be formed.
Q2. The necessity of considering multiple paths when a single evidence path is annotated.
Thank you for this important question. We chose these datasets because they are widely used benchmarks in GraphRAG, making it easier to compare with existing methods. Although these datasets usually provide one annotated evidence path supporting the answer, the KGs we search are extracted from text by an LLM. The same entity may appear under different names, and the same relation may be expressed in different ways. Therefore, finding a clearly defined annotated path in the actual KG may still require exploring different entity matches and connections.
For example, consider the question: “What state was the dissenter in Kessler v. Treat from?” We would expect to start from the matched judge, Justice John Marshall Harlan, and find his home state. However, in the saved candidate graph, this node was only connected to a legal opinion in another case and did not lead to the answer. BubbleRAG instead started from another candidate, the case Kessler v. Treat, and followed a relation identifying the dissenting judge to find John Marshall Harlan. It then obtained the answer, Kentucky, from his linked biography.
Here, Justice John Marshall Harlan and John Marshall Harlan are the same person, but the KG stores them as two disconnected nodes. The latter was not among the initial anchors. It was found during the connection search and included in the top-ranked subgraph. Thus, even after matching the correct person, the method may still need to start from another entity to find the evidence corresponding to the annotated path.
The KG analysis in the appendix further shows the variety of relation expressions. The KGs for 2Wiki, HotpotQA, and MuSiQue contain about 76,900, 150,800, and 157,800 triples, respectively, but have about 14,600, 36,500, and 45,000 distinct relation expressions. About 70% of these expressions appear only once. BubbleRAG keeps multiple candidate matches and connection options to handle these uncertainties during retrieval and avoid ruling out the correct evidence too early.
Reply to the rebuttal.
First, I shall apologize for mistakenly copying the weaknesses into the strengths block. I used a Word document to write my initial drafts for my reviews, and the strengths and weaknesses sections look similar.
Thank you for your efforts in improving the manuscript and addressing my questions. Now, this reviewer believes that the strengths weight higher than weaknesses, and adjusted the overall scoring.
Nevertheless, this reviewer remains unconvinced that a paper proposing a pipeline RAG system for question answering offers sufficient novelty for acceptance at a top conference, particularly in light of recent trainable RAG systems (e.g. G-Reasoner, GFM-RAG) that achieve better performance without relying on larger-scale models. In addition, the evaluation lacks experiments on real-world questions with multiple explicitly valid evidence paths.
Regarding these issues, this reviewer would like to defer the decision to the AC/PC.
Rebuttal by Authors (1/2)
We thank the reviewer for the careful reading and constructive feedback. Below, we address each concern and explain the corresponding clarifications and revisions.
W1. Excessive technical terminology and unclear connections between concepts.
Thank you for pointing this out. We agree that we introduced too many technical terms before explaining the overall idea, without clearly showing how these concepts are connected. This made the paper harder to follow. We used these terms to describe the roles of different stages in our method, but our explanation was not clear enough.
To address this issue, we have revised our paper to simplify the explanation and add the necessary introduction at the beginning of the method section. We now first explain how BubbleRAG finds candidate entities for the concepts in a question, connects them into subgraphs, selects useful subgraphs, and then adds further facts needed to answer the question, before introducing the corresponding technical terms. We have also improved the transitions between stages to explain why each next step is needed and how it uses the results of the previous step. These changes help readers understand the overall idea before going into the details.
W2. Missing comparisons with GFM-RAG and G-Reasoner.
Thank you for drawing our attention to GFM-RAG and G-Reasoner. Both are closely related to our work. However, our experiments mainly compare methods that use off-the-shelf models without additional training of retrieval or reasoning components. GFM-RAG and G-Reasoner introduce trained components for graph retrieval or reasoning, while BubbleRAG focuses on handling uncertainty in concept matching and evidence connection at query time, without training additional components. We therefore limit our experimental comparisons to training-free methods to reduce the influence of additional training data and supervision. We agree that both works are important for understanding recent progress in GraphRAG. We have added a discussion of these methods in Related Work.
We have also added experiments with two recent training-free GraphRAG methods, QAFD-RAG and LinearRAG. The results with the 30B model are shown below:
W3. Insufficient analysis of why BubbleRAG improves over the baselines.
Thank you for your suggestion. We have added a more detailed analysis of BubbleRAG’s gains in Section 4.2. Using the original run logs of BubbleRAG and HippoRAG2 with the 30B model, we compared their retrieval results and answers question by question. We explain the gains from two aspects. First, BubbleRAG selects candidate entities together with their connections, allowing evidence that ranks lower but is important for answering the question to be selected. Second, RAE uses the question to guide further retrieval of specific information, helping the model find the relations, attributes, and dates needed to answer it.
On 2Wiki, BubbleRAG’s gains mainly come from questions where HippoRAG2 did not rank key supporting documents in its top 15 results. Some supporting documents for these questions were ranked between 16 and 200 by HippoRAG2. BubbleRAG’s subgraph ranking allows these documents to be selected based on their connections to other concepts in the question. The case study in the appendix shows this difference: by connecting a film to its director, BubbleRAG selected a subgraph containing the director’s biography and included this answer-bearing biography in the context for answering the question. HippoRAG2 ranked the same biography only 65th.
On MuSiQue, about 57% of the net F1 gain also comes from questions where HippoRAG2’s top 15 results did not cover all annotated supporting document titles. The remaining gain comes from questions where all supporting document titles were already covered. After examining the answers and saved traces, we found that even when relevant documents were retrieved, the answers could still mix up attributes of different entities or facts from different time periods. RAE continues retrieval based on the question. For example, after finding a relevant person, it searches for the specific family relationship or dates of holding office asked about in the question, providing more targeted evidence for the final answer.