- Rating
- 3
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes HyperReCo, a hypergraph-based framework for multi-hop retrieval-augmented generation. It introduces a query-conditioned HyperGNN to learn document and entity relevance through hypergraph message passing, and Gradient-Guided Hyper-Path Decoding (GGHD) to construct explicit evidence connections for LLM reasoning. Experiments on six benchmarks demonstrate strong retrieval and downstream QA performance, with ablation studies and cross-domain evaluations supporting the effectiveness of the framework.
Critical strengths
-
Strong and consistent empirical performance. HyperReCo achieves consistent improvements in both retrieval recall and downstream QA accuracy across three multi-hop benchmarks. The gains over HGRAG are substantial, and the additional experiments with different embedding models further support the robustness of the approach. These results provide convincing evidence of the framework's practical effectiveness.
-
Effective integration of retrieval and evidence connection. The paper addresses a meaningful limitation of multi-hop RAG: retrieving the correct documents does not guarantee that an LLM can connect their evidence. The path ablation shows that explicit evidence connections improve mean EM from 55.0 to 57.6 under the same retrieved documents. This provides useful evidence for the value of the proposed retrieval-and-connection framework.
Critical weaknesses
-
Limited methodological novelty relative to prior work. The main components are closely related to existing approaches: HGRAG adopts a similar document–entity hypergraph representation, while GFM-RAG and HyperRAG have explored learned structural retrieval and evidence-path construction. Although HyperReCo combines these ideas effectively, the paper does not sufficiently establish what new capabilities its HyperGNN provides beyond existing graph-based approaches. This limits the significance of the methodological contribution.
-
Insufficient evidence for the distinctive contribution of GGHD. While explicit paths clearly improve QA performance, GGHD only achieves a 0.6–0.7 point mean EM improvement over the simpler S-forest and G-paths alternatives. The statistical significance of these gains is not established, and the faithfulness of the gradient-based connection scores remains unclear. Consequently, the necessity of the proposed decoding mechanism is not yet fully demonstrated.
Questions for authors
-
What is the key methodological advantage of HyperReCo over closely related methods such as HGRAG and GFM-RAG? Could the authors compare HyperGNN against a standard query-conditioned document–entity bipartite GNN with matched supervision, input features, and model capacity? This would help isolate the contribution of the proposed architecture.
-
Figure 3 examines message passing and query conditioning, but does not isolate the contributions of entity-level supervision and KL-based semantic distillation. Could the authors provide ablations of these objectives under the same HyperGNN architecture? This would help clarify which components drive the retrieval improvements.
-
How faithfully do the gradient-based bridge scores in Eq. (6) capture specific cross-document dependencies? Could the authors validate this by masking high-attribution connections in the frozen HyperGNN and comparing their effects on target-document relevance with low-attribution or random connections?
Overall, I lean toward weak acceptance given the consistent empirical improvements. My main reservations concern methodological novelty and the independent contribution of GGHD. Convincing responses to these questions would strengthen my confidence in supporting acceptance.
6 comments
Response to Reviewer QK1C [Part 2/4]
Proposition 1 concerns what the hypergraph preserves. We now further examine how a retrieval mechanism uses query-relevance signals on hypergraph. Existing hypergraph-based methods (e.g., HGRAG) rely on predefined diffusion to propagate relevance over these higher-order evidence associations. However, fixed propagation may fail to fully exploit these associations, as it can collapse distinct entity-level relevance distributions into identical document scores.
Theorem 1 (Coverage and strict separation from fixed diffusion). For every valid , every , and every ,
If, in addition, contains two distinct entities with identical hyperedge memberships, the inclusion is strict:
Proof. Step 1: coverage. Choose linear HGMP messages and readout such that
u^{(t)}&=W_pD_E^{-1}H^\top D_V^{-1/2}h^{(t)},\ h^{(t+1)}&=D_V^{-1/2}Hu^{(t)},\ \widehat y^{(t)}&=W_pH^\top h^{(t)}. \end{aligned}$$ Substituting the first equation into the second yields $$h^{(t+1)}=S_qh^{(t)}.$$ With $h^{(0)}=x$, induction gives $$h^{(t)}=S_q^tx \quad\Longrightarrow\quad \boxed{\widehat y^{(t)}=W_pH^\top S_q^tx=y^{(t)}.}$$ Therefore, fixed hypergraph diffusion is exactly recoverable as a special case of HGMP. *Step 2: Strict separation.* Suppose two distinct nonisolated entities $v_1,v_2$ have identical hyperedge memberships. Let $z=\mathbf e_{v_1}-\mathbf e_{v_2}$. Since their incidence rows and degrees coincide, $$H^\top D_V^{-1/2}z=0 \quad\Longrightarrow\quad S_qz=0,\qquad W_pH^\top z=0.$$ For any $x'=x+\alpha z$, the diffusion scores satisfy $$\begin{aligned} y^{(t)}(x')-y^{(t)}(x)&=\alpha W_pH^\top S_q^tz\\ &=0,\qquad \forall t\geq0. \end{aligned}$$ Therefore, every deterministic post-processing of the diffusion scores takes the same value on $x$ and $x'$. Now consider two relevance vectors $x_c,x_s$, identical elsewhere, with $$(x_{c,v_1},x_{c,v_2})=(0,1), \qquad (x_{s,v_1},x_{s,v_2})=(\frac12,\frac12).$$ Since $x_s=x_c+\frac12z$, their diffusion scores are identical at every depth. For any hyperedge $e_d$ containing these entities, consider a GELU-based nonlinear HGMP readout with uniform attention: $$F_d(x)=\frac1{\delta_d}\sum_{v\in e_d} \operatorname{GELU}(x_v-\frac12),$$ where $\delta_d=|e_d|$. For any hyperedge containing $v_1,v_2$, using $\operatorname{GELU}(s)=s\Phi(s)$, we obtain $$\begin{aligned} F_d(x_c)-F_d(x_s)&=\frac{1}{\delta_d}\left[\operatorname{GELU}\left(\frac12\right)+\operatorname{GELU}\left(-\frac12\right) -2\operatorname{GELU}(0)\right]\\ &=\frac{1}{\delta_d} \left[\Phi\left(\frac12\right)-\frac12\right]>0. \end{aligned}$$ Therefore, we can derive $$\boxed{ y^{(t)}(x_c)=y^{(t)}(x_s),\ \forall t\geq0, \qquad F_d(x_c)\neq F_d(x_s). }$$ $\square$ ## Isolating the contributions of entity-level supervision and KL-based semantic distillation (Question 2) Thank you for this helpful suggestion. To isolate these objectives, we retrained the same HyperGNN architecture after removing either the entity-supervision loss (*w/o Entity*) or the KL-based semantic distillation loss (*w/o KL*), with all other settings fixed. Following Figure 3, we report R@5. From the results, **both objectives contribute to every dataset.** Removing entity supervision lowers mean R@5 by 2.1 points, with the largest effect on 2Wiki (2.9 points), and removing KL distillation lowers it by 1.4 points, with the largest effect on MuSiQue (2.9 points). *Ablation of the training objectives* | Variant | HotpotQA | MuSiQue | 2Wiki | Mean R@5 | | :---- | :---- | :---- | :---- | :---- | | HyperReCo | 96.8 | 74.5 | 97.8 | **89.7** | | w/o Entity | 95.2 | 72.6 | 94.9 | 87.6 | | w/o KL | 96.3 | 71.6 | 97.0 | 88.3 | **We have added these results in Section 5.3.**Response to Reviewer QK1C [Part 3/4]
Comparison with bipartite GNNs (Question 1)
We thank the reviewer for bringing up the connection with a standard query-conditioned document–entity bipartite GNN. We want to clarify that our HyperGNN is a specialized, type-aware bipartite GNN with direction-specific, query-conditioned message passing (detailed proof in Appendix C.8.1). This design explicitly models entity–document membership roles and captures complementary evidence interactions essential for multi-hop retrieval. It offers more specialized representations than homogeneous bipartite GNNs, while remaining expressively equivalent in principle to sufficiently expressive heterogeneous bipartite GNNs. We further provide a theoretical analysis to clarify these expressive relationships.
(1) HyperGNN is a heterogeneous bipartite GNN. If document and entity nodes have type-specific message-passing and aggregation functions, a heterogeneous bipartite GNN can emulate our HyperGNN, which follows the two-stage hypergraph message-passing formulation in AllSet (Chien et al., 2022) and the incidence-graph view of UniGNN (Huang and Yang, 2021). We have added detailed theoretical analysis in Appendix C.8.1
(2) Advantages over Homogeneous Bipartite GNN. To evaluate the practical benefit of explicitly distinguishing the two propagation directions, we compared HyperGNN with a standard query-conditioned bipartite GNN (Homogeneous Bipartite GNN), which treats documents and entities as a single node type with shared message-passing and aggregation functions. Both models use the same input features, supervision, number of layers, and hidden dimension. From the results, HyperGNN improves average R@2 by 4.6 points and average R@5 by 3.0 points over the matched baseline. This is empirical evidence that explicitly distinguishing document and entity roles through direction-specific message passing benefits multi-hop evidence retrieval.
Retrieval recall of the HyperGNN and a matched homogeneous bipartite GNN
| Avg. R@2 | Avg. R@5 | |
|---|---|---|
| Homogeneous BiGNN | 70.6 | 86.7 |
| HyperReCo | 75.2 | 89.7 |
We have added the experimental results and detailed discussion in the Appendix C.8.2 of the revised manuscript.
Independent contribution of GGHD (Weakness 2)
We thank the reviewer for this important concern. We agree that GGHD yields only modest EM improvements over S-forest and G-paths. However, the primary objective of GGHD is to construct high-quality, compact, and useful hyper-paths. Since EM measures final answer correctness, it cannot directly capture the quality or usefulness of the generated evidence paths.
To evaluate this objective more directly, we employ LLM-as-a-judge to assess path-level helpfulness. GGHD achieves the highest helpful-path ratio (59.2%) while generating the fewest paths (1.68 per question), compared with 50.6% helpfulness and 2.11 paths for S-forest. These results suggest that GGHD produces more compact and useful evidence connections, providing complementary evidence beyond answer-level EM. To further validate the reliability of LLM-based evaluation, two PhD researchers independently annotated the decoded paths from 300 QA samples. Their helpful-path ratios were 66.4% and 68.6%, respectively, with over 82% agreement with the LLM judge (83.5% on average). These results demonstrate encouraging consistency between human and LLM judgments, further supporting the usefulness of GGHD-generated evidence paths.
Table 11. LLM-judged path helpfulness and number of paths
| Method | Helpful-path ratio (%) ↑ | Mean paths per question ↓ |
|---|---|---|
| GGHD | 59.2 | 1.68 |
| w/o Q-PCSF | 28.4 | 5.39 |
| G-paths | 43.1 | 2.97 |
| S-forest | 50.6 | 2.11 |
GGHD adopts gradient-based attribution to identify evidence connections explicitly. Specifically, gradient attribution estimates the sensitivity of predicted relevance to candidate bridges, while Q-PCSF globally organizes these connections into compact hyper-paths. To validate this design, we conduct frozen-model message masking (detailed in our response to Question 3). Masking high-attribution bridges causes substantially larger target-logit drops than masking random or low-attribution bridges (0.154 vs. 0.067 and 0.017 on average), supporting the effectiveness of gradient attribution in identifying prediction-sensitive connections.
We have added the detailed discussion in Appendices C.4, C.5 of the revised manuscript.
Response to Reviewer QK1C [Part 1/4]
We sincerely thank you for your positive and detailed review of our paper. We address the reviewer’s concerns point-by-point below.
Methodological contribution beyond HGRAG and GFM-RAG (Weakness 1 & Question 1)
We thank the reviewer for raising this concern. We clarify that our novelty lies in a novel framework that introduces a query-conditioned HyperGNN to learn higher-order evidence associations and translates learned structural relevance into explicit hyper-paths for LLM reasoning through gradient-guided hyper-path decoding. Key advantages over closely related methods are summarized as follows.
Here, we provide additional theoretical analyses to further substantiate the advantages of our approach from two perspectives:
The core proofs are presented below, and we have added detailed theoretical analysis and proofs in Appendix D.2.
Let H be the entity–document incidence matrix, DV the entity-degree matrix, and A(H)=HH⊤−DV the pairwise projection. Define the document-level query coverage as C(H,Q)=∑d1[Q⊆ed]. There exist two hypergraphs with identical pairwise projections and query-dependent entity features but different query coverage.
Proposition 1 (Representational Advantage of Hypergraphs for GNN-Based Evidence Retrieval). Let C(H,Q) denote the document-level query coverage function. There exist two document-induced hypergraphs HA and HB with identical pairwise projections and query-dependent entity features, such that A(HA)=A(HB),C(HA,Q)=C(HB,Q).
Proof. Consider two hypergraphs on V=1,…,6: EA=1,2,3,1,4,5,2,4,6,3,5,6
EB=1,2,4,1,3,5,2,3,6,4,5,6
In both hypergraphs, every entity pair co-occurs exactly once, except 1,6,2,5,3,4, which never co-occur. Therefore, A(HA)=A(HB).
For Q=1,2,3, assign identical query-dependent entity features xv=1[v∈Q]. Any deterministic KG-GNN g operating solely on the pairwise projection and these features must satisfy
g(A(HA),X)=g(A(HB),X).
However, HA contains a document covering all query entities, whereas HB does not:
C(HA,Q)=1=0=C(HB,Q). □