acceptodds
Under review as a conference paper at ICLR 2027

DhiRAG: Learning Soft Hierarchical Indices for Retrieval-Augmented Generation

Abstract

Hierarchical indices can provide corpus-level context for retrieval-augmented generation, but their usefulness depends on both how communities are constructed and how community information is combined with passage-level relevance. We present DhiRAG, a framework that learns soft hierarchical affiliations and uses them to contextualize evidence ranking. Starting from a semantic graph over text chunks, a graph neural assignment network is optimized using a differentiable structural-entropy objective. The learned affiliations serve two connected roles: they produce community prototypes through weighted aggregation of chunk embeddings, and they transfer query–community relevance to the original chunks. The resulting structural scores are combined with local matching signals to retrieve textual evidence, without generating natural-language community summaries. On PopQA, HotpotQA, 2WikiMultiHopQA, and MuSiQue, DhiRAG achieves the highest average Str-Acc and LLM-Acc among the evaluated methods, improving over the strongest average baseline by and percentage points, respectively. Retrieval-level evaluation and component ablations provide complementary evidence for the effectiveness of the learned index. These results support soft hierarchical context as a useful complement to local evidence matching.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
GraphRAG

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_ywKLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

DhiRAG builds a KNN graph over chunks and trains a GCN assigner with a differentiable structural-entropy (SE) loss to get soft hierarchical chunk-to-community affiliations. These are reused to (i) form community prototypes without LLM summaries and (ii) give chunks a structural prior that is fused with dense+lexical scores (γ=0.2). It reports the best average accuracy on PopQA, HotpotQA, 2Wiki, and MuSiQue.

The efficiency-oriented direction is good and the performance looks nice. However, the novelty is limited (essentially GNN + SeRAG), no code is available, the paper is not rigorously written and contains many numerical errors, and important ablations are missing.

Critical strengths
  1. Summary-free indexing gives low token cost (Table 7).
  2. Using the same soft affiliations for both prototypes and scoring is a clean design, and single-pass ranking avoids hard-filtering errors.
  3. It achieves the best average accuracy across four benchmarks, and the paper includes retrieval-level metrics and run-to-run variance.
Critical weaknesses

Major

  1. Limited novelty. SeRAG already does SE-based, summary-free hierarchical indexing, and the differentiable SE formulation follows LSEnet/HyperSED. The contribution reduces to adding a GNN and soft assignments.
  2. Missing ablations.
    • No γ=0 result (hybrid scoring only). Since 80% of the score comes from the dense+lexical term, it is unclear whether the hierarchy helps at all.
    • No ablation of the lexical/NER term, and SeRAG is not run with the same hybrid scoring.
    • No comparison of the SE objective against Leiden or k-means.
    • No sensitivity analysis for λ, τ, H, K, or γ, even though the paper claims sensitivity analyses.
  3. "Soft" is unverified. The orthogonality regularizer and low-τ Gumbel-Softmax both push assignments toward one-hot, and the paper never reports how soft the learned S actually is.
  4. Possible unfair comparison. A 1,200-token chunk merges about 8–9 HotpotQA passages (about 1,073 chunks vs. 9,221 passages), so top-3 retrieval and Hit@3 may not be comparable with passage-level baselines.
  5. No code, and key details are missing: the lexical/NER scoring, which level is used in Eq. 10, N_h, λ, and τ.
  6. Efficiency is overclaimed. DhiRAG is slower than SeRAG in both indexing (939 s vs. 771 s) and retrieval (0.433 s vs. 0.188 s).

Minor

  • Table 1 caption says "three datasets"; there are four.
  • all-MiniLM-L6-v2 is 384-d, but Table 5 says 768.
  • HotpotQA passages: 9,221 (Table 4) vs. 9,811 (A.6).
  • A.3 says pools range from 8,676 to 11,656, but 2Wiki has 6,119. A.3 also mentions "conversational contexts" that are not evaluated.
  • Indexing time: 939.5 s (Table 7) vs. about 15 min + 78 s (Table 8).
  • "HOC dilemma" (RQ2) is undefined in the main text.
  • Overclaims: "guarantees", "purely attributable", "sets a new standard" (the method is second on HotpotQA).
  • LaTeX errors ("Equation equation 1"); the indexing in Eq. 3 is confusing.

References

  • DiffPool (Ying et al., 2018); MinCutPool (Bianchi et al., 2020): the CᵀAC pooling and orthogonality regularizer closely mirror these.
  • SEP (Wu et al., ICML 2022): structural-entropy-guided graph pooling.
  • Leiden (Traag et al., 2019): a natural clustering baseline.
Questions for authors
  1. What are the results with γ=0 and without the lexical/NER term? How does SeRAG perform with the same hybrid scoring?
  2. What is the entropy or overlap of the learned S per level?
  3. Is "w/o Soft Assignment" the argmax of the trained S, or retrained with hard assignments?
  4. Which level(s) are used in Eq. 10, and how is N_h chosen?
  5. What is the retrieval unit and context length for each baseline?
  6. Will the code be released?
Reviewer_DPUROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DhiRAG, a hierarchical retrieval-augmented generation framework that learns soft chunk-to-community affiliations using graph neural networks and differentiable structural entropy. The learned affiliations are used both to construct community embedding prototypes and to transfer query–community relevance back to individual passages during retrieval. The method combines this structural relevance with local semantic and lexical matching, without requiring LLM-generated community summaries.

The paper reports the highest average accuracy among the evaluated methods across four QA benchmarks, outperforming SeRAG by 2.4 percentage points in string-based accuracy and 2.7 points in LLM-judged accuracy.

Critical strengths
  • Good architectural contribution. The shared use of learned soft affiliations for community representation and passage ranking provides an interesting connection between hierarchical indexing and retrieval.
  • Good empirical performance. The method achieves strong results across factual and multi-hop QA benchmarks, with comparisons against relevant recent baselines, including SeRAG, HippoRAG2, and LinearRAG.
  • Multiple evaluation perspectives. The paper includes end-to-end QA results, retrieval-level measurements, component ablations, and sensitivity analyses.
  • Summary-free indexing. The indexing approach avoids LLM-generated community summaries while retaining original passages as evidence for answer generation.
Critical weaknesses

1. The central learning contribution is insufficiently isolated

The framework builds on GNN-based clustering, differentiable structural entropy, and soft assignments. Its main contribution is the shared use of learned affiliations for hierarchical representation and retrieval scoring. However, the experiments do not establish whether structural-entropy-trained affiliations offer advantages over simpler soft clustering or community-similarity scoring.

The existing ablations remove message passing, node semantics, or soft assignments, but do not directly isolate the proposed learning objective or retrieval mechanism. Important missing comparisons include a flat retriever with identical local scoring, simpler soft clustering under the same retrieval scorer, and alternative community representations without structural-entropy optimization. These comparisons are important for establishing the learning contribution expected at ICLR.

2. The mathematical formulation and learned hierarchy require stronger justification

The continuous structural-entropy formulation (Equations 5–6) needs a clearer probabilistic interpretation, particularly regarding how products of soft memberships represent expected graph cuts and how the relaxation relates to the original partition-tree objective.

Additionally, the orthogonality regularizer in Equation 8 encourages separation between assignment columns, potentially limiting the overlap that motivates soft affiliations. The paper does not quantify assignment entropy, effective community overlap, or sensitivity to regularization strength. Consequently, it is unclear whether the learned hierarchy exhibits the intended non-exclusive structure and whether that structure contributes to retrieval performance. The boundary-risk diagnostic in Appendix C.2 provides complementary evidence, but does not directly quantify the overlap learned by the affiliations.

3. The retrieval evaluation does not directly establish multi-hop evidence completeness

Gold Hit@3 measures whether at least one gold evidence unit appears among the retrieved passages. For multi-hop questions requiring multiple supporting facts, this metric does not distinguish partial evidence retrieval from retrieval of the complete evidence chain.

Since the motivation for soft affiliations is to capture relationships across multiple semantic contexts, supporting-fact recall and complete-evidence coverage would provide a more direct evaluation of the central claim.

4. Statistical reliability of the smaller retrieval improvements remains unclear

Table 9 and Appendix C.1 report mean and standard deviation over three runs for end-to-end QA accuracy, providing useful evidence of run-to-run stability. However, comparable variation estimates are not reported for the retrieval-level results, where several improvements are small, including differences of 0.1–0.5 percentage points.

Confidence intervals or paired significance tests over queries would help establish whether these smaller retrieval improvements are reliable. Clarifying which sources of randomness vary across the three runs, including whether the hierarchy is retrained with different seeds, would also strengthen the interpretation of the reported variation.

5. Efficiency trade-offs and implementation details need clarification

DhiRAG avoids LLM-generated summaries but incurs additional computational costs relative to its closest summary-free baseline, SeRAG: approximately 22% greater indexing time and 2.3 times the retrieval latency. This accuracy–efficiency trade-off deserves more explicit discussion.

Some implementation details are also insufficiently specified, including the precise hybrid retrieval scoring function, the Gumbel-Softmax temperature configuration, and the role of NER mentioned in the appendix. The embedding-dimension configuration also needs clarification, and the HotpotQA passage counts differ between Table 4 and Appendix A.6.

Questions for authors
  1. Can the authors isolate the benefit of structural-entropy-trained soft affiliations by comparing against simpler community representations under identical retrieval scoring?
  2. What assumptions justify the continuous structural-entropy objective, and how much meaningful community overlap is actually learned?
  3. Does DhiRAG improve retrieval of complete supporting-evidence chains, rather than merely retrieving at least one relevant passage?
  4. Are the smaller retrieval-level improvements statistically reliable across queries and training seeds? Which sources of randomness are varied in the three runs reported in Table 9?
  5. Why does Equation 10 use maximum aggregation over communities? How does it compare with weighted summation or log-sum-exp, which could incorporate contributions from multiple relevant communities?
Reviewer_5uU4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces DhiRAG, a framework that—given an undirected graph constructed via K-NN and cosine similarity—assigns "soft affiliations" shared among chunks/nodes using an SENet, and leverages these to adjust the chunks' scores during the retrieval stage. At the query stage, DhiRAG assigns a score to each chunk based on its association with communities relevant to the query; it also calculates a score for the relationship between each chunk and the query, and these scores are then combined. The chunks are ranked according to this final score, and the top kk are selected.

Critical strengths

S1) The idea of ​​using SENet to assign “soft communities” to chunks is quite interesting.

S2) The mathematical aspects and the general concept of the paper are relatively clear.

Critical weaknesses

W1) There is a clear incompatibility between the operations defined in equations 7 and 8 and the paper’s stated objective of “assigning soft communities” to the chunks. Given how the adjacency matrix at each level is constructed (using graph pooling Ah−1A^{h-1}), the regularization term in the loss function (λ∑∣∣(Ch)TCh−I∣∣\lambda \sum ||(C^h)^TC^{h}−I||), the temperature parameter τ\tau of the Gumbel-Softmax, and the loss coefficient λ\lambda itself, the minimization process effectively forces the columns of (Ch)TCh(C^h)^TC^h to become orthogonal—a condition that only occurs when each row contains exactly one value of 11. I suggest that the authors rigorously investigate the community assignments by varying the values ​​of τ\tau and λ\lambda to verify whether a truly “soft” assignment is actually being achieved. A heatmap visualization would also help clarify these points.

W2) The paper lacks experimental evidence (figures, variations in loss parameters) to support the significance of the (soft) community score’s role in the quality of RAG-based query responses.

W3) There are clear issues with some of the claims in the paper, such as: 1) I do not understand how the authors obtained 768-dimensional embeddings, given that the all-MiniLM-L6-v2 model from SentenceTransformer produces 384-dimensional embeddings. 2) How did the authors arrive at a figure of 9,221 for the HotpotQA dataset in Table 4, while reporting 9,811 in Appendix 6? 3) Why does the caption for Table 1 state that there are 3 datasets, when there are clearly 4 (PopQA, HotpotQA, 2WikiMultiHopQA, and MuSiQue)? 4) What is meant by the term “long-term conversational contexts” mentioned in Appendix 3? Which of the datasets in Table 4 possesses this characteristic? 5) Appendix B and the main text say different things regarding scalability.

W4) Eq. 3 is dimensionally inconsistent (the product involving CkC^k appears to be ill-defined), and the claim that SENet “guarantees” the joint optimization of topological density and semantic similarity does not hold, as the loss function is purely structural, with semantics entering only as a feature for the GNN. Consequently, it is unclear whether the “soft” affiliations analyzed in the ablation studies are, in fact, a result of the described loss function. The observed improvements in metrics seem to stem from other phenomena rather than from any genuinely “learned hierarchy.”

Questions for authors

Q1) Why is k=10k=10 used for the kNN graph?

Q2) Was this "str acc" (string accuracy) calculated in both directions—i.e., checking if the prediction is in the ground truth or if the ground truth is in the prediction?

Q3) Since "str acc" is a more lenient metric than EM (Exact Match) and F1, I would like to see the EM and F1 scores for your method, as well as for LinearRAG and HippoRag2, across two datasets.

Q4) Other papers used as baselines report R@2 and R@5. I would like to see your method evaluated using R@2 and R@5, compared against LinearRAG and HippoRag2, on the Musique and HotpotQA datasets.

Q5) Just to clarify: did the baselines also use the same chunks as those employed in DhiRAG? Why not use the original chunks provided by the benchmarks (such as Musique and HotpotQA)?

Q6) There are some state-of-the-art (SOTA) papers on RAG from NeurIPS 2026 that should be included in the literature review and/or the comparative analysis.

Q7) Could you provide a flowchart illustrating your method, covering stages such as indexing, training, etc.? (An anonymous link is acceptable.)

Q8) The paper "HYPERRECO: Retrieving and Connecting Evidence with Hypergraph Neural Networks for LLM Multi-hop Reasoning" reports results significantly better than those of DhiRAG in terms of EM; the authors should consider this work as well.

Q9) In what scenarios does your method fail? For which types of queries? Are these the same queries that cause failures in other methods? Why do these failures occur—what is the underlying limitation? Is it the LLM, or the data processing pipeline?

Q10) I would like to see some results based on the Arena benchmark datasets.

Q11) I would like to see the proportion of queries answered correctly by DhiRAG compared to the baselines, as well as the proportion of incorrect answers, broken down by the number of hops. For example, if DhiRAG correctly answered 100 3-hop queries, how many of those 100 did LinearRAG and HippoRAG2 also answer correctly? This could be done specifically for the Musique and HotpotQA datasets.

Q12) Please explain how the authors adjusted the relationship between τ\tau and λ\lambda; providing some results based on different variations would help clarify this.

Q13) Could the authors provide an image or table illustrating the final matrix of the learned communities?

Reviewer_5JC4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DhiRAG learns a summary-free hierarchical index for RAG. A graph neural assignment network, trained with a differentiable structural-entropy objective, produces cumulative soft chunk-to-community affiliations. The same affiliations construct community embedding prototypes and transfer query–community relevance back to original chunks, and this structural signal is fused with dense and lexical matching for single-pass ranking. The paper includes baseline comparisons, component ablations, retrieval measurements, three-run statistics, a boundary diagnostic, and timing results. I view the contribution as the shared-affiliation indexing/retrieval design and its empirical utility, not a new structural-information theory.

Critical strengths
  1. A coherent integration that preserves original evidence. Sharing soft affiliations between community prototypes and passage scoring avoids both generated community summaries and irreversible coarse-to-fine filtering. The hard-assignment ablation is a relevant intervention on precisely this coupled mechanism: its changing both prototypes and scores is expected, not a reason to dismiss the control. Removing soft assignment reduces LLM-Acc by 0.7/2.3/1.8 points on PopQA/HotpotQA/MuSiQue; removing message passing or explicit node semantics also reduces all three reported scores (Table 2).

  2. Meaningful and reasonably broad empirical support. The method leads six of eight dataset-level accuracy entries and improves average Str-Acc/LLM-Acc over SeRAG by 2-3 percentage points. Table 9 reports means and small standard deviations over three runs, making results statistically sound. A shared reader, encoder and three-context budget are reported, and Table 7 reports the same aggregate prompt-token count for DhiRAG and SeRAG. These controls, together with retrieval measurements, support the utility of the integration. The paper also acknowledges cases where another method is better, which indicates their integrity on results, i appreciate this.

Critical weaknesses
  1. The marginal value of the hierarchy over the exact local scorer is not isolated. Eq. 11 combines the structural prior with FhybridF_{hybrid}, but the dense/lexical fusion is not fully defined and an otherwise identical gamma=0 control is not reported. Thus, Table 2 supports components within the proposed architecture, while the comparison to Vanilla RAG does not establish the hierarchy's net benefit over that exact local retriever. This limits precise contribution attribution; it does not negate the component evidence or establish that the gain comes only from lexical matching.

  2. Several central implementation choices and retrieval interpretations need clarification. Table 5 does not specify hierarchy widths, assignment temperature/schedule, the orthogonality weight/normalization, the coarsened feature update, or which levels enter Eq. 10. The passage-to-chunk and gold-evidence mapping also needs to be explicit. Gold Hit@3 and First-Gold MRR measure finding a first supporting unit, not recovering the complete evidence needed for a multi-hop answer. The appendix appropriately limits its boundary diagnostic, and its separate overlap experiment should not be mistaken for a direct main-result replication. These gaps restrict the strength of the mechanistic claims more than they undermine the observed system-level improvement.

Questions for authors
  1. Implementation clarification: Please specify FhybridF_hybrid, including lexical/NER weighting and normalization; hierarchy widths, temperature and regularizer settings; coarsened feature pooling; the levels used in Eq. 10; and how the hard-assignment variant is produced. These details would resolve the main reproducibility concern.

  2. Focused attribution check: What happens when gamma=0 with exactly the same local scorer, corpus, reader and evidence budget? This single control would materially sharpen the assessment of the hierarchy's contribution. A same-scorer fixed-hierarchy comparison would be useful additional evidence, but is not a separate prerequisite for my current recommendation.

  3. Evaluation clarification: Please identify the main query subsets, passage/chunk mapping and gold matching, explain the 9,221 HotpotQA passages in Table 4 versus 9,811 in the timing setup, and state what varies across the three runs. If complete-support coverage is available, it would help determine whether the improvement reflects fuller multi-hop evidence rather than only first-hit retrieval.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.