acceptodds
Under review as a conference paper at ICLR 2027

HKVM: Learning to Select Multi-Hop Evidence with Hypergraph Key-Value Memory

Abstract

Multi-hop retrieval-augmented generation depends on selecting a small set of passages that jointly support an answer within a limited context budget. Relational indexes expose connections across passages, but those connections do not by themselves determine which evidence should be retained. We introduce HKVM (Hypergraph Key-Value Memory), a framework that couples relational indexing with learned evidence selection. HKVM groups locally extracted triples around shared bridge entities to form hypergraph retrieval keys, while retaining the original passages as values. A supervised selector combines structural and text-retrieval signals to rank candidate passages. We evaluate HKVM on the development splits of 2WikiMultiHopQA, MuSiQue, and HotpotQA with fixed candidate pools and an up-to-five-passage selection budget. Compared with a learned controller built on the same ColBERTv2 retrieval source, HKVM improves support-selection F1 (precision-recall F1 over supporting passages) by 1.61, 1.52, and 0.36 points and increases the fraction of questions retaining all supporting passages by 6.54, 3.57, and 1.16 percentage points, respectively. Across six retrieval sources, supervised reranking improves support-selection F1 for every source, while comparisons among learned alternatives vary by dataset and by whether performance is measured by average support or complete-support retention. These results show that relational indexing is most useful as a learned evidence-control layer: its value lies in how structural signals guide passage selection and preserve complete multi-hop support.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Retrieval-Augmented Generation, Multi-Hop Question Answering, Evidence Selection, Hypergraph Learning, Key-Value Memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_2VU1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the multi-hop evidence selection problem: given a pool of candidate passages, select a small set that jointly provide complete support for an answer within a limited context budget. The authors argue that relational indexes expose connections across passages but do not by themselves determine which evidence should be retained. They propose HKVM, which extracts provenance-preserving relation triples from candidate passages, groups them around shared bridge entities to form bridge-group hypergraph keys while keeping original passages as values, performs query-conditioned weighted propagation over the hypergraph to compute structural signals per passage, and trains a supervised passage-level selector that combines structural signals with text retrieval features to rank candidates under a fixed context budget.

Critical strengths
  1. Exemplary experimental methodology with matched comparison designs that separate learning effects from organizational effects, and diagnostic analyses that go far beyond standard ablation studies to reveal mechanism.
  2. The key-value separation means HKVM can work with any base retrieval source, and the supervised selector can integrate any available features.
Critical weaknesses
  1. Hypergraph construction and propagation add complexity, and the paper does not adequately explore whether simpler features fed into the same supervised selector could achieve similar results.
  2. Evaluation uses a fixed budget of at most five passages. In practice context budgets vary, and HKVM’s performance across different budget sizes or adaptive budget strategies is unclear.
  3. Modest absolute improvement magnitude, support-selection F1 gains, while consistent, are numerically small, and it is unclear how much this selection-level improvement translates into final answer quality gains.
Questions for authors
  1. Has HKVM been evaluated in an end-to-end RAG setting where the candidate pool is generated by a first-stage retriever rather than fixed? How much do selection-level improvements translate into final answer accuracy gains?
  2. How does HKVM perform compared to simply adding entity overlap or relation count features to a text passage ranker? Does hypergraph propagation provide unique signals that simple structural features cannot capture?
  3. The set-level comparison found no significant advantage of group-completion features over pairwise interactions. In what scenarios do you expect explicit set-level modeling to yield greater benefits?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 2VU1

We thank the reviewer for the focused questions about simpler features, context budgets and answer quality. They prompted new controlled comparisons and four-reader QA, and helped us reorganize the paper around the information that relational membership contributes to budgeted evidence selection.

1. Comparison with simple structural features (W1, Q2). New Table 2A compares eight conditions in the same 34-slot passage MLP, with identical training rows, initialization and feature positions. Simple8 adds tuple/entity counts, question–entity overlap and coverage, relation-token coverage and cross-passage shared-entity counts to the text features. It improves selection on every dataset. Adding unit-weight endpoint/group readouts further raises all-support retention by 1.68/0.59/3.18 pp and support F1 by 0.44/0.26/0.96 pp on 2Wiki/MuSiQue/HotpotQA. The joint readout exceeds either updated path alone on both metrics in all three datasets (§5.2; Appendix E.1).

These controls show that which passages participate in particular connections carries information beyond aggregate counts and overlap. Table 2B separately examines readout after edge learning: one update improves retention over direct seed readout by 7.41/2.47/3.08 pp. The stronger-text experiment adds an important distinction: unit readouts do not improve a fixed fine-tuned BGE selector (Appendix F.2). Relational information is useful in the common learner, while its additional utility depends on the text model and fusion.

2. Context budget (W2). New Table 4 and Appendix H.1 report precision, recall and all-support retention at budgets 1/3/5/8/10; Figure 9 compares all seven methods at intermediate budgets. Moving from five to eight passages raises native retention from 93.31% to 97.94% on 2Wiki, 34.71% to 44.79% on MuSiQue and 79.96% to 93.48% on HotpotQA, with lower precision. Five provides a compact operating point in this precision–completeness trade-off. This experiment evaluates prefixes of fixed rankings; adaptive stopping and budget-dependent answer quality remain open.

3. Final answers and retrieved candidates (W3, Q1). Table 3 now reports EM/token F1 for five selection policies and four readers on the same 2,048 questions per dataset. HKVM achieves the highest answer F1 with three of four readers on both 2Wiki and MuSiQue, and exceeds learned ColBERT in both metrics in all twelve dataset–reader comparisons. These gains show that the retained evidence improves downstream answers across distinct readers. BM25 leads on HotpotQA. Appendix G includes the full reader results and the stronger single-Qwen fine-tuned BGE reference.

Appendix L additionally evaluates first-stage BM25 top-10/20/50 retrieval, five-passage selection and QA on 512 questions over a bounded structured corpus whose membership derives from an earlier query-conditioned retrieval union. This directly evaluates selection after first-stage retrieval. BGE outperforms HKVM in this retrieved-pool setting, complementing the original-pool results with a stronger text-selector comparison.

4. When set-level modeling may help (Q3). The main selector learns passage scores informed by relational context. Appendix J.2 separately compares group-completion and pairwise heads under a shared joint utility, with matched parameter counts; their F1 intervals span zero. Explicit set modeling is most motivated when individually weak passages become useful together, the budget is tight and the connections contain few distractors. The group-composition analysis (§5.5; Appendix I.2) shows why selectivity matters: real groups contain support-related signal but can also connect irrelevant passages. These observations motivate learning selection-dependent complementarity; the current diagnostic leaves its advantage over pairwise interactions unresolved (§6).

The accompanying Source data package contains complete structural, budget, QA and set-diagnostic results. manuscript_map.csv links each current table/figure to its CSV and relevant filters.

Thank you for the detailed response and additional experiments. The matched simple-feature ablations and downstream QA evaluation partially address my concerns. My remaining concern is the practical significance of the structural stage, that is, fine-tuned BGE remains stronger, the tested readout fusion provides no additional gain, and the bounded retrieved-pool evaluation does not establish an advantage for HKVM. The proposed scenarios for set-level modeling are plausible, but its benefit over pairwise interactions remains unverified.

Reviewer_o7tCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HKVM studies supervised passage-level support selection for multi-hop QA under a limited passage budget. It extracts provenance-preserving relation tuples, groups tuples around shared entities into bridge-group keys, and projects query-conditioned structural activation back to the original passages. A learned scorer combines structural and retrieval features to select an available prefix of at most five passages. Evaluation uses fixed candidate pools on the development splits of 2WikiMultiHopQA, MuSiQue, and HotpotQA. Supervised selection improves all six retrieval sources, while HKVM’s performance relative to learned alternatives depends on the dataset and support metric. Additional diagnostics examine candidate completeness, group contamination, linear propagation equivalence, and a separately trained set-level selector.

Critical strengths
  1. The evaluation distinguishes meaningful failure modes. Reporting support F1, recall, and all-support retention separates average evidence coverage from preservation of a complete annotated support set. Comparing candidate completeness with selected completeness also distinguishes missing evidence from evidence discarded during selection (Sections 4.2 and 5.3–5.4).

  2. The learned-baseline comparison has useful breadth. The study covers six retrieval sources, three benchmarks, and three seeds, with shared candidate pools and extraction artifacts. The positive raw-to-calibrated changes in all 18 source–dataset combinations provide informative evidence that passage-level supervision materially changes the baseline for comparison (Appendix B).

  3. HKVM shows concrete benefits in several comparisons. Relative to the learned ColBERTv2 comparator, support F1 improves by 1.61, 1.52, and 0.36 percentage points across the three datasets. HKVM also achieves the highest reported mean all-support retention on 2Wiki and MuSiQue. These observations support conditional usefulness of the proposed selection pipeline.

  4. The diagnostic interpretation is appropriately candid. The paper reports contaminated groups, unfavorable baseline comparisons, linear propagation equivalence, and an inconclusive group-completion versus pairwise comparison. It clearly distinguishes the gold-assisted group diagnostic and separate set-level study from the main inference procedure (Appendix C).

Critical weaknesses
  1. The central learned pipeline is insufficiently specified. Sections 3.4 and D.2 identify feature families, logistic-regression baseline controllers, and out-of-fold predictions, but do not fully specify the native/final scorer, exact feature construction, training configuration, or fold protocol. The operational distinction between “calibrated” and “learned comparator” is particularly important: Hyper-RAG-HG obtains 38.1484 versus 41.8877 support F1 on MuSiQue in these two conditions (Table 4). The table explicitly distinguishes these conditions, but their implementation differences need explanation to make the comparison families reproducible and interpretable.

  2. The main candidate pools are not fully reconstructable from the manuscript. The paper verifies shared candidate views, and the Discussion identifies the retrieval source supplying the pool. However, the retrieval universe, candidate cutoff, treatment of benchmark-provided contexts, filtering, and candidate-size distribution are not sufficiently specified. These details are central to reproducing the evidence substrate and interpreting the different candidate-visibility ceilings, particularly on MuSiQue.

  3. Attribution to the proposed structural components remains unresolved. Section 3.4 describes adding structural features, and the learned ColBERTv2 comparison is relevant evidence. However, the incomplete pipeline specification prevents confirming that this comparison holds the final scorer, text features, stacking procedure, and training configuration fixed while changing only the structural inputs. The oracle group controls, algebraic propagation check, and separate set-level heads address different questions. Main-protocol component ablations would clarify the contributions of confidence weighting and the endpoint/group projections.

  4. The smallest comparative advantages have limited evidential weight. The authors appropriately present the main results as descriptive and report all three seeds. Nevertheless, HKVM’s 0.1802-percentage-point all-support advantage over Static-HG on 2Wiki changes sign across seeds (Tables 3 and 5). Such near ties provide limited evidence of a stable advantage beyond the evaluated runs. An uncertainty analysis addressing shared-passage dependence and training variation would strengthen their interpretation; the separate set-level confidence intervals do not quantify uncertainty in these main comparisons.

  5. The practical tradeoff of the structural stage remains incompletely characterized. HKVM’s strongest comparative result is complete-support retention on 2Wiki and MuSiQue. On HotpotQA, learned BM25 exceeds it by 1.98 F1 points and 6.61 all-support points. These outcomes support the paper’s conditional interpretation. Although Appendix F.4 gives construction complexity, measured extraction/preprocessing cost and query-time overhead relative to the learned text baselines are not reported. This leaves the practical benefit of the additional structural stage difficult to assess and is a secondary concern about significance.

Questions for authors
  1. Clarification — learning pipeline: What precisely distinguishes the calibrated, learned-comparator, and native HKVM conditions? Please specify their feature maps, learning stages, objectives, regularization, OOF fold construction, and the data used for fitting and model selection.

  2. Clarification — candidate and budget protocol: How are the main candidate pools constructed for each dataset, including the retrieval universe, cutoff, filtering, and use of provided contexts? Please also confirm whether every main condition selects exactly min(5, |C(q)|) passages.

  3. Clarification and evidence request — structural attribution: Is the learned ColBERTv2 comparison an otherwise identical version of the native pipeline with only structural inputs removed? Please document this matching explicitly and report any main-protocol ablations that isolate confidence weighting and the endpoint/group projections.

  4. Evidence request — reliability of near ties: How should the smallest main-table differences be interpreted in light of the reported seed variation? Can an uncertainty analysis accounting for shared passages and training variation distinguish reliable advantages from descriptive near ties?

  5. Clarification — measured overhead: Beyond the construction-complexity discussion in Appendix F.4, what are the measured one-time extraction/preprocessing costs and query-time overhead of HKVM relative to the learned text baselines? These measurements would help assess the practical value of the observed support-selection gains.

2 comments
AuthorsOpenReview ↗

Response to Reviewer o7tC

We thank the reviewer for the detailed assessment of reproducibility, attribution and practical value. Your questions led us to rewrite the learning specification and add two complementary structural controls, measured component costs and downstream QA. The revised main text now connects each experimental comparison to the question it answers.

1. Learning pipeline and development use (W1, Q1). §§3.3–4.3 and Appendix C distinguish three conditions. Calibrated uses a source-specific 20→96→48→1 passage MLP. A learned comparator combines that output with raw ColBERT through a 23-feature logistic controller. Native HKVM instead learns a 20-feature edge MLP, retains/reweights the top four groups, and supplies its passage readout to the same controller family. Appendix C enumerates features, targets, sampling, optimization and stage-specific settings, explaining why calibrated and learned-comparator scores differ.

Both MLP families use 18 epochs, learning rate 0.002 and weight decay 10⁻⁴; the controller uses 80-epoch logistic SGD, learning rate 0.01 and L2 10⁻⁴. Five-fold OOF training predictions fit the controller; full-training source models produce development features. Native and generic fold hashes use different keys, as specified in Appendix C. All datasets use fixed 10,000-query training subsets. §4.1 and Appendix C.2 explicitly disclose MuSiQue development reuse in the earlier parameter screen; fixed-epoch fitting does not remove this research-level reuse.

2. Candidate universe and exact budget (W2, Q2). §4.1 and Appendix A define the benchmark-provided passages U(q), frozen ColBERT candidates C(q) of at most ten, and selection of exactly min(5, |C(q)|) in every main condition. Appendix H.1 gives the complete size distribution, including all 37 HotpotQA pools below five; Appendix I.1 reports visibility ceilings. The native structural source uses the complete query-indexed tuple cache over U(q); the common-learner controls restrict tuples to C(q). Saved candidate IDs and rankings specify the evaluated pools.

3. Structural attribution (W3, Q3). The native–learned-ColBERT comparison changes upstream learning as well as structural inputs. We now distinguish complete-selector comparisons from the new controls. Table 2A holds the 34-slot passage learner, training rows, initialization and other features fixed. Beyond Simple8 counts/overlap/coverage, unit endpoint/group readouts improve all-support retention by 1.68/0.59/3.18 pp on 2Wiki/MuSiQue/HotpotQA. Joint readouts exceed either updated path alone on both F1 and retention in all three datasets.

Table 2B fixes learned edge models, regenerates OOF sources and refits the final controller per path. One update improves retention over direct seed readout by 7.41/2.47/3.08 pp, with positive F1 and retention changes in all nine dataset–seed cells. Confidence and both projection removals are reported in the same table and Appendix E. Removing endpoint projection helps MuSiQue; removing group projection helps 2Wiki/HotpotQA. Their complementary value in separate feature blocks can coexist with different optima after summation in the native chain (§5.2). The common learner identifies information beyond simple features; the reconstructed chain tests its use after edge learning. Appendix J.1 gives the linear operator equivalence.

4. Near ties and practical benefit (W4–W5, Q4). We now identify the 0.18 pp 2Wiki retention difference from Static-HG as a descriptive near tie, consistent with its sign change across seeds (§5.1; Appendix D.3/Figure 5). A dependence-aware uncertainty analysis covering shared passages and training variation has not been performed; the set-level intervals concern a different comparison. Table 1 retains BM25's HotpotQA advantage. New Table 3 adds answer quality across four readers, with HKVM achieving the highest F1 among five policies for three readers on each of 2Wiki and MuSiQue. The revised contribution draws on these answer outcomes and Table 2's controlled contrasts. Stronger fine-tuned BGE results appear in §§5.1/5.3 and Appendices F–G.

5. Measured overhead (W5, Q5). §5.4 and Appendix H separate upfront extraction from cached computation. Reported extraction totals are 601,667 passage/request slots, 542.7M tokens and 591.6 CNY. On the common-learner substrate, cached tuple/group construction averages 0.246/0.282/0.283 ms/query and unit propagation 0.034/0.037/0.037 ms/query. Table 15 also reports text-feature preparation and hot text/joint scorers. These results show lightweight cached structural operations and make the extraction investment explicit; they exclude encoding, retrieval and reader generation. Original extraction wall-clock and itemized billing are unavailable.

The Source data CSVs retain aggregate and seed summaries, coefficients and timings; manuscript_map.csv maps current tables/figures to their data and filters.

Thank you for the detailed clarifications and the additional controls, downstream QA results, and cost measurements, which address several of my original concerns. The new experiments provide positive evidence for structural readouts and propagation; however, the justification for the specific native design remains less clear: its comparison with learned ColBERT changes upstream learning, the common-learner controls use a different tuple substrate, and the native ablations improve performance when the endpoint projection is removed on MuSiQue or the group projection is removed on 2Wiki/HotpotQA. Thus, the demonstrated complementarity of separate feature blocks does not yet clearly justify their additive combination in the native system. The acknowledged absence of dependence-aware uncertainty estimates also limits the weight of the smallest comparative advantages. These remaining concerns about the native design’s incremental benefit lead me to maintain my score, while recognizing the substantive improvements.

Reviewer_TRvzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes HKVM, a hypergraph-based framework for multi-hop evidence selection. It extracts factual triples, constructs hypergraph groups through shared entities, and combines structural relevance with text-retrieval features using a supervised passage selector. Experiments on three multi-hop QA benchmarks show modest improvements over ColBERTv2, but inconsistent advantages over other learned baselines.

Critical strengths
  1. Addresses the important problem of selecting complementary evidence for multi-hop QA under limited context budgets.

  2. Combines relational evidence organization with supervised passage selection while preserving original passages.

  3. Uses fixed candidate pools and multiple learned baselines, with useful analyses of candidate coverage and distractors.

Critical weaknesses
  1. Triple extraction, hypergraph propagation, and supervised reranking are established techniques. The proposed integration offers limited algorithmic innovation.

  2. The propagation is equivalent to incidence-based linear diffusion, and explicit group-level selection fails to significantly outperform pairwise alternatives.

  3. Gains over learned ColBERTv2 are only 0.36–1.61 F1 points, while simpler baselines outperform HKVM on several datasets.

  4. Although motivated by complementary multi-hop evidence, the main selector independently ranks passages without explicitly modeling their joint contributions.

  5. No downstream QA results establish whether improved evidence selection translates into better answer quality.

Questions for authors
  1. Can the authors isolate the contributions of structural features, propagation, and learned reranking through ablations?

  2. Would explicitly optimizing passage-set complementarity improve complete-support retention?

  3. Do the improvements translate into downstream QA accuracy?

  4. Since the method depends on LLM-extracted triples and cannot recover missing supporting passages, sensitivity analyses involving extraction noise, different extractors, and alternative candidate pools would be valuable.

  5. A stronger comparison with entity–passage bipartite graphs, pairwise relational graphs, and their learned variants would help establish whether higher-order grouping provides unique benefits.

2 comments
AuthorsOpenReview ↗

Response to Reviewer TRvz

We thank the reviewer for recognizing the importance of complementary evidence under limited context budgets and for identifying where the original experiments left its contribution unclear. Your questions motivated new structural controls, four-reader QA and input-sensitivity experiments, together with a substantial rewrite of the argument and methods.

1. Contribution and comparative gains (W1, W3). HKVM preserves which facts and original passages belong to a shared-entity connection, learns which connections matter to the query, and reads their activation back to passage addresses for budgeted selection. The central question is what this membership information adds to a learned selector. The revised §§1–3 develop this contribution, and §5.2 directly tests it. Table 1 presents all seven controllers: HKVM has the highest mean all-support retention on 2Wiki and MuSiQue; BM25 leads on HotpotQA. The 2Wiki Static-HG difference is a near tie. Fine-tuned BGE is stronger on all three datasets (Appendix F). Our contribution is a learnable route from relational membership to original-passage selection, with controlled evidence for its information value and downstream answer utility.

2. Structural features, readout and learning (Q1). In new Table 2A, eight conditions share the same 34-slot passage MLP, training rows and initialization. Simple8 contains tuple/entity counts, overlap, coverage and shared-entity summaries. Adding unit endpoint/group readouts raises all-support retention by 1.68/0.59/3.18 pp and F1 by 0.44/0.26/0.96 pp on 2Wiki/MuSiQue/HotpotQA. Joint readouts outperform either updated path alone on both metrics in every dataset. Specific membership therefore supplies information beyond simple structural summaries.

Table 2B fixes learned edge models, regenerates OOF sources and refits the passage controller per path. One readout update exceeds direct seed readout by 7.41/2.47/3.08 pp in all-support retention, with positive F1 and retention changes in all nine dataset–seed comparisons. This comparison includes restart, normalization and refitting. Removing individual projections can help: the common learner weights their feature blocks separately, whereas the native chain sums them before text fusion. This suggests that their relative contribution should be learned during fusion (§5.2; Appendix E).

3. Linear propagation, graph comparisons and set objectives (W2, W4, Q2, Q5). Appendix J.1 explicitly formulates the incidence-based linear operator. Equivalent graph factorizations preserve its memberships and weights; the useful object is the membership information delivered to passage selection. Table 1 compares learned controllers over weighted pairwise PPR and two hypergraph sources. A separately trained, fully matched entity–passage bipartite comparison remains open.

The main controller learns pointwise support scores informed by relational context (§3.4): shared connections can change a passage's score before set selection. Appendix J.2 separately compares equally sized group-completion and pairwise heads under the same joint support/completeness/distractor utility. Their F1 intervals span zero, so this experiment does not establish a group-completion advantage. Tables 20–22 report these heads and fixed-cardinality pointwise/listwise/MMR results. Set modeling is particularly motivated when individually weak passages are jointly necessary under a tight budget and groups contain few distractors. §6 develops these conditions as hypotheses for selection-dependent modeling.

4. Answer quality (W5, Q3). New Table 3 evaluates five selection policies with OLMo-3-7B, Llama3.1-8B, Gemma3-12B and Qwen3.8-27B on the same 2,048 questions per dataset. HKVM achieves the highest F1 with three of four readers on both 2Wiki and MuSiQue, and improves EM/F1 over learned ColBERT in all twelve dataset–reader comparisons. BM25's HotpotQA advantage and the additional single-Qwen BGE results are retained (§5.3; Appendix G).

5. Extraction sensitivity and alternative candidates (Q4). With fitted models fixed, 30% tuple deletion lowers all-support retention by 9.24/2.18/2.33 pp; endpoint replacement lowers it by 13.28/3.31/3.46 pp (§5.5; Appendix I.3). The effects connect selection quality to the relations forming the groups. This tests corruption of one extraction cache; cross-extractor robustness remains open. Appendix L adds BM25-generated top-10/20/50 pools over a bounded, historically query-conditioned structured corpus and separate 512-question QA. BGE remains stronger there; the comparison broadens the candidate setting without claiming full-Wikipedia coverage.

Complete scores and contrasts accompany the revision as task-specific CSVs, indexed by current table/figure in manuscript_map.csv.

Thank you for the detailed response.

My main concern remains the novelty of the overall idea and techniques. Although HKVM aims to preserve which facts and original passages belong to a shared-entity connection and learns which connections matter to the query, similar ideas have been explored in adaptive graph retrieval and selective node selection. Despite some technical refinements, the overall novelty appears limited.

Therefore, I will maintain my current rating.

Reviewer_D4ZtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, author proposes HKVM (Hypergraph Key-Value Memory) for budgeted passage selection to support a particular question. It is a multi-hop RAG framework for decoupling relations from passage text by extracting entities across relation tuples. Across different benchmarks, under a five-passage context budget, the authors propose combining structural signals from hypergraph diffusion with standard retrieval features in a supervised selector. Evaluation on six retrieval sources shows that supervised selection improves every source over its raw ranking, while HKVM’s gains over other learned selectors are mixed.

Critical strengths

-> Metrics employed are realistic where, all matched cells are reported with seed-level values. The group-composition diagnostic is with randomized controls. Also clarifys that the hypergraph readout reduces to a linear operator with HotpotQA losses.

-> The split between candidate-pool visibility and selector decisions is a useful, realistic evaluation. Separating evidence that was never in the pool from evidence the selector dropped is the right way to show failures in multi-hop retrieval.

Critical weaknesses

-> Every experiment re-orders a benchmark supplied pool of around ten passages (candidate recall is already at 100% on 2Wiki and HotpotQA). The relational index is never evaluated on a corpus (that's the setting it's designed for and the only one in which the per-passage LLM extraction cost is actually justifiable).

-> Every learned comparator is a logistic regression and there is no fine-tuned selector. The abstract leads with the weakest text comparator (ColBERTv2) and HKVM is beaten by BM25 on HotpotQA on all metrics.

-> The hyperparameter screen (Appendix F.3) was run on the same MuSiQue dev queries that Table 1 reports. No answer level metric is provided (there was one in an earlier protocol).

-> The salience and bridge potential scalars are never validated. Static-HG performs the same as HKVM on 2Wiki and HotpotQA. Section 5.6 shows the readout is a one step linear operator, and there is no ablation or inspection whatsoever (no example groups, no analysis of the learned coefficients, no per-hop analysis) to show what the hypergraph actually contributes.

Questions for authors

-> See Weaknesses

Also,

-> How you construct the candidate pools?

-> Why are results missing when retrieving over the full corpus, along with top-20 and top-50 retrieved pools, to test for relational indexing?

-> Could you please test the retrieval for cross-dataset transfer w or w/o fine-tuning?

-> Also, a five-passage budget is enough. If yes, please also report precision and cost of an extra passage in the context.

-> Also, report computation cost with different small open-source models.

1 comment
AuthorsOpenReview ↗

Response to Reviewer D4Zt

We thank the reviewer for prompting stronger comparisons and closer investigation of the structural contribution. We added controlled experiments and downstream QA, and reorganized the main text and appendices around their findings.

1. Learned comparators and stronger text selection (W2). §§3.3–4.3 and Appendix C now distinguish passage calibration, native edge learning and the final logistic controller. We added zero-shot and fine-tuned BGE-reranker-v2-m3; fine-tuned BGE leads native HKVM on all three datasets in support F1/retention and the supplementary Qwen QA comparison. Unit-readout fusion adds no gain to fixed BGE (Appendices F–G). The abstract now foregrounds structural controls and four-reader QA.

Table 1 retains learned BM25's HotpotQA advantage. Explicit entities in comparison questions and documented word-matching shortcuts make lexical matching useful in this setting (Yang et al., 2018; Jiang & Bansal, 2019). These findings offer a plausible explanation for its strength in our small distractor pool (§5.5): locating supporting passages and reasoning across them impose different demands. The controls below test the additional information supplied by relations.

2. Structural contribution and inspection (W4). Table 2A fixes the passage learner, rows and feature positions. Beyond eight count/overlap/coverage features, unit endpoint/group readouts raise all-support retention by 1.68/0.59/3.18 percentage points (pp) on 2Wiki/MuSiQue/HotpotQA. Joint readouts exceed either updated path alone in F1 and retention on every dataset, identifying information in specific membership beyond aggregate summaries.

Table 2B fixes learned edge models and refits the controller per readout. One update improves retention over direct seed readout by 7.41/2.47/3.08 pp. Individual path removals can help; separate feature blocks in the common learner versus summed projections in the native chain plausibly explain the different balances (§5.2; Appendix E). Gains hold without salience/bridge weighting; their individual contributions were not separately tested. Appendix J.1 gives the linear operator; Appendix K adds hop strata, nine reconstructed groups and complete coefficients. §5.1 identifies the Static-HG near tie as seed-dependent.

3. Downstream answers (W3). Table 3 adds EM/token F1 for four readers on the same 2,048 questions per dataset. Among five selection policies, HKVM has the highest answer F1 with three of four readers on both 2Wiki and MuSiQue, and exceeds learned ColBERT in EM/F1 in all twelve dataset–reader pairs (§5.3; Appendix G).

4. Candidate construction and retrieved pools (W1, Q1–Q2). The main study asks whether relational membership helps retain complete evidence under a passage budget. Shared candidates separate missing evidence from selection errors; Table 2A further fixes the learner to test membership beyond simple features. §4.1 and Appendix A specify the provided-context universe, frozen ColBERT top-ten candidates and short pools.

Following your suggestion, Appendix L adds BM25-generated top-10/20/50 pools over 60,763 structured passages, with five-passage selection and separate 512-question QA. BGE leads HKVM at top-50. Corpus membership derives from an earlier query-conditioned retrieval union; full-Wikipedia HKVM would require tuple coverage beyond the current cache. This extension tests retrieval-generated candidates, while the controlled comparisons directly support the core selection question. Corpus-wide effectiveness and indexing amortization remain deployment questions.

5. Development-set use (W3). §4.1 and Appendix C.2 disclose MuSiQue development reuse in the earlier parameter screen and retain its original answer-aware results separately from support/QA comparisons. Appendix C specifies fitting schedules and all three selector seeds.

6. Transfer (Q3). Appendix M.3–M.4 reports earlier answer-aware last-stage controller transfer and target calibration. Pooled transfer exceeds RRF on all datasets. Target labels are excluded from the final controller, while upstream calibration can use target training data; full-pipeline transfer remains open. Calibration varies with sample size and composition.

7. Budget and cost (W1, Q4–Q5). Table 4 and Appendix H report precision/recall/retention at budgets 1/3/5/8/10. Five is a compact operating point; larger budgets recover more support. §5.4 separates reported extraction expenditure from cached preparation/scoring. Marginal reader cost per extra passage and smaller-extractor costs remain unmeasured (Appendix H); the four-reader study varies readers.

The accompanying Source data package provides task-specific CSVs; manuscript_map.csv links each current table/figure to its data and filters.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Revisions and Additional Evidence

We thank all four reviewers for their constructive assessments. Your questions prompted a broader experimental investigation and substantial reorganization of the manuscript. The revision strengthens the evidence for our original research question: how relational membership can help select complementary evidence under a finite passage budget. Shared candidates and budgets separate candidate availability from selection errors, allowing the controlled experiments to examine the information that relations add to passage selection.

Structural information beyond simple features. New Table 2A uses a common passage learner to separate relational information from learning capacity. Beyond eight entity/tuple, overlap and coverage summaries, unit endpoint/group readouts improve all-support retention by 1.68/0.59/3.18 percentage points on 2Wiki/MuSiQue/HotpotQA. Joint readouts outperform either updated path alone in both F1 and retention on every dataset. The result identifies the contribution of specific membership in shared connections beyond aggregate structural summaries (§5.2; Appendix E.1).

Readout after group learning. New Table 2B fixes learned edge models and refits the final controller for each readout path. One update improves retention over direct seed readout by 7.41/2.47/3.08 points, with positive F1 and retention changes in all nine dataset–seed comparisons. The full path comparison also reveals that removing an individual projection can help. Together with the common-learner results, this makes the fusion of passage-local attribution and shared group context an important part of the explanation (§5.2; Appendix E.2).

Evidence selection translates into answer quality. New Table 3 evaluates five selection policies with four readers on the same 2,048 questions per dataset. HKVM achieves the highest answer F1 with three of four readers on both 2Wiki and MuSiQue, and improves EM/F1 over learned ColBERT in all twelve dataset–reader comparisons. Learned BM25 leads on HotpotQA; §5.5 discusses its strength in relation to the dataset's documented lexical cues. These results connect the selection study to independent-reader answer quality (§5.3; Appendix G).

Broader comparisons and a clearer method. We added fine-tuned BGE support and single-Qwen QA comparisons, fixed-BGE feature fusion, variable passage budgets, tuple-input sensitivity and retrieval-generated candidate pools. Fine-tuned BGE is stronger than native HKVM; the tested unit-readout fusion adds no gain. These comparisons distinguish the structural information established by the common-learner controls from the performance of the complete selector (Appendices F–L). The retrieved-pool study extends evaluation to first-stage BM25 top-10/20/50 candidates in a bounded, historically query-conditioned corpus, complementing the fixed-pool controls. §§3–4 and Appendices A–C now specify the learning stages, features, OOF construction, candidate scopes and development-set use. §5.4 and Appendix H separate reported extraction expenditure from measured cached computation.

The main text now places structural controls, answer quality and budget results in Tables 2–4; the appendices provide complete comparisons and diagnostics. The accompanying Source data package supplies task-specific numerical summaries, with manuscript_map.csv linking current tables and figures to their CSVs and filters. Our individual replies address each reviewer's questions and point to the relevant revised material.

We are preparing the complete extracted relation data and implementation code for public release. Regardless of the acceptance decision, we will release both in full after the review process concludes and the venue's anonymity requirements permit.

Dear Reviewers,

Please take a moment to read the authors’ rebuttal and participate in the discussion.

Thanks,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.