acceptodds
Under review as a conference paper at ICLR 2027

EviPort: From Passage Relevance to Evidence Sufficiency in Budgeted Multi-Hop Retrieval

Abstract

Retrieval-augmented generation (RAG) grounds Large Language Models (LLMs) in external evidence, but multi-hop questions require a complementary set of passages rather than several individually relevant ones. Under a tight context budget, one-shot dense retrieval can repeatedly retrieve evidence for the same readily retrievable subproblem while missing semantically distant bridge or terminal facts. Existing multi-hop approaches improve evidence discovery through iterative retrieval, query decomposition, or structured corpus reasoning, but generally do not explicitly allocate limited context slots across dependent evidence needs. We introduce EviPort, a framework that represents multi-hop questions as dependency-aware evidence ports without task-specific training or corpus graph construction. Passage-derived answers ground downstream ports, enabling each evidence need to trigger an independent full-corpus search before evidence assembly under a fixed reader budget. Across HotpotQA, 2WikiMultiHopQA, and MuSiQue, EviPort consistently improves both evidence retrieval and downstream question answering over strong dense, iterative, and graph-based baselines. These results support framing multi-hop retrieval as budgeted evidence-set construction rather than independent passage ranking.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
retrieval-augmented generation, multi-hop retrieval, evidence retrieval, question answering, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jr97OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces EviPort, a retrieval framework for constructing complementary evidence sets under a fixed reader budget. It decomposes questions into dependency-aware evidence ports, binds downstream queries to answers extracted from retrieved passages, and assembles the final context through round-robin selection. The framework reuses a frozen dense index without task-specific training or corpus graph construction. Experiments on HotpotQA, 2WikiMultiHopQA, and MuSiQue report improvements in evidence retrieval and question answering over dense, iterative, and graph-based baselines.

Critical strengths
  1. The paper addresses a meaningful limitation of passage-level ranking, where individually relevant passages can occupy the context budget without covering all required evidence.
  2. The reported QA improvements are consistent across three benchmarks under a shared reader protocol. The frozen transfer evaluations on HotpotQA and 2WikiMultiHopQA strengthen the empirical results.
  3. The mechanism interventions, alternative selectors, and oracle diagnostics provide useful insight into the sources of improvement and the remaining failures.
Critical weaknesses
  1. The methodological contribution beyond existing question decomposition approaches is not sufficiently established. Evidence ports are subquestions with explicit dependencies, and their execution follows familiar retrieval and intermediate-answer extraction steps. The matched iterative control is useful, but it does not replace a direct comparison with a dependency-aware decomposition method such as Q-DREAM under the same retrieval and reader settings.

  2. The allocation component has limited support as a distinct contribution. With identical plans, bindings, and candidate lists, the simpler port-heads plus dense-fill strategy achieves higher EM and F1 than round-robin in Table 4. Round-robin improves FullEvidence@5 by only 1.10 points, without reported uncertainty estimates. These results support preserving coverage across subquestions, but the additional value of the proposed allocation rule remains unclear.

  3. The fixed execution plan is unreliable on longer reasoning chains. For four-hop questions, 53.01% of plans contain fewer steps than the annotated hop count, and FullEvidence@5 reaches only 7.23%. Appendix C also specifies that unsupported answer strings are propagated into dependent queries without correction. Although EviPort improves over the baselines, the absence of plan revision or recovery leaves a substantial weakness in the dependency mechanism central to the method.

  4. The evaluation leaves important generalization questions unresolved. All corpora contain approximately 6,000 to 12,000 passages, while the main selector and matched iterative comparisons are confined to MuSiQue, which served as the development set. The frozen transfer results are valuable, but repeating these comparisons on held-out data and larger corpora would better establish the robustness of the conclusions.

  5. The accuracy gains involve a substantial increase in online computation. Despite matching search and LLM-call counts, EviPort requires 7.53 seconds per question versus 4.92 seconds for the iterative control in Table 13. A comparison under equal latency or token budgets would clarify whether explicit port planning remains advantageous when the iterative baseline has access to comparable computational resources.

Questions for authors

Please refer to the summary of weaknesses.

Reviewer_ALc1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to retrieve a complete set of supporting evidence for multi-hop RAG under a fixed context budget. It proposes EviPort, which decomposes a complex question into a dependency-aware retrieval plan and executes it using evidence-grounded intermediate answers to instantiate subsequent queries. It then assembles five passages through a round-robin selection strategy across subquestion-specific candidate lists and provides them to the reader model to answer the original question. Based on evaluation on three benchmarks, EviPort demonstrate highest performance among all compared baselines.

Critical strengths
  1. The paper highlights an important distinction: high passage-level retrieval recall does not necessarily imply complete coverage of the evidence needed to answer a multi-hop question.
Critical weaknesses
  1. The proposed method relies heavily on dependency-aware planning and the use of intermediate answers to guide subsequent retrieval, both of which have been explored in prior work[1, 2, 3]. Using directed acyclic graph to represent a plan is a well-established method, so the paper does not clearly establish its technical novelty beyond these existing approaches.

  2. The effectiveness of the proposed budget allocation strategy is not fully established. It is unclear from the experiments why round-robin selection should be preferred over simpler alternatives. It needs more explanations and analysis to support this design choice.

[1] Plan*RAG: Efficient Test-Time Planning for Retrieval Augmented Generation Prakhar Verma, Sukruta Prakash Midigeshi, Gaurav Sinha, Arno Solin, Nagarajan Natarajan, Amit Sharma [2] ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, Dongkuan Xu [3] An LLM Compiler for Parallel Function Calling Sehoon Kim, Suhong Moon, Ryan Tabrizi, Nicholas Lee, Michael W. Mahoney, Kurt Keutzer, Amir Gholami

Questions for authors
  1. How does EviPort differ from prior planning-based retrieval methods, such as Plan*RAG and Q-DREAM? What specific technical contributions does it introduce beyond these approaches, and how does it compare with them under the same experimental conditions?

  2. The paper should include additional baselines, particularly prior planning-based retrieval methods, to better establish EviPort’s effectiveness and contribution.

Reviewer_YNjEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the tendency of standard pointwise dense retrieval to fill limited context budgets with redundant early-hop passages while missing semantically distant downstream evidence in multi-hop question answering, the authors propose EviPort, a training-free and graph-free framework that formulates multi-hop retrieval as budgeted evidence-set construction. The central idea is to explicitly plan dependent information needs and allocate fixed context slots across them. EviPort consists of four sequential components: an LLM-based evidence-demand planner that decomposes a query into an acyclic graph of symbolic sub-questions (ports); a sequential grounded executor that uses an LLM passage judge to extract answers from retrieved texts and bind them to downstream ports; a port-wise full-corpus candidate generator that runs independent dense searches for each grounded port; and a port-balanced budget assembler that uses a round-robin policy to select the final Top-K context. Experiments on 1,000-question subsets of HotpotQA, 2WikiMultiHopQA, and MuSiQue demonstrate that EviPort consistently outperforms dense, iterative, and graph-based baselines in both passage recall and end-to-end QA accuracy, primarily by improving complete evidence-set coverage rather than first-hit discovery.

Critical strengths

S1. The paper provides a clear problem formulation that distinguishes pointwise passage relevance from setwise evidence sufficiency, backed by diagnostic metrics that effectively isolate where retrieval gains occur. S2. The experimental design is rigorous and transparent, maintaining a strict separation between the inspected development dataset and frozen cross-dataset transfer evaluations, while utilizing search-and-call-matched baselines and oracle diagnostics. S3. EviPort achieves strong empirical improvements in both retrieval completeness and downstream QA performance across three benchmarks without requiring expensive corpus-graph construction or task-specific retriever training.

Critical weaknesses

W1. The motivation and discussion of existing works are insufficient. See D1-D2. W2. The proposed budget allocator has limited support as a substantive methodological contribution beyond a simple coverage heuristic. See D3. W3. The strongest controlled analyses rely on the inspected development set, and the reported results leave uncertainty about robustness and longer-hop effectiveness. See D4-D5.

Questions for authors

D1. The Introduction introduces the fixed-budget problem, but leaves it unclear how existing works fail to address this challenge. D2. The experimental baselines do not cover the categories outlined in the first two sections; for example, none of the “question decomposition and adaptive retrieval” methods are included. In particular, the method mentioned in the Introduction as “addressing a related problem” is neither discussed further nor evaluated. D3. In Table 4, the simpler “port heads + dense fill” allocator exceeds round-robin in Recall@5, EM, and F1, while round-robin’s advantage is confined to 1.10 points of FullEvidence@5; this provides limited evidence for the particular allocation rule presented as a central component. D4. The paper reports point estimates without confidence intervals or significance tests, leaving small differences such as the 0.9-point HotpotQA recall gain over HippoRAG 2 and the 1.10-point completeness difference between allocators statistically unresolved. D5. The search-and-call-matched iterative comparison and alternative-allocation experiments are conducted only on the repeatedly inspected MuSiQue development/audit set (Tables 4 and 9), so their conclusions lack frozen cross-dataset validation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.