acceptodds
Withdrawn from ICLR 2027

OriginRAG: Harnessing GraphRAG by Tracing Evidence Origins

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

GraphRAG gathers supporting evidence for multi-hop questions from corpus graphs. However, prior methods do not explicitly expose where retrieved evidence originates and how retrieval decisions connect to source grounding. We propose OriginRAG, a novel framework that harnesses GraphRAG by maintaining a set of demand-specific provenance lineages as its retrieval state. It derives evidence demands from the query and initializes one provenance lineage per demand. Each lineage is expanded by demand-conditioned graph retrieval, so every candidate preserves its origin: the demand that triggered its retrieval. Such origin tracing helps an agent choose complementary passages while completing a lineage. Once a demand is satisfied, it is grounded in a verbatim span from the selected evidence. On representative multi-hop QA benchmarks, OriginRAG surpasses leading GraphRAG baselines in R@5, EM, and F1, with an average improvement of 4.90 points.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Retrieval-Augmented Generation, Graph Retrieval-Augmented Generation, Evidence Lineage, Multi-hop Question Answer.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_x6r4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OriginRAG induces evidence demands from a multi-hop question and runs graph retrieval separately for each demand. A passage keeps the demand IDs and ranks by which it was retrieved. An LLM selector uses these tags to choose five complementary passages, after which another call extracts demand-linked verbatim spans for the QA reader. The evaluation uses 1,000 questions each from HotpotQA, 2WikiMultiHopQA, and MuSiQue, reporting retrieval and answer metrics, component ablations, provenance perturbations, a sampled span audit, and token usage.

Critical strengths
  1. The use of per-demand retrieval tags is a sensible way to preserve information about why each passage was retrieved. Figure 1 gives a clear example where whole-question ranking misses a second-hop passage, while demand-specific retrieval can recover it.

  2. Table 5 provides a useful controlled test of the provenance information. With the candidate pool fixed, R@5 is 89.65 with the correct provenance, 86.15 without provenance, 86.05 with equal-length neutral text, and 80.45 with shuffled provenance. This suggests that the selector benefits from the actual demand–passage association rather than simply from receiving a longer prompt.

Critical weaknesses
  1. I could not clearly isolate the effect of lineage completion from the reported experiments. In Table 2, “w/o Lineage Completion” averages 88.15 R@5 / 54.60 EM / 66.48 F1, which matches the results in Table 3, while its EM/F1 also match the ungrounded reader in Table 7. Section 5.3 appears to say that removing lineage completion can change the retrieved passages, whereas Section 5.4 describes Table 7 as a fixed-passage comparison. However, Table 7 does not report passage IDs or R@5. If both Table 7 rows use the passages selected by the full OriginRAG system, I would expect them to share the full system's 89.65 R@5. The current presentation makes it difficult to tell whether the reported +4.47 EM / +4.30 F1 improvement comes from the extracted spans themselves or partly from changes in the selected evidence.

  2. I also think the paper overstates what is guaranteed by the lineage validation step. Equation (9) checks containment, schema consistency, and retrieval provenance, but it does not verify that the extracted span actually supports the associated demand. This distinction matters because the manual audit in Table 6 finds full support for 420 of 500 spans, while 50 are only partially supportive and 30 are unsupported or linked to the wrong demand.

  3. The headline improvements should also be interpreted more carefully. The abstract reports gains against the average of four baselines, while the gap to the strongest baseline, G-reasoner, is much smaller: 1.12 R@5, 3.57 EM, and 2.91 F1 on average. No paired uncertainty is reported for these differences. Taken together, I think the evidence supports the usefulness of provenance-aware retrieval, but is less conclusive about the semantic grounding provided by the extracted spans and the overall size of the improvement.

Questions for authors
  1. Could you clarify exactly which passage IDs are given to the reader in the relevant rows of Tables 2, 3, and 7? Please also report the R@5 for both rows in Table 7. Most importantly, what is the QA difference when the selected passage set is fixed to the full OriginRAG output and only the presence or absence of demand-linked spans is changed?

  2. Could you provide per-query supporting-passage hits and answer predictions for the Table 2 ablations, together with paired uncertainty estimates? I would also appreciate an explanation for why the reported R@5 and F1 drops are identical across the three datasets.

  3. On exactly the same selected passage sets, how does the proposed demand-linked extraction compare with a generic span extractor that does not receive demand IDs? It would also be useful to report a human audit focused specifically on whether the retained spans semantically support their associated demands.

Reviewer_m9nHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes OriginRAG, a GraphRAG pipeline for multi-hop question answering built on HippoRAG 2. An LLM first decomposes the question into several "evidence demands", each describing a fact that a passage must establish. The system retrieves passages separately for each demand and records which demand retrieved each candidate. An LLM then uses these records to select a complementary set of passages, and it extracts a verbatim supporting span for each satisfied demand before the answer is generated. On three multi-hop QA benchmarks, the method reports better retrieval and answer accuracy than several graph-based RAG baselines.

Critical strengths
  1. The evaluation uses three standard multi-hop QA benchmarks under an established protocol. It reports both retrieval and answer metrics, and it includes a HippoRAG 2 baseline that shares the same graph index and retrieval settings.
  2. The provenance controls in Table 5 (removing the provenance, replacing it with neutral text of equal length, and shuffling it across passages) are a thoughtful way to test whether the correct demand–passage correspondence matters, rather than just the extra prompt text.
  3. The paper reports the token cost of the additional query-time steps, and it audits whether the extracted spans actually support their linked demands, using two independent judges.
Critical weaknesses
  1. The central notion of "origin" could be made clearer, and its novelty seems limited. The abstract first states that prior methods do not explicitly expose where retrieved evidence originates, which may suggest source documents; many RAG systems already keep these through document identifiers. Only later does it define origin as the demand that triggered a passage's retrieval. Under this meaning, the key mechanism amounts to recording which sub-query retrieved each candidate, and passing this record to the selection and grounding steps. This is closer to careful bookkeeping than to a new retrieval principle, and the paper would benefit from explaining what is conceptually new beyond this.
  2. Apart from this record, the pipeline consists of components that are already common in multi-hop RAG: decomposing a question into sub-questions, retrieving for each sub-question, selecting a passage set with an LLM, and extracting supporting spans for the reader. The controls in Tables 3 and 4 show an empirical benefit over decomposition and set-selection baselines, but they do not by themselves clarify what is conceptually new. Moreover, apart from running the HippoRAG 2 retriever once per demand, the steps the paper adds, such as demand induction, set selection and span grounding, do not use the graph structure. It is therefore unclear what is specific to GraphRAG, and the same pipeline could be placed on top of almost any retriever.
  3. I had difficulty reconciling some of the numbers in the analysis. In Table 2, each of the three ablations lowers R@5 by exactly the same amount on all three datasets (2.50, 5.00 and 1.50 points, respectively, as also stated in the text), which is unusual for datasets that differ this much. In addition, Table 3 reports an average R@5 of 88.15 for evidence-set construction, and its EM and F1 equal those of the "Selected passages" condition in Table 7, which according to the paper differs from the full model only in whether grounded spans are added. The full model, however, reaches an average R@5 of 89.65 (Tables 1 and 5). Since source grounding is applied only after the passages are selected, I would expect the two R@5 values to be the same. Because these tables support the paper's analysis of why the method works, clarification would be important.
  4. Some key details are missing. Section 4.1 describes the induction instructions as summarized, and Appendix A notes that repeated input wrappers are omitted, although Section 5.1 states that full prompts appear in Appendices A and B. The paper itself also does not show how the question and candidate information are formatted in the actual LLM calls. The quality of demand induction is not evaluated on its own, for example by checking whether questions are decomposed correctly, or how the results vary with the number of hops.
Questions for authors
  1. Could the authors explain why each ablation in Table 2 lowers R@5 by the same amount on all three datasets, and why the R@5 of evidence-set construction in Table 3 (88.15) differs from that of the full model (89.65)?
  2. Beyond the empirical comparisons in Tables 3 and 4, could the authors clarify what is conceptually new about recording the demand that retrieved each passage, compared with existing pipelines that decompose a question and retrieve for each sub-question, and which parts of the method rely specifically on the graph structure?
  3. How accurate is the demand induction on its own, and how does performance vary with the number of hops, for example on 2-, 3- and 4-hop MuSiQue questions?
Reviewer_aVdkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes OriginRAG, a GraphRAG pipeline with three stages. First, an LLM decomposes the query into “Evidence Demands.” Second, HippoRAG 2-style PPR retrieval runs separately for each demand, and each candidate passage keeps the IDs and ranks of the demands that retrieved it. Third, an LLM selects k = 5 passages using this provenance and extracts verbatim spans for each demand, which are given to the reader together with the passages. The paper evaluates on 1,000 queries each from HotpotQA, 2WikiMultiHopQA and MuSiQue, and reports a macro R@5/EM/F1 of 89.65/59.07/70.78.

Critical strengths

The paper includes controls for the provenance signal (removed, neutral replacement and shuffled; Table 5), and a fixed-passage comparison that isolates the effect of grounded spans on the reader (Table 7).

Critical weaknesses
  1. Limited novelty: Evidence-Demand Induction (Section 4.1) is essentially query decomposition, which has long been studied. The paper itself cites several such methods, including question decomposition (Ammann et al., 2025), LogicRAG and DeCoR. The remaining additions are incremental: attaching demand IDs and ranks to candidates in the selector prompt, and extracting verbatim spans.

  2. The paper also omits RSF-GLLM (Bandyopadhyay & Muppidi, ICML 2026), which addresses a closely related problem of retrieving bridge nodes in multi-hop reasoning and grounding generation in explicit reasoning paths. It should be cited and discussed.

  3. Weak baseline comparison. The headline gains in the abstract (5.59 points) and in Section 5.2 (3.59/6.81/6.37) are measured against the mean of four GraphRAG baselines. The contributions list (lines 86–88) nevertheless describes them as gains “over the leading GraphRAG baselines.” Against the strongest baseline, G-reasoner, the macro gains are only 1.12 R@5, 3.57 EM and 2.91 F1.

There are further comparability problems. ETS-7B and Relink are copied from their papers with different readers, so they are not comparable. The QD and QD + SETR comparisons (Table 4) are reported on MuSiQue only.

  1. Writing and notation. The graph index G\mathcal{G} is not defined in Section 3 (line 140); its construction appears only in Section 5.1 and Table 10. The notation in Eqs. 2–3 is confusing, and it is unclear how δj\delta_j is obtained.

  2. Redundancy and diversity. Retrieval runs independently for each demand, and PPR does not promote diversity. Only exact duplicates and same-title passages are removed, not semantically repeated ones. Pruning 24 merged candidates down to 5 can then discard useful evidence; Table 4 reports a selection loss of 8.72 R@5.

  3. Cost analysis. Table 8 reports only token counts, which exclude the QA reader, and gives no running time.

Questions for authors
  1. What exactly is the graph index G\mathcal{G}? How is δj\delta_j in Eq. 3 obtained: is it an input or a derived signal?
  2. How do you handle semantic repetition among passages retrieved independently for different demands?
  3. How does OriginRAG differ from existing query-decomposition methods and from RSF-GLLM?
  4. Can you report gains over the strongest baseline rather than the mean, and run the QD and SETR comparisons on all three datasets? Can you report wall-clock running time alongside token counts in Table 8?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.