acceptodds
Under review as a conference paper at ICLR 2027

CHASE: Chain-Guided Hypothetical Supporting-Fact Generation for Graph-Free Multi-Hop Dense Retrieval

34 views

Abstract

Graph-based retrieval-augmented generation (RAG) outperforms conventional dense-retrieval RAG on multi-hop questions by capturing cross-document connections through an explicit graph index, at the cost of offline graph construction and, for some systems, task-specific training. We ask whether the reasoning structure such a graph supplies can instead be elicited from the question itself at query time. We propose CHASE (Chain-guided Hypothetical Supporting-fact gEneration), a training-free framework in which a single LLM call infers the latent reasoning chain of a multi-hop question and emits one hypothetical supporting fact (HSF) per hop, written in the declarative form a supporting passage would take. Each HSF is issued as an independent dense query and the per-anchor results are consolidated by max-score fusion, so the generator produces a chain while the retriever consumes a set and all vector searches run in parallel. Under a matched first-stage protocol that holds the encoder, corpus and index fixed, HSF anchors raise Recall@2 over question-only retrieval by 13.9 points on MuSiQue, 11.1 on 2WikiMultiHopQA and 7.0 on HotpotQA, and anchor-aware reranking adds a further 9.5, 12.2 and 4.0 points over question-only reranking. Against published graph-based and iterative systems CHASE is competitive with GFM-RAG and IRCoT+HippoRAG, although those systems use different encoders, so we report that comparison as context rather than as a controlled result. We also map where the method applies: anchor generation calls for a generator above a capability threshold that, at Recall@2, no open model we tested at 35B parameters or below reaches on two of the three benchmarks, which makes distillation of anchor generation the clearest route to a deployable version.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-hop Question Answering, Retrieval-Augmented Generation, Graph-free Retrieval, Multi-hop Retrieval, Evidence Coverage

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_oZeWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CHASE, a training-free framework for multi-hop dense retrieval that generates hypothetical supporting facts (HSFs) from questions using an LLM. These HSFs serve as independent retrieval queries, whose results are combined through max-score fusion and anchor-aware reranking without requiring explicit graph construction. Experiments on three multi-hop QA benchmarks demonstrate substantial improvements over standard dense retrieval, while additional analyses investigate representation gaps, generation quality, and model capability requirements.

Critical strengths

(1) The paper addresses an important limitation of multi-hop dense retrieval by explicitly modeling hop-specific information needs without requiring graph construction or retriever training. (2) The experimental design includes controlled comparisons under identical retrieval settings, along with extensive ablations that effectively isolate the contributions of hypothetical supporting facts, fusion strategies, and reranking. (3) The analysis of embedding-space representation gaps and generator capability provides useful insights into why the approach works and under which conditions its effectiveness deteriorates.

Critical weaknesses

(1) The effectiveness of CHASE heavily depends on a powerful proprietary LLM for generating high-quality supporting facts. Smaller open-source models frequently fail to outperform question-only retrieval, limiting the approach's practical accessibility. (2) Despite using only one LLM call per question, the method incurs substantial generation overhead, requiring approximately 9K–16K tokens per query. This raises concerns about deployment efficiency and scalability in high-volume retrieval scenarios. (3) The proposed approach builds primarily on existing techniques such as hypothetical document generation, multi-query retrieval, and score fusion. Its methodological novelty is therefore somewhat limited, particularly since simpler interrogative sub-queries outperform HSFs on 2WikiMultiHopQA.

Questions for authors

Can the authors investigate whether shorter prompts or smaller distilled generators can preserve CHASE's retrieval improvements while significantly reducing its inference cost?

2 comments
AuthorsOpenReview ↗

Response to Reviewer oZeW (1/2)

Thank you. You asked whether shorter prompts or smaller generators could preserve CHASE's gains while cutting inference cost. We tested both, and both work, which addresses the deployment objection in your W1 and W2.


Q — Can shorter prompts or smaller/distilled generators preserve the gains?

Shorter prompts: yes, almost free. We varied demonstrations per question type from five to zero, holding generator, encoder, corpus and index fixed. The zero-shot condition keeps only the system prompt, which contains no dataset-specific content and no examples.

DemosMuSiQue R@2 / tokens2Wiki R@2 / tokensHotpotQA R@2 / tokens
5 per type59.81 / 20,16977.28 / 11,72383.10 / 19,316
2 per type58.75 / 10,97576.40 / 6,18983.50 / 9,308
1 per type58.78 / 6,17676.13 / 4,38382.70 / 6,111
0 (none)58.04 / 2,26775.70 / 2,02281.80 / 2,212
Question only44.12 / 063.88 / 071.30 / 0

Going from five demonstrations to zero costs 1.77 / 1.58 / 1.30 Recall@2 while cutting tokens 8.9× / 5.8× / 8.7×, to roughly 2,000–2,300 per question, and still clears question-only retrieval by 13.9 / 11.8 / 10.5 points. The demonstrations buy the last point or two of accuracy; they are not what makes the method work.

Cheaper generators: yes, correcting our W1 claim. A flash-tier hosted model (DeepSeek-v4.1-flash) matches or exceeds our frontier-model configuration on all three benchmarks:

Recall@2, no rerankMuSiQue2WikiHotpotQA
Qwen3.8-Max (submitted main)58.0474.9778.30
DeepSeek-v4.1-flash59.8177.2883.10

The advantage persists after anchor-aware reranking (+0.78 / +0.88 / +1.20).

Combined, this answers your W1 and W2 together. The submitted paper claimed the method "rests on a frontier-scale generator." That claim was wrong, and we have withdrawn it. A flash-tier model prompted zero-shot reaches 58.04 / 75.70 / 81.80 Recall@2, at or above our frontier-model configuration on two of three benchmarks, at roughly one ninth of the token cost. It retains 88–89% of the same generator's five-shot gain, and exceeds the gain of our frontier-model main configuration on MuSiQue and 2Wiki. The open models below 35B fail for want of instruction-following reliability, not parameter count: the task needs a well-formed, entity-instantiated chain, and a cheap hosted model that emits one reliably suffices. We have rewritten §4.7, §4.8 and the conclusion on this basis. Appendix F.

The residual ~2K tokens is also the part most amenable to prefix caching, since the prompt prefix is identical for every question, a saving we do not exploit in the reported numbers.


AuthorsOpenReview ↗

Response to Reviewer oZeW (2/2)

W3 — Methodological novelty, particularly since sub-queries outperform HSFs on 2Wiki

You are right, and the new controls make the point sharper than the submitted version did. We ran two experiments that bear on it.

First, a matched HyDE baseline (same generator, encoder, index, budget), which the submitted paper lacked:

Recall@2MuSiQue2WikiHotpotQA
Question only44.1263.8871.30
HyDE (one global hyp. doc.)49.1262.6372.05
CHASE58.0474.9778.30

HyDE loses 1.25 points on 2Wiki and trails CHASE by 8.92 / 12.34 / 6.25. The separation from the nearest generation-based precedent is therefore large and not an artifact of multi-query retrieval in general.

Second, on your specific observation about 2Wiki: we removed the confound in our sub-query comparison (the old prompt let HSFs guess entities but restricted sub-queries to entities named in the question). With information content held fixed, the gap is +2.09 / −1.35 / +3.00, and under anchor-aware reranking it disappears entirely, with the two forms within 0.60 points on every benchmark.

Rather than defend declarative form, we have restated the contribution. Reranking the sub-query pool against its own anchors gains 10.00 / 12.10 / 3.70 over question-only reranking of the same 80-candidate pool. Anchor-aware reranking is thus a contribution independent of anchor form, the one component whose benefit is uniform across benchmarks, and it now appears as a named contribution in §1 rather than an optional stage. The claim the paper makes is that anchors must be hop-local and instantiate omitted entities, and that candidates should be scored against their best-matching anchor; phrasing is a per-benchmark tuning choice.


The practical answer to your question is that zero-shot prompting with a flash-tier model costs about 2K tokens per question and stays at or above our frontier-model configuration on two of three benchmarks. If this addresses your concerns, we would be grateful if you would consider revisiting your assessment.

Reviewer_J5D7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CHASE, a training-free framework for multi-hop dense retrieval that bridges cross-hop representation gaps without explicit knowledge graphs or iterative retrieval. CHASE uses an LLM to generate hypothetical supporting facts as retrieval queries, combining results through max-score fusion and anchor-aware reranking. Experiments on MuSiQue, 2WikiMultiHopQA, and HotpotQA show improved retrieval recall, with additional analyses of query representations, fusion strategies, and generator capabilities.

Critical strengths
  1. CHASE avoids graph construction and model training, while enabling parallel retrieval instead of sequential search.

  2. The method achieves clear improvements in retrieval recall, especially on MuSiQue and 2WikiMultiHopQA.

  3. The paper includes useful ablations on HSF generation, fusion strategies, and generator capabilities.

Critical weaknesses
  1. Limited novelty. CHASE largely combines existing ideas from HyDE, question decomposition, and multi-query retrieval. The main difference seems to be generating declarative facts instead of questions.

  2. The generated HSFs often already contain the intermediate facts and even the final answer. It is unclear how much retrieval contributes beyond the LLM's existing knowledge.

  3. Lack of explicit evidence coverage. Max-score fusion ranks individual passages but does not ensure that the retrieved evidence covers all reasoning steps or forms a coherent chain.

  4. Heavy reliance on LLM knowledge. CHASE depends on the generator correctly predicting missing entities before retrieval. This may simply shift the core difficulty from retrieval to the LLM.

Questions for authors
  1. Is retrieval really necessary when the LLM already generates the supporting facts and final answer? Does CHASE discover new evidence, or mainly retrieve documents supporting what the LLM already knows?

  2. What is fundamentally new compared with HyDE and question decomposition? Since HSFs do not consistently outperform standard sub-queries, what makes this representation particularly useful?

  3. How does CHASE actually benefit from the reasoning-chain structure? If HSFs are retrieved independently without considering their dependencies, how is this different from standard multi-query retrieval?

  4. Does CHASE solve the cross-hop representation gap or simply shift it to the LLM? If the LLM does not already know the missing entities or facts, how can the method reliably retrieve them?

  5. Does max-score fusion truly address multi-hop retrieval? It can rank passages highly based on a single HSF, but how does it ensure coverage of all required reasoning steps?

2 comments
AuthorsOpenReview ↗

Response to Reviewer J5D7 (1/2)

Thank you. Each of your five questions turned out to be answerable with an experiment rather than an argument, so we ran all five. One of them changed our claim.


Q1 — Is retrieval really necessary when the LLM already generates the facts and the answer?

We tested this directly, by giving the answer generator the anchors instead of the retrieved passages.

EM / F1MuSiQue2WikiHotpotQA
Question + sub-queries only11.4 / 17.9519.1 / 26.2824.9 / 35.15
Question + HSFs only (no retrieval)24.6 / 37.3450.8 / 60.6437.8 / 50.57
Retrieved top-5 (CHASE)36.1 / 47.9959.2 / 68.5959.7 / 72.93
Retrieval − HSF-only+11.5+8.4+21.9

Retrieval adds 11.5 / 8.4 / 21.9 EM over reading the anchors alone. We then measured how often the anchors are actually right (Appendix E): bridge entities are correct 65.1 / 70.7 / 64.6% and answer entities 35.1 / 65.6 / 56.1%.

The MuSiQue pair answers your question most directly: the anchors name the correct answer only 35.1% of the time, yet end-to-end EM is 36.1. CHASE is not retrieving documents that confirm what the generator knows; it is retrieving passages that correct a partly wrong anchor. The anchors are good enough to retrieve with, not good enough to answer from.

Q2 — What is fundamentally new compared with HyDE and question decomposition?

We ran the matched HyDE baseline that was missing (same generator, encoder, corpus, index, budget; one global hypothetical document as the single query):

Recall@2MuSiQue2WikiHotpotQA
Question only44.1263.8871.30
HyDE49.1262.6372.05
CHASE58.0474.9778.30

HyDE loses 1.25 points on 2Wiki, where it is worse than embedding the question, and CHASE leads it by 8.92 / 12.34 / 6.25. The separation from HyDE is therefore large.

On decomposition, you are right that the HSF-vs-sub-query difference does not carry the claim, and we have stopped making it. With information content held fixed (letting sub-queries also instantiate guessed entities) the gap is +2.09 / −1.35 / +3.00, and under anchor-aware reranking it disappears entirely: the two forms land within 0.60 points everywhere.

We have rewritten the paper's claim accordingly. The evidence supports three things: (i) decomposing into hop-local anchors, which separates CHASE from a global hypothetical document by 6–12 points; (ii) instantiating entities the question omits, which the perturbation study shows carries essentially the whole gain; and (iii) anchor-aware reranking, which lifts either anchor form ~10 / 12 / 4 points above question-only reranking of the same pool. Declarative phrasing is a second-order tuning choice, and the revised paper says so.

Q3 — How does CHASE actually benefit from the reasoning-chain structure?

Tested by removing the dependency. We regenerated MuSiQue anchors with m separate calls, the i-th seeing only the question, the total m and its own index. No call sees another's output. The independent arm was additionally given the gold hop count, which the chain-guided arm must infer, so the comparison is conservative against us.

MuSiQueNIndep. R@2Chain R@2Indep. R@5Chain R@5
All1,00058.4358.8772.9275.81
2-hop51870.8570.9580.7983.01
3-hop31651.0052.3770.7375.21
4-hop16633.8433.5352.5154.47

The dependency is worth 0.43 points at R@2 and 2.89 at R@5. By chain length the R@5 gap is 2.22 at two hops, 4.48 at three and 1.96 at four. It is largest where the design predicts, at three hops, since a two-hop question has no bridge entity for a later anchor to inherit, but it is not monotone, and the four-hop subset (N=166) is too small for us to read a trend into. We report this as modest support, not a decisive effect, obtained while the baseline enjoys oracle knowledge of m. Appendix D.

AuthorsOpenReview ↗

Response to Reviewer J5D7 (2/2)

Q4 — Does CHASE solve the cross-hop gap or simply shift it to the LLM?

It shifts it, and we now say so explicitly in §4.5. But the measurements let us say how far the shift goes, which we think is the useful answer:

  • Measured anchor error rates: 34.9 / 29.3 / 35.4% of bridge entities are wrong.
  • The error budget (Appendix J) shows CHASE stays at or above question-only retrieval up to roughly 90% corruption on MuSiQue, 59% on 2Wiki, 41% on HotpotQA.
  • Every measured error rate falls inside its budget, and HotpotQA, with the tightest margin (35.4% against 41%), is also the benchmark with the smallest gain (+7.0).

So the task is relocated from "match a question to heterogeneous evidence" to "guess entities well enough to retrieve", a materially easier task, since the measurements show a third of guesses can be wrong without losing the advantage. Where the generator cannot instantiate plausible entities at all, the budget predicts failure, and we now name that as the limitation (specialist medical or industrial corpora).

Q5 — Does max fusion ensure coverage of all reasoning steps?

A fair objection to average Recall@k, so we added the metric that answers it directly: All-Covered@5, the fraction of questions whose every gold passage is in the top five.

R@5 / AC@5MuSiQue2WikiHotpotQA
Question-only dense56.92 / 25.671.28 / 41.884.95 / 71.4
HSF, max fusion74.33 / 49.689.15 / 77.188.95 / 79.9
HSF, round-robin75.63 / 50.290.18 / 78.189.20 / 80.3
Pool ceiling91.24 / 79.494.45 / 86.897.15 / 94.6

Max fusion nearly doubles whole-chain coverage on MuSiQue (25.6 → 49.6) and 2Wiki (41.8 → 77.1). We also implemented the coverage-aware alternative your question implies: round-robin interleaving of each anchor's list, which enforces per-anchor representation instead of taking the maximum. It is slightly better (+0.6 / +1.0 / +0.4), so your concern is real but small at this budget. The pool ceiling (79.4 / 86.8 / 94.6) shows what reranking still has to work with. New §4.5 and Table 4.


Your questions redirected the paper's claim rather than confirming it. The contribution we now state is the anchor-aware pipeline, namely hop-local decomposition plus entity instantiation plus anchor-aware reranking, not the declarative form we originally emphasized.

If these responses address your concerns, we would be grateful if you would consider revisiting your assessment.

Reviewer_Ywn8OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper points out that previous dense retrieval methods have limitations because questions often lack sufficient information to retrieve relevant evidence. Furthermore, since each supporting passage contains information required for different reasoning steps, semantic discrepancies arise between the question and the relevant passages. The paper defines this mismatch as the cross-hop representation gap and proposes CHASE (Chain-guided Hypothetical Supporting-fact gEneration) to address this problem. Unlike previous methods that decompose a question into multiple sub-queries, CHASE generates hypothetical supporting facts (HSFs) in the form of declarative sentences. Through this process, CHASE aims to identify bridge entities and hop-local evidence needs that are not explicitly mentioned in the original question. Each generated HSF is then used as an independent query for parallel dense retrieval. Furthermore, the paper proposes an anchor-aware cross-encoder reranking method that utilizes both the original question and the generated HSFs to evaluate and rerank the retrieved passages. Experimental results on multi-hop QA benchmarks show that the proposed method improves retrieval performance and end to end question answering performance over previous methods. The results also demonstrate that the generated HSFs are better aligned with gold supporting passages in the embedding space than the original questions.

Critical strengths
  • This paper addresses an important problem in multi-hop retrieval, particularly the cross-hop representation gap between questions and supporting passages.
  • This paper proposes an anchor-aware cross-encoder reranking method that utilizes both the original question and the generated HSFs.
  • The proposed method shows improved results compared to previous graph-based and iterative retrieval methods.
Critical weaknesses
  • CHASE uses an LLM to generate declarative HSFs, but previous works such as HyDE[1] and DiffusionRet[2] have already explored hypothetical document generation for retrieval. Therefore, the technical novelty of the generation-based approach appears limited. Although CHASE generates multiple hop-local HSFs instead of a single hypothetical document, the advantages of this design over previous generation-based approaches could be further clarified.

  • In Table 8, declarative HSFs do not consistently outperform interrogative sub-queries. In particular, sub-queries achieve higher Recall@2 on 2WikiMultiHopQA. Furthermore, there are knowledge-graph-based approaches that construct query-specific subgraphs to improve alignment between questions and graph-based evidence, but the paper does not provide a direct comparison with such approaches[3,4].

  • Although this paper evaluates the proposed method on three multi-hop QA benchmarks, all of them are derived from Wikipedia. Furthermore, the results in Table 4 show that CHASE strongly depends on generator capability. Therefore, it remains unclear how robust the method would be in more specialized domains, such as medical or semiconductor, where the LLM may lack sufficient domain knowledge.

[1] Asai et al., Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection, ICLR, 2024

[2] Qiao et al., DiffusionRet: Diffusion-Enhanced Generative Retriever using Constrained Decoding, Findings of EMNLP 2023

[3] Cai et al., SimGRAG: Leveraging Similar Subgraphs for Knowledge Graphs Driven Retrieval-Augmented Generation, Findings of ACL, 2025

[4] Yu et al., STEM: Structure-Tracing Evidence Mining for Knowledge Graphs-Driven Retrieval-Augmented Generation, ACL, 2026

Questions for authors
  • Could CHASE use BM25 combined with constrained decoding, similar to the approach explored in DiffusionRet[2], instead of embedding-based retrieval?
  • How robust is the proposed method when applied to datasets from domains that the LLM may be unfamiliar with, such as medical or semiconductor domains?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Ywn8 (1/2)

Thank you for the review. You rated the paper sound but judged that it taught you little that was new. We think that is a fair reading of the submitted version, which kept its new findings in appendices and foregrounded a claim about declarative form that our own data does not support. We have run the controls you asked for and restructured the paper around what they show.


W1 — Technical novelty over HyDE and other generation-based retrieval

We agree this needed a direct test rather than an argument, so we ran a matched HyDE baseline: same generator (Qwen3.8-Max), same BGE-M3 index and corpus, same budget, no reranking, one global hypothetical document as the single query.

Recall@2 / R@5, matchedMuSiQue2WikiHotpotQA
Question only44.12 / 56.9263.88 / 71.2871.30 / 84.95
HyDE (one global hyp. doc.)49.12 / 63.9862.63 / 73.6572.05 / 84.70
CHASE (hop-local anchors)58.04 / 74.3374.97 / 89.1578.30 / 88.95

The informative case is 2Wiki, where HyDE is worse than embedding the question (−1.25): a single fluent document blends two endpoints into one vector and matches neither passage cleanly. Hop-local anchors do not have that failure mode, and lead HyDE by 8.92 / 12.34 / 6.25. On comparison-style questions the difference is not incremental; it separates a configuration that helps from one that does not. Table 1 and Appendix B.

We also address Self-RAG (Asai et al., ICLR 2024) and constrained-decoding generative retrieval in the revised related work (§2). On your question about BM25 with constrained decoding in the manner of DiffusionRet: that line changes the retrieval paradigm itself, generating identifiers over a learned index, whereas CHASE deliberately leaves the index and encoder untouched and changes only the query representation. The two are orthogonal rather than competing: CHASE's anchors are plain text and could in principle be issued against a sparse index or a constrained decoder, which we now note as a direction we did not evaluate.

W2 — HSFs do not consistently outperform sub-queries; no comparison with KG subgraph methods

You were right, and we have stopped making the claim. The submitted comparison was also confounded: the prompt let HSFs guess missing entities while restricting sub-queries to entities named in the question, so the two arms differed in information content as well as in form. With that confound removed (both allowed to instantiate guessed entities), the Recall@2 gap is +2.09 / −1.35 / +3.00, and under anchor-aware reranking it disappears entirely: the two forms land within 0.60 points on all three benchmarks.

We have rewritten the abstract, introduction, §4.4 and the conclusion around what the evidence does support: anchors must be hop-local and must instantiate the entities the question omits, and candidates should be scored against their best-matching anchor rather than the global question. That last component lifts either anchor form ~10 / 12 / 4 points above question-only reranking of the same pool, and is now a named contribution rather than an optional stage.

On the knowledge-graph line you raise: SimGRAG (Cai et al., Findings of ACL 2025) builds a query-specific subgraph at retrieval time, which shares CHASE's question-conditioned, ephemeral character while still requiring a constructed graph. We now cite and discuss it in the revised §2. We did not add it as a numerical row because it does not report on the curated 1,000-question subsets of Gutiérrez et al. (2024) that every row of our cross-system block uses, and re-running it under our protocol was not feasible within the discussion period; we prefer to say so than to present a comparison across mismatched protocols.

AuthorsOpenReview ↗

Response to Reviewer Ywn8 (2/2)

W3 — All benchmarks are Wikipedia-derived; dependence on generator capability

On encoder and generator dependence, we have new results that revise our claim.

Replacing BGE-M3 with GTE-large-en-v1.5, re-encoding each corpus and reusing the anchors unchanged:

Recall@2Question-onlyCHASEΔ (GTE)Δ (BGE-M3)
MuSiQue45.2357.48+12.25+13.92
2Wiki60.8878.68+17.80+11.09
HotpotQA66.4580.95+14.50+7.00

The gain is slightly larger under the second encoder, so the cross-hop representation gap is a property of single-vector question encoding rather than of one encoder family.

On generator capability, the submitted claim that the method "rests on a frontier-scale generator" was wrong and has been withdrawn. A flash-tier hosted model matches or exceeds our main configuration under the same prompt (59.81 / 77.28 / 83.10 Recall@2 vs. 58.04 / 74.97 / 78.30), and prompted zero-shot still reaches 58.04 / 75.70 / 81.80 at ~2,000–2,300 tokens per question, about one ninth of what we reported. What the sub-35B open models lack is instruction-following reliability for well-formed chain output, not scale as such.

On specialist domains we have no result, but the paper now supports a prediction. We did not have time to evaluate a medical or industrial corpus, and the revised paper says so. What it provides instead is the quantitative basis for that case. We measured how often the generator's entities are actually correct (65.1 / 70.7 / 64.6% for bridges, Appendix E) and we measure how much entity error the method tolerates before losing its advantage: roughly 90% corruption on MuSiQue, 59% on 2Wiki, 41% on HotpotQA (Appendix J). Every measured error rate falls inside its budget. A domain where the generator cannot instantiate plausible entities at all is where that budget is exceeded, and the revised conclusion names it as the predicted failure mode instead of leaving it open.


On informativeness

The findings we think are worth a reader's time were under-presented in the submitted version, and are now in the main text:

  • A single global hypothetical document can be worse than no expansion at all on comparison-style questions (2Wiki, −1.25).
  • Declarative form, which the paper was named for, is second-order; entity instantiation carries essentially the whole gain, and we show this by refuting our own H3.
  • The method works while ~35% of its bridge entities are wrong, and the error budget quantifies how much error it absorbs.
  • Retrieval adds +11.5 / +8.4 / +21.9 EM over answering from the anchors alone, so the generated facts are good enough to retrieve with but not to answer from.
  • A zero-shot, flash-tier configuration matches our frontier-model result on two of three benchmarks at ~1/9 the cost.

If these responses address your concerns, we would be grateful if you would consider revisiting your assessment.

Reviewer_JuQ7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CHASE generates a sequence of hypothetical declarative supporting facts in one LLM call, independently retrieves passages for these anchors from an unchanged dense index, combines candidates by maximum similarity and optionally applies anchor-aware cross-encoder reranking. Experiments use BGE-M3 and three curated 1,000-question multi-hop QA subsets, with anchor-form, fusion, reranking, generator-scale and entity-perturbation studies. Large gains over question-only retrieval support the retrieval recipe. The principal unresolved contribution question is what chain-guided hypothetical facts add beyond similarly equipped generative expansion and sub-question approaches. The paper explicitly treats published graph/iterative retrieval results as contextual rather than controlled state-of-the-art evidence.

Critical strengths
  1. A practical fixed-index method with sizeable controlled gains. First-stage Recall@2 improves over question-only BGE-M3 by 7-14 points on benchmarks. Reranking the same HSF candidate pool with anchor-max rather than question-only scoring improves Recall@2 by non-marginal gains. This genuine same-pool control demonstrates that the reranking query representation matters, though it involves additional scoring work.

  2. Informative disclosure of consequential limitations. The paper acknowledges that sub-queries outperform HSFs on 2Wiki, all tested open generators fail to beat question-only Recall@2 on two datasets, and full entity corruption removes the advantage. The paper also reports approximately 9,200–15,900 generation tokens per question and does not equate one LLM call with a cheap pipeline. These negative results improve the study's practical value.

Critical weaknesses
  1. The distinctive contribution is not sufficiently isolated against the nearest alternatives. Section 2 identifies HyDE as the closest precedent but does not evaluate a matched global hypothetical-document baseline. More importantly, the included sub-query comparison yields HSF Recall@2 differences of only +2.22/-2.25/+0.30 points; at Recall@10, sub-queries are slightly better on all three datasets (Tables 3 and 8). This is not evidence of statistically established sub-query superiority, but it does not establish a consistent HSF advantage. Appendix J jointly generates paired sub-queries/HSFs, supporting matched anchor counts, yet restricts sub-queries to explicit question entities while allowing HSFs to guess entities. The contrast therefore changes information content as well as declarative form. Sub-queries are not tested with the same anchor-aware reranking. The strong question-only gains establish generated-cue utility more convincingly than the distinctive chain-guided design's incremental value.

  2. Important ablation and evaluation definitions are unresolved. With a fixed m and the same complete score vector per passage, mean equals sum/m and must induce the same ranking; Table 3 reports MuSiQue Recall@2 of 47.99 versus 25.48. A mean over only returned hits can legitimately explain this, since its denominator varies by passage. The missing-score convention must therefore be specified before the fusion mechanism can be interpreted; the discrepancy does not invalidate the HSF or same-pool reranking gains. Similarly, 32 results per adaptive anchor and an 80-candidate reranking cap need explicit unique-candidate and scored-pair accounting, not an assumption that the comparison is unfair. The reported EM values 59.32/59.76 also require run/denominator clarification, since one binary-EM run over exactly 1,000 questions has 0.1-point granularity. Averaging or different retained subsets could explain them. Finally, the five dataset-specific HSF demonstrations underlying the long prompts are not printed in Appendix J, leaving prompt reconstruction and development/evaluation separation unresolved.

Questions for authors

see above weaknesses part

2 comments
AuthorsOpenReview ↗

Response to Reviewer JuQ7 (1/2)

Thank you for a review that identified the controls we were missing and, in two places, correctly diagnosed the cause of an anomaly. We ran every experiment you asked for. One of them changed our claim, and the revised paper states the claim the new evidence supports.


W1a — No matched global hypothetical-document baseline

You were right that this was the missing control. We ran HyDE under the matched protocol: same generator (Qwen3.8-Max, T=0.1), same BGE-M3 index and corpus, same budget, no reranking, one generated document as the single query.

Recall@2 / R@5, matchedMuSiQue2WikiHotpotQA
Question only44.12 / 56.9263.88 / 71.2871.30 / 84.95
HyDE (one global hyp. doc.)49.12 / 63.9862.63 / 73.6572.05 / 84.70
CHASE58.04 / 74.3374.97 / 89.1578.30 / 88.95
CHASE − HyDE+8.92+12.34+6.25

HyDE gains 5.00 on MuSiQue and 0.75 on HotpotQA but loses 1.25 on 2Wiki, where a single fluent document blends two endpoints into one vector and matches neither passage cleanly. Hop-local anchors do not have that failure mode. The ordering holds end-to-end: CHASE leads HyDE by 5.1 / 14.4 / 5.6 EM. Added to Table 1, Table 2 and Appendix B.

W1b — The sub-query comparison confounds form with information content

We accept this point. The prompt restricted sub-queries to entities named explicitly in the question while instructing HSFs to guess missing ones, so the two arms differed in information content as well as in declarative form.

We rewrote the prompt so both fields may instantiate guessed bridge entities, generated the pair in one call, and retrieved with each field separately:

Recall@2 / R@5Sub-query (entity-inst.)HSFΔ R@2
MuSiQue57.35 / 75.5459.44 / 76.23+2.09
2Wiki78.03 / 91.3876.68 / 90.93−1.35
HotpotQA80.05 / 90.8583.05 / 92.65+3.00

With information content held fixed, declarative form is a second-order effect that is signed differently per benchmark. We no longer claim declarative phrasing is what closes the gap. The abstract, introduction, §4.4 and conclusion now state that what matters is that anchors are hop-local and instantiate omitted entities; phrasing is a tuning choice. Appendix C.

W1c — Sub-queries are not tested with the same anchor-aware reranking

Run. Same 80-candidate pool, same cross-encoder, anchors drawn from each arm's own generation:

Recall@2MuSiQue2WikiHotpotQA
Question-only reranking50.2566.4383.05
Anchor-aware, sub-query anchors60.2578.5386.75
Anchor-aware, HSF anchors59.7278.6387.00
vs. question-only reranking+10.00 / +9.47+12.10 / +12.20+3.70 / +3.95

The two anchor forms land within 0.60 points everywhere, and both are ~10 / 12 / 4 points above question-only reranking of the same pool. The implication is that anchor-aware reranking is a contribution separable from anchor form, and the one component whose benefit is uniform across benchmarks. It is now a named contribution in §1 rather than an optional stage. Table 3(c).

AuthorsOpenReview ↗

Response to Reviewer JuQ7 (2/2)

W2a — mean = sum/m should induce the same ranking

Your inferred explanation was correct. Scores exist only over retrieved pairs: s(hᵢ,p) enters fusion only when p ∈ R(hᵢ;k), since a passage outside an anchor's top-k has no computed similarity. Mean averages over the anchors that actually retrieved p, so its denominator varies by passage and mean ≠ sum/m. Sum and CombMNZ treat an absent pair as zero and therefore reward a passage for being retrieved by many anchors. The candidate pool is identical across all rows of Table 3(b); the rows differ in the fusion rule only. Now stated precisely in §3.2.

W2b — 32 per anchor and the 80-candidate cap need explicit accounting

All conditions retrieve top-32 per anchor over the question plus its m anchors, deduplicate, and truncate to the same cap of 80 unique candidates. The cap is identical for the question-only, sub-query and HSF arms, so Table 3(c) is same-pool by construction. First-stage Recall@2 is insensitive to k beyond the fusion cut, since deepening a per-anchor list cannot change the top two fused positions unless the new entries outscore them, which max fusion would already have surfaced.

We also now state the asymmetry explicitly rather than leave it implicit: question-only reranking scores one (query, passage) pair per candidate; anchor-max scores m+1, so a pool of C candidates costs C(m+1) forward passes instead of C. Our claim is that which query a candidate is scored against matters, not that the two configurations cost the same. §3.3 and Appendix A.

W2c — EM values 59.32 / 59.76 require run/denominator clarification

You were right, and your second hypothesis, different retained subsets, was the correct diagnosis. Our QA harness silently dropped questions whose answer-generation call failed: HotpotQA was scored over 999 questions (597/999 = 59.76) and 2Wiki over 998 (592/998 = 59.32). MuSiQue was unaffected. Thank you for catching this.

We re-ran the failed calls and re-scored all datasets over a fixed denominator of 1,000:

CorrectedEMF1
MuSiQue36.147.99
2WikiMultiHopQA59.268.59
HotpotQA59.772.93

These are single runs, not averages. No ranking, comparison or conclusion changes: CHASE remains ahead of all listed baselines on MuSiQue and HotpotQA and remains behind GFM-RAG on 2Wiki, which we continue to report as a boundary of the graph-free approach. All QA numbers in this response use the 1,000-question denominator. Table 2 and Appendix A.

W2d — The five demonstrations are not printed

Now printed verbatim in Appendix Q, together with an explicit statement that they were written from Wikipedia articles and the training splits, and that none is drawn from the 1,000-question evaluation subsets.

We would add that the method no longer depends on them: a zero-shot prompt containing no dataset-specific content at all reaches 58.04 / 75.70 / 81.80 Recall@2, clearing question-only retrieval by 13.9 / 11.8 / 10.5 points, at ~2,000–2,300 tokens per question. Appendix F.


What changed in the paper

Your W1 controls did not confirm our framing; they redirected it. The submitted paper claimed declarative form closes the cross-hop gap. The matched evidence says the gain comes from hop-local decomposition plus entity instantiation, with form a second-order, benchmark-dependent effect, and that anchor-aware reranking is a separable contribution that works for either form. We have rewritten the abstract, introduction, §4.4 and the conclusion accordingly.

If these responses address your concerns, we would be grateful if you would consider revisiting your assessment. We are happy to run further controls in the remaining discussion time.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response to All Reviewers (1/2)

We thank all four reviewers. Every weakness raised has been addressed with a new controlled experiment, a correction, or both. A revised PDF is uploaded; new text is marked in blue.

Three reviewers identified the same gap: the paper did not isolate what chain-guided hypothetical facts add over similarly-equipped alternatives. We ran the missing controls. They changed our claim, and the revised paper states the claim the new evidence supports.


1. What the new controls show

(a) A matched HyDE baseline, the control that was missing (R-JuQ7 W1, R-Ywn8 W1, R-J5D7 Q2, R-oZeW W3). Same generator, encoder, corpus, index and budget; one global hypothetical document as the single query. Recall@2:

Matched, BGE-M3, no rerankMuSiQue2WikiHotpotQA
Question only44.1263.8871.30
HyDE (one global hyp. doc.)49.1262.6372.05
CHASE (hop-local anchors)58.0474.9778.30
CHASE − HyDE+8.92+12.34+6.25

HyDE gains 5.00 on MuSiQue and 0.75 on HotpotQA but loses 1.25 on 2Wiki, where it is worse than embedding the question. Hypothetical generation as such does not produce our result; hop-local decomposition does, separating CHASE from its nearest precedent by 6–12 points.

(b) Declarative form is not the mechanism, and we no longer claim it is (R-JuQ7 W1, R-Ywn8 W2, R-J5D7 Q2, R-oZeW W3). R-JuQ7 correctly observed that our sub-query comparison confounded declarative form with information content: the prompt restricted sub-queries to entities named in the question while letting HSFs guess. With the confound removed (both generated in one call, both allowed to instantiate guesses), the Recall@2 gap narrows to +2.09 / −1.35 / +3.00, and under anchor-aware reranking it vanishes: the two forms land within 0.60 points everywhere.

We have restated the contribution: anchors must be hop-local and must instantiate the entities the question omits; phrasing is a per-benchmark tuning choice. Abstract, §1, §4.4 and §5 rewritten on this basis.

(c) Anchor-aware reranking is a contribution independent of anchor form (R-JuQ7 W1, R-oZeW W3). Reranking the sub-query pool against its own anchors gains 10.00 / 12.10 / 3.70 over question-only reranking of the same 80-candidate pool, within 0.60 of the HSF configuration everywhere. This is the one component whose benefit is uniform across benchmarks, and it now appears as a named contribution rather than an optional stage.

(d) Retrieval does real work; it is not confirming what the generator knows (R-J5D7 Q1, Q4). Giving the answer generator the anchors instead of retrieved passages:

EMMuSiQue2WikiHotpotQA
Question + sub-queries11.419.124.9
Question + HSFs (no retrieval)24.650.837.8
Retrieved top-5 (CHASE)36.159.259.7

Retrieval adds +11.5 / +8.4 / +21.9 EM over the anchors alone. Measured anchor entity accuracy: bridges 65.1 / 70.7 / 64.6%, answer entities 35.1 / 65.6 / 56.1%. On MuSiQue the anchors name the right answer only 35.1% of the time while EM is 36.1. They are good enough to retrieve with, not to answer from.

(e) The frontier-generator requirement was overstated, and we have withdrawn it (R-oZeW W1/W2, R-Ywn8 W3, R-JuQ7). A flash-tier hosted model (DeepSeek-v4.1-flash) matches or exceeds our main configuration under the same prompt (59.81 / 77.28 / 83.10 Recall@2 vs. 58.04 / 74.97 / 78.30). Separately, dropping from five demonstrations to zero costs only 1.77 / 1.58 / 1.30 points, bringing cost to ~2,000–2,300 tokens per question, roughly one ninth of the 9.2K–15.9K we reported, while still reaching 58.04 / 75.70 / 81.80, at or above our main configuration on two of three benchmarks. The zero-shot configuration retains 88–89% of the same generator's five-shot gain, and exceeds our frontier-model main configuration's gain on MuSiQue and 2Wiki.

(f) Chain structure contributes, modestly (R-J5D7 Q3). Generating each anchor in an isolated call, while giving that baseline the gold hop count that CHASE does not receive, costs 0.43 Recall@2 and 2.89 Recall@5 on MuSiQue, with the largest gap at three hops (4.48). We report this as modest support, not a decisive effect.

(g) Chain coverage, not just average recall (R-J5D7 Q5). All-Covered@5 (every gold passage in the top five) rises from 25.6 → 49.6 on MuSiQue and 41.8 → 77.1 on 2Wiki. A round-robin rule that enforces per-anchor representation is slightly better still (+0.6 / +1.0 / +0.4), so the concern that max fusion lets one anchor dominate is real but small at this budget.

(h) The gains are not specific to one encoder (R-Ywn8 W3). Under GTE-large-en-v1.5 with anchors unchanged, Recall@2 improves by +12.25 / +17.80 / +14.50, slightly more than under BGE-M3.

AuthorsOpenReview ↗

Global Response to All Reviewers (2/2)

2. Corrections we are making on the record

EM granularity (R-JuQ7 W2). The reviewer was right, and their second hypothesis, different retained subsets, was the correct diagnosis. Our QA harness silently dropped questions whose generation call failed, scoring HotpotQA over 999 (597/999 = 59.76) and 2Wiki over 998 (592/998 = 59.32). We re-ran the failed calls and re-scored everything over a fixed denominator of 1,000: HotpotQA 59.7 / 72.93, 2Wiki 59.2 / 68.59 (MuSiQue unaffected at 36.1 / 47.99). No ordering or conclusion changes. All QA numbers in this response use the full 1,000-question denominator.

Fusion missing-score convention (R-JuQ7 W2). The reviewer's inferred explanation was also correct. Scores exist only over retrieved pairs. Mean averages over the anchors that actually retrieved a passage, so its denominator varies by passage and mean ≠ sum/m; sum and CombMNZ treat an absent pair as zero. The candidate pool is identical across those rows. Now stated precisely in §3.2.

Budget accounting (R-JuQ7 W2). All conditions retrieve top-32 per anchor and truncate to the same cap of 80 unique candidates before reranking, so the reranking rows are same-pool by construction. We also now disclose the asymmetry we were asked about: question-only reranking scores one pair per candidate, anchor-max scores m+1. The claim is that which query a candidate is scored against matters, not that the two cost the same. Details in Appendix A.

Prompts and development/evaluation separation (R-JuQ7 W2). All demonstrations are now printed verbatim in Appendix Q, with an explicit statement that they come from the training splits and that none appears in the 1,000-question evaluation subsets. We note that the zero-shot configuration in (e) contains no demonstrations at all and still clears question-only retrieval by 13.9 / 11.8 / 10.5 points.


3. Where the revised paper stands

The contribution we now claim is narrower and better supported than the submitted one:

  1. A diagnosis measured directly: the cross-hop representation gap, positive for 90.9% of examples in embedding space, widening exactly where question-only retrieval is weakest.
  2. A plan-once, retrieve-in-parallel framework with anchor-aware reranking, where hop-local decomposition beats a global hypothetical document by 6–12 points, and anchor-aware scoring lifts both anchor forms ~10/12/4 points above question-only reranking.
  3. A stated operating envelope: form is second-order; entity instantiation carries the gain; the error budget predicts where the method fails; a cheap zero-shot generator suffices.

The revised claim is narrower than the submitted one and follows what the controls show. We are glad to run further controls in the remaining discussion time.

Location of changes: Abstract; §1 (contributions); §2 (related work); §3.2 (fusion convention), §3.3 (reranking cost); Table 1 (HyDE row); Table 2 (corrected EM, HyDE row); §4.4 + Table 3 (blocks a′, a″, new rerank row); §4.5 + Table 4 (coverage, necessity); §4.7 (flash-tier); §4.8 (cost); §5 (conclusion); Appendices A–G (new).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.