acceptodds
Under review as a conference paper at ICLR 2027

BindRAG: Binding-Aware Retrieval For Multi-Hop Question Answering

Abstract

Large language models (LLMs) often struggle with knowledge-intensive questions that require retrieving and combining evidence across multiple reasoning steps, often from different documents. Retrieval-augmented generation (RAG) mitigates this limitation by providing external evidence, but conventional retrieval is poorly suited to multi-hop questions where evidence needed at a later step depends on an intermediate answer discovered earlier. Decomposition and iterative retrieval methods address this by generating follow-up queries step by step, but they typically leave the dependency on intermediate answers implicit or commit to a single intermediate answer, allowing early errors to propagate to later retrieval. We present BindRAG, a multi-hop retrieval framework that explicitly binds intermediate answers to downstream retrieval queries, making sequential dependencies directly executable. BindRAG further maintains a query-aware memory that preserves alternative bindings and hop-structured evidence while retrieval proceeds. The highest-ranked binding advances the primary retrieval path, while alternative candidates and their supporting evidence are retained for final resolution. This allows uncertain intermediate decisions to be reconsidered without repeatedly changing the retrieval trajectory. Across HotpotQA, MuSiQue, and 2WikiMultiHopQA, BindRAG improves answer accuracy by an average of 10.3 percentage points over the strongest baseline.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Retrieval-Augmented Generation (RAG), Multi-Hop Question Answering, Query Decomposition, Explicit Binding, Multi-Hop Retrieval, LLM Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uunUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work presents BindRAG, a RAG framework for multi-hop question answering. The method first prompts an LLM for a decomposed retrieval plan, executes the plan by binding and storing retrieved entities from previous hops/steps and leverages them for subsequent hops dependent on said entity. Throughout, the method maintains alternative options, if produced, and completes a final full resolution step considering them before producing an answer. The authors formalize the problem at hand and present experimental results where BindRAG is evaluated on three 1k question subsets of common multi-hop QA datasets and compared against a set of graph-based RAG methods. The work reports that the presented method beats out every baseline in every single setting.

Critical strengths

The presentation of the problem with running examples and the formalization are good. However, it is evident that many parts of the submitted work are strongly LLM-edited.

Critical weaknesses

I’m mainly concerned with the novelty of the method. Analyzing the main contributions:

(1) Binding-aware execution has been proposed in ReWOO (Xu et al., 2023) and LLMCompiler (Kim et al., 2024) which are both correctly cited, but also other un-cited works like Break It Down (Wolfson et al., 2020) present the idea of re-inserting previous answers in their BREAK method, and more recently pi-CoT (Gong et al., 2026) leverages prolog for binding answers.

(2) Uncertainty-aware execution has been targeted by Multi-Chain Reasoning (Yoran et al., 2024) and Beam Retrieval (Zhang et al., 2024), which are both not cited.

(3) While the works’ experimental results may be a valid, they don’t compare to the most competitive and most highly related methods.

Additionally, it is hard to verify the correctness of the evaluation results when no code artifacts are submitted. Also, the perfect trend in Table 1 of Str-Acc < LLM-Acc for HotpotQA and MusiQue, whereas the inverse holds perfectly for 2Wiki, with no deviations whatsoever raises questions.

Questions for authors
  1. Why was temperature 0 chosen? This does not guarantee determinism and also does not reflect how a regular user would use the language model with default, model-provider suggested settings.
  2. Why is the LightRAG reference listed with author Tian Ao, instead of Tu Ao? I was unable to find an online reference where the author is listed as Tian instead of Tu.
  3. Can you explain the choice of models used? It is not clear to me why Qwen2.5 was chosen over 3.5 or 3.6 and why gpt-4o-mini was leveraged as the judge.
  4. How do you explain the perfect scores in Table 1 (please see weaknesses)?
4 comments

[Weaknesses 1–3: Novelty and relationship to prior work]

Thank you for highlighting these closely related methods. We agree that variable substitution, dependency-aware execution, and uncertainty handling are not individually novel. We have therefore clarified our contribution and expanded the related-work discussion to include Break It Down, π-CoT, Multi-Chain Reasoning (MCR), and Beam Retrieval.

Our key distinction lies in how intermediate dependencies are represented and used during final answer synthesis. ReWOO preserves execution plans and tool observations, while IRCoT accumulates reasoning steps and retrieved evidence. BindRAG instead maintains an explicit typed dependency program, resolved answer bindings, and hop-wise supporting evidence as structured inputs to the final reader. Thus, intermediate answers are not only substituted into downstream retrieval queries but also retained with their dependency relationships for final synthesis.

Our controlled experiments help identify the contribution of this design. Removing binding propagation reduces MuSiQue accuracy by 17.0 points, while removing query-aware memory reduces it by 4.4 points with the answer prompt fixed. Moreover, removing the program and resolved bindings from the final reader reduces accuracy by 4.8 points with retrieved evidence held constant. These results support the importance of explicitly preserving and using dependency information beyond retrieval.

We also added direct comparisons with ReWOO, IRCoT, SeqProp, and adapted MCR. BindRAG outperforms ReWOO by 25.0 points with each method's own reader and 17.7 points with a shared flat reader. However, BindRAG and SeqProp are statistically tied under the shared reader, indicating that the additional end-to-end benefit comes primarily from binding-aware final synthesis rather than a fundamentally different substitution mechanism. Against IRCoT, BindRAG achieves comparable flat-reader accuracy with substantially fewer tokens.

Method (MuSiQue)BindRAG − method, own readerBindRAG − method, shared flat readerTokens per question
SeqProp+7.6+1.3 [−0.5, +3.2]5.0k
IRCoT+9.2+1.5 [−1.2, +4.2]29.7k
ReWOO+25.0+17.7 [+14.6, +20.8]3.2k
MCR (adapted)+10.1−2.7 [−5.5, +0.0]67.4k

BindRAG uses 5.8k tokens per question. Str-Acc differences with 95% CIs (Table 2).

We do not include three of the methods you mention in our comparison. LLMCompiler's main advantage is parallel execution, whereas most MuSiQue (86.9% chain) and HotpotQA (81.1% bridge) questions are sequential; ReWOO represents this plan-and-execute family. Beam Retrieval requires supervised retriever training, unlike the training-free methods compared here. For π-CoT, we found no public code (its latest arXiv version gives the prompts in an appendix), so we discuss the differences in the related-work section without an experimental comparison.

Finally, our ablations show that intermediate candidate enumeration does not provide a statistically significant accuracy gain in isolation. We therefore position uncertainty handling as a supporting mechanism rather than an independent novelty claim.

In summary, our contribution is not the invention of variable binding or evidence accumulation, but a structured retrieval-and-synthesis framework that preserves explicit answer dependencies from execution through final answer generation. We have revised the manuscript to make this distinction and its empirical support clearer.

[Weakness: Code availability]

We have uploaded an anonymized supplementary package containing the implementation, prompts, configurations, question IDs, predictions, evaluation scripts, and documented baseline adaptations. This is intended to make the experimental setup and reported comparisons reproducible.

[Weakness: Writing issues and limitations]

Thank you for pointing out the writing and presentation issues. We have corrected the identified errors and clarified the scope of our conclusions. In particular, we now explicitly acknowledge settings where BindRAG does not outperform the baselines, including certain 2WikiMultiHopQA question types and shared-reader comparisons.

[Question 1: Why temperature 0?]

We followed DIGIMON's greedy-decoding configuration to keep generation settings consistent across methods and reduce sampling variance. We agree that temperature 0 does not guarantee determinism or represent typical provider defaults; our objective was a controlled comparison rather than a deployment-oriented evaluation.

[Question 2: LightRAG author name]

Thank you for catching this error. The correct author name is Tu Ao, and we have corrected the reference.

[Question 3: Choice of backbone and evaluation judge]

We selected Qwen2.5-32B-Instruct because it is a publicly available backbone supported by DIGIMON, enabling consistent comparisons with the graph-based baselines while balancing model capability and computational cost. We additionally evaluated GPT-4o-mini and Llama-3.1-8B-Instruct to examine backbone dependence (Table B.1).

We used gpt-4o-mini as the LLM judge for consistency with DIGIMON and LogicRAG. We acknowledge potential self-evaluation bias when GPT-4o-mini is also the generator; therefore, we report Str-Acc as a complementary judge-independent metric for these settings.

[Question 4: Consistent Str-Acc and LLM-Acc patterns across datasets]

Thank you for highlighting this unusual pattern. We verified that Str-Acc is consistently lower than LLM-Acc on HotpotQA and MuSiQue, while the reverse holds on 2WikiMultiHopQA across all methods in Table 1. We therefore examined question-level disagreements between the two metrics.

The metrics agree on 88–99% of questions, depending on the method and dataset. On HotpotQA and MuSiQue, disagreements commonly involve shorter but semantically correct predictions that fail string containment against longer gold answers. On 2WikiMultiHopQA, gold answers are often short entities or categories, so string containment may accept predictions with additional, semantically incorrect qualifiers that the LLM judge rejects.

For example, "Jean Cocteau" may fail string matching against a longer full-name gold answer, whereas "Chinese American" may pass substring matching against "Chinese" despite not being equivalent.

These findings suggest that the consistent direction across methods reflects dataset-specific answer characteristics interacting with the two evaluation metrics, rather than a method-specific effect. We have clarified this behavior and the limitations of both metrics in the revised manuscript.

Thank you for the extensive response to my review. Please see my response below.

W1-3:

  • Regarding bullet point 1, it is still unclear to me what is novel about the formulation/formalization. The problem BindRAG tackles has been formalized in very similar ways in previous works, including in the ones you cite yourself (bullet point 1).
  • I appreciate the revision of the contributions section and, based on my understanding, the key methodological contribution lies in providing the final reader (prompt) with more context than existing works have done (bullet point 2). Looking at Table H.1, the biggest contribution (+16.1) comes from slot substitution (see my concerns about related works in my initial review on this) and +4.8 from providing the program, bindings and grouping the intermediate results by hops.
  • With Table 2 added, it seems now that your interpretation of the experimental results has shifted from pure performance over other methods, to include reduced token cost (bullet point 3)? I believe a separation of this would be beneficial.

I understand that comparisons to methods that require training or don't provide the source code, so your revisions to the related works section resolves my concern for these two.

Writing issues: While I appreciate the revisions made and the authors conceding on the contribution claims, I still believe there need to be key revisions done outside of the results section (this is where the current version shows most blue-colored differences). I am less concerned with the revised introduction and the related works, but starting in section 3, a lot of the writing could be drastically simplified.

Q1: Understood. Q2: I can confirm this has been corrected.

Q3: Digimon appears to support a wide variety of open and closed-weight models. This does not resolve my question for the choice of Qwen version. I understand the choice of wanting to use a consistent judge, but have you evaluated any discrepancies in judging when using other models as the judge (even on a small subset, not even all evaluation results)?

Q4: This addresses my question.

Reviewer_EWVaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BindRAG compiles multi-hop questions into retrieval programs with explicit variables, passes intermediate answers into later queries, and retains evidence and alternative bindings for final answer generation.

Critical strengths
  1. The gains on MuSiQue are substantial. In Table 1, Str-Acc rises from the strongest evaluated baseline's 29.3 to 48.3, and LLM-Acc from 36.8 to 52.6. Agreement between the two metrics makes a purely lexical explanation unlikely.

  2. Explicit answer propagation addresses a practical retrieval problem: later evidence may be indexed under an entity absent from the original question. The larger gains on sequential questions in Table 2 and Figure 3 fit this motivation. The appendix also reports alternative backbones, including settings where performance deteriorates.

Critical weaknesses
  1. The closest execution baseline is missing, and the retrieval budgets are not adequately controlled. Table 1 focuses on graph-based RAG, although the main claim concerns sequential retrieval. IRCoT already studies interleaved retrieval and reasoning on these three datasets. LogicRAG is useful, but does not establish the gain over a simple system that answers each subquestion and uses that answer in the next query. BindRAG also retrieves five passages per hop and expands ambiguous branches, whereas Vanilla RAG retrieves at most five passages once. Varying BindRAG's own k does not resolve this difference.

  2. The QAM ablation changes an insufficiently specified set of inputs. The final reader receives accumulated passages, hop grouping, intermediate answers, alternatives, and a copy-then-answer prompt. Table 2 does not make clear which of these disappear without QAM. The 15-point drop on 2WikiMultiHopQA is large, but could partly reflect lost evidence or a different answering prompt. Appendix D also attributes an important role to final candidate enumeration without reporting the corresponding numbers.

Questions for authors

On MuSiQue, how much of the gain remains against sequential answer propagation with the same final reader and a matched total token budget? With the retrieved passages fixed, what are the separate effects of hop grouping, alternative bindings, and copy-then-answer? Please specify exactly what the QAM ablation removes.

2 comments

[Weakness 1: IRCoT and fair retrieval-budget comparison]

We agree that varying only BindRAG's retrieval parameter does not establish a fair budget comparison. We therefore implemented IRCoT using its public code and prompts, added SeqProp as a direct sequential answer-propagation baseline, and measured total token consumption across all pipeline stages.

Figure 3 and Table E.1 now compare methods by total tokens, retrieval calls, and final-context size. Under a shared flat reader and matched retrieval-call counts, BindRAG outperforms IRCoT by 7.0 points (95% CI [4.2, 9.8]). It also achieves comparable accuracy to IRCoT with approximately one-fifth of the tokens.

Shared flat reader, MuSiQueTotal tokens per questionBindRAG − method
BindRAG5.3k—
IRCoT (tuned)25.1k+1.5 [−1.2, +4.2]
IRCoT (retrieval calls matched to BindRAG)17.9k+7.0 [+4.2, +9.8]

Str-Acc differences with 95% CIs (Table E.1).

[Question: Gain over sequential propagation with the same reader and token budget]

This is an important distinction. Under the same flat reader, BindRAG and SeqProp are statistically tied (+1.3 points, 95% CI [−0.5, +3.2]). This comparison uses SeqProp at k=5, which uses fewer tokens than BindRAG (4.8k vs. 5.3k per question). Against the smallest SeqProp setting that uses at least as many tokens (k=10, 7.7k), BindRAG's gain is 4.3 points (95% CI [1.9, 6.7]); since SeqProp is less accurate at k=10 than at k=5, this matched comparison favors BindRAG. We therefore do not claim that BindRAG's retrieval procedure alone is superior to sequential answer propagation.

Instead, our controlled reader analysis shows where the additional contribution lies: explicitly providing the program and resolved bindings to the final reader improves accuracy. The step-by-step experiment (Table H.1) further separates the gains from binding propagation (+16.1 points) and binding-aware memory (+4.8 points).

Thus, BindRAG's contribution is the integration of dependency-aware retrieval with binding-aware final synthesis, rather than substitution alone.

[Weakness 2 / Question: Individual QAM components with fixed evidence]

We isolated the effects of individual reader components while holding retrieved passages fixed. As reported in Table 3, removing the program and resolved bindings reduces Str-Acc by 4.8 points on MuSiQue and 5.3 points on 2WikiMultiHopQA. Replacing copy-then-answer with direct answering reduces accuracy by 5.3 and 2.3 points, respectively.

Additional analyses with the same fixed evidence show that hop grouping and uncertain bindings have no statistically significant individual effects.

Removed from the reader input (evidence fixed)MuSiQue2WikiMultiHopQA
Program and resolved bindings−4.8 [−7.1, −2.5]−5.3 [−7.7, −2.9]
Copy-then-answer (→ direct answering)−5.3 [−7.3, −3.4]−2.3 [−4.0, −0.6]
Hop grouping−0.3 [−2.0, +1.4]+0.5 [−1.2, +2.2]
Uncertain bindings+0.4 [−0.8, +1.6]+0.5 [−0.4, +1.4]
Earlier-hop passages+0.3 [−1.6, +2.2]−1.6 [−3.5, +0.3]

Str-Acc change with 95% CIs. The first two rows are from Table 3 (lower block); the other rows are from the same fixed-evidence experiments.

The final candidate enumeration that the original Appendix D refers to lowers Str-Acc by 2.3 [0.5, 4.1] (MuSiQue) and 2.2 [0.5, 4.0] (2WikiMultiHopQA) points when reduced from three copied spans to one (details in our response to Reviewer qHEH). We have revised Section 4.6 to emphasize the components with demonstrated accuracy contributions without overstating the effects of the remaining components.

[Question: What does the QAM ablation remove, and why is the 2WikiMultiHopQA drop so large?]

The QAM ablation removes resolved and uncertain bindings, hop-wise evidence organization, and passages from earlier retrieval hops. It retains the decomposition program, binding propagation during retrieval, and the final answer-generation procedure.

We also corrected a confound in the original experiment by keeping the answer prompt fixed. Even after this correction, removing QAM reduces 2WikiMultiHopQA accuracy by 14.5 points (95% CI [11.6, 17.4]), confirming that the effect is not explained by the prompt change.

Our fixed-evidence analysis suggests that bindings and earlier-hop passages provide complementary information. Removing either alone has a relatively small effect, whereas removing both together with the other memory components reduces accuracy by 22.7 points (79.6 to 56.9; Appendix I).

Removed (2WikiMultiHopQA, evidence fixed)Str-Acc change
Program and resolved bindings only−5.3 [−7.7, −2.9]
Earlier-hop passages only−1.6 [−3.5, +0.3]
Both, with hop grouping and uncertain bindings−22.7 (79.6 → 56.9)

This is particularly relevant to comparison and aggregation questions, where the final reader must preserve evidence about multiple entities. We now explain this interaction rather than attributing the entire decrease to a single QAM component.

Reviewer_qHEHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents BindRAG, a framework for multi-hop QA that decomposes a question into a typed retrieval program and uses intermediate answers to instantiate subsequent queries. Alternative answers and evidence from each hop are retained for final answer generation. Experiments on three QA benchmarks show improvements.

Critical strengths
  1. BindRAG makes retrieval dependencies explicit by passing intermediate answers into subsequent queries. It also retains alternative candidates and their evidence for the final reader, allowing uncertainty to remain available after retrieval has followed a primary path.

  2. The reasoning-type analysis gives a useful account of where the full system helps. Gains are more pronounced on questions with sequential dependencies than on comparison questions (Figure 3). This is consistent with the motivation for binding, although it does not by itself isolate the contribution of each component.

Critical weaknesses
  1. The paper would benefit from a closer comparison with existing multi-step retrieval methods. ReWOO is discussed as prior work, but neither a comparable plan-and-execute system nor IRCoT is evaluated. BindRAG also makes additional retrieval calls for alternatives and recovery. A comparison using similar retrieval budgets would help show what the binding and memory design adds.

  2. The binding and memory ablations need clearer descriptions. How are dependent queries constructed without binding propagation, and which passages remain available without memory? These changes could affect the reader's access to evidence as well as the execution mechanism.

Questions for authors
  1. How are dependent queries and reader inputs constructed in the binding and memory ablations? Is the answer-generation prompt held fixed?

  2. Could you report the candidate-enumeration results discussed in Appendix D, particularly the effect of copy-then-answer with the retrieved evidence held fixed?

  3. Do you have results for a comparable plan-and-execute baseline, together with its total retrieval budget and final-context length?

3 comments

[Weakness 1 / Question 3: Comparable baselines and computational budget]

Thank you for pointing out the need for stronger comparisons. We added SeqProp, IRCoT, ReWOO, and an adapted Multi-Chain Reasoning baseline, evaluating them with both their own readers and a shared flat reader (Table 2).

Method (MuSiQue)BindRAG − method, own readerBindRAG − method, shared flat readerRetrieval steps / LLM calls / tokens per question
BindRAG——3.7 / 3.8 / 5.8k
SeqProp+7.6+1.3 [−0.5, +3.2]2.6 / 3.6 / 5.0k
IRCoT+9.2+1.5 [−1.2, +4.2]4.6 / 5.6 / 29.7k
ReWOO+25.0+17.7 [+14.6, +20.8]1.3 / 3.8 / 3.2k
MCR (adapted)+10.1−2.7 [−5.5, +0.0]25.9 / 27.7 / 67.4k

Str-Acc differences with paired bootstrap 95% CIs; Qwen2.5-32B, 1,000 questions. All own-reader differences are significant (Holm-corrected). Final-context sizes are in Table E.1.

Importantly, BindRAG does not significantly outperform SeqProp under the shared reader (+1.3 points, 95% CI [−0.5, +3.2]). Its larger end-to-end advantage (+7.6 points) primarily comes from the binding-aware final reader, rather than retrieval alone. We now make this distinction explicit.

We also report retrieval steps, LLM calls, final-context size, and total tokens (Table E.1). BindRAG uses 5.8k tokens per question, compared with 29.7k for IRCoT, while achieving higher end-to-end accuracy. These comparisons clarify both the source of the improvement and its computational cost.

[Weakness 2 / Question 1: Ablation definitions and prompt control]

We agree that the original ablations did not fully isolate individual components.

The original 'w/o binding' ablation asked the decomposer for self-contained sub-questions without slots, but the substitution code was unchanged: slots the decomposer still emitted (45% of questions on MuSiQue, 32% on HotpotQA, 15% on 2WikiMultiHopQA) were substituted as in the full model, and resolved and uncertain bindings remained visible to the reader. This ablation therefore removed binding only partially. We added a strict variant that retrieves each sub-question with its slots removed and never substitutes earlier answers. This reduces MuSiQue Str-Acc by 17.0 points, confirming the importance of propagating intermediate answers into dependent retrieval queries.

The 'w/o QAM' ablation leaves retrieval unchanged, including binding substitution, and changes only the reader input: the reader receives the question, the program, and the five last-hop passages as a flat list, without resolved and uncertain bindings, hop grouping, or earlier-hop passages. It therefore changes the reader's access to evidence rather than the execution mechanism. Exact definitions are in Appendix F.

We also identified a prompt confound in the original ablations: the number of copied answer candidates changed alongside the removed components. We therefore repeated the ablations with the final answer prompt fixed.

Under this controlled setting, removing QAM reduces Str-Acc by 4.4 points on MuSiQue and 14.5 points on 2WikiMultiHopQA (Table 3). Removing ambiguity handling, however, does not yield a statistically significant accuracy change (Section 4.6).

Ablation (answer prompt fixed)MuSiQue2WikiMultiHopQA
w/o QAM−4.4 [−7.0, −1.9]−14.5 [−17.4, −11.6]
w/o binding, strict (independent retrieval)−17.0 [−19.9, −14.0]—

Str-Acc change from the full model, with 95% CIs (Table 3).

We have revised Section 4.6 to distinguish the demonstrated contribution of query-aware memory from the auxiliary role of ambiguity handling.

[Question 2: Copy-then-answer and candidate enumeration]

We isolated the effects of these components while keeping retrieved evidence fixed.

Replacing copy-then-answer with direct answering reduces Str-Acc by 5.3 points on MuSiQue and 2.3 points on 2WikiMultiHopQA (Table 3). This confirms that the final answer-generation strategy contributes independently of retrieval.

Removing the final candidate enumeration (three copied spans reduced to one) lowers Str-Acc by 2.3 points on MuSiQue (95% CI [0.5, 4.1]) and 2.2 on 2WikiMultiHopQA ([0.5, 4.0]); on HotpotQA the change is not significant (−1.0 [−2.5, +0.5]).

ChangeMuSiQue2WikiMultiHopQA
Copy-then-answer → direct answering−5.3 [−7.3, −3.4]−2.3 [−4.0, −0.6]
Final candidates 3 → 1 (copied spans)−2.3 [−4.1, −0.5]−2.2 [−4.0, −0.5]

Str-Acc change, with 95% CIs, with the retrieved evidence fixed.

We therefore distinguish the demonstrated benefit of copy-then-answer and final candidate enumeration from intermediate candidate enumeration, which remains part of the execution procedure but has not been shown to independently improve accuracy.

Thanks for response. After reading the rebuttal, I decide to keep my score unchanged.

Comments on the paper

1 comment

Tables added for readability

For the reviewers' convenience, we have added tables to our responses, which previously presented the results only in running text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.