acceptodds
Under review as a conference paper at ICLR 2027

StageRAG: Answer-Supervised Cascade Optimization for Retrieval-Augmented Generation

Abstract

Retrieval-augmented generation (RAG) is commonly deployed as a cascade: a retriever recalls candidates, a reranker selects a compact context, and a language model generates the answer. Optimizing this cascade from answer feedback alone faces two challenges. Corpus-specific supporting chains are costly and ambiguous to annotate, creating a Gold-Evidence Bottleneck; meanwhile, final-answer feedback does not reveal where evidence was lost, creating Cascade Credit Ambiguity. We introduce StageRAG, an answer-supervised framework that improves both the retriever and reranker without gold supporting-document annotations. StageRAG records candidate progression in a logged RAG execution, uses an answer-and-citation verifier to determine answer correctness and support attribution, converts the resulting states into query-level preferences, and balances them with per-query caps. We also formalize a pipeline-aware ranking objective, derive a pointwise rule under idealized conditions, and bound its regret under bounded evidence interactions. On HotpotQA, 2WikiMultiHopQA, and TAT-QA, StageRAG improves F1 over the strongest compared answer-supervised baseline by 5.65, 12.44, and 6.43 points, respectively. Cascade diagnostics show improved alignment with downstream trajectory preferences and recovery of supporting evidence into the final context.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Retrieval-Augmented Generation, Multi-Stage Cascade Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YBTyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies answer-supervised optimization of a retriever–reranker–generator cascade without gold supporting-document labels. StageRAG logs document retention, verifier-attributed support, and answer correctness, converts the resulting states into within-query preferences, and separately trains the retriever and reranker using capped preference sampling. The generator and inference topology remain fixed. Experiments on HotpotQA, 2WikiMultiHopQA, and RAGBench-TATQA show improvements in retrieval recall and answer F1, with additional component, supervision, verifier, and backbone analyses.

Critical strengths
  1. The problem is practically relevant. Improving both retrieval stages from question–answer pairs avoids task-specific evidence annotation, and the method adds no inference-time verifier or generation calls.
  2. The main improvements are substantial and consistent across the three evaluated datasets. The component ablations in Table 2 support complementary benefits from retriever and reranker optimization. The evidence-transition analysis is particularly useful: on HotpotQA, complete gold-evidence coverage in the final context increases from 30.0% to 41.8%.
  3. The experimental analysis goes beyond the main comparison. Table 3 controls the training budget when changing supervision, and the appendices examine the ambiguous failure state, verifier substitution, and a Qwen-based pipeline. I appreciate the explicit distinction between observable support attribution and causal evidence use, as well as the stated limitations of the theoretical assumptions.
Critical weaknesses
  1. The additional answer-quality benefit of the full state construction over simpler supervision needs stronger support. In Table 3, binary z4 supervision already reaches 0.5216 F1, compared with 0.5273 for the full method: a difference of 0.57 percentage points. The recall improvements are clearer, but this central comparison is reported only on HotpotQA with one training seed. The development-set confidence intervals in Appendix A.7 concern alternative treatments of z3, rather than this comparison.
  2. The theory motivates downstream-aware ranking but does not establish that the proposed state preferences estimate the derived pipeline value. Appendix A.7 correctly acknowledges that even the core ordering is an empirical surrogate. In particular, retained but unattributed documents are preferred over filtered documents despite neither having verified support. The contribution should therefore be judged primarily by its empirical evidence, rather than as a theoretically validated preference-learning rule.
  3. The stronger-backbone evidence is narrower than the main results might suggest. On Qwen-based TAT-QA, full optimization reaches 0.5040 F1, while reranker-only optimization reaches 0.5038 (Table 10). This supports compatibility, but provides little evidence of additional answer-quality benefit from retriever optimization in that setting. Likewise, the verifier audit reports only 46.74% gold precision for the original attributed support set on TAT-QA. Gold mismatch is not necessarily a verifier error, but representative disagreements would help interpret the supervision on this dataset.
Questions for authors
  1. Can you report paired uncertainty for the full-state versus binary-z4 comparison, and provide the corresponding comparison on one additional dataset? This would help establish whether the finer state distinctions reliably improve answer quality.
  2. How much of the gain comes from transferring the original reranker’s retention preferences, versus verifier-derived answer support? A retention-only control using all logged queries and the same training budget would clarify the value of answer supervision beyond ranker distillation; the existing answer-only control uses only successful trajectories.
  3. For Qwen-based TAT-QA, is the near equality between full and reranker-only F1 stable across runs? For the TAT-QA verifier audit, could you show a few support/gold disagreements indicating whether they reflect alternative valid evidence, incomplete annotations, or attribution errors?
2 comments
AuthorsOpenReview ↗

Response to Reviewer YBTy - Part 1

Dear Reviewer YBTy,

Thank you for your careful review and for noting the complementary stage gains, the difference between support attribution and causal use, and our clear limitation section. Below we report the requested statistical tests, new controls, and audit examples, and note the updates in our revised manuscript.

W1 & Q1: Paired uncertainty and a second dataset (Full-stage vs. Binary-z4z_4)

To check if fine-grained trajectory states improve retrieval and answer quality over Binary z4z_4, we tested both on 1,000 fixed queries on HotpotQA and 1,000 fixed queries on 2WikiMultiHopQA. We report 10,000-sample paired bootstrap 95% CIs for Full-stage minus Binary-z4z_4 (in percentage points). To check if the fixed reader (GPT-4o-mini) limits answer gains, we kept retrieved contexts fixed and also ran GPT-5.6 Luna:

HotpotQA (test@1,000):

ReaderMethodRecall@20 ↑Recall@3 ↑EM ↑F1 ↑
GPT-4o-miniBinary z4z_40.70450.67200.41900.5220
StageRAG (Full-stage)0.74400.69000.42600.5311
Gain (pp; 95% CI)+3.95 [2.85, 5.10]+1.80 [0.70, 2.90]+0.70 [-1.00, 2.40]+0.91 [-0.81, 2.57]
GPT-5.6 LunaBinary z4z_40.70450.67200.42900.5520
StageRAG (Full-stage)0.74400.69000.45600.5680
Gain (pp; 95% CI)+3.95 [2.85, 5.10]+1.80 [0.70, 2.90]+2.70 [0.90, 4.50]+1.60 [0.05, 3.15]

2WikiMultiHopQA (test@1,000):

ReaderMethodRecall@20 ↑Recall@3 ↑EM ↑F1 ↑
GPT-4o-miniBinary z4z_40.623000.561000.36000.40040
StageRAG (Full-stage)0.689750.621750.35700.39787
Gain (pp; 95% CI)+6.675 [5.475, 7.850]+6.075 [4.825, 7.350]-0.300 [-2.300, 1.600]-0.253 [-2.129, 1.669]
GPT-5.6 LunaBinary z4z_40.623000.561000.27100.2995
StageRAG (Full-stage)0.689750.621750.29600.3238
Gain (pp; 95% CI)+6.675 [5.475, 7.850]+6.075 [4.825, 7.350]+2.500 [0.300, 4.700]+2.428 [0.329, 4.559]

(Note: Within each dataset, both readers receive the exact same retrieved contexts, so the recall scores and gains are identical.)

Taken together, StageRAG beats Binary z4z_4 in recall on both datasets (all four recall 95% CIs above zero, up to +6.68 pp). With GPT-4o-mini, HotpotQA answers improve slightly (+0.70 EM, +0.91 F1) while 2Wiki does not, because the reader fails to use the extra evidence, not our retrieval pipeline. To test this, we kept the retrieved contexts unchanged and switched only to GPT-5.6 Luna. On the exact same contexts, StageRAG improves both EM and F1 on both datasets (+2.70 EM and +1.60 F1 on HotpotQA; +2.50 EM and +2.43 F1 on 2Wiki, all four 95% CIs above zero).

W2: Link between the theory and empirical preferences

We agree with your view and state this boundary clearly in Section 3.5 of the Revised Paper: our theory proves the necessity of cascading optimization of RAG retrieval system and motivates pipeline-aware ranking, rather than directly deriving the empirical state order. However, directly solving this objective requires heavy online RL or repeated counterfactual LLM rollouts, which is often not practical. StageRAG offers an effective design that bridges theory and practice: it avoids complex training algorithms (such as RL), keeps annotation costs low, and delivers strong empirical gains.

Specifically, viπ(q,d)=piπ(q,d)r(q,d)v_i^\pi(q,d) = p_i^\pi(q,d)r(q,d) separates potential contribution rr from downstream survival probability pp. Here, z2≻z1z_2 \succ z_1 serves as a practical proxy for pp: neither document is cited, yet z2z_2 passed reranking while z1z_1 was filtered. Even without numerically estimating vv, Table 3 shows this proxy is essential: removing z2≻z1z_2 \succ z_1 pairs drops Recall@3 by 2.80 pp (67.70% → 64.90%).

Thus, the theory establishes why upstream retrieval must account for downstream survival, and StageRAG turns that insight into a simple, low-cost, and effective design across the cascade.

AuthorsOpenReview ↗

Response to Reviewer YBTy - Part 2

W3 & Q3: Qwen TAT-QA multi-seed stability and verifier audit disagreements

1. Three-seed check on Qwen TAT-QA (Table 11 in Revised Paper):

We repeated the Full-stage vs. Reranker-only test on TAT-QA test@500 across three training seeds (42, 43, and 44):

SeedFull-stage R@3 / F1 ↑Reranker-only R@3 / F1 ↑ΔR@3 (pp)ΔF1 (pp)
420.4768 / 0.50400.4528 / 0.5038+2.40+0.02
430.4668 / 0.50130.4478 / 0.4998+1.90+0.15
440.4738 / 0.50560.4518 / 0.5004+2.20+0.52
Average0.4725 / 0.50360.4508 / 0.5013+2.17+0.23

Across all three seeds, Full-stage beats Reranker-only in every run, raising Recall@3 by +2.17 pp (0.4725 vs. 0.4508) and F1 by +0.23 pp (0.5036 vs. 0.5013) on average. Compared with the unoptimized Qwen cascade (Table 10), StageRAG lifts Recall@3 by +5.47 pp and F1 by +1.77 pp. This confirms stable gains on Qwen3, while end-task F1 over Reranker-only stays modest due to the fixed Qwen3-8B reader.

2. Audit examples of TAT-QA support/gold mismatches (Table 20 in Revised Paper):

For reference, Table 20 in the Revised Paper (Table 19 in the original submission) summarizes the cross-model audit on 200 queries per dataset:

DatasetAnswer agreement (%)Support confirmation (%)Gold precision (%)
HotpotQA99.5092.2589.80
2Wiki100.0077.3888.78
TAT-QA97.0095.2946.74

Note that 46.74% (43/92) measures overlap with the annotated gold set, not verifier accuracy. Using a fixed random seed, we checked 20 of the 49 non-matching records (across 18 queries) and found three clear cases:

  • Valid evidence outside the gold set (z4z_4): Asked for the principal of senior notes due June 2021, the system answered USD 800 million. The cited non-gold passage explicitly states that same USD 800 million principal for those notes.
  • Correct answer with wrong citation (z4z_4): The system correctly answered 2019 as having the highest diluted net EPS. However, it cited a non-gold table of weighted-average share counts instead of the gold EPS table.
  • Literal support with a scope mismatch (z3z_3): Asked for the change in total R&D expenses, the cited passage reports a 22% increase excluding impairment charges. It literally supports the stated number, but misses the question's scope—and the verifier properly marked the answer as wrong (z3z_3).

Hence, gold mismatches on TAT-QA come from a mix of valid unannotated passages, citation errors, and scope mismatches caught by answer verification. Notably, Case 1 shows that even benchmarks with evidence annotations can miss valid supporting passages. By learning directly from answer-supervised trajectories instead of static gold labels, StageRAG naturally discovers and uses this unannotated evidence.

Q2: Retention-only control versus answer-supervised stages

As suggested, we added a Retention-only baseline under the same training budget (Table 3 and Appendix A.6 in Revised Paper). It uses all logged training queries without answer filtering or verifier support, and matches optimizer update steps separately for the retriever and reranker. It learns only to rank initial top-3 retained documents above filtered ones:

Method (HotpotQA test@500)Recall@20 ↑Recall@3 ↑EM ↑F1 ↑
Retention-only0.71700.65700.39600.4929
StageRAG (Full-stage)0.73100.67700.43400.5273
StageRAG Gain (pp; 95% paired CI)+1.40 [0.10, 2.70]+2.00 [0.60, 3.50]+3.80 [1.20, 6.40]+3.44 [0.89, 6.03]

Overall, StageRAG significantly beats Retention-only on all four metrics (+3.44 F1, 95% CI [0.89, 6.03]). This demonstrates that StageRAG does more than copy the initial reranker. Turning answer correctness and citation support into stage-aware preferences improves both retrieval and answer quality without gold labels.

We hope these additional analyses address your remaining concerns and further strengthen your support for our work.

Reviewer_Q1ofOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper supervises both the retriever and reranker using answers in RAG, basically combining trajectory-state labeling and pairwise preference learning.

Critical strengths

S1. Extensive experiments. Three datasets and two backbones (MiniLM and Qwen3) are used. Ablation and robustness results are reported.

Critical weaknesses

W1. Limited novelty.

  • Essentially the paper combines trajectory-state labeling and pairwise preference learning in a straightforward day. Its technical difference from RRPO and other recent advences is marginal.
  • The presented document state ordering is a bit heuristic and not always reasonable.
  • The beginning of the paper argues that RAG faces a gold-evidence bottleneck. This may be true as of 2024, but we have seen many related efforts since 2025, such as GaRAGe (ACL 2025), mmRAG (ISWC 2025), and utility-oriented passage annotations (EMNLP 2025).

W2. Writing requires improvement.

  • The state definitions (Equation 2) are overly formalized and hard to understand. Directly reading the text is easier.
  • Many symbols are used without explanation, such as dd at the beginning of Section 3.2.
  • Many concepts lack explanation, such as pair type in Section 3.4.
  • Theoretical analysis is a claimed contribution, but the analysis is in the appendix, which is unacceptable. Its connection with the method is also weak.
Questions for authors

Q1. Do you believe the gold-evidence bottleneck still exists today?

Q2. What is the technical contribution of your work, compared with RRPO and other recent methds?

Q3. Could you discuss when your document state ordering rules become ineffective?

Q4. What is a pair type?

16 comments
AuthorsOpenReview ↗

Response to Reviewer Q1of - Part 1

Dear Reviewer Q1of,

Thank you for noting the breadth of our experiments across datasets and backbones. Since several concerns stem from how our trajectory states and pair types were presented in the original draft, we would clarify these core definitions first (W2 & Q4) and then W1.1–W1.3 and Q1–Q3.

W2 & Q4: Clarifying trajectory states, pair types, and main-text theory

Thank you for these helpful writing suggestions. We rewrote Section 3 in the revised manuscript (in blue) to make every concept clear:

  • Four trajectory states (Secs. 3.2–3.3, Eq. 2): Given a question–answer input (q,a∗)(q, a^*), we run the base RAG pipeline once and attach one state to each retrieved document d∈Cqd \in \mathcal{C}_q:

    • z1z_1 (Filtered): retrieved in top-KrK_r, but filtered out by the reranker (retained(d) = 0).
    • z2z_2 (Retained, unattributed): kept in the top-KcK_c context, but not verified as support (retained(d) = 1, attributed(d) = 0).
    • z3z_3 (Attributed, wrong answer): verified as supporting the generated answer, which is wrong (attributed(d) = 1, correct(q) = 0).
    • z4z_4 (Attributed, correct answer): verified as supporting the generated answer, which is correct (attributed(d) = 1, correct(q) = 1).
  • What a pair type is (Q4, Sec. 3.4): A preference pair (q,d+,d−)(q, d^+, d^-) compares two documents for the same query. Its pair type is their ordered state combination (e.g., z4≻z1z_4 \succ z_1). For example, suppose the reranker filters d1,d2d_1, d_2 and keeps d3,d4,d5d_3, d_4, d_5. If the answer is correct and only d5d_5 is cited as support, then d1,d2d_1, d_2 are z1z_1, d3,d4d_3, d_4 are z2z_2, and d5d_5 is z4z_4. Thus, (d5,d1)(d_5, d_1) and (d5,d2)(d_5, d_2) are different pairs of type z4≻z1z_4 \succ z_1, while (d5,d3)(d_5, d_3) has type z4≻z2z_4 \succ z_2. We cap sampled pairs per query and per type so common types do not drown out rare ones.

  • Simpler state definitions (Secs. 3.2–3.3, Eq. 2): Replaced hard-to-read indicators with direct checks (retained(d), attributed(d), correct(q)) and plain-text labels for z1z_1–z4z_4.

  • Complete symbol definitions (Secs. 3.2–3.4): Checked and defined every symbol at first use, including d∈Dd \in \mathcal{D}, RθR_\theta, CϕC_\phi, Eq,Ed,simE_q, E_d, \text{sim}, and σ\sigma.

  • Theory in the main text (Sec. 3.5): Rewrote Section 3.5 with explicit notation (r(q,d)r(q,d), piπ(q,d)p_i^\pi(q,d)) to present our core theoretical results directly in the main text. Due to the page limit, full assumptions and proofs remain in Appendix A.1. This theory directly guides StageRAG: since local relevance rr alone can fail when downstream survival pp differs, we supervise the upstream retriever with downstream signals—retention (z2≻z1z_2 \succ z_1) and verified support (z4≻z2z_4 \succ z_2). Thus, the theory explains why upstream training must follow downstream feedback, while Table 3 validates the discrete state ordering empirically.

AuthorsOpenReview ↗

Response to Reviewer Q1of - Part 2

W1.3 & Q1: Does the gold-evidence bottleneck still exist?

We respectfully explain why the gold-evidence bottleneck still exists and even grows worse as RAG corpora become larger. Collecting verified supporting labels faces a clear scaling trade-off:

  1. Full-corpus annotation is too expensive. True gold labels require checking supporting evidence across the entire corpus. Whether done by humans or LLMs, money, time, and context-length costs grow rapidly with corpus size.
  2. Pool-based annotation misses valid evidence. Retrieving a small candidate pool first and labeling only inside it (by humans or LLMs) cuts cost. However, any valid evidence missed by initial retrieval is never labeled, leaving the dataset biased and incomplete.

How StageRAG avoids this trade-off (Our contribution): StageRAG does not try to rebuild gold document labels. Using only existing question-answer pairs and one offline pass (2N2N calls: one answer generation + one verification), it trains both the retriever and reranker directly from cascade trajectories, addressing both full-corpus annotation cost and pool-label bias discussed above.

In contrast, the three 2025 papers cited by the reviewer still fall into one of the two cases above:

  • Case 1 — High full-corpus cost: GaRAGe [1]
    • Feature: Provides human-written answers with human-verified citations to supporting passages.
    • Shortcoming: Reaching this gold quality costs $9.65 per example (~$22,832 for 2,366 examples), which is too expensive for training on new or growing corpora.
  • Case 2 — Pool-based omission: mmRAG [2] and Utility-Focused LLM Annotation [3]
    • Feature: Both retrieve a candidate pool first and use LLMs to label passages inside it.
    • Shortcoming: Both produce biased pseudo-labels rather than true gold evidence. Any valid evidence missed by initial retrieval stays unlabeled, and neither checks whether passages jointly support a fixed reference answer.

In short, StageRAG avoids both expensive full-corpus labeling and biased pool pseudo-labels, offering a scalable way to optimize RAG retrieval system using only question–answer pairs in a cascading manner.

References

[1] I.-T. Sorodoc, L. F. R. Ribeiro, R. Blloshmi, C. Davis, and A. de Gispert. GaRAGe: A Benchmark with Grounding Annotations for RAG Evaluation. Findings of ACL, 2025.

[2] C. Xu, Q. Chen, Y. Feng, and G. Cheng. mmRAG: A Modular Benchmark for Retrieval-Augmented Generation over Text, Tables, and Knowledge Graphs. ISWC, 2025.

[3] H. Zhang, M. Tang, K. Bi, J. Guo, S. Liu, D. Shi, D. Yin, and X. Cheng. Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation. EMNLP, 2025.

AuthorsOpenReview ↗

Response to Reviewer Q1of - Part 3

W1.1 & Q2: Technical contribution relative to RRPO and answer-feedback methods

Recent answer-feedback methods (e.g., RRPO [4]) reduce the need for gold labels, but they usually optimize only one stage and ignore where each document stops in the cascade retrieval system. StageRAG makes three distinct contributions:

  1. Cascade-aware joint optimization: StageRAG jointly optimizes both the retriever and reranker of any RAG pipeline using shared trajectory states (filtering, retention, verifier support, and answer correctness), guided by our pipeline-aware ranking theory (v=p⋅rv = p \cdot r, Sec. 3.5 in Revised Paper). By training the upstream retriever with downstream signals, StageRAG recovers missed evidence directly into the candidate pool and final context, raising HotpotQA All Gold@20 from 37.6% to 50.2% and All Gold@3 from 30.0% to 41.8% (Table 14 in Revised Paper).
  2. Balanced preference learning (per-query caps): StageRAG limits pair counts per query and per type. This keeps common pairs (z2≻z1z_2 \succ z_1) from drowning out rare verified signals (z4z_4). In Table 3, this raises F1 by +2.47 and +10.99 points over natural and uniform sampling.
  3. Offline supervision with low cost: StageRAG collects trajectory states in one offline pass (2N2N calls for NN queries; $0.799 for 5,000 HotpotQA queries, Sec. 4.5) and reuses balanced preference pairs for training, greatly reducing API calls by avoiding repeated online RL rollouts.

Comparing directly with RL-based rerankers such as RRPO [4]:

  • Feature: RRPO avoids gold labels by training a reranker with RL. For a reranked list of kk documents, it feeds the top-1, top-2, ..., top-kk documents step by step to the reader and uses answer quality as the reward.
  • Shortcoming: 1) It keeps the initial retriever fixed, so it cannot recover missed evidence and relies only on answer rewards without stage-level attribution. 2) Evaluating both the policy and reference rankings step by step takes roughly 2Nk2Nk online reader calls across NN queries, compared with 2N2N offline calls in StageRAG.

We adapted RRPO to the same MiniLM pipeline across all three datasets (Table 1 in Revised Paper) and ran a controlled 2×2 swap on HotpotQA test@500:

Method (Main Protocol, Table 1)HotpotQA R@20 / R@3 / F1 ↑2Wiki R@20 / R@3 / F1 ↑TAT-QA R@20 / R@3 / F1 ↑
Naive (Frozen Cascade)0.6427 / 0.5945 / 0.44250.5585 / 0.5095 / 0.29380.4307 / 0.3232 / 0.4191
REPLUG (Best Baseline)0.6480 / 0.6040 / 0.44890.5500 / 0.5030 / 0.27730.4553 / 0.3368 / 0.4279
RRPO (Reranker-only RL)0.6427 / 0.5980 / 0.43330.5585 / 0.5095 / 0.29520.4307 / 0.3327 / 0.4155
StageRAG (Ours, Joint)0.7310 / 0.6770 / 0.52730.6850 / 0.6205 / 0.40170.5673 / 0.4548 / 0.4922
Controlled 2×2 Swap (HotpotQA test@500)R@20 ↑R@3 ↑EM ↑F1 ↑
Original Retriever + Original Reranker0.64270.59450.35800.4425
Original Retriever + RRPO Reranker0.64270.59800.34800.4333
StageRAG Retriever + RRPO Reranker0.73100.66600.40600.5042
StageRAG Retriever + StageRAG Reranker0.73100.67700.43400.5273

Taken together, joint training beats every baseline shown above across all three benchmarks (+7.84, +10.65, and +6.43 F1 over the strongest baseline shown on each dataset). Even with the RRPO reranker held fixed, switching to StageRAG's retriever boosts Recall@3 by +6.80 pp and F1 by +7.09 pp. On that same candidate pool, StageRAG's reranker further beats RRPO by +2.31 F1 and +2.80 EM.

References

[4] Y. Wu, X. Shen, F. Wang, C. Zhou, Z. Wu, X. Dai, and R. Xia. Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning. ACL, 2026.

Reviewer_kTTPOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

STAGERAG uses answer feedback and document trajectories to train a RAG retriever and reranker without gold evidence labels. Experiments on three QA benchmarks show improved retrieval and answer quality.

Critical strengths
  1. The method improves both retrieval stages without changing the generator or adding inference modules, outperforming matched W-RAG and REPLUG baselines on three datasets.
  2. On HotpotQA, complete evidence coverage rises from 30.0% to 41.8%, showing that training improves evidence availability.
Critical weaknesses
  1. The preference ordering is heuristic and lacks theoretical justification (Appendix A.7), potentially reinforcing the initial reranker’s errors.
  2. The added gains are small: full-state supervision improves HotpotQA F1 by only 0.57 points over binary supervision, while joint training beats reranker-only training by 0.02 points with Qwen. Their reliability remains unclear.
Questions for authors
  1. Under the same training budget, does STAGERAG outperform reranker-score distillation combined with binary verified-support supervision?
  2. Are the gains over binary z4z_4 supervision consistent across seeds or a larger test set? Can you report paired confidence intervals?
2 comments
AuthorsOpenReview ↗

Response to Reviewer kTTP - Part 1

Dear Reviewer kTTP,

Thank you for recognizing StageRAG's gains across both retrieval stages without changing the generator or inference pipeline, as well as our evidence-coverage analysis. Below we answer your questions with new controlled ablations (Tables 3 and 11 in Revised Paper) and paired confidence intervals on 1,000-query test sets.

W1: Justification of the ordering and avoiding initial reranker errors

1. Why the state ordering is justified: Our core order z4≻corez2≻corez1z_4 \succ_{\text{core}} z_2 \succ_{\text{core}} z_1 follows our pipeline-aware ranking rule (v=p⋅rv = p \cdot r, Sec. 3.5 in Revised Paper). Here, z2≻z1z_2 \succ z_1 reflects downstream survival (pp), and z4≻z2z_4 \succ z_2 reflects verified support (rr). Table 3 confirms both are needed: removing z2≻z1z_2 \succ z_1 or z4≻z2z_4 \succ z_2 drops Recall@3 by 2.80 pp and 1.60 pp.

2. Why StageRAG fixes rather than copies initial reranker errors: First, verified support overrides initial reranking. Within the top-3 context, StageRAG ranks supported evidence (z4z_4) above unused or wrong-answer docs (z2,z3z_2, z_3). Per-query caps also limit z2≻z1z_2 \succ z_1 to 2 pairs per query, versus 4 for z4≻z1z_4 \succ z_1 (Appendix A.5).

Second, StageRAG fixes initial filtering errors on held-out test queries (Figure 3b, Table 18 in Revised Paper). It promotes 14 of 43 gold documents dropped by the initial reranker, and recovers 116 gold documents missed by Top-20.

3. Direct control against copying the initial reranker (Retention-only): To test what happens if we only copy the initial reranker, we added a Retention-only baseline (Table 3, Appendix A.6 in Revised Paper). It trains on all logged queries with matched update steps, using no answer or citation labels. It learns only to rank top-3 kept documents above filtered ones:

Method (HotpotQA test@500)Recall@20 ↑Recall@3 ↑EM ↑F1 ↑
Retention-only0.71700.65700.39600.4929
StageRAG (Full-stage)0.73100.67700.43400.5273
StageRAG Gain (pp; 95% paired CI)+1.40 [0.10, 2.70]+2.00 [0.60, 3.50]+3.80 [1.20, 6.40]+3.44 [0.89, 6.03]

Overall, StageRAG significantly beats Retention-only on all four metrics (all 95% paired intervals above zero). Answer-and-citation supervision clearly adds value beyond copying the initial reranker.

Q1: Comparison against reranker-score distillation + Binary-z4z_4

We added the requested Binary-z4z_4 + score distillation (KD) control under the same training budget (Table 3 and Appendix A.6 in Revised Paper). It trains the retriever with binary z4z_4 supervision plus score distillation from the initial reranker (λ=0.1\lambda = 0.1), paired with the binary-z4z_4 reranker. Both methods match the number of retriever document exposures during training:

Method (HotpotQA test@500)Recall@20 ↑Recall@3 ↑EM ↑F1 ↑
Binary z4z_40.69400.65500.42800.5216
Binary z4z_4 + KD (λ=0.1\lambda=0.1)0.70700.66700.41000.5111
StageRAG (Full-stage)0.73100.67700.43400.5273
Full − (Binary z4z_4 + KD) (pp)+2.40+1.00+2.40+1.62

In short, score distillation raises recall over Binary z4z_4 alone (+1.30 Recall@20, +1.20 Recall@3). Even so, StageRAG leads on all four metrics (+2.40 Recall@20, +1.00 Recall@3, +2.40 EM, +1.62 F1). Trajectory states capture richer cross-stage signal than binary support plus score distillation, showing the importance of stage-aware cascading optimization of RAG retrieval system.

AuthorsOpenReview ↗

Response to Reviewer kTTP - Part 2

W2 & Q2: Reliability across larger test sets, a second dataset, and Qwen seeds

1. Full-stage vs. Binary-z4z_4 on 1,000-query test sets (HotpotQA & 2Wiki):

We tested both methods on 1,000 fixed queries each from HotpotQA and 2Wiki. We report 10,000-sample paired bootstrap 95% CIs for Full-stage minus Binary-z4z_4 (in percentage points). To check if the original reader (GPT-4o-mini) limits answer conversion, we kept each method's retrieved contexts fixed and also ran GPT-5.6 Luna:

HotpotQA (test@1,000):

ReaderMethodRecall@20 ↑Recall@3 ↑EM ↑F1 ↑
GPT-4o-miniBinary z4z_40.70450.67200.41900.5220
StageRAG (Full-stage)0.74400.69000.42600.5311
Gain (pp; 95% CI)+3.95 [2.85, 5.10]+1.80 [0.70, 2.90]+0.70 [-1.00, 2.40]+0.91 [-0.81, 2.57]
GPT-5.6 LunaBinary z4z_40.70450.67200.42900.5520
StageRAG (Full-stage)0.74400.69000.45600.5680
Gain (pp; 95% CI)+3.95 [2.85, 5.10]+1.80 [0.70, 2.90]+2.70 [0.90, 4.50]+1.60 [0.05, 3.15]

2WikiMultiHopQA (test@1,000):

ReaderMethodRecall@20 ↑Recall@3 ↑EM ↑F1 ↑
GPT-4o-miniBinary z4z_40.623000.561000.36000.40040
StageRAG (Full-stage)0.689750.621750.35700.39787
Gain (pp; 95% CI)+6.675 [5.475, 7.850]+6.075 [4.825, 7.350]-0.300 [-2.300, 1.600]-0.253 [-2.129, 1.669]
GPT-5.6 LunaBinary z4z_40.623000.561000.27100.2995
StageRAG (Full-stage)0.689750.621750.29600.3238
Gain (pp; 95% CI)+6.675 [5.475, 7.850]+6.075 [4.825, 7.350]+2.500 [0.300, 4.700]+2.428 [0.329, 4.559]

(Note: Within each dataset, both readers receive the exact same retrieved contexts, so the recall scores and gains are identical.)

Taken together, StageRAG beats Binary z4z_4 in recall on both datasets (all four recall 95% CIs above zero, up to +6.68 pp). With GPT-4o-mini, HotpotQA answers improve slightly (+0.70 EM, +0.91 F1) while 2Wiki does not, because the reader fails to use the extra evidence, not our retrieval pipeline. To test this, we kept the retrieved contexts unchanged and switched only to GPT-5.6 Luna. On the exact same contexts, StageRAG improves both EM and F1 on both datasets (+2.70 EM and +1.60 F1 on HotpotQA; +2.50 EM and +2.43 F1 on 2Wiki, all four 95% CIs above zero).

2. Multi-seed stability on Qwen TAT-QA (Table 11 in Revised Paper):

We repeated the Full-stage vs. Reranker-only comparison on TAT-QA test@500 across three training seeds (42, 43, 44):

SeedFull-stage R@3 / F1 ↑Reranker-only R@3 / F1 ↑ΔR@3 (pp)ΔF1 (pp)
420.4768 / 0.50400.4528 / 0.5038+2.40+0.02
430.4668 / 0.50130.4478 / 0.4998+1.90+0.15
440.4738 / 0.50560.4518 / 0.5004+2.20+0.52
Average0.4725 / 0.50360.4508 / 0.5013+2.17+0.23

Across all three seeds, Full-stage beats Reranker-only in every run, raising Recall@3 by +2.17 pp (0.4725 vs. 0.4508) and F1 by +0.23 pp (0.5036 vs. 0.5013) on average. Compared with the unoptimized Qwen cascade (Table 10), StageRAG lifts Recall@3 by +5.47 pp and F1 by +1.77 pp. This confirms stable gains on Qwen3, while end-task F1 over Reranker-only stays modest due to the fixed Qwen3-8B reader.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Revisions

Dear Area Chair,

We sincerely thank you and the reviewers for the constructive feedback. Below we note our discussion status (ratings: 3, 3, 2, with positive movement) and key revisions:

  • Positive assessments (kTTP, YBTy; rating 3: Weak acceptance): Both recognize our practical motivation, zero inference overhead, and thorough diagnostics. We address all requests via new controls, 1,000-query CIs, 3-seed Qwen runs, and audits.
  • Rating increase from Reviewer Q1of (1 → 2): Across two rounds, we substantially address all concerns:
    • We add 4 new baselines (beating all) and analyze 3 recent annotation works, reinforcing our core contribution: low-cost, offline cascade-state optimization without gold document labels or online RL overhead.
    • We discuss concrete state-ordering failure cases, clarifying boundaries and how per-query caps and z4z_4 signals protect performance.
    • We rewrite Section 3 for clearer, more coherent presentation.

Consensus Among Reviewers

  • Practicality & Zero Inference Overhead (kTTP, YBTy): Optimizing retriever and reranker from QA pairs alone avoids gold evidence annotation with no extra inference calls.
  • Strong Empirical Gains (Q1of, kTTP, YBTy): Extensive evaluation across HotpotQA, 2Wiki, and TAT-QA on MiniLM and Qwen3 (+5.65, +10.65, +6.43 F1 over matched baselines; HotpotQA full-evidence coverage 30.0% → 41.8%).
  • Thorough Analysis & Clarity (Q1of, YBTy): YBTy highlights budget-controlled ablations, verifier substitution, and cross-backbone studies; Q1of confirms improved writing.

Summary of New Experiments and Revisions

RL and Mutual Distillation Baselines (Q1of):

  • RRPO: We add RRPO across all three benchmarks; in a HotpotQA swap, adding StageRAG's retriever boosts F1 by +7.09 pp, and StageRAG's reranker adds +2.31 F1.
  • CORE-Style Mutual KD: Adapting bidirectional distillation to our cascade still trails StageRAG on all four metrics, showing score exchange alone cannot match cascade states.
  • MMOA-RAG & CoRAG-Style Joint RL: Adapting MAPPO and GRPO to jointly train retriever and reranker under the same 10,000-call API budget, StageRAG beats MAPPO by +5.81 F1 and GRPO by +6.27 F1 (with 72.70% of GRPO groups skipped due to identical rewards and zero relative advantage). Even at 20,000 calls (~11.6h), GRPO trails by 4.78 F1.

Supervision Controls (kTTP, YBTy):

  • Retention-Only: Under matched updates without answer/citation labels, StageRAG beats Retention-only on all four metrics (+3.44 F1, all four 95% CIs > 0).
  • Binary z4z_4 + Distillation: Under the same budget, StageRAG leads on all four metrics (+2.40 R@20, +1.62 F1).

Scale, Reader Conversion & Stability (kTTP, YBTy):

  • 1,000-Query HotpotQA & 2Wiki: Full-stage improves recall over Binary z4z_4 on both datasets (all four 95% CIs > 0); on identical contexts, GPT-5.6 Luna yields gains on all four EM/F1 metrics (all four 95% CIs > 0).
  • 3-Seed Qwen TAT-QA: Full-stage beats Reranker-only across all three seeds (+2.17 R@3 avg) and improves the base Qwen cascade.

Writing, Theory, Failure Cases & Audits (Q1of, YBTy):

  • Section 3 & Theory: We simplify state definitions, symbols, and pair types with an example, and clarify in Sec. 3.5 how StageRAG bridges pipeline-aware theory and offline practice.
  • Failure Cases (Q1of): We add concrete z2≻z1z_2 \succ z_1 and z3z_3 analyses in Appendix A.7.
  • TAT-QA Audit (YBTy): Auditing 20 gold-mismatch records (46.74% overlap) reveals valid unannotated evidence alongside citation errors and z3z_3 scope mismatches.
  • Pooled Pseudo-Labels (Q1of): We show via Proposition S1, baselines (W-RAG, REPLUG, Answer-only), and cost (~6.7× fewer tokens; 2N vs. ~35N calls) why pooled labels fall short.

All revisions are highlighted in blue in the updated manuscript. We believe these additions thoroughly address all reviewer concerns.

Best regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.