acceptodds
Under review as a conference paper at ICLR 2027

Relevance Says Yes, but Sufficiency Says No: Efficient Adaptive Retrieval via Latent Evidence Sufficiency

11 views

Abstract

Adaptive retrieval-augmented generation (RAG) must decide whether additional retrieval is needed, which evidence is worth acquiring, and when retrieval should stop. Existing approaches typically rely on retrieval relevance, uncertainty, or explicit model judgments, yet relevance does not indicate whether the current evidence is sufficient, and highly relevant passages may still be redundant. We further find that explicit sufficiency judgments can be systematically misaligned with the sufficiency information encoded in hidden representations. Building on these observations, we introduce **SuREST**, an adaptive RAG framework that uses a lightweight probe to recover latent evidence sufficiency and employs it as a unified control signal throughout retrieval. Across four multi-hop QA benchmarks, SuREST achieves the best answer quality among the evaluated baselines. With Qwen2.5-7B-Instruct, it improves macro F1 by 2.36 points over IRCoT, the strongest baseline by macro F1, while reducing retained passages by 56.7% and end-to-end inference time by 53.6%. Evidence analysis suggests that these gains come from reducing redundant evidence while largely preserving coverage. These results demonstrate that latent evidence sufficiency provides an effective basis for jointly improving retrieval effectiveness, evidence quality, and efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Adaptive Retrieval-Augmented Generation, Evidence Sufficiency, Latent Representations, Retrieval Control, Retrieval Efficiency, Multi-hop Question Answering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WostOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies adaptive RAG through the lens of evidence sufficiency. The authors identify a systematic mismatch between a language model's explicit sufficiency judgments and the sufficiency information encoded in its hidden representations: on controlled paired data, the native output is heavily biased toward predicting insufficiency, whereas a lightweight linear probe trained on the same final-layer representation achieves substantially better discrimination. Building on this observation, the authors propose SuREST, which recovers latent evidence sufficiency via a linear probe and uses it as a unified control signal for retrieval decisions. Evaluated on four multi-hop QA benchmarks, SuREST achieves the best macro F1 among all baselines while reducing retained passages.

Critical strengths
  1. The identification of a representation–output mismatch for evidence sufficiency is conceptually compelling. The controlled paired-data construction and the careful comparison between the native LM head and a linear probe on the same hidden state (Table 1) convincingly demonstrate that this is a readout mismatch rather than an absence of information. The analysis of bias persistence across answer-correctness conditions further strengthens the claim.

  2. Using a single latent sufficiency signal to jointly control retrieval triggering, evidence selection, and termination is a clean architectural choice. The SMG scoring function, which measures the reduction in latent insufficiency conditioned on the current evidence set, naturally discourages redundancy and favors complementary passages. This stands in contrast to approaches that combine multiple heterogeneous signals.

  3. SuREST achieves the highest EM and F1 on all four datasets while substantially reducing retrieval cost (Table 2). The extensive ablations isolate the contribution of each component, providing clear evidence that the gains are not attributable to any single factor.

  4. The probe transfers to Mistral and Qwen with competitive improvements and the linear-probe AUROC scales with model size (Figure 4).

Critical weaknesses
  1. All experiments are conducted on Wikipedia-based multi-hop QA datasets. While these are standard benchmarks, the paper frames its contribution as broadly applicable to adaptive RAG. It remains unclear whether the latent sufficiency signal and the SMG selection strategy would transfer to other retrieval scenarios, such as single-hop fact verification, long-document QA, where evidence sufficiency may manifest differently in hidden representations.

  2. Building the sufficiency probe requires substantial supervised effort: paired data construction with filtering, domain balancing, etc. This raises the question of whether the practical advantage over a simple fine-tuned classifier on sufficiency judgment is as large as the paper implies, especially given that the native-readout ablation requires offline calibration using the probe data.

  3. Several relevant baselines are discussed but not directly compared, such as SetR, which explicitly selects passages as a set to cover complementary information. A direct comparison against SetR would help clarify whether the gains come from the sufficiency signal itself or from the state-dependent, set-aware selection that SMG implicitly provides.

Questions for authors

see weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer Wost (Part 1/2)

We thank the reviewer for the insightful feedback and valuable comments!

[W1] Generalization Beyond Multi-Hop QA.

Thank you for highlighting this limitation. We additionally evaluated SuREST on 500 SQuAD v1.1 questions in the open-domain setting, without retraining the probe or changing the original backbone, thresholds, or retrieval budgets. Single-hop QA offers less room for adaptive evidence acquisition, as the initial BM25 results may already contain sufficient evidence. Nevertheless, SuREST improves F1 by 3.73 points over BM25 top-3 (paired-bootstrap 95% CI: [0.88, 6.60]). Compared with IRCoT, it achieves higher F1 while reducing retrieval calls, retained passages, and inference time by 35.9%, 61.2%, and 61.0%, respectively. These results support transfer beyond multi-hop QA, although SuREST incurs additional cost relative to fixed single-step retrieval.

Table a: Single-hop QA results on SQuAD v1.1.

MethodEM ↑F1 ↑Retrieval calls ↓Retained passages ↓Time (s) ↓
No retrieval12.6020.710.000.000.22
BM25 top-120.0027.861.001.000.63
BM25 top-323.8032.171.003.000.68
IRCoT24.8034.764.9614.8916.25
SuREST (Ours)27.0035.903.185.776.33

We acknowledge that SQuAD remains Wikipedia-based and does not establish transfer to different retrieval domains. For long-document QA, a natural adaptation would be to split the target document into passages and restrict retrieval to that document. SuREST could then use the same sufficiency-guided selection and stopping mechanism. Extending this approach to different document structures and capturing evidence dependencies across distant sections are directions for our future work. We have added the single-hop evaluation and clarified these limitations in Appendix C.3.

[W2] Supervision Cost and Comparison with an Explicit Sufficiency Judge.

The probe supervision is derived from existing benchmark annotations rather than new manual sufficiency labels. HotPotQA pairs use annotated supporting facts, while MuSiQue provides paired answerable/unanswerable contexts (Appendix A.1). Filtering relies on automated answer checks with deterministic EM/F1 criteria, and domain balancing uses simple oversampling (Appendices A.1 and A.3). The backbone remains frozen during probe training. We acknowledge the one-time costs of data filtering and representation extraction.

To assess the practical trade-off, we additionally trained an explicit sufficiency judge by LoRA fine-tuning Qwen2.5-7B-Instruct using the same 6,200 unique training pairs, four prompt formulations, and two label mappings as the probe (99,200 training views; 156,800 examples per epoch after domain balancing). Its decision thresholds were independently calibrated using ground-truth labels from the frozen development set.

On the independent MuSiQue confirmatory set, the fine-tuned judge achieves stronger offline sufficiency discrimination.

Table b: Sufficiency discrimination on the MuSiQue confirmatory set.

MethodAUROC ↑BAcc ↑Paired Acc. ↑
Probe L210.81373.5%84.1%
Explicit Judge (LoRA)0.89482.2%89.4%
AuthorsOpenReview ↗

Response to Reviewer Wost (Part 2/2)

We compare the judge with SuREST without uncertainty-aware lookahead, since the judge lacks the five-seed ensemble required by this component.

Table c: QA performance and retrieval cost.

MethodMacro F1 ↑Calls ↓Passages ↓Time (s) ↓
SuREST (Full)33.253.536.447.99
Probe w/o Lookahead32.353.586.027.16
Explicit Judge (LoRA)31.282.474.645.28

Despite stronger offline discrimination, the judge achieves lower F1 than the probe without lookahead (31.28 vs. 32.35), with declines across all four benchmarks. Further analysis reveals more frequent sufficiency-based stopping by the judge: on MuSiQue, it stops retrieval in 397 cases, compared with 84 for the probe without lookahead.

The probe is also substantially cheaper to train. The LoRA judge has 40.37M trainable parameters and takes ~24 hours for training alone, exceeding the approximately 10 hours required for the probe's entire pipeline, including data preparation, feature extraction, and training. The five deployed L21 probe heads contain only 17,925 trainable parameters. Thus, while the judge achieves stronger offline discrimination and lower retrieval cost, the probe delivers higher end-to-end answer quality at substantially lower training cost. Detailed training configurations and implementation details are provided in Appendix C.3.2.

Finally, the calibration in Appendix C.3.1 applies only to the native-readout replacement ablation, not to standard SuREST. It uses offline probe scores to align native margins with the controller's thresholds and is distinct from the supervision-matched LoRA comparison above.

[W3] Comparison with SetR and Contribution Attribution.

We agree that SetR is a relevant baseline, as both methods aim to acquire complementary evidence, but their approaches differ. SetR selects a passage subset from a candidate pool based on explicit information requirements. In contrast, SuREST explicitly conditions SMG on the current evidence set and recomputes candidate gains after each addition, enabling incremental, state-dependent evidence acquisition. SuREST further uses the same latent sufficiency signal to control retrieval triggering and termination.

Our analyses provide evidence that both the latent sufficiency signal and state-dependent selection contribute to performance. The learned probe achieves stronger sufficiency discrimination than the native readout (AUROC 0.797 vs. 0.678; Table 1). Replacing SMG with BM25 rank-order selection reduces macro F1 from 33.25 to 31.52 (Table 3). Moreover, under identical evidence states and candidate pools, SMG selects missing supporting evidence more frequently than BM25 top-1 (10.91% vs. 8.37%; Table 5).

We agree that a direct comparison with SetR would further clarify the relative contributions of latent sufficiency and set-aware evidence selection, but time and resource constraints during the rebuttal phase prevent us from adding this baseline. We believe the current ablation studies and controlled evidence-selection results already provide evidence for the benefits of both components, although they do not fully isolate their individual effects.

Thank you very much for your reply. I keep my score of 3 unchanged.

Reviewer_7K8sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SuREST, which uses a hidden-state sufficiency probe to guide retrieval triggering rather than the relevance signals only. Estimating whether an additional document improves the current evidence set is an important problem in RAG, and the experimental results show its effectiveness. However, the interpretation of sufficiency-score differences as document utility appears stronger than the current training objective and empirical evidence support. Please see the comments below.

Critical strengths
  • The paper addresses an important challenge in RAG: determining whether a candidate document adds useful evidence beyond the current context.
  • The method improves answer quality while reducing retained passages and inference time across multiple benchmarks. The evidence-selection analysis also provides useful support that the method reduces redundancy and identifies missing supporting evidence.
Critical weaknesses
  1. Firstly, we think the training objective does not establish that score differences reflect marginal utility (a slight overclaim). Although sufficient and insufficient contexts are constructed in pairs, the probe is trained with pointwise binary cross-entropy. For example, suppose a question requires three supporting facts. Contexts containing one and two of these facts are both labeled insufficient. BCE does not explicitly require the two-fact context to receive a better score, nor does it require the score difference to reflect progress toward answering the question. Consequently, a useful document could receive a negligible or negative SMG despite adding a necessary reasoning step. Pairwise ranking is not necessarily required, but the paper needs to justify why its classification score provides a reliable utility ordering.

  2. The discussion and evaluation of document-utility attribution are incomplete. Existing approaches estimate document contributions through leave-one-out ablation, Shapley values [1,2], subset-based surrogate models, or gradient-based attribution. The paper should discuss these alternatives and compare SMG with representative methods. The author maybe should clearly state why the proposed method is better or just an incremental work for existing probing methods. If it is just an incremental improvement, I think the experimental analysis may not be enough.

  3. Efficiency concern. The reported speedup depends substantially on cache reuse. Table 13 shows that full-prefill recomputation increases latency from 7.99 to 27.33 seconds, exceeding IRCoT’s reported 17.23 seconds, while answer quality remains similar. The authors should separate the savings from document selection and early stopping from those enabled by caching and batching, and clarify whether comparable optimizations are applied to the baselines. In this version, it seems that the efficiency is most gained from the cache, not the methods.

[1] Nematov et al. Source Attribution in Retrieval-Augmented Generation. 2025. [2] Chen et al. Shapley Context Pruning: A Cooperative Game Perspective for Context Reranking and Pruning. 2026.

Questions for authors
  • Why should differences in a BCE-trained sufficiency score reflect document utility? When contexts containing one and two necessary facts are both labeled insufficient, what encourages the probe to rank the latter higher?

  • When multiple documents are useful only jointly but receive small or negative individual SMGs, how often does the bounded lookahead recover them?

  • How much of the latency improvement remains under comparable caching and batching implementations? How does candidate-scoring cost grow with larger candidate pools and longer evidence contexts?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 7K8s (Part 1/3)

We appreciate your thorough evaluation of our paper. Below, we address your concerns and provide our responses and we have carefully revised the manuscript.

[W1/Q1] Sufficiency Scores and Evidence Utility.

We agree that BCE does not explicitly enforce an ordering among incomplete contexts, and SMG does not guarantee that every useful passage receives a positive gain. Our rationale is that the probe reads sufficiency information already encoded in the frozen language model, rather than learning document utility directly. The model's ability to assess evidence completeness is reflected in its hidden representations. Consistent with this view, our cross-backbone experiments (Figure 4) show higher sufficiency-probe AUROC for Mistral-24B and Qwen3.8-27B than for Qwen2.5-7B. We further examine whether the probe score reflects degrees of evidence incompleteness. On the independent MuSiQue confirmatory set, contexts missing two supporting hops receive higher insufficiency scores than those missing one, with the same trend across different reasoning depths.

Table a: Probe insufficiency scores by missing supporting hops.

Total hopsMissing 1 (n)Mean scoreMissing 2 (n)Mean scoreDifference95% CI
All2710.422242.42+2.00[1.66, 2.35]
22110.551542.82+2.27[1.84, 2.67]
347−0.09501.55+1.64[0.96, 2.31]
4130.16201.55+1.39[0.44, 2.40]

Differences are mean scores for two missing hops minus one; intervals are 95% bootstrap CIs. Five examples missing three hops are excluded.

These results suggest that the probe captures graded information about evidence incompleteness despite binary supervision. However, comparisons across questions do not establish monotonicity when evidence is added to the same question.

SMG uses changes in predicted insufficiency to guide passage selection. When no candidate has a confidently positive gain, uncertainty-aware lookahead evaluates whether complementary candidates are useful together. Our evidence-selection analysis (Table 5) shows that SuREST selects missing gold evidence more often than BM25 top-1 under the same evidence states and candidate pools (10.91% vs. 8.37%) and achieves greater recall gain (4.95 vs. 3.81 percentage points). Removing SMG also reduces macro F1 from 33.25 to 31.52 while increasing retained passages from 6.44 to 11.60 (Table 3).

Together, these findings support the usefulness of sufficiency-score differences for evidence selection, without implying that BCE guarantees monotonic evidence progress.

AuthorsOpenReview ↗

Response to Reviewer 7K8s (Part 2/3)

[W2] Document Utility Attribution and Methodological Contribution.

Our contribution is not a general-purpose document attribution method. We identify a mismatch between explicit sufficiency judgments and the sufficiency information encoded in hidden representations, then use this latent signal to jointly control retrieval triggering, evidence selection, and termination. SMG is the evidence-selection component of this framework. Unlike leave-one-out (LOO), which measures the effect of removing a passage from the full candidate set, or Shapley values, which average contributions across subsets, SMG measures the gain from adding a passage to the evidence actually accumulated so far. This directly matches the sequential decision made during adaptive retrieval.

To compare these approaches, we conducted an additional controlled experiment on 400 questions (100 per dataset). For each question, we froze SuREST's first candidate-selection state, consisting of three retained passages and ten candidates. All attribution methods use the same sufficiency probe. Random reports expected performance under uniform selection, BM25 follows the original retrieval ranking, SMG measures conditional addition gain, LOO measures removal from the full candidate set, and Shapley-16 averages addition gains over 16 random permutations while keeping the initial evidence fixed. All three use KV-cache reuse and micro-batches of five.

Table b: Evidence-selection quality and scoring cost under fixed retrieval states.

MethodTop-1 Gold Hit(%) ↑Top-1 Recall Gain(pp) ↑Top-1 Complete(%) ↑Top-3 Recall Gain(pp) ↑Time (s/state) ↓
Random11.955.417.0110.16—
BM2518.098.0811.0511.61—
SMG25.9111.7617.0714.130.374
LOO17.007.629.7212.062.341
Shapley-1626.3711.8616.2414.3019.788

SMG substantially outperforms LOO and achieves selection quality close to Shapley-16, with only a 0.46-point difference in Top-1 gold hit rate and a 0.17-point difference in Top-3 recall gain. However, Shapley-16 requires 52.9× more scoring time (19.788 vs. 0.374 s/state), while LOO is 6.26× slower and less effective. These results support SMG as an efficient method for selecting evidence that improves sufficiency given the current evidence state, rather than a general-purpose document attribution method. The full experimental setup and results have been added to Appendix C.5 of the revised manuscript.

[Q2] Recovery of Complementary Evidence by Lookahead.

We analyzed retrieval traces from all 2,000 evaluation questions. Bounded lookahead (beam width 3, depth 2) was invoked at 2,022 selection states and found admissible two-passage plans in 401 cases (19.8%). Among these, 372 plans (92.8%) involved passages with non-positive individual SMGs at the same evidence state, but whose combined evidence achieved a joint SMG above 0.5 or reached the sufficiency threshold. Both passages were eventually retained in 355 of these 372 cases (95.4%) under rolling commitment.

Removing lookahead reduces macro F1 from 33.25 to 32.35 (−0.90 points; 95% paired-bootstrap CI [0.34, 1.48] for the improvement), with the largest decrease on MuSiQue (−1.84 points). These results show that bounded lookahead can recover complementary passage combinations that would be rejected by individual SMG scoring.

AuthorsOpenReview ↗

Response to Reviewer 7K8s (Part 3/3)

[W3/Q3] Efficiency and Computational Scalability.

Thank you for raising this concern. As detailed in Sections 5.1 and C.2, all methods share the same backbone, retrieval corpus, global budgets, and hardware setup, while preserving their method-specific inference procedures.

SuREST repeatedly evaluates candidates against a shared question–evidence prefix. This structure naturally supports KV-cache reuse and batched candidate scoring, making them integral to its efficient implementation (Appendix B.3). Unlike methods such as IRCoT, which do not perform equivalent candidate-branching evaluations, SuREST can directly exploit shared-prefix computation across candidates.

To separate retrieval efficiency from computational optimization, we compare SuREST with a full-prefill variant (Table 13). Table 13 compares SuREST with a full-prefill variant that recomputes the complete context for each candidate. The two variants achieve similar macro F1 (33.25 vs. 32.91), retrieval rounds (3.53 vs. 3.50), and retained passages (6.44 vs. 6.31). However, inference time increases from 7.99s to 27.33s without cache reuse. Thus, the reductions in retrieval rounds and retained evidence arise from sufficiency-guided control, while KV-cache reuse substantially reduces the computational cost of candidate scoring. We therefore view KV-cache reuse as a natural optimization enabled by SuREST's design, while acknowledging that the reported wall-clock advantage reflects the evaluated implementations rather than fully optimized versions of every baseline.

We additionally evaluate different candidate pool sizes while keeping the remaining configuration unchanged.

Table c: Candidate pool size(K) ablation

KEMF1RoundsPassagesTime (s)Scored candidates
525.9032.553.695.897.2826.30
1026.2033.253.536.447.9961.33
1526.0532.953.446.659.7396.32
2025.8532.983.356.7310.57130.73

Increasing K from 5 to 20 raises the average number of scored candidates from 26.30 to 130.73 (4.97×), while end-to-end latency increases from 7.28s to 10.57s (+45.2%). Performance peaks at K=10, indicating that larger candidate pools increase scoring cost without consistent accuracy gains. These results support the choice of K=10 as an effectiveness–efficiency trade-off. Longer evidence contexts may increase prefix-processing and attention costs. However, SuREST's shared-prefix structure enables KV-cache reuse across candidate evaluations, avoiding redundant prefix computation and mitigating this overhead. Latency is therefore expected to increase with context length, but the growth is partially offset by cache reuse. The full candidate-pool ablation and computational analysis have been added to Appendix C.8.

Reviewer_52GQOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SuREST, an adaptive RAG framework that leverages latent evidential sufficiency in the hidden states of large models to uniformly control retrieval triggering, evidence selection, and retrieval termination. The method evaluates the current evidence through a lightweight linear probe and, based on the marginal gain in sufficiency brought by candidate passages, selects complementary evidence and reduces redundancy. On four multi-hop question-answering datasets, SuREST with Qwen2.5-7B improves macro-average F1 by 2.36 points over IRCoT, while reducing the number of retained passages and inference time by 56.7% and 53.6%, respectively.

Critical strengths

This paper addresses the problem in adaptive RAG that "relevance does not equal evidential sufficiency," and proposes using latent evidential sufficiency in hidden states to guide retrieval triggering, evidence filtering, and retrieval termination. The approach is clear and demonstrates a certain degree of innovation.

Critical weaknesses

The comparison between the probe and native readout does not match the task supervision, and lacks an explicit sufficiency judge trained with the same training data and supervision budget as a control. Therefore, the source of the performance improvement has not been sufficiently isolated. In addition, the efficiency advantage relies relatively heavily on engineering optimizations such as KV-cache reuse, and it still needs to be clarified whether each baseline receives the same degree of implementation optimization.

Questions for authors
  1. When using the same training data and supervision budget, does the latent sufficiency probe still outperform the explicit sufficiency judge?
  2. After all baselines receive the same KV-cache reuse and inference optimizations, does SuREST's efficiency advantage still hold?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 52GQ (Part 1/2)

We thank the reviewer for the careful reading of our paper and for the detailed and insightful comments and we have carefully revised the manuscript.

[W/Q1]: Supervision-Matched Explicit Sufficiency Judge.

We additionally trained an explicit sufficiency judge by LoRA fine-tuning Qwen2.5-7B-Instruct on the same 6,200 unique training pairs, four prompt formulations, and two label mappings as the probe (99,200 training views; 156,800 examples per epoch after domain balancing). The judge's thresholds were independently calibrated using ground-truth labels from the frozen development set.

On the independent MuSiQue confirmatory set, the fine-tuned judge achieves stronger sufficiency discrimination than the linear probe.

Table a: Sufficiency discrimination on the MuSiQue confirmatory set.

MethodAUROC ↑BAcc ↑Paired Acc. ↑
Linear Probe L210.81373.5%84.1%
Explicit Judge (LoRA)0.89482.2%89.4%

We compare the judge with SuREST without uncertainty-aware lookahead, since the judge lacks the five-seed ensemble required by this component.

Table b: QA performance and retrieval cost.

MethodHotPotQA F1MuSiQue F1IIRC F12Wiki F1Macro F1 ↑Calls ↓Passages ↓Time (s) ↓
SuREST (Full)48.2817.6527.6139.4633.253.536.447.99
Probe w/o Lookahead47.8715.8026.6739.0632.353.586.027.16
Explicit Judge (LoRA)47.0315.5125.6836.9131.282.474.645.28

Despite stronger offline discrimination, the judge achieves lower macro F1 than the probe without lookahead (31.28 vs. 32.35), with declines on all four benchmarks. The gap is larger on out-of-domain datasets (−1.57 F1 on average) than on in-domain datasets (−0.56). Further analysis shows that the judge stops retrieval more frequently: on MuSiQue, it triggers sufficiency-based stopping in 397 cases, compared with 84 for the probe without lookahead, alongside fewer retained passages and lower F1.

The probe also requires substantially less training: the LoRA judge uses 40.37M trainable parameters and nearly 24 hours of training, compared with 17,925 parameters for the five deployed L21 heads and approximately 10 hours for probe extraction and training. Thus, while the judge achieves stronger offline discrimination and lower retrieval cost, the probe delivers higher end-to-end answer quality with substantially lower training cost. Detailed training configurations and implementation details are provided in Appendix C.4.2.

AuthorsOpenReview ↗

Response to Reviewer 52GQ (Part 2/2)

[W/Q2] Implementation Fairness and Efficiency.

Thank you for raising this concern. As described in Section 5.1 and Appendices C.1–C.2, all methods use the same backbone, retrieval corpus, global budgets, and hardware setup, while preserving their method-specific inference procedures.

SuREST repeatedly evaluates candidate passages against a shared question–evidence prefix. This structure naturally enables KV-cache reuse and batched candidate scoring, making these optimizations an integral part of SuREST's efficient implementation (Appendix B.3). In contrast, methods such as IRCoT do not perform equivalent candidate-branching evaluations, so identical candidate-scoring optimizations are not directly applicable.

To isolate the effect of KV-cache reuse, we additionally evaluate a full-prefill variant that recomputes the complete context for each candidate. As shown in Table 13, removing cache reuse preserves similar macro F1 (32.91 vs. 33.25), retrieval rounds (3.50 vs. 3.53), and retained passages (6.31 vs. 6.44), while increasing inference time from 7.99s to 27.33s. These results show that SuREST's retrieval efficiency gains arise from its sufficiency-guided control rather than cache reuse, whereas its latency advantage benefits substantially from the optimization.

We therefore view KV-cache reuse as a natural optimization enabled by SuREST's design, while acknowledging that the reported wall-clock advantage reflects the evaluated implementations rather than fully optimized versions of every baseline. We have clarified these implementation differences in Appendix C.2.

Reviewer_52GQOpenReview ↗

Reply to Author

Thank you very much for your reply. As I am not an expert in this field, I will temporarily keep my score of 3 unchanged.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of reviewers feedback and authors response

We thank all reviewers for their constructive feedback and careful evaluation. We appreciate their recognition of the strengths of our work and the opportunity to further clarify and strengthen our contributions.

Strengths. Reviewers recognized the importance of evidence sufficiency in adaptive RAG and the value of identifying useful evidence beyond retrieval relevance. In particular, they highlighted the representation–output mismatch in sufficiency judgments, the unified use of latent sufficiency for retrieval triggering, evidence selection, and termination, and the effectiveness of state-dependent evidence acquisition. Reviewers also acknowledged the strong empirical performance, including improved answer quality with fewer retained passages across four multi-hop QA benchmarks, extensive ablation studies, and cross-backbone generalization.

Responses and Revisions. In response to the reviewers' concerns, we have revised and strengthened the manuscript:

  • Sufficiency supervision: We introduced a supervision-matched LoRA sufficiency judge. Although the LoRA judge achieves stronger offline discrimination, the linear probe achieves better QA performance with substantially lower training cost.
  • Evidence selection: We analyzed graded sufficiency scores and complementary evidence recovery, and compared SMG against LOO and approximate Shapley attribution, demonstrating its favorable selection quality–cost trade-off.
  • Generalization: We extended evaluation to open-domain single-hop QA on SQuAD v1.1 without task-specific adaptation.
  • Efficiency: We clarified the distinct contributions of sufficiency-guided retrieval and KV-cache optimization, and examined the effects of candidate pool size on performance and computational cost.

The revised manuscript incorporates the new experiments and implementation clarifications, including the single-hop evaluation (Appendix C.3), supervision-matched judge (Appendix C.4.2), document attribution comparison (Appendix C.5), and inference-time measurement details (Appendix C.2).

These additions provide further evidence for the effectiveness of latent sufficiency as a unified retrieval control signal, while clarifying the method's limitations and implementation-dependent efficiency gains. We believe the revised manuscript presents a more complete and carefully supported evaluation of SuREST.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.