- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces RIPPLE, a learned retrieval planner that combines a persistent, query-specific corpus snapshot with feedback from unsuccessful retrieval attempts. The snapshot is constructed by retrieving a broad candidate set and applying maximal marginal relevance to select diverse documents. The planner generates subqueries conditioned on this snapshot and previous failure summaries. It is trained using GRPO with answer-quality rewards at individual search states, alongside format, query-relevance, and diversity rewards.
The paper reports improvements over retrieval, planning, and RL-based baselines on ANTIQUE, TechQA, and MSQA. Additional experiments investigate component ablations, multi-hop QA, zero-shot transfer, different planner and generator backbones, independent evaluation, and inference latency.
Critical strengths
-
The planner-state design addresses a useful practical problem. Combining a diversity-aware snapshot with summaries of unsuccessful retrievals gives the planner both an initial view of potentially complementary information and feedback about previously unproductive directions. This is a concrete approach to improving retrieval planning for underspecified and non-factoid questions.
-
The reported empirical investigation is broad. Beyond the three primary benchmarks, the paper evaluates multi-hop tasks, zero-shot transfer, multiple planner and generator models, and an independent evaluator. The snapshot and feedback ablations also provide encouraging evidence that these components contribute to the reported gains. If the evaluation protocol is clarified, this breadth would strengthen the case that the approach is useful beyond a single dataset or model configuration.
Critical weaknesses
-
The experiments do not clearly isolate the central round-level supervision claim. The terminal-only ablation in Table 2 is described as also removing the failure-conditioned training mode. It is therefore unclear whether the difference comes from reward granularity, training-state coverage, or the number of optimization updates. Furthermore, under the stated stopping rule, search terminates once at least one document is judged relevant. Consequently, any trajectory that continues to another round must have an empty previously accepted relevant-evidence pool. Under this literal formulation, later-state answer utility depends on the current round’s accepted evidence, rather than progressive accumulation of accepted partial evidence across rounds. This can support learning to retry after retrieval failures, but does not establish the stronger claim about credit assignment for progressively closing evidence gaps.
-
The evaluation protocol leaves important questions about both numerical reliability and metric interpretation. Appendix A.3 defines LLM-Eval and CAR as means of binary judgments. With 200 ANTIQUE test questions, single-run scores must be multiples of 0.005, which cannot produce the reported 0.418 LLM-Eval or 0.762 CAR. Repeated runs or a different aggregation scheme could explain these values, but the repetition count, aggregation rule, and uncertainty are not reported. This limits confidence in the main comparisons and smaller ablation differences. Additionally, CAR only measures whether the generator abstains; it does not independently verify that the evidence supports the answer. Its improvement therefore cannot by itself substantiate the paper’s evidence-sufficiency claims.
Questions for authors
Clarification and validation requests:
-
Please specify the exact evaluation denominators, number of training and evaluation repetitions, and aggregation rules behind Tables 1–3. How are the ANTIQUE scores of 0.418 and 0.762 obtained under the binary metric definitions? Per-run binary counts and uncertainty for the principal comparisons would help resolve this concern.
-
Does the implemented training and inference procedure stop immediately when the cumulative relevant-document count reaches one? If so, how does the round-level objective supervise progressive accumulation of partial evidence? In the terminal-only ablation, do all decisions in failure-conditioned states still receive optimization updates? Please clarify whether state coverage and optimization budgets are matched.
-
What evidence supports interpreting increased CAR as improved evidence sufficiency, beyond increased willingness to answer? Please also clarify how the 310-example TechQA split was used, how hyperparameters were selected, what retrieval corpus was available, and how unanswerable cases were handled.
7 comments
Author Response to Reviewer 5uGY (2/6)
"W1 - ...Furthermore, under the stated stopping rule, search terminates once at least one document is judged relevant. Consequently, any trajectory that continues to another round must have an empty previously accepted relevant-evidence pool. Under this literal formulation, later-state answer utility depends on the current round's accepted evidence, rather than progressive accumulation of accepted partial evidence across rounds. This can support learning to retry after retrieval failures, but does not establish the stronger claim about credit assignment for progressively closing evidence gaps."
"Q2 - Does the implemented training and inference procedure stop immediately when the cumulative relevant-document count reaches one? If so, how does the round-level objective supervise progressive accumulation of partial evidence?"
We describe the precise mechanism on how the relevance threshold based stopping works and discuss how failure summaries and round-level credit assignment operates on failed rounds to enable the planner to learn to progressively close evidence gaps. Finally, we also discuss an example from TechQA to illustrate this -
How relevance threshold-based search stopping rule works - During a search round, the relevance judge inspects the retrieved documents to determine if the retrieved evidence is relevant and sufficient to answer the question. If the judge finds at least one such relevant and sufficient document (hence the threshold of 1), then we stop the search early, do not execute any further search rounds (both during training as well as during inference) and generate the answer using the accepted evidence collected so far (as also shown in figure 2 in main paper). However, in scenarios where a document or more than one document are found to be relevant but collectively not sufficient enough to answer the question by the judge, it is deemed as a failed round and the relevant documents are added to the accepted relevant-evidence pool. The subsequent search round is then executed conditioned on the failure summary of previous rounds allowing for further cumulative accumulation of evidence. Hence, answer utility of the round would depend on accepted partial evidence accumulated so far. We have modified the paper pdf now to clarify the relevance threshold mechanism and now discuss how round level objective supervise accumulation of partial evidence -
Why failed rounds contribute towards effective training of planner: Post the completion of a failed round, the round-level reward determined based on answer generated using the evidence gathered till completion of that round can still carry a training signal and provides supervision to the planner to learn to close the evidence gap owing to the following reasons:
- On a failed round, an intermediate answer is still generated and scored, so the round carries a real training signal. Even when relevance threshold is not being met in a failed round, the evidence gathered can be relevant and partially complete but insufficient. The evidence gathered till that round's completion is used to generate an answer and receives its round-level reward. Because the reward on this answer is graded rather than tied to the relevance threshold, it reflects exactly this partial usefulness: a plan whose evidence yields a better answer (even if still incomplete) scores higher than one whose evidence is unhelpful.
- This per-round signal complements the failure summaries rather than restating them. The failure summaries sit in the state and tell the planner which search directions already failed giving planner the context to redirect search plan accordingly. The round-level reward then grades the new plan by the quality of the answer generated from the evidence it accumulated. Failure summaries supply the context for plan redirection; round level objective supplies the trainable signal reflecting the quality of the answer generated from the evidence the redirected plan gathered.
- Because the planner is rewarded on the failed rounds as well as on the round that finally succeeds, it can tell which plans held the search back and which moved it forward. Each failed round's low answer-based reward pushes the planner away from the plan that produced it, while the succeeding round's higher reward pulls it toward the plan that retrieve answer-supporting evidence. Credit is therefore assigned to each round's plan individually. Based on the reward of the round that eventually succeeded against those of the failed rounds that preceded it, the planner implicitly learns what separated the productive plan from the unproductive ones.
Answer to be continued ...
Author Response to Reviewer 5uGY (3/6)
We now discuss an example to illustrate the relevance threshold working mechanism and how round level credit assignment supervises progressive accumulation of partial evidence:
An example RIPPLE trajectory from TechQA. A sample question is taken which is about reporting a WebSphere Portal 8.5 startup failure after a database transfer to Oracle, with errors EJPDB0085W, EJPDB0160E, and EJPFD0016E and the database character set shown as WE8MSWIN1252 (not UTF-8). The ground truth answer is that the database was created with an unsupported character set and must be dropped and recreated with a Unicode configuration; a successful transfer does not fix this. We discuss the evidence retrieved after each search round and corresponding relevance judge labels below -
- Round 1 (fails the relevance threshold bar with all irrelevant documents). The planner's subqueries retrieve only documents about datasource and database-URL fixes; the relevance judge deems all the documents as irrelevant, so the accepted-evidence pool is empty. An intermediate answer is still generated from the round's documents and scored: the answer misreads the encoding warning as a connection-mapping issue and recommends retaining the database, which contradicts the ground-truth answer, so it gets a very low answer correctness reward. The failure summaries are written into the planner state.
- Round 2 (fails the relevance threshold bar with a relevant document). Conditioned on those summaries, the planner redirects toward the encoding question and retrieves D1 (swg21383033), which establishes the right diagnostic branch, that a successful transfer is insufficient and the unsupported code set must be checked against UTF8 / AL32UTF8 / AL16UTF16, but does not contain the exact recovery procedure. The relevance judge labels this document as relevant but not sufficient, so the document gets added to the accepted evidence pool but relevance threshold is still not met; the answer generated using the accepted evidence pool gets slightly higher correctness this time for correctly identifying the unsupported-encoding branch and ruling out the URL-fix remedy. At the same time, further failure summaries are added regarding lacking recovery procedure.
- Round 3 (clears the relevance threshold). The refined subqueries retrieve D2 (swg21317981), the gold document, which supplies the drop-and-recreate remedy and the Oracle character-set checks. The evidence now clears the relevance bar, and the final answer gives the complete, correct recovery which gets the highest correctness score reward for updating the learner accordingly.
The above case study illustrates the progressive improvement in the searched evidence across rounds and quality of the answer generated from each round's gathered evidence operating under the relevance threshold rule. The above case study has been added to Appendix A.8 and referred in main paper.
W2 + Q1 (combined) — The evaluation protocol accounts for the reported metrics: denominators, aggregation, and repetitions
"W2 - The evaluation protocol leaves important questions about both numerical reliability and metric interpretation. Appendix A.3 defines LLM-Eval and CAR as means of binary judgments. With 200 ANTIQUE test questions, single-run scores must be multiples of 0.005, which cannot produce the reported 0.418 LLM-Eval or 0.762 CAR. Repeated runs or a different aggregation scheme could explain these values, but the repetition count, aggregation rule, and uncertainty are not reported. This limits confidence in the main comparisons and smaller ablation differences."
"Q1 - Please specify the exact evaluation denominators, number of training and evaluation repetitions, and aggregation rules behind Tables 1–3. How are the ANTIQUE scores of 0.418 and 0.762 obtained under the binary metric definitions? Per-run binary counts and uncertainty for the principal comparisons would help resolve this concern."
Aggregation rule and denominators (how 0.418 and 0.762 arise).
Each of LLM-Eval and CAR is the mean of five evaluation runs. On ANTIQUE this is five runs over 200 questions, or 1000 binary judgments, so the reported values are multiples of 0.001 and 0.418 and 0.762 are exactly 418/1000 and 762/1000, which resolves the steps-of-0.005 concern. The same construction fixes the denominators on the other datasets: 5 x 310 = 1550 binary judgments on TechQA and 5 x 572 = 2860 on MSQA. This averaging rule and these denominators apply identically to every entry in Tables 1 to 3.
Answer to be continued ...
Author Response to Reviewer 5uGY (1/6)
We thank the reviewer for the close reading and for recognising the core of RIPPLE's design: the planner-state formulation that combines a diversity-aware corpus snapshot with summaries of unsuccessful retrievals, giving the planner both an initial view of complementary information and feedback on previously unproductive directions, together with the breadth of the empirical investigation across component ablations, multi-hop tasks, zero-shot transfer, multiple planner and generator backbones, and an independent evaluator.
Below we address the concerns raised: whether the experiments isolate the round-level supervision claim and how the stopping rule relates to it (W1, Q2); the evaluation protocol behind the reported metrics, covering the denominators, aggregation rule, and number of runs (W2, Q1); and the requested clarifications on the TechQA split, hyperparameter selection, the retrieval corpus, and unanswerable cases, together with an extensive analysis to establish that RIPPLE improves retrieved evidence sufficiency (Q3). Here we use the notation W to depict weakness and Q to depict Question.
W1 + Q2 (combined) — The terminal-only ablation isolates round-level supervision, and the stopping rule supports progressive credit assignment
The terminal-only variant retains the failure-conditioned training mode such that the failure summaries of previous rounds are used in subsequent rounds. The only change in terminal-only ablation is the reward granularity. Training-state coverage and the number of optimization updates are held fixed, so the comparison does isolate round-level supervision. We explain this elaborately in the following points -
Section 5 (lines 411-417) has been updated to make explicit that the terminal-only variant retains all failure-conditioned training states along with failure summaries and changes only the reward granularity.
Answer to be continued ...