- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SR-Search proposes action-specific credit assignment for search-and-refine agents trained with GRPO. The old policy evaluates the gold-answer log-likelihood before search, after retrieval, and after refinement. The resulting search and refinement information gains are normalized separately and assigned to their corresponding action spans, with a fraction of refinement credit also propagated back to the same-cycle search. The paper evaluates Qwen2.5-3B and 7B Base/Instruct models on seven QA benchmarks, reporting improvements over AutoRefine and step-reward baselines. Validation includes component ablations, training dynamics, search-frequency analysis, and a reported 11.2% increase in per-step training time.
Critical strengths
-
Separating evidence acquisition from evidence refinement is a useful and practically motivated credit-assignment design. The method provides dense supervision using the existing policy and gold answers, without requiring a separate critic, process reward model, or step-level annotations.
-
The reported improvements extend across seven benchmarks and two model sizes. The component ablations and computational-cost analysis provide useful evidence about the method’s practical behavior and tradeoffs, beyond final-answer accuracy alone.
Critical weaknesses
-
The main comparisons are insufficiently specified to support the claimed attribution of gains to credit assignment. Sections 4.2 and A.2 describe a matched AutoRefine comparison, but the original AutoRefine uses a refinement-dependent retrieval reward in addition to final-answer F1 and reports 250 training steps, versus 300 here. The AutoRefine rows in Table 1 match the original published results. It is therefore unclear which baselines were rerun under matched settings and which results were taken from prior work. The evaluation protocol also needs clarification: with 125 Bamboogle questions, single-run EM changes in increments of 0.008, whereas reported values such as 0.645, 0.641, and 0.383 cannot result from rounding a correct-answer count divided by 125. Averaging across runs could explain this, but repetitions, seeds, and uncertainty are not reported. These issues limit confidence in the comparative conclusions, particularly the 0.003 average improvement over GiGPO with the 7B Instruct backbone.
-
The experiments do not sufficiently isolate the benefit of structured routing from the benefit of adding dense answer-aware supervision. IGPO already uses gold-answer probability gains and backward credit propagation; the distinctive contribution here is the separation and routing of search/refinement credit. Table 2 removes channels or replaces normalized gains with raw gains, but does not keep the information signals and their scale comparable while changing the routing. A matched cycle-level or uniform-routing control would directly test the central contribution. Moreover, the answer-containment metrics in Figure 3 do not establish that retrieved or refined content supplies the evidence needed for the inference. The reported analyses therefore support a useful training configuration more clearly than the proposed credit-assignment mechanism.
Questions for authors
-
Clarification: Which baseline rows were retrained and reevaluated in your environment, and which were taken from published results? For AutoRefine specifically, was its original retrieval-specific reward retained, and what training budget was used?
-
Clarification: What evaluation procedure produces the reported Bamboogle values given its 125 questions? Please specify the metric implementation, number of runs, aggregation procedure, and uncertainty, including for the comparison with GiGPO.
-
Evidence request: Under identical interaction, outcome-reward, and training-budget settings, does structured SIG/RIG routing outperform a control that uses the same scoring checkpoints but assigns a cycle-level information-gain signal uniformly to search and refinement actions, with comparable reward scaling?
2 comments
Dear Reviewer, thank you again for your review. We have posted our detailed response and added new three-seed results and baseline-source clarifications to the revised manuscript. With limited time remaining for discussion, we would appreciate your feedback and any remaining questions as soon as possible, so that we can address them before the discussion closes. Thank you for your time and consideration.
Thank you for the specific questions. The revised paper adds baseline provenance and a three-seed study that reproduces SR-Search's main performance result.
Q1 / Weakness 1. Baseline sources, AutoRefine reward, and budget
Section 4.1 and Appendix A.3, Table 6 (pp. 13-14) distinguish quoted results from our experiments. All non-SR-Search main-table rows are quoted, not retrained or reevaluated by us. SR-Search results, ablations, sweeps, behavior/cost measurements, and new seed runs are our experiments.
Table 6 lists AutoRefine as the numerical source for Direct Generation, SFT, R1, Naive RAG, IRCoT, Search-o1, Search-R1, and AutoRefine; GiGPO for StepSearch and GiGPO; and IGRPO for IGPO. The appendix specifies that IGPO EM values come from the IGPO rows in IGRPO v1. The baseline seed statistics come from AutoRefine Appendix C.2, Table 7.
AutoRefine's quoted results include its original refinement-dependent retrieval reward; we did not replace it with an outcome-only reward.
SR-Search trains for 300 steps on eight A100-80GB GPUs; AutoRefine's published hyperparameter table lists 250 steps. Figure 3(a) shows SR-Search's training reward reaching a plateau before step 250, with limited later improvement. Its reward advantage is already visible before the additional 50 steps. This addresses convergence timing, rather than constituting a matched-budget final-EM rerun.
Q2 / Weakness 1. Bamboogle aggregation and variability
Evaluation uses greedy decoding (temperature=0, n=1), one rollout per question, and binary EM after answer extraction and normalization. Our loader is:
It shuffles a mixed-dataset pool and drops the final incomplete batch. EM is computed over retained questions, so Bamboogle's denominator can differ from 125. Archived EM and action-count metrics reconstruct counts of 118-125 for Bamboogle and 484-498 for the other six datasets. This explains non-1/125 values from this evaluator, without multiple-rollout averaging. The overall score is the unweighted mean of seven dataset scores.
The quoted IGPO/GiGPO values were not generated by this loader. GiGPO's public search-QA code retains the last batch but averages episode rewards over records expanded by interaction step, which is not constrained to multiples of 1/125. Table 6 identifies the sources rather than implying a common evaluator.
Appendix B.1, Table 7 (pp. 14-15) reports three independent SR-Search runs, one per seed. Qwen2.5-3B-Base achieves 0.441 +/- 0.009, compared with published three-seed results of 0.408 +/- 0.014 for AutoRefine and 0.315 +/- 0.015 for Search-R1: gains of 3.3 and 12.6 percentage points. SR-Search's mean is higher than AutoRefine's on every benchmark, and closely matches the original 0.442.
Given the limited rebuttal period and multi-GPU cost, we prioritized SR-Search repetitions and quoted baseline seed statistics from AutoRefine Appendix C.2, Table 7. Table 7's two-sided paired tests use seven benchmark means (N=7, df=6): p=7.51e-4 versus AutoRefine and p=3.65e-3 versus Search-R1. These are cross-benchmark tests; the standard deviations separately describe seed variation.
The new repetitions concern 3B Base, not the Instruct models. They do not supply uncertainty estimates for the small 7B Instruct gap against GiGPO.
Q3 / Weakness 2. Simpler information-gain assignment and routing
We tested a trajectory-level alternative during development, aggregating action-level information gains into the outcome reward. Its overall performance was approximately at the AutoRefine level, without the improvement of the final action-specific design.
SR-Search preserves which stage produced each gain: SIG goes to search, RIG to refinement, and bounded same-cycle RIG feedback goes to the search supplying the input. This differs from adding answer-aware supervision alone or propagating a single turn-level gain. Table 2's channel ablations provide complementary support. We have not completed the scale-matched cycle-uniform control.
Figure 3 and Appendix E connect the signals to answer-related behavior. RIG is a likelihood-based utility signal; answer containment is not itself proof of document support. The QA, ablation, and repeated-run results support the training method's practical effectiveness.
We hope the new evidence and clearer provenance address your concerns, and respectfully ask you to consider a more favorable score.