- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Answer-Bearing Frontier (ABF) Training for multi-turn search QA. It mines states immediately before the first answer-bearing tool observation, restores their histories, samples one-decision continuations, and rewards immediate acquisition with search, document, and terminal rewards . ABF updates alternate with full-trajectory Outcome GRPO updates. Across 6 benchmarks at 2B and 9B, ABF reports the highest average EM, 37.4% and 48.8% respectively.
Critical strengths
- The author gives clear problem framing and precise ABF definition, the first answer-bearing transition is formalized with a witness action and frontier state.
- Coherent local reward design for search, document opening, and terminal answers, including discouraging premature answering.
- Alternating ABF and Outcome updates are well motivated, with rollout expansion/compression dynamics shown across training cycles.
Critical weaknesses
- The acquisition predicate is lexical and restricted to named-entity training answers. The paper acknowledges misleading matches for numerical/Boolean answers and misses paraphrases, so frontier selection and rewards may not track semantic acquisition.
- Cost accounting is easy to misread. The 12.1% sampling and 19.3% work ratios use selected-prefix lower bounds; full recorded mining is 88.4% and 69.2%, with retrieval, validation, and overhead excluded.
- The authors only conduct experiments on the Qwen3.5 series and do not validate their method on the weaker Qwen2.5 base model. Since most baseline [1, 2] comparisons in the paper are evaluated on Qwen2.5-7B-Instruct, the generalization ability of the proposed approach cannot be fully verified.
[1] Wang et al. Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents. ICLR 2026.
[2] Ma et al. TSPO: Breaking the Double Homogenization Dilemma in Multi-turn Search Policy Optimization. ACL Findings 2026.
Questions for authors
- How does ABF behave if the lexical answer-bearing predicate is replaced by a semantic verifier or LLM judge, and what happens on numerical/Boolean questions excluded from training?
- What are matched full-compute comparisons against baselines, given that the reported ABF cost ratios are lower bounds or selected-prefix proxies?
- Could the authors supplement experiments based on the Qwen2.5 base model and conduct comparative evaluations under the experimental settings strictly aligned with baseline methods such as IGPO?
6 comments
I appreciate the authors' rebuttal, which provides helpful additions regarding semantic judges, Qwen2.5 baseline experiments, and cost metric clarifications. However, several core concerns remain unresolved: the numerical/Boolean evaluations are not explicitly validated within semantic frontiers or rewards, and the DeSA setting is not strictly aligned. Additionally, the paper's presentation and figure still require refinement. Therefore, I maintain my original score.
Thanks for getting back to us. We'd like to clarify three points so we can better understand and address your remaining concerns.
1) Numerical and Boolean questions. Do you mean evaluating evidence acquisition on these questions with a semantic judge, or including them in local training with semantic frontiers and rewards?
For semantic evaluation, we have just broken down the existing Qwen3-32B judgments by answer type. Below are the results for Search-R1 and our original, lexically trained ABF, using the metrics in Table 2:
| Answer type | Semantic metric | Search-R1 | ABF |
|---|---|---|---|
| Boolean | First-search hit (%) | 88.6 | 93.8 |
| Boolean | All-search hit (%) | 79.8 | 78.9 |
| Boolean | Supporting docs / rollout | 3.29 | 4.25 |
| Boolean | Open hit (%) | 61.4 | 72.1 |
| Numerical | First-search hit (%) | 4.5 | 5.4 |
| Numerical | All-search hit (%) | 7.8 | 7.0 |
| Numerical | Supporting docs / rollout | 0.34 | 0.44 |
| Numerical | Open hit (%) | 9.2 | 12.9 |
ABF has higher first-search hit rates, more supporting documents, and higher open hit rates in both groups, though all-search hit rates are slightly lower.
If you mean including these question types in local training with semantic supervision, that is beyond this paper's claims and remains a direction for future work.
2) DeSA settings. We aligned the added experiment with DeSA's original setup, using its official code, Qwen2.5-3B model, NQ+HotpotQA training data, Wiki-18/E5 retrieval with top-3 results, tool interface, and full-parameter optimization. ABF's local training and alternating Outcome schedule are implemented within this setup. With the same evaluation questions and official scoring, ABF reaches 37.63 mean EM vs. 35.74 for DeSA's released Stage 2 model across seven datasets. Could you tell us which specific setting you still consider misaligned, or what concern remains about this comparison?
3) Presentation and figures. Could you point us to the passages or figures you found unclear in the revised version, and tell us what was hard to follow? We're happy to revise them, but need specific examples to understand what needs changing.
Before undertaking further experiments, we would appreciate knowing which remaining concern they should address and what result would resolve it.
Response to reviewer Cg2W
Thank you for your feedback. We took your concerns seriously and completed substantial additional experiments. We hope the new evidence and manuscript revisions address your concerns.
Q1: Can a semantic judge replace lexical matching, and how does ABF perform on numerical/Boolean questions?
We also added independent semantic evaluation with a Qwen3-32B judge on all 2,130 evaluation questions, including Boolean questions. ABF remains higher on all four acquisition metrics, now included in Table 2:
Together, these results show that lexical matching provides a useful training signal despite its noise, while semantic supervision offers a trade-off between quality and cost.
Q2: Is ABF's cost advantage preserved when frontier mining is included?
Yes, in sampling tokens, even including all recorded mining. A local update uses 1.509M tokens, versus 1.706M for a paired Outcome update. Training one decision saves enough sampling to offset the mining overhead.
The training curves also show better accuracy at similar sampling budgets: 47.6% validation EM at 312M tokens for ABF, versus 41.9% at 308M for Search-R1 (Figure 5(a)).
We revised the cost description to make the accounting explicit. Figure 5 counts selected frontier prefixes, fresh local continuations, and Outcome rollouts, including model-call inputs and outputs. The 12.1% ratio uses this accounting; 88.4% includes all recorded mining. Appendix E gives both calculations. These are sampling costs, excluding optimizer updates, retrieval, and evaluation.
Q3: Do the gains hold on Qwen2.5 and in the baselines' original settings?
ABF gains 3.62 mean EM and 4.49 mean F1 points, with higher EM on five of six datasets, supporting its benefit beyond Qwen3.5.
IGPO's live Google-search environment is harder to reproduce, so we have not run an original-setting comparison. In our shared setup, we retain its information-gain supervision; Appendix C.1 explains the adaptations.
W1–W3: Addressed in Q1 (lexical reliability and answer types), Q2 (cost), and Q3 (Qwen2.5 and original-setting comparisons).