- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a new harness for agentic search scenarios that abstracts away long evidence contexts into concise observations, and proposes a paired OPSD training method to encourage the base LLMs to familiarize themselves with such a harness. Experiments using Qwen3.6-27B as the base policy across multi-hop and open-domain QA benchmarks show the effectiveness of the harness compared to the Search-R1 harness as well as the OPSD training.
Critical strengths
- The problem formulation is clean in Section 3.1.
- Figures are easy to follow across the paper.
- Experiments consider multiple benchmarks and different base LLMs to show general applicability and effectiveness.
Critical weaknesses
W1 The challenges are not clearly stated. Challenges i) and ii), memory and context degradation and policy errors and failures, are common bottlenecks for long-context scenarios, not specifically tied to agentic search; and iii) the credit-assignment issue is a general and open problem for policy training, as different tasks/policies may require careful, distinct designs to balance correct rewards while avoiding possible hacking issues.
W2 For methodology, my understanding is that the authors propose a new harness that lets an external module manage the context while the policy only cares about the next action conditioned on the abstracted context, together with OPSD training to become familiar with such a harness. That is fair. Yet how the context is compacted and abstracted is unclear or hard to follow, and how it compares with existing agentic context management works like summary, context folding, etc., is unclear.
W3 The biggest drawback of this paper is the empirical validation of the method. I think to show H-Search is effective, there are two aspects: i) the effectiveness of the H-Search harness alone, and ii) the effectiveness of the proposed OPSD in mastering the H-Search harness compared to other learning paradigms. For both conditions, the authors should use the same base policy and identical training/eval configurations for a fair comparison, and report end-to-end QA performance (e.g., answer accuracy) for policy capability, while intermediate evidence recall can serve as an extra metric (currently the paper only reports retrieval recall and leaves downstream answer quality to future work). However, the current experiments in the main paper are very mixed, with mixed-up agent harnesses and different base policies, and only upon reading the Appendix can I find the base policy direct inference scores. At least in the main paper, the authors should clearly provide:
- Base model direct inference, e.g., the standard ReAct workflow;
- Base model direct inference w/ H-Search and w/ Search-R1 harness across the same set of policies to show the general effectiveness of the harness;
- Then for training, under the same base policy and the H-Search harness, compare OPSD, OPD (where the teacher can be a more capable Qwen model), standard GRPO, and offline imitation methods like SFT/BC. Comparing with these in the main text can better show the actual utility of OPSD. I saw the BC-only and GRPO results in Appendix Table 3, GRPO should be more expensive, and that efficiency advantage and training details should be highlighted in the main paper.
Beyond performance alone, different harnesses/training paradigms may shape different agentic behaviors, like the avg. searches, and avg. thinking and output lengths. All of these can help us understand the policy behaviors, what it learns, and how to better improve it, yet all of them are lacking in the paper. I still do not know the actual context length for training/eval in the paper.
W4 Finally, a minor point: in my experience, all these benchmarks can mostly fit within a 32K context, which is quite small and does not align with the long-context bottlenecks repeatedly mentioned in the paper. More challenging agentic benchmarks, like BrowseComp, should definitely be considered to strengthen the evaluation. Also, the harness is limited to corpus search only; I am not sure whether it sacrifices generalization capability across broader agentic tasks.
Questions for authors
Please see my weaknesses
Response to Reviewer uEwn
We thank the reviewers for their thoughtful feedback and respond point by point below.
Weaknesses & Detailed Review
W1: How the three challenges arise in agentic search
We agree that these challenges also arise in other sequential tasks. In agentic search, they have concrete consequences. (i) As new results arrive, the agent must retain evidence from earlier turns because it may be needed to formulate the next query. (ii) The appropriate action depends on the current evidence state: the agent must decide whether to inspect a candidate, search again, or stop. (iii) The same low final recall can result from failing to retrieve, inspect, or retain a relevant document. OPSD provides action-level guidance at the search states visited by the student.
W2: Context compaction and its relation to context folding
Appendix C.1 describes the state representation, and Appendix D provides the prompts.
Document summaries and reading. The harness deterministically extracts query-conditioned, 200-character excerpts after retrieval. The policy invokes
read_documentbased on the question and available evidence when snippets seem insufficient. The tool returns full content when available and the available snippet in snippet-only mode or after full-text retrieval fails. The context-budget threshold is approximately 8,192 tokens.How H-Search compacts context. Each policy input renders the current query plan, selected document IDs, a bounded set of top candidates with read status and brief summaries, remaining information gaps, and recent searches and outcomes, as illustrated in Appendix D’s Working-Memory Representation. Tool actions update this record:
Readmarks candidates as inspected,Selectadds evidence, andJudge Sufficiencydetermines whether to stop or continue. The policy acts on the updated state, and OPSD uses the same representation during training and inference. Context Folding summarizes completed subtask trajectories. H-Search renders the search state needed for the next decision.W3: Controlled comparisons for the harness and OPSD
We will adjust the descriptions of the key controlled comparative experiments—presented in Appendix Tables 5 and 3—within the main text.
Harness effectiveness. Table 5 compares H-Search, FrugalRAG, and CoRAG using the same Qwen3.6-27B checkpoint without task-specific training, achieving macro-average document recall of 76.3%, 64.7%, and 62.3%, respectively. Table 2 compares H-Search with Search-R1 on shared backbones, obtaining 83.1% vs. 72.0% with GLM-5.2 and 71.6% vs. 70.5% with GPT-OSS-120B. These comparisons control for backbone choice.
Training effectiveness. Table 3 fixes the Qwen3.6-27B backbone and H-Search harness. Macro-average recall is 76.3% without training, 77.3% with BC alone, 77.6% with GRPO, and 79.8% with OPSD, showing gains over BC and GRPO under the evaluated setup. Distillation from a stronger teacher would test a separate question.
Inference budgets. We additionally report TriviaQA recall, maximum rounds, and mean token consumption under our experimental configurations.
With thinking disabled and an eight-round limit, H-Search-Lite achieves 73.9% recall, exceeding the strongest baseline, FrugalRAG, by 1.4 percentage points. H-Search-OPSD reaches 79.2% with a larger budget.
To assess the effect of round limits, we reran all four baselines with a maximum of 20 rounds.
All baselines improve, while H-Search-OPSD retains a 5.4-percentage-point lead over FrugalRAG, the strongest baseline. The advantage thus persists on TriviaQA under a common 20-round limit. We will add these results and budget settings to the revision.
W4
Context length. Even within 32K tokens, earlier evidence can become harder to use as unrelated tool outputs accumulate (Liu et al. [1]). Removing structured state reduces average document recall from 76.3% to 46.7% (Table 4), measuring its value on the evaluated tasks.
Generalization. H-Search’s state fields and action constraints target corpus retrieval. Explicit state tracking could transfer to other agentic tasks with task-specific state representations and tools.
[1] Liu, N. F., Lin, K., Hewitt, J., et al. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 2024, 12, 157–173.