acceptodds
Under review as a conference paper at ICLR 2027

SearchPLM: Towards Search-Proficient Language Models

Abstract

Despite recent advances in reasoning and tool use, Large Language Models (LLMs) still struggle with complex information-seeking tasks. Through systematic analysis, we identify a key failure mode in current LLM-based search agents: they tend to issue overly specific queries that closely overlap with the original question and terminate search prematurely after several retrieval failures. We term this phenomenon the Golden Document Fallacy, an implicit assumption that a single document can fully answer the original question , which reduces multi-turn search to a shortcut-like search process. To address this issue, we first collect, identify, and summarize search experience from human experts, and distill it into a general Exploration-Coverage-Synthesis (E-C-S) paradigm, which structures effective search into strategic exploration, iterative evidence coverage, and multi-source synthesis. Based on this paradigm, we develop SearchPLM, a Search-Proficient Language Model trained with strategy-guided supervised distillation and curriculum-based reinforcement alignment. Experiments show that SearchPLM outperforms strong open-source baselines with more parameters, highlighting the importance of search strategy optimization for more generalizable information-seeking tasks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Deep Search, Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_aQj5OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines overly specific queries and premature stopping in LLM search agents, termed the Golden Document Fallacy. It proposes an Exploration–Coverage–Synthesis (E-C-S) strategy and trains SearchPLM through supervised distillation and curriculum GRPO, with BM25-based webpage refinement. Experiments across seven benchmarks report gains over several larger models.

Critical strengths
  1. E-C-S provides concrete guidance for query refinement, failure recovery, and source verification. The prompting and guided-trajectory ablations support its usefulness.

  2. SearchPLM-14B achieves the highest scores on six of seven benchmarks in the shared search environment. RL further improves performance while reducing average searches from 6.64 to 5.75 relative to the SFT model.

Critical weaknesses
  1. Table 10 shows an advantage from E-C-S-guided trajectories after SFT. However, the unguided SFT model is not evaluated after the same RL training, so it is unclear whether this advantage remains in the final model.

  2. The averaging rule in Table 3 is unclear. SearchPLM-14B has an unweighted mean of 58.74 and a sample-weighted mean of 67.51 using Table 12, versus the reported 67.43; the corresponding values for MiroThinker are 52.70 and 59.68, versus 62.03.The summarizer is also listed as DeepSeek-V3 in Section 4.2 and V3.1 in Appendix H.

Questions for authors
  1. How does the unguided-trajectory SFT model in Table 10 perform after the same RL procedure, using identical RL data and training budgets?

  2. Please specify how Avg. is computed, explain or correct the numerical discrepancies, and confirm the summarizer used for all compared models.

3 comments

We thank the reviewer for recognizing the practical value of E-C-S guidance and SearchPLM's benchmark performance. We have added the requested vanilla-SFT+RL comparison and clarified the averaging protocol and summarizer configuration.

W1 & Q1. Vanilla SFT followed by RL.

The additional SFT+RL comparison is now included in Appendix E, Table 16 (lines 848–877) and reproduced below. The SFT-stage comparison appears in Appendix E, Table 14 (lines 837–845). Both models start from Qwen3-14B and undergo SFT followed by the same RL procedure; SearchPLM uses E-C-S-guided SFT.

Both checkpoints use the same RL set of approximately 1,300 questions from the ASearcher training split, after excluding SFT questions and questions with zero accuracy across repeated teacher sampling (§3.3.2, lines 294–300). All RL hyperparameters and the training budget are identical, following Appendix G.2 (lines 908–921).

Training pipelineMuSiQueHotpotQA2WikiBamboogleBrowseCompGAIAWebWalkerMicro Avg.Avg. searches
Vanilla SFT + RL37.1175.2076.9080.406.2548.5351.2560.779.22
SearchPLM (E-C-S SFT + RL)39.8477.8083.8084.0012.1150.4963.1267.515.75

SearchPLM improves all seven benchmark scores and raises Micro Avg. by 6.74 percentage points, while reducing average searches by 37.6%. This supports the value of strategy-guided SFT initialization in the full SFT+RL pipeline.

W2 & Q2. Avg. calculation and summarizer typo.

Thank you for identifying these discrepancies. They arose from an aggregation-script bug and manual transcription errors. All numerical entries in the manuscript tables were entered by hand, and some values were mistyped during this process. The script excluded instances without valid scores from the denominator, including runs that reached the maximum number of search iterations without generating a final answer. Such runs have no final answer to judge and now receive zero. We fixed the script, recomputed Micro Avg. over all 2,163 questions with unscored instances contributing zero, and corrected the transcription errors. Before submitting the final version, we will recheck every table entry and reported aggregate against the underlying evaluation outputs. The submitted Table 3 is now Table 5 (lines 324–348). §4.2 (lines 410–413) defines the scores, and Appendix J, Table 19 (lines 1033–1042) gives the evaluation counts.

SearchPLM-14B. The submitted version reported 67.43%. Our recomputed Micro Avg. in Table 5 (line 348) is 67.51%, which agrees with your sample-weighted calculation from the seven benchmark scores.

Qwen3-14B. Three instances without valid scores now contribute zero. The corrected Micro Avg. is 54.90%, with 36.76% on GAIA and 46.91% on WebWalker (Table 5, lines 333–334). SearchPLM-14B improves over this baseline by 12.61 percentage points.

MiroThinker. Two incomplete GAIA runs were excluded from the original aggregation. Counting them as zero changes GAIA from 43.00% to 42.16% and the pooled score from 62.03% to 61.97%. We also corrected the WebWalker entry to 54.71% (Table 5, line 346). The updated benchmark scores agree with the recomputed Micro Avg., using the evaluation counts in Appendix J, Table 19 (lines 1033–1042).

Corrected results in Table 5 (lines 333–334, 346, and 348).

ModelMuSiQueHotpotQA2WikiBamboogleBrowseCompGAIAWebWalkerMicro Avg.Avg. searches
SearchPLM-14B39.8477.8083.8084.0012.1150.4963.1267.515.75
MiroThinker-v1-30B32.8172.6081.0082.409.7742.1654.7161.977.84
Qwen3-14B24.2267.3073.0078.000.7836.7646.9154.901.49

Webpage summarizer. We corrected the typo in §4.2 (lines 403–408) to DeepSeek-V3.1, which summarized webpages for all models, consistent with Appendix H (lines 953–956). DeepSeek-V3 judged final answers (§4.2, lines 407–413); Kimi-K2.5 and GLM-5 independently re-scored predictions (§5, Table 8, lines 432–438 and 482–485).

The added comparison shows that E-C-S-guided initialization retains its advantage after identical RL training. We appreciate your careful examination of the evaluation details and hope the additional results and protocol clarifications address your main concerns.

Dear Reviewers,

We would appreciate your feedback on our rebuttal, additional experiments, and revised manuscript, particularly whether they address your main concerns. If any questions remain, we would welcome the opportunity to discuss them and provide further clarification during the discussion period.

Your feedback will help us further improve the paper. Thank you again for your time and effort.

AuthorsOpenReview ↗

A Kindly Reminder

Dear Reviewer,

With only one day remaining in the discussion period, we would sincerely appreciate it if you could find time to read our rebuttal and participate in the discussion.

We understand that this is a busy and stressful period for everyone, and we appreciate the time and effort you have already devoted to reviewing our work. In response to your concerns, we have added a controlled intervention, a vanilla-SFT+RL comparison, and fixed-budget evaluations, and incorporated these results into the revised manuscript.

We have not yet received feedback on these additions. Even a brief response indicating whether the new evidence addresses your concerns, and which questions remain, would be very helpful. This would give us an opportunity to clarify unresolved points before the discussion closes.

We remain available for further questions and would greatly value your assessment. Thank you again for your time and consideration.

Reviewer_333TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SearchPLM aims to improve how language models search for information. It proposes an Exploration-Coverage-Synthesis strategy that encourages broad initial searches, query refinement, and evidence synthesis. The strategy guides teacher-generated trajectories for supervised fine-tuning, followed by curriculum-based GRPO. The paper reports improvements across seven QA benchmarks and analyzes changes in search behavior.

Critical strengths
  1. The results are promising. SearchPLM-14B improves the reported average score from 54.98 to 67.43 over Qwen3-14B, with clear gains on GAIA and WebWalker.

  2. The paper includes useful controls for prompting, teacher distillation, curriculum ordering, and BM25 refinement. It also checks the LLM judge against human annotations and other judges.

Critical weaknesses
  1. The novelty is limited. E-C-S organizes familiar practices such as query refinement, decomposition, and cross-source checking. The training pipeline combines teacher distillation, curriculum GRPO, and document filtering. The main proposed insight is the Golden Document Fallacy, but query length, lexical overlap, and retrospective failure labels do not establish this specific explanation. A controlled intervention on failed queries would provide stronger evidence that over-specification causes the failures. As presented, the paper offers a useful recipe but limited new understanding of search behavior.

  2. The benefit of the proposed strategy is not fully isolated. Appendix E compares ordinary and E-C-S-guided distillation and shows gains, but only before RL. A comparison after the same RL training is missing. SearchPLM also uses 5.75 searches on average versus 1.49 for the base model. Table 7 provides useful query-quality evidence, but selects only questions where the base model fails. It therefore does not establish better search quality across the full evaluation set or at a matched cost.

Questions for authors
  1. On a sample of failed trajectories, does replacing an over-specific query with a broader one improve retrieved evidence and final answers under the same remaining search budget? How were the failure labels in Table 2 assigned and checked?

  2. Does E-C-S-guided distillation still outperform ordinary teacher distillation after the same RL training, with matched data size and correctness filtering?

  3. Can you compare accuracy at several fixed search budgets and repeat the useful-query analysis on a randomly sampled test subset? Please also clarify whether the latency and token counts in Table 9 include the external summarizer.

5 comments

We thank the reviewer for recognizing the performance gains and experimental controls. We have added a controlled intervention, a vanilla-SFT+RL baseline, and fixed-budget evaluations to clarify the contribution of the GDF analysis and the value of strategy-guided training.

W1 & Q1. Novelty and evidence for the Golden Document Fallacy.

We disagree with the assessment of limited novelty. Identifying and studying the Golden Document Fallacy (GDF) is a core contribution of this work. GDF characterizes a recurring pattern in which agents seek a single document satisfying multiple evidence requirements, adapt insufficiently to uninformative retrieval, and stop prematurely or answer from incomplete evidence.

To our knowledge, prior work has not systematically characterized this single-document-oriented failure pattern across LLM search agents. We study GDF prevalence, associated search decisions, and response to strategy guidance through failure analysis, cross-model comparisons, and a controlled intervention. These behavioral findings provide the basis for E-C-S.

Appendix B (lines 768–790) reviews query rewriting and iterative retrieval (e.g., Rewrite–Retrieve–Read, RQ-RAG, and IR-CoT) and RL-based search agents (e.g., StepSearch, Search-R1, and ASearcher). In fixed query-rewriting workflows such as Rewrite–Retrieve–Read, the pipeline prescribes the search sequence; optimizing the rewriter leaves strategy orchestration outside the learned policy. SearchPLM jointly optimizes reasoning and search decisions through E-C-S-guided SFT and RL, allowing the model to adapt query formulation, search depth, and evidence synthesis to remaining information gaps. E-C-S provides explicit strategy supervision for these decisions (Appendix B, lines 791–794).

Failure labels and expert agreement. The analysis in §2.3, Table 3 (lines 170–177) covers 245 failed WebWalker trajectories from Qwen2.5-72B-Instruct, the strongest model in our original preliminary evaluation. DeepSeek-V3 assigned initial labels using Appendix L.4 (lines 1227–1248). Three human experts examined the trajectories and checked whether the LLM's attributions were correct, considering query usefulness and substantial differences from human queries at the same search stage. We did not quantify inter-expert agreement for these attribution labels. The reported 199/245 (81.2%) is an observational categorization of the failure sample. §2.4 (lines 182–197) discusses the associated stopping and answering behavior under insufficient evidence.

Additional rebuttal analysis 1: failure attribution across models. We extended the automated trajectory analysis to five additional models from the preliminary evaluation (Appendix O, Table 20, lines 1630–1646):

ModelFailure trajectoriesOver-specific countOver-specific shareMean retrieval failure rate
Qwen2.5-72B-Instruct (original analysis)24519981.2%88.0%
ASearcher-v2-32B28724685.7%85.1%
GLM-4-32B30525683.9%90.3%
QwQ-32B30724479.5%82.0%
Qwen3.5-35B-A3B19615780.1%86.5%
Qwen3.6-27B22917375.5%83.8%

Retrieval failure rates average the within-trajectory proportion of responses flagged as lacking relevant information. Across six models, 75.5–85.7% of the analyzed failure trajectories were labeled as exhibiting over-specific queries, supporting the recurrence of this pattern across instruction-, reasoning-, and search-enhanced models.

Additional rebuttal analysis 2: query length and answer accuracy. In §2.3, Table 2 (lines 159–166), we correlate first-query and mean query length with final accuracy across 17 baseline models, with one observation per model.

Evaluation scopeFirst-query length vs. accuracy (Pearson r)Mean query length vs. accuracy (Pearson r)First-query length vs. accuracy (Spearman ρ)
MuSiQue, GAIA, BrowseComp, and WebWalker (pre-experiment split)−0.28−0.41−0.23
All 7 benchmarks−0.36−0.34−0.36

Results show that longer queries are associated with lower accuracy at the model level in both evaluation scopes.

Q1. Does broadening an over-specific query improve results after a failed search under the same remaining search budget?

Controlled intervention on query constraints. We addressed this question by prompting the original model to relax secondary query constraints: “For your next search, preserve the core entity and information need, and relax or defer secondary constraints when appropriate.” In §2.4, Table 4 (lines 200–215), we paired continuations of 100 randomly sampled Qwen2.5-72B-Instruct WebWalker failure trajectories from identical histories after an uninformative search. The intervention branch receives this one-time guidance, which also allows decomposition and evidence-guided follow-up search; the control continues directly. Qwen2.5-72B-Instruct generates all subsequent queries in both branches, using the same environment and identical remaining budgets from the original 20 search iterations.

ConditionAnswer score (%)Correct / 100Partially correct / 100Avg. additional searches
Control11.06103.97
One-time strategy prompt20.510214.82

Guidance to relax query constraints and adapt subsequent searches improves final-answer scores under the same remaining search budget: 11.0% → 20.5%. The paired gain is 9.5 percentage points (bootstrap 95% CI [3.5, 16.0]; permutation p=0.0041). This estimates the combined effect of the prompt's constraint relaxation, decomposition, and follow-up search guidance.

From GDF analysis to E-C-S. The failure pattern in §2.4 (lines 182–197) motivates the three search decisions in §3.1 (lines 235–256) and Figure 2 (lines 216–230): choose an initial search target, revise a query to address a specific evidence gap, and combine evidence across sources. §3.2 (lines 270–275) explains how the teacher demonstrates these decisions in training trajectories. Appendix E, Table 14 (lines 837–845) compares SFT checkpoints trained on trajectories from the same teacher: E-C-S guidance raises the GAIA score from 42.65 to 45.10 and the WebWalker score from 59.63 to 59.93.

W2 & Q2. Vanilla SFT followed by RL.

The additional SFT+RL comparison is now included in Appendix E, Table 16 (lines 848–877) and reproduced below. The SFT-stage comparison appears in Appendix E, Table 14 (lines 837–845). Both models start from Qwen3-14B and undergo SFT followed by the same RL procedure; SearchPLM uses E-C-S-guided SFT.

Both checkpoints use the same RL set of approximately 1.3k questions from the ASearcher training split, after excluding SFT questions and questions with zero accuracy across repeated teacher sampling (§3.3.2, lines 294–300). All RL hyperparameters and the training budget are identical, following Appendix G.2 (lines 908–921).

Training pipelineMuSiQueHotpotQA2WikiBamboogleBrowseCompGAIAWebWalkerMicro Avg.Avg. searches
Vanilla SFT + RL37.1175.2076.9080.406.2548.5351.2560.779.22
SearchPLM (E-C-S SFT + RL)39.8477.8083.8084.0012.1150.4963.1267.515.75

SearchPLM improves all seven benchmark scores and raises Micro Avg. by 6.74 percentage points, while reducing average searches by 37.6%. This supports the value of strategy-guided SFT initialization in the full SFT+RL pipeline.

W2 & Q3. Fixed search budgets, random-subset query analysis, and cost accounting.

We evaluated both models on the same random sample from seven benchmarks, treating WebWalker's easy, medium, and hard splits as separate subsets: 100 questions per subset, 900 in total (seed=42). The maximum budgets are 3 and 5 search iterations. To extend §5, Table 9 (lines 451–459 and 486–490) beyond baseline failures, we report scores, average searches, and retrieval coverage—the fraction of questions obtaining at least one non-empty search result (§5, Table 10, lines 492–508).

IterationsModelAnswer score (%)Avg. searchesRetrieval coverage (%)
3Qwen3-14B38.891.5367.2
3SearchPLM-14B41.172.6181.3
5Qwen3-14B41.831.5869.9
5SearchPLM-14B48.113.7486.7

Under the same iteration budgets, SearchPLM improves answer scores by 2.28 and 6.28 percentage points and retrieval coverage by 14.1 and 16.8 points. Higher scores and broader coverage support more effective query strategies under comparable search-budget constraints.

Cost accounting and the external summarizer (Appendix D, Table 13, lines 821–832). The updated cost statistics for these fixed-budget runs are:

IterationsModelPolicy-model tokens per questionEnd-to-end inference time (s)
3Qwen3-14B5,60840.2
3SearchPLM-14B3,05625.5
5Qwen3-14B5,29048.5
5SearchPLM-14B5,51035.2

Latency measures end-to-end inference, including model generation, search, and external summarization. Token counts cover the policy model; summarizer tokens were omitted.

The controlled intervention and matched-RL comparison strengthen the link between GDF analysis and learning adaptive search strategies. We hope these results, together with the fixed-budget comparisons and cost clarifications, address your concerns and help inform your reassessment of the contribution.

Comments on the paper

7 comments

We thank the reviewer for the constructive assessment and for recognizing the evidence from our prompting, distillation, and training ablations. We have expanded the failure analysis and experimental controls, with particular attention to annotation procedures, retained supervision, and search cost.

W1 & Q1. Failure sample and annotation protocol.

Failure labels and expert agreement. The analysis in §2.3, Table 3 (lines 170–177) covers 245 failed WebWalker trajectories from Qwen2.5-72B-Instruct, the strongest model in our original preliminary evaluation. DeepSeek-V3 assigned initial labels using Appendix L.4 (lines 1227–1248). Three human experts examined the trajectories and checked whether the LLM's attributions were correct, considering query usefulness and substantial differences from human queries at the same search stage. We did not quantify inter-expert agreement for these attribution labels. The agreement values in §5, Table 8 (lines 432–438) concern final-answer judging. The reported 199/245 (81.2%) is an observational categorization of the failure sample. §2.4 (lines 182–197) discusses the associated stopping and answering behavior under insufficient evidence.

Additional rebuttal analysis 1: failure attribution across models. We extended the automated trajectory analysis to five additional models from the preliminary evaluation (Appendix O, Table 20, lines 1630–1646):

ModelFailure trajectoriesOver-specific countOver-specific shareMean retrieval failure rate
Qwen2.5-72B-Instruct (original analysis)24519981.2%88.0%
ASearcher-v2-32B28724685.7%85.1%
GLM-4-32B30525683.9%90.3%
QwQ-32B30724479.5%82.0%
Qwen3.5-35B-A3B19615780.1%86.5%
Qwen3.6-27B22917375.5%83.8%

Retrieval failure rates average the within-trajectory proportion of responses flagged as lacking relevant information. Across six models, 75.5–85.7% of the analyzed failure trajectories were labeled as exhibiting over-specific queries, supporting the recurrence of this pattern across instruction-, reasoning-, and search-enhanced models.

Additional rebuttal analysis 2: query length and answer accuracy. In §2.3, Table 2 (lines 159–166), we correlate first-query and mean query length with final accuracy across 17 baseline models, with one observation per model.

Evaluation scopeFirst-query length vs. accuracy (Pearson r)Mean query length vs. accuracy (Pearson r)First-query length vs. accuracy (Spearman ρ)
MuSiQue, GAIA, BrowseComp, and WebWalker (pre-experiment split)−0.28−0.41−0.23
All 7 benchmarks−0.36−0.34−0.36

Longer queries are associated with lower accuracy at the model level in both evaluation scopes.

Additional controlled intervention after uninformative retrieval. In §2.4, Table 4 (lines 200–215), we paired continuations of 100 randomly sampled Qwen2.5-72B-Instruct WebWalker failure trajectories from identical histories after an uninformative search. One branch receives a single prompt guiding constraint adjustment, decomposition, and evidence-guided follow-up search. The original model continues both branches in the same environment under a total budget of 20 search iterations, including prior searches.

ConditionAnswer score (%)Correct / 100Partially correct / 100Avg. additional searches
Control11.06103.97
One-time strategy prompt20.510214.82

The intervention raises the average score by 9.5 percentage points (paired bootstrap 95% CI [3.5, 16.0]; paired permutation p=0.0041), demonstrating the benefit of strategy-guided continuation after uninformative retrieval.

From GDF analysis to E-C-S. The failure pattern in §2.4 (lines 182–197) motivates the three search decisions in §3.1 (lines 235–256) and Figure 2 (lines 216–230): choose an initial search target, revise a query to address a specific evidence gap, and combine evidence across sources. §3.2 (lines 270–275) explains how the teacher demonstrates these decisions in training trajectories. Appendix E, Table 14 (lines 837–845) compares SFT checkpoints trained on trajectories from the same teacher: E-C-S guidance raises the GAIA score from 42.65 to 45.10 and the WebWalker score from 59.63 to 59.93.

W2 & Q2. Vanilla SFT followed by RL.

The additional SFT+RL comparison is now included in Appendix E, Table 16 (lines 848–877) and reproduced below. The SFT-stage comparison appears in Appendix E, Table 14 (lines 837–845). Both models start from Qwen3-14B and undergo SFT followed by the same RL procedure; SearchPLM uses E-C-S-guided SFT.

Both checkpoints use the same RL set of approximately 1.3k questions from the ASearcher training split, after excluding SFT questions and questions with zero accuracy across repeated teacher sampling (§3.3.2, lines 294–300). All RL hyperparameters and the training budget are identical, following Appendix G.2 (lines 908–921).

Training pipelineMuSiQueHotpotQA2WikiBamboogleBrowseCompGAIAWebWalkerMicro Avg.Avg. searches
Vanilla SFT + RL37.1175.2076.9080.406.2548.5351.2560.779.22
SearchPLM (E-C-S SFT + RL)39.8477.8083.8084.0012.1150.4963.1267.515.75

SearchPLM improves all seven benchmark scores and raises Micro Avg. by 6.74 percentage points, while reducing average searches by 37.6%. This supports the value of strategy-guided SFT initialization in the full SFT+RL pipeline.

W2 & Q3. Fixed search budgets, random-subset query analysis, and cost accounting.

We evaluated both models on the same random sample from seven benchmarks, treating WebWalker's easy, medium, and hard splits as separate subsets: 100 questions per subset, 900 in total (seed=42). The maximum budgets are 3 and 5 search iterations. To extend §5, Table 9 (lines 451–459 and 486–490) beyond baseline failures, we report scores, average searches, and retrieval coverage—the fraction of questions obtaining at least one non-empty search result (§5, Table 10, lines 492–508).

IterationsModelAnswer score (%)Avg. searchesRetrieval coverage (%)
3Qwen3-14B38.891.5367.2
3SearchPLM-14B41.172.6181.3
5Qwen3-14B41.831.5869.9
5SearchPLM-14B48.113.7486.7

Under the same iteration budgets, SearchPLM improves answer scores by 2.28 and 6.28 percentage points and retrieval coverage by 14.1 and 16.8 points. Higher scores and broader coverage support more effective query strategies under comparable search-budget constraints.

Cost accounting and the external summarizer (Appendix D, Table 13, lines 821–832). The updated cost statistics for these fixed-budget runs are:

IterationsModelPolicy-model tokens per questionEnd-to-end inference time (s)
3Qwen3-14B5,60840.2
3SearchPLM-14B3,05625.5
5Qwen3-14B5,29048.5
5SearchPLM-14B5,51035.2

Latency measures end-to-end inference, including model generation, search, and external summarization. Token counts cover the policy model; summarizer tokens were omitted.

The controlled intervention and matched-RL comparison strengthen the link between GDF analysis and learning adaptive search strategies. We hope these results, together with the fixed-budget comparisons and cost clarifications, address your concerns and help inform your reassessment of the contribution.

We thank all reviewers for their time, careful reading, and constructive suggestions. We have responded to each review and updated the manuscript, with revisions highlighted in red.

Identifying and systematically studying the Golden Document Fallacy is a core contribution of this work. The paper connects this behavioral diagnosis to E-C-S, which teaches models to adapt queries, gather evidence across sources, and allocate search effort through strategy-guided training. The expanded cross-model analysis and query-length correlations further characterize this failure pattern.

Following the reviewers' suggestions, we have added:

  • A controlled intervention on 100 paired failure trajectories, improving answer scores from 11.0% to 20.5% under identical remaining search budgets (§2.4, Table 4, lines 200–215).
  • A vanilla-SFT+RL baseline under the same RL data, hyperparameters, and training budget. E-C-S initialization improves Micro Avg. by 6.74 percentage points and reduces average searches by 37.6% (Appendix E, Table 16, lines 848–877).
  • Fixed-budget comparisons on a shared random sample of 900 questions, showing higher answer scores and retrieval coverage with maximum budgets of three and five search iterations (§5, Table 10, lines 492–508).

We have also clarified the annotation procedures, training controls, averaging protocol, summarizer configuration, and cost accounting, and corrected the numerical reporting errors discussed in our responses.

We would appreciate your feedback on the new experiments and revised manuscript, and welcome further questions and discussion. Your responses will help us address remaining concerns and further improve the work. Thank you again for the time and effort you have devoted to this review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.