acceptodds
Under review as a conference paper at ICLR 2027

SEEDS: Preview-Guided Evidence Admission for Search Agents

37 views

Abstract

Search agents extend large reasoning models with multi-round access to external knowledge, yet most existing systems largely conflate retrieving evidence with consuming it. Retrieved documents are commonly incorporated according to fixed or rank-based policies, despite the fact that evidence utility is conditioned on the agent's current reasoning state. Consequently, a lower-ranked candidate may be critical for the current step, while a higher-ranked candidate may provide redundant, irrelevant, or misleading information. We formulate this overlooked problem as evidence admission, which asks, at each search step, which retrieved candidates should be allowed to enter the agent’s evolving reasoning context. To enable such selective control, we introduce SEEDS (Selective Evidence Entry During Search), a preview-guided evidence admission framework for search agents. SEEDS first exposes lightweight previews of retrieved candidates and then lets the agent jointly determine which candidates and how many should be fully inspected, conditioned on its current reasoning state. Unlike fixed or adaptive prefix policies, SEEDS can select arbitrary subsets with flexible cardinality, and it can be instantiated directly with an existing reasoning agent or further optimized through learning. Across nine knowledge-intensive QA benchmarks under both local corpus and live web search, consistently improves search effectiveness while substantially reducing full-evidence consumption. Controlled analyses show that more effectively recovers answer-bearing evidence and frequently selects useful candidates beyond the corresponding retrieval prefix. These results establish evidence admission as a complementary dimension of agentic search, separating what information is retrieved from what information is ultimately allowed to shape reasoning. Anonymized code is available at https://anonymous.4open.science/r/SEEDS-5C1C.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Search, Agent

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_prVnOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SEEDS, a preview-guided evidence admission framework for multi-round search agents. After retrieval, the agent reads lightweight previews and selects an arbitrary subset of candidates for full inspection, conditioned on its current reasoning state. The framework is evaluated with both local retrieval and live web search, with an additional trainable variant based on supervised fine-tuning and reinforcement learning.

Critical strengths
  1. The distinction between retrieving candidates and deciding which ones to inspect is practically meaningful. The framework is simple and can be integrated into existing search agents without additional training.

  2. The experiments cover multiple QA benchmarks and both local and live-web environments. The reduction in full-document consumption is substantial: SEEDS-ReAct reduces average reads from 7.7 to 3.1 relative to ReAct dyn-k while improving average EM from 36.9 to 41.6.

  3. The appendix provides useful implementation details, preview ablations, and an explicit discussion of the limitations of answer occurrence as an evidence-quality proxy.

Critical weaknesses
  1. The core mechanism is not sufficiently validated. The paper emphasizes evidence selection conditioned on the evolving reasoning state. However, ReAct dyn-k determines the reading depth before observing retrieval results, whereas SEEDS first reads all previews and then selects an arbitrary subset. This comparison changes both the available information and the selection mechanism. It remains unclear whether the improvement comes from preview access, non-prefix selection, or reasoning-state conditioning. Please include a preview-aware prefix baseline and a selector using only the question, current query, and previews without the reasoning history.

  2. The main efficiency evaluation is incomplete. The document-token statistics in Table 4 exclude previews, selection reasoning, and other generated outputs. This overhead is potentially substantial because local passages contain approximately 100 words, while each search exposes ten previews of up to 60 tokens each. Although Table 13 includes preview overhead for SEEDS-R1, equivalent accounting is missing for the main QwQ-32B experiments. Please report total inference tokens, including selection and document-refinement calls, together with end-to-end latency, to establish whether fewer full-document reads translate into practical computational savings.

  3. The contribution of evidence expansion is unclear. All candidate previews enter the reasoning context and may already contain sufficient information to answer the question. Appendix E.3 also explicitly accommodates correct answers without expanding the relevant document. Therefore, reduced full-document consumption does not by itself establish the benefit of selecting documents for expansion. Please include a preview-only baseline that retains iterative search but disables expansion, and report how often SEEDS answers correctly without opening any documents. This would distinguish the benefit of compact previews from the additional value of selective expansion.

  4. The learning experiment does not isolate the source of improvement. SEEDS-R1 introduces supervised warm-start training and an auxiliary selection reward based on gold-answer occurrence or supporting-fact annotations. These provide additional supervision beyond the admission interface itself. It is therefore unclear how much of the improvement over Search-R1 comes from the proposed interaction mechanism versus the training recipe. Please compare SFT-only, SFT followed by EM-only PPO, and PPO with the selection reward under matched budgets, and clarify whether the Search-R1 baseline receives comparable demonstrations and evidence-level supervision.

Questions for authors

See Weakness

4 comments

W1: Isolating the Core Mechanism.

R1: We thank the reviewer for this suggestion. To disentangle the contributions of preview access, non-prefix selection, and reasoning-state conditioning, we add two baselines under the same QwQ-32B local-corpus setting:

  • ReAct dyn-k-preview: Observes the same candidate previews as SEEDS but selects only a top-k retrieval prefix.
  • Non-history selector: Selects an evidence subset using only the question, current search query, and candidate previews, without access to the reasoning history.
MethodNQTriviaQAHotpotQABamboogleAvg.#Doc
ReAct dyn-k-preview32.666.632.253.646.32.1
Non-history selector37.867.433.654.448.33.8
SEEDS39.669.635.656.850.43.1

Non-prefix selection and reasoning-state conditioning each contribute. (1) Preview access alone is insufficient: with previews but prefix-only selection, ReAct dyn-k-preview underperforms SEEDS by 4.1 average EM. (2) Reasoning-state conditioning adds gains: SEEDS improves average EM by another 2.1 points over the non-history selector while reading fewer documents (3.1 vs. 3.8). SEEDS achieves the highest EM on all four datasets. We will include these controls in the revised paper.

W2: Computational Efficiency and Overhead.

R2: We extend our efficiency analysis to report total inference tokens, including previews, admission decisions, and document-refinement calls, together with end-to-end latency per question. All values are averages per question as the setting in Table4.

SettingMethodTotal tok.End-to-end latency (s)EM / Acc.
LocalReAct top-324695737.5
LocalReAct dyn-k26016336.9
LocalReAct top-1039537135.3
LocalSEEDS-ReAct35607441.6
Live webSearch-o1619532209.3
Live webSEEDS-o14759917311.8

Local retrieval: the overhead is small. Passages are short, so previews and admission decisions add some cost, but the overall cost of SEEDS-ReAct stays within the range of the ReAct baselines: its total tokens are below ReAct top-10, and its latency is within 3 s of it. At this cost, SEEDS-ReAct achieves the highest EM among all methods.

Live-web search: clear savings. Each page must be fetched and summarized, so skipping pages saves much more than admission costs. Compared with Search-o1, SEEDS-o1 uses 23.2% fewer total tokens and is 21.4% faster, with higher accuracy by 2.5 points.

These results highlight the benefits of selective evidence admission, particularly when full-evidence acquisition is computationally expensive.

W3: Contribution of Evidence Expansion.

R3: Following the reviewer's advice, we conduct two more analyses to separate the contribution of lightweight previews from that of selective evidence expansion.

(1) Preview-only Ablation. We introduce a preview-only baseline that retains iterative search but disables full-document expansion, along with an expand-all variant that expands every candidate. Exact Match (EM) and Avg.#Doc per question are reported.

VariantNQTriviaQAHotpotQABamboogleAvg.#Doc
Preview-only36.666.633.655.248.00
Preview + expand-all39.268.033.052.848.312.1
SEEDS39.669.635.656.850.43.1

Selective expansion improves accuracy beyond preview exposure alone. SEEDS outperforms the preview-only baseline by 2.4 average EM, which shows that expansion adds value beyond what previews provide. SEEDS also outperforms expand-all by 2.1 average EM while reading 74% fewer full documents (3.1 vs. 12.1), which shows that the gain comes from selecting which documents to expand and not from reading more evidence.

(2) Correct Answers Without Expansion. We further examine how often SEEDS-ReAct answers correctly without full-document expansion. Across all six datasets, only 4.5% of all questions are answered correctly without opening any document.

Together, these results show that SEEDS benefits from selective full-evidence expansion and does not rely on preview information alone.

W4: Source of Improvement in the Learning Experiment.

To separate the supervised warm-start, outcome-only RL, and the auxiliary selection reward, we compare SFT-only, SFT followed by EM-only RL, and the full SEEDS-R1 with the selection reward, alongside Search-R1.

MethodNQTriviaQAPopQA2WikiMuSiQueOverall
Search-R143.661.040.028.214.940.5
SEEDS SFT-only40.659.341.033.710.840.8
SEEDS SFT + EM-only RL44.562.543.537.913.044.0
SEEDS-R1 (SFT + RL with selection reward)45.261.844.438.412.744.4

Without the selection reward, the gain remains. Trained with the same EM-only reward as Search-R1, SEEDS SFT + EM-only RL reaches 44.0 overall EM, 3.5 points above Search-R1. Adding the selection reward contributes only a further 0.4 points.

The warm-start alone gives no advantage. SEEDS SFT-only is on par with Search-R1 (40.8 vs. 40.5), so the demonstrations do not by themselves improve accuracy. The improvement therefore arises when outcome-based RL, the same training signal used by Search-R1, is applied to the admission interface.

Regarding the baseline, we reproduce Search-R1 following its original training protocol, which uses only the final-answer EM reward without evidence-level supervision. We totally agree that adding evidence-level supervision would provide a stronger comparison and will include this enhanced Search-R1 baseline in the revised paper.

My issue has been resolved. Thank you for your reply.

Dear Reviewer prVn,

Thank you so much for your encouraging feedback and support! Your thoughtful suggestions have helped us substantially improve the paper. We are truly grateful for your recognition of our work!

Best regards, The Authors

Reviewer_ee4WOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SEEDS, a preview-guided evidence admission framework for multi-round search agents.

Given lightweight previews of retrieved candidates, the agent selects which candidates—and how many—for reasoning, conditioned on its current reasoning state.

SEEDS supports non-prefix selection with flexible cardinality and can be instantiated through prompting or further optimized through learning. Experiments under local-corpus and live-web search, showing improved average answer accuracy and substantially reduced full-evidence consumption.

Critical strengths

Clear problem formulation. The distinction between evidence availability and full-evidence consumption addresses a practical issue in multi-round search: retrieval rank alone may not reflect a candidate’s utility given previously acquired evidence.

Good experimental results. In Table 1, SEEDS-ReAct improves average EM from 36.9 to 41.6 over ReAct dyn-k while reducing document reads from 7.7 to 3.1. SEEDS-o1 improves average EM from 42.5 to 43.3 over Search-o1 while reducing document reads from 30.2 to 12.6.

Critical weaknesses
  1. Insufficient coverage and differentiation of existing search agents undermine the central motivation and novelty claim.

My main concern is that the paper does not adequately account for existing search agents that already separate retrieving candidates from accessing their detailed contents. The abstract and introduction argue that existing systems “largely conflate retrieving evidence with consuming it,” motivating preview-guided evidence admission as an overlooked decision.

However, I think several established search agents already expose lightweight search results and allow subsequent selective browsing, here are some examples:

  • DeepResearcher returns titles, URLs, and snippets through its search tool, while a separate browsing tool accesses pages requested by the agent.
  • Tongyi DeepResearch returns the top-10 titles, snippets, and URLs through Search, and uses Visit to extract information from specified pages according to agent-provided information-seeking goals.
  • MiroThinker separates structured web search through google_search from conditional information extraction at target URLs through scrape_and_extract_info.
  • WebThinker’s Deep Web Explorer receives titles, URLs, and snippets, then reasons about whether to search further or click a promising link for additional information.

These interfaces already implement an important part of the proposed separation between evidence availability, lightweight observation, and detailed evidence consumption. They also permit selective access beyond a fixed retrieval-rank prefix. Although these systems do not necessarily implement SEEDS’s single-step subset-selection action, their capabilities directly challenge the breadth of the claimed limitation.

  1. The evaluation does not resolve this concern. The experiments demonstrate improvements over the evaluated passage-retrieval and document-refinement baselines, while the live-web comparison in Table 2 includes only Search-o1. These comparisons do not establish an advantage over agents already equipped with preview-based search and selective browsing. Consequently, it remains unclear whether the reported gains reflect a distinct contribution beyond existing search–browse mechanisms or primarily the introduction of selective access into baselines that lack it.

The authors should clarify what SEEDS adds beyond this existing methods and evaluate that distinction under different current baselines.

Questions for authors

N/A

4 comments

W1 & W2: Evidence Admission vs. Existing Search–Visit Agents

R: We thank the reviewer for highlighting these related systems. We agree that DeepResearcher, Tongyi DeepResearch, MiroThinker, and WebThinker already support lightweight search previews and selective page access, and acknowledge that our original wording "largely conflate retrieving evidence with consuming it" was too broad.

Based on our examination of their papers and released code, we clarify three key points: (1) the distinction in problem formulation, (2) the differences in agentic workflows, and (3) our revisions and additional experiments. Code references are provided at the end.

(1) Formulation: page access on the live web vs. admission over a retrieved pool

In these systems, the browse or visit tool exists because a web search engine returns only snippets, and the page must be fetched before it can be read. Browse/Visit tools enable selective access to web content, but they do not by themselves define an explicit, pool-scoped evidence-admission decision. According to their original code for the local Wikipedia retrieval setting, they are equivalent to standard ReAct, without any browse or visit operations in the agent loop. That said, browser or visit tools are inherently provided in live web, and therefore, the browse/visit tool is a means of obtaining page content, rather than deciding which retrieved candidates should be consumed.

  • DeepResearcher gives the clearest evidence. On its local Wikipedia corpus (search_engine="rag"), the agent receives only web_search. browse_webpage is added only in the online-search setting [DR].
  • MiroThinker and WebThinker are evaluated only on the live web. MiroThinker v1.0's main agent is configured with web search, scrape_and_extract_info and a Python sandbox [MT]. WebThinker's search supports only Bing or Serper [WT].

SEEDS defines admission as a decision over the retrieved pool, independent of how the admitted content is obtained: a passage lookup on a local corpus (E5 over Wikipedia), or a page fetch on the live web (Serper + Jina). The same decision therefore exists, and is studied, in both settings.

(2) Agentic Workflow Differences: optional Browse/Visit vs. mandatory Admit

In these systems, Browse/Visit tools are provided to access detailed webpage content, but their invocation remains optional within the agentic workflow. Existing agents may visit one or multiple URLs after searching, or proceed without visiting any. In contrast, SEEDS explicitly enforces a Reason → Search → Admit → Observation loop, requiring an admission decision after every search, even when no document is selected. This distinction is illustrated by the workflows below.

DeepResearcher: Reason → web_search([q1..q3]) → {title, URL, snippet}×10 per query
                → [optional] browse_webpage(url_list), |url_list| ≥ 1 → Obs(per-URL content) → …

Tongyi DR:      Reason → search([q1..qn]) → {title, snippet, URL}×10 per query
                → [optional] visit(url | url_list, goal) → Obs(per-URL blocks) → …

MiroThinker:    Reason → google_search(q) → results
                → [optional] scrape_and_extract_info(url, info_to_extract) → Obs(one URL) → …

WebThinker:     Main LRM → Explorer: search → {title, URL, snippet}×10
                → [optional] click_link(url) → Obs(one page) → click_link(url′) / search …
                → Final Information → Main LRM

SEEDS:          Reason → Search(q) → K previews {id, title, snippet}
                → ▶[mandatory] Admit E ⊆ D (one step, E = ∅ allowed)
                → Obs(E) via host agent: passages | per-URL content | per-URL blocks → Reason

The detailed differences between SEEDS and these systems are as follows:

(a) DeepResearcher & Tongyi DeepResearch: optional access vs. an enforced admission round.

Both accept URL lists, so batching is not the difference. The differences are these:

  • Access is optional and unordered. Each turn is either an answer or one tool call, and nothing requires a page-access step after a search [DR, TY]. Which candidates are read is therefore an emergent property of the trajectory. In SEEDS, every search is followed by exactly one admission over its pool, so each round has a defined pool Dₜ and decision Eₜ.
  • There is no explicit empty admission. The access call must name at least one URL: DeepResearcher asserts len(url_list) >= 1 [DR], and Tongyi's Visit schema sets minItems: 1 [TY]. Declining is expressed only by taking another action, which cannot be told apart from "the snippets suffice" or "search more." In SEEDS, E = ∅ is an explicit outcome that leads to query reformulation.
  • The pool is scoped differently. Tongyi's Visit accepts any URL and does not check it against earlier results [TY]. DeepResearcher restricts browsing to the latest search [DR], which makes it the closest prior workflow to ours. However, that search batches up to three queries [DR], so the pool is not tied to a single query, and browsing remains optional.

(b) MiroThinker & WebThinker: sequential one-URL acquisition vs. one joint decision.

  • One URL per step. MiroThinker's scrape_and_extract_info takes a single url: str [MT], and its system prompt permits one tool per message [MT]. WebThinker's Explorer extracts one URL per click_link action [WT]. Reading M pages therefore takes M turns, u₂ is chosen after u₁'s content is already in context. Full-document access decisions are sequential and may be updated after each visit, rather than being committed as a single subset before full-content acquisition.
  • Arbitrary URLs. Both accept any URL. WebThinker also follows links found inside pages [WT], which makes access a navigation step rather than an admission over the retrieved pool.

SEEDS: Explicit and Measurable Evidence Admission. SEEDS jointly determines which subset of the current K candidates should be consumed after each search, with all K previews available and the empty set allowed. In the four systems above, evidence selection arises implicitly from optional browsing actions rather than a standalone admission decision. Making this decision explicit allows the agent to directly assess which evidence is worth consuming, while enabling systematic analysis and optimization of the admission policy. Each SEEDS search round has a defined candidate pool and selection outcome. Our results show that 77.7% / 80.2% of SEEDS-ReAct / SEEDS-o1 rounds depart from the corresponding retrieval-rank prefix, with 44.1% / 39.6% of admitted documents selected beyond that prefix.

Code references

(3) Revisions and additional experiments

Manuscript revisions: We have expanded the Related Work to discuss search agents with separate page-access tools (DeepResearcher, Tongyi DeepResearch, MiroThinker, and WebThinker), including the workflow distinctions outlined above.

Additional baselines: We conduct two sets of experiments to evaluate SEEDS against existing Search–Visit workflows, using matched reasoning backbones, retrievers, and candidate pool sizes within each comparison:

  • SEEDS on search-visit agents (live web): We compare existing search-visit agents with their SEEDS-enhanced variants on HLE in a live web search setting. Specifically, we plug SEEDS's explicit evidence-admission step into each existing backbone.
Agent / SettingAccuracy#Visited URLs
WebThinker-R1-7B9.79.70
SEEDS-WebThinker-R1-7B11.43.07
Tongyi DeepResearch 30B-A3B27.24.32
SEEDS-Tongyi DeepResearch 30B-A3B27.94.73
DeepResearcher-7B9.42.84
SEEDS-DeepResearcher-7B10.72.62
DeepResearcher-Qwen3-30B-A3B7.42.13
SEEDS-DP-Qwen3-30B-A3B8.12.18
  • Search-visit on the local corpus: We implement the search-visit workflow on the local Wikipedia corpus, where search returns the same K previews as SEEDS and a visit tool returns the full passages, under the same backbone and retriever, and compare it with SEEDS. EM / F1 and average #Doc per question are reported.
AgentNQBamboogleHotpotQA#Doc
ReAct k=335.2 / 45.252.8 / 63.932.0 / 44.76.5
ReAct dyn-k33.0 / 43.151.2 / 65.631.6 / 43.86.3
ReAct Search–Visit (|V|=1)35.6 / 45.655.4 / 67.733.0 / 44.31.84
ReAct Search–Visit (|V|≥1)33.6 / 44.753.6 / 64.832.4 / 44.92.62
SEEDS-ReAct39.6 / 51.756.8 / 68.635.6 / 47.23.26

Results show that SEEDS brings additional gains when applied on top of agents that already separate search from evidence access, and that SEEDS-ReAct outperforms both search–visit variants on all three datasets, indicating that a search–visit workflow does not by itself constitute effective evidence admission.

In summary

Browse/Visit tools are introduced to meet the information-acquisition needs of live-web search, where search APIs typically return only titles, snippets, and URLs. However, in these agents, browsing is part of a flexible tool-use policy: the agent is not explicitly required to invoke Browse/Visit after every search, nor to visit a fixed number of pages. In contrast, SEEDS explicitly enforces a Reason → Search → Admit → Observation loop, requiring an admission decision after every search, including the option to admit no candidates. This makes evidence admission a mandatory, standalone decision rather than an optional action within the broader browsing workflow. SEEDS does not claim to introduce the split between search and page access. Its contribution is to formulate admission as an explicit, enforced and measurable decision that applies to both local and live-web search, and to show that this decision matters. Our additional experiments confirm its effectiveness: SEEDS improves all four search-visit agents on the live web and outperforms both search–visit variants on the local corpus.

Dear Reviewer ee4W,

Sorry to bother you. As the rebuttal period is approaching its end, we would like to kindly follow up and ask if you could take a moment to review our rebuttal. We have carefully addressed your concerns with additional explanations and experiments, and would greatly appreciate any further feedback or questions you may have.

Thank you so much for your time and consideration!

Best regards, The Authors

Reviewer_oHywOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Search agents usually read whatever the retriever ranks highest, this paper argues that retrieving evidence and consuming it should be separate decisions. In SEEDS, the agent first sees short previews of all candidates, such as titles and query-matched snippets. It then picks proper subset to read. The method works with prompting alone and can be further trained with SFT and RL. On nine QA benchmarks with local and web search, SEEDS matches or improves accuracy while reading far fewer full documents.

Critical strengths
  1. The method is clear and well motivated. Separating evidence availability from evidence consumption is intuitive, and the paper shows that neither fixed nor adaptive prefix sizes are sufficient.
  2. The method is simple and retriever-agnostic. The preview–admit–expand loop plugs into existing agents without architectural changes and works without training.
  3. Full-document consumption drops consistently across all local and live-web benchmarks, which matters for agents whose context fills up over many search rounds.
Critical weaknesses
  1. The efficiency accounting excludes the overhead SEEDS adds. The reported document tokens omit preview tokens, yet in the local setting each preview is a sizable fraction of a 100-word passage and all ten previews are shown at every search. Including previews, SEEDS-ReAct consumes more retrieved tokens per question than the top-3 and dynamic-prefix ReAct baselines. Each search also costs an extra generation turn, and total tokens and latency are not reported.
  2. SEEDS sees all candidates as previews, whereas the prefix baselines see only the top few, so the improvement may come from wider coverage rather than from the admission decision. Controls such as preview-only, preview plus expand-all, truncated full-pool reading, and a reranker under a matched token budget are absent.
  3. The supporting analyses are weakly designed. "Answer-bearing" is defined by gold-answer string match, and the query-matched preview may already expose the answer, so a high hit rate may only mean the answer was visible in the preview, not that the agent judged utility well. In the robustness study, accuracy increases as noise grows, and answer-bearing documents are always retained using gold knowledge, so it does not show robustness to realistic noise.
Questions for authors
  1. What are the total tokens per question (previews, documents, reasoning, and output) and the wall-clock latency of SEEDS vs. the baselines?
  2. How do preview-only, expand-all, truncated full-pool reading, and reranker baselines perform under a matched token budget?
  3. Can you report mean and standard deviation over multiple runs for the main tables, and explain the HLE F1 drop?
4 comments

W1 & Q1: Computational Efficiency and Overhead.

R1: We thank the reviewer for pointing out this in our efficiency accounting. We report preview tokens, full-document tokens, generated tokens, total tokens, and latency in both local-corpus and live-web settings. All values are averages per question as the setting in Table4.

SettingMethodPrev.Doc.Retrieved (Prev. + Doc.)ReasoningTotal TokensLatency (s)EM / Acc.
LocalReAct top-3-12541254121624695737.5
LocalReAct dyn-k-12671267133526016336.9
LocalReAct top-10-24292429152439537135.3
LocalSEEDS-ReAct13195281847171335607441.6
Live webSearch-o1-41869418698230619532209.3
Live webSEEDS-o19788620959869584759917311.8

Local retrieval: the overhead is small. Passages are short, so previews and admission decisions add some cost, but the overall cost of SEEDS-ReAct stays within the range of the ReAct baselines: its total tokens are below ReAct top-10, and its latency is within 3 s of it. At this cost, SEEDS-ReAct achieves the highest EM among all methods.

Live-web search: clear savings. Each page must be fetched and summarized, so skipping pages saves much more than admission costs. Compared with Search-o1, SEEDS-o1 uses 23.2% fewer total tokens and is 21.4% faster, with higher accuracy.

These results highlight the benefits of selective evidence admission particularly when full-evidence acquisition is computationally expensive.

W2 & Q2: Distinguishing Coverage from Evidence Admission.

R2: We appreciate the reviewer's suggestion to further disentangle the benefits of broader candidate coverage from those of evidence admission. We clarify that ReAct top-10 already accesses all ten retrieved candidates but does not achieve the same performance as SEEDS, suggesting that broader coverage alone is insufficient.

To further isolate the contribution of evidence admission, we conduct additional experiments with preview-only reasoning, expanding all candidates, truncated full-pool reading, and a widely used reranker (BGE-reranker-v2-m3) under the same retrieval setup. Exact Match (EM) metric and # Doc per question consumed are reported.

VariantNQTriviaQAHotpotQABamboogle#Doc
SEEDS39.669.635.656.83.1
Preview-only36.666.633.655.20
Preview + expand-all39.268.033.052.812.1
Truncated full pool (40%)37.865.233.651.213.0
Truncated full pool (60%)39.667.434.051.212.7
Reranker + top-333.265.635.257.65.8
Reranker + dyn-k32.866.234.853.65.9

The ablation results demonstrate the effectiveness of selective evidence expansion beyond broader candidate coverage. SEEDS outperforms preview-only by 2.4 average EM, showing that preview access alone is insufficient, and achieves higher accuracy than expand-all and truncated full-pool variants while reading approximately 4× fewer documents. These results support the contribution of evidence admission beyond simply exposing the agent to more candidates.

The reranker comparison further supports the benefit of explicit evidence admission over relevance-based selection. We additionally employ BGE-reranker-v2-m3 to rerank retrieved candidates before selecting either the top three documents (Reranker + top-3) or a dynamically determined prefix (Reranker + dyn-k). SEEDS outperforms both variants by 2.5–3.6 points in average EM while consuming nearly half as many documents, demonstrating the advantage of explicit evidence admission over conventional relevance-based reranking.

W3: Supporting Analyses and Robustness.

R3: (1) Preview Answer Exposure and Evidence Selection. We acknowledge that previews may directly expose the gold answer. To isolate the effect, we exclude documents whose previews contain the gold-answer string from the analysis and recompute the proportion of selected documents that are answer-bearing.

MetricReAct dyn-kSEEDS-ReAct
Answer-bearing fraction (%)5.712.9

SEEDS-ReAct selects answer-bearing documents at more than twice the rate of ReAct dyn-k even after excluding previews that explicitly reveal the answer. This shows that its advantage in selecting answer-bearing evidence is not solely attributable to direct answer exposure.

(2) Robustness to Retrieval Noise. Our original experiment preserves answer-bearing evidence to isolate distractor robustness from evidence unavailability, consistent with controlled perturbation evaluations [1]. The non-monotonic accuracy trend is also consistent with prior findings that certain retrieval noise can improve RAG performance [2].

To further address the reviewer's concern, we conduct a more challenging experiment that randomly replaces a fraction ρ of the top-10 candidates with lower-ranked documents, without protecting answer-bearing evidence.

MethodCleanρ = 30%ρ = 50%ρ = 80%
SEEDS48.947.848.544.2
ReAct k=344.544.943.441.8
ReAct k=1043.243.641.040.5

SEEDS consistently outperforms both baselines across all perturbation levels, including when answer-bearing evidence may be removed. We will clarify the distinction between answer-preserving distractor perturbations and unrestricted candidate replacement in the revised paper.

[1] Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Data. Yang et al., ACL 2026.

[2] The Power of Noise: Redefining Retrieval for RAG Systems. Cuconasu et al., SIGIR 2024.

Q3: Multi-run Variance and HLE F1 Drop.

(1) Multi-run Variance. We reran SEEDS-ReAct and the baselines with three random seeds and report the mean ± standard deviation of EM below.

MethodRunsNQTriviaQAHotpotQABamboogleAvg.
SEEDS-ReAct339.9 ± 0.669.3 ± 0.935.7 ± 1.558.7 ± 2.650.9 ± 0.5
MAIN-RAG334.5 ± 0.965.8 ± 1.232.3 ± 0.753.9 ± 2.446.6 ± 0.8
ReAct dyn-k333.0 ± 0.265.6 ± 2.032.4 ± 1.754.9 ± 3.646.5 ± 1.2
ReAct k=3334.1 ± 0.964.8 ± 0.932.5 ± 0.853.9 ± 1.246.3 ± 0.2

SEEDS-ReAct achieves the highest mean EM on all four datasets, with a 4.3-point average improvement over the strongest baseline (50.9 vs. 46.6). The variance across seeds is small, and the average improvement over every baseline is statistically significant.

(2) HLE F1 Drop. HLE contains expert-level questions whose answers often involve numerical values, formulas, or specialized expressions with multiple semantically equivalent forms. Token-level F1 is sensitive to such surface-form variations and may underestimate answer correctness. Therefore, we adopt LLM-judged accuracy as the primary evaluation metric, under which SEEDS-o1 improves from 10.34 to 13.79, despite the decrease in token-level F1 (10.60 to 6.66). We will clarify this distinction in the revision.

Thanks for your rebuttal. I will keep the positive score.

Dear Reviewer oHyw,

Thank you so much for your encouraging feedback and support! Your thoughtful suggestions have helped us substantially improve the paper. We are truly grateful for your recognition of our work!

Best regards, The Authors

Comments on the paper

1 comment

We sincerely thank all Reviewers, Area Chairs, and Program Chairs for their time and thoughtful feedback. We have provided detailed point-by-point responses to all concerns raised by Reviewer oHyw\color{#4A6FA5}{\text{oHyw}}, Reviewer ee4W\color{#B5524A}{\text{ee4W}}, and Reviewer prVn\color{#5B8C5A}{\text{prVn}}, supported by extensive additional experiments and clarifications.

We are particularly encouraged by the initial positive ratings from reviewers oHyw\color{#4A6FA5}{\textbf{oHyw}} and prVn\color{#5B8C5A}{\textbf{prVn}}, both of whom also scored our paper 3 in Presentation, Informativeness, and Soundness. Following our rebuttal, both reviewers maintained their positive ratings and confirmed that their concerns had been resolved. Regarding reviewer ee4W\color{#B5524A}{\textbf{ee4W}}'s only concern about the distinction from existing Search–Visit agents, we have provided detailed clarifications of the problem formulation, methodology, and implementation, supported by additional direct experimental comparisons, although we have not yet received further feedback.

We are encouraged by the reviewers' positive recognition of our work:

  • Motivation: A clear, well-motivated, and practically meaningful problem formulation for multi-round search-augmented reasoning that distinguishes evidence availability from consumption and highlights the limitations of existing rank-based evidence selection (Reviewers oHyw\color{#4A6FA5}{\text{oHyw}}, ee4W\color{#B5524A}{\text{ee4W}}, prVn\color{#5B8C5A}{\text{prVn}}).
  • Methodology: A simple, retriever-agnostic framework that supports flexible evidence admission and integrates into existing search agents without architectural changes or additional training (Reviewers oHyw\color{#4A6FA5}{\text{oHyw}}, prVn\color{#5B8C5A}{\text{prVn}}).
  • Evaluation: Strong results across local-corpus and live-web benchmarks, demonstrating improved answer accuracy with substantially reduced full-document consumption (Reviewers oHyw\color{#4A6FA5}{\text{oHyw}}, ee4W\color{#B5524A}{\text{ee4W}}, prVn\color{#5B8C5A}{\text{prVn}}).
  • Presentation: Clear methodological explanations and detailed appendix materials, including implementation details, preview ablations, and discussion of the limitations of evidence-quality proxies (Reviewers oHyw\color{#4A6FA5}{\text{oHyw}}, prVn\color{#5B8C5A}{\text{prVn}}).

In response to the reviewers' feedback, we have conducted additional experiments and revised the manuscript. The key updates are summarized below:

  • Distinction from Existing Search–Visit Agents (ee4W\color{#B5524A}{\text{ee4W}}, W1&2): Building on the reviewer's recognition of evidence admission as a practically meaningful problem, we clarify how SEEDS differs from existing search agents with Browse/Visit tools through problem formulation and workflow analyses of the four mentioned systems. We expanded the Related Work and added experiment results in Tables 9 and 18, showing that a Search–Visit workflow alone does not reproduce SEEDS's effectiveness, while explicit admission further improves existing Search–Visit agents.
  • Evidence Admission Effectiveness (oHyw\color{#4A6FA5}{\text{oHyw}}, W2&3; prVn\color{#5B8C5A}{\text{prVn}}, W1&3): Added controlled baselines in Tables 1 and 8, including preview-aware prefix, history-free, preview-only, expand-all, truncated full-pool, and reranking variants. We also excluded gold-answer exposure in previews, further validating the benefits of non-prefix selection, reasoning-state conditioning, and selective expansion beyond broader candidate coverage.
  • Computational Efficiency and Evaluation Reliability (oHyw\color{#4A6FA5}{\text{oHyw}}, W1&Q3; prVn\color{#5B8C5A}{\text{prVn}}, W2): Added detailed token and latency accounting in Tables 13 and 17, three-run evaluations in Table 15, and clarification of the HLE F1 drop, providing a more complete assessment of computational trade-offs and result consistency.
  • Robustness and Training Analysis (oHyw\color{#4A6FA5}{\text{oHyw}}, W3; prVn\color{#5B8C5A}{\text{prVn}}, W4): Added realistic noise experiments in Table 14 and SFT/EM-only RL ablations in Table 21, demonstrating robustness without protecting answer-bearing evidence and clarifying the contributions of outcome-based learning and selection rewards.

Overall, SEEDS formulates evidence admission as a distinct decision problem in agentic search, determining which retrieved evidence should enter and influence the evolving reasoning trajectory. Our simple, training-free, and general framework improves average EM by +4.7 points with ↓60% full-document reads over adaptive ReAct on local QA benchmarks, while also demonstrating gains in live-web search.

We sincerely appreciate the reviewers' constructive feedback, which has helped strengthen our paper. We hope our additional experiments and clarifications further support the contributions of SEEDS.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.