acceptodds
Under review as a conference paper at ICLR 2027

Show or Stow: Learning Evidence Admission from Stage-Specific Utility in Multi-Agent Search

11 views

Abstract

Agentic search has become a central approach to reasoning-intensive information retrieval. When multiple agents carry it out, the evidence they collect guides further search and supports final-answer generation, but the same evidence can have different utility at these two stages. Existing approaches organize shared evidence or learn what to share without separately learning its value for each use. To mitigate this mismatch between stages, we propose StageAdmit, an evidence admission framework that keeps every collected item in an evidence pool and regresses stage-specific keep-mask utilities to learn separate admission policies for continued search and final-answer generation. A predictor with stage-specific heads decides which evidence enters each search agent's panel during search and which enters the writer's panel for the final answer. The predictor is trained on paired comparisons that keep or mask a piece of evidence, measuring its effect on final-answer quality and subsequent tool calls for continued search, and its marginal gain for final-answer generation. Because all evidence remains in the pool, evidence withheld from search agents can still reach the writer. Experiments on WideSearch and GISA show that StageAdmit outperforms all evaluated baselines, surpassing the strongest baseline by 1.5 and 1.6 Item-F1 points with GLM-5, while reducing completion time by 13.1% and writer input tokens by 36.4%. The same frozen predictor also improves GPT-5.1 and Qwen3.8-27B agents without retraining.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, information seeking, agentic search, evidence selection, stage-specific utility

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_L4RrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes StageAdmit, an evidence admission framework for multi-agent search that distinguishes evidence utility for continued search and final-answer generation. It retains all collected evidence and learns separate admission policies for the two stages using keep–mask comparisons of downstream utility. Experiments on WideSearch and GISA show improvements in answer quality and inference efficiency.

Critical strengths
  • S1: The paper distinguishes evidence utility for continued search from that for final-answer generation, offering a novel perspective on multi-agent search and motivating separate admission policies for the two stages.
  • S2: The results show consistent quality gains across two benchmarks and three backbones through stage-specific evidence admission, alongside reductions in tool calls and writer input.
Critical weaknesses
  • W1: The paper lacks analysis of how predicted utility relates to actual evidence contributions. Figure 5(c,e) shows that Item-F1 declines as positive thresholds increase, but it remains unclear whether this reflects the cumulative loss of many small-gain items or the exclusion of important evidence whose utility is underestimated.
Questions for authors
  • Q1: Can you provide concrete examples of the same evidence item with its measured utilities at both stages, explaining why it helps one stage but not the other? These examples would help interpret the reported stage-specific utility differences.
  • Q2: If a slightly positive threshold reduces admission without affecting Item-F1 in Figure 5(c,e), would it offer a better quality–cost trade-off than zero? Does this suggest that near-zero utilities are systematically overestimated, or are the marginal gains simply too small to be visible in the aggregate results?
Reviewer_SCfsOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

StageAdmit learns which shared evidence to show to workers during execution and which to show to the final writer in multi-agent search. It trains on paired keep/mask continuations, using separate utility targets for the two stages. Execution utility includes answer quality and tool cost; synthesis utility measures an item's marginal contribution to the writer. A lightweight predictor uses shared representations with stage-specific residual heads. Withheld evidence remains in the underlying pool. Experiments on WideSearch and GISA show improvements over unrestricted sharing and other admission rules, including across agent backbones. A separate counterfactual study examines how an item's utility changes between execution and synthesis.

Critical strengths
  1. The paper examines a useful distinction: evidence that helps the final writer may distract workers or lead them to make unnecessary tool calls. The held-out analysis finds frequent disagreements in utility between stages. The appendix checks whether this could simply be measurement noise, using repeated measurements, split-half reliability, and a same-stage reference. The roughly 47% cross-stage opposite-sign rate is substantially higher than the approximately 23% same-stage rate. Including this reference makes the result more convincing.

  2. Tables 1–2 show gains on both benchmarks and several backbones. For GLM, full admission reaches 79.3/76.0, compared with 77.5/74.1 for Share-all and 78.6/75.3 for synthesis-only admission. The resource analysis includes predictor overhead and suggests reductions in writer context and tool use. Keeping withheld items in the underlying pool is also a practical design choice, since evidence can remain available to other recipients.

Critical weaknesses
  1. The evaluation is limited to a specific multi-agent setup. The system has separate search workers and a final writer. It is unclear whether the same benefits would hold for a single agent that alternates between searching and writing.

  2. The ablations do not clearly show why separate stage-specific prediction helps. The shared-head baseline still receives stage information, while using one target for both stages also changes the training labels. These comparisons leave open whether the gains come from distinguishing the stages or from the different supervision used for each.

  3. Deployment reliability and adaptation cost need more evidence. The system admits evidence whenever its predicted benefit is above zero, so small errors near zero can reverse the decision. The paper does not report how often this admits harmful evidence. Collecting labels also requires running continuations with and without an item, but the cost analysis does not show how much usage is needed to offset that expense.

Questions for authors
  1. Which parts of StageAdmit require separate search workers and a final writer? Could the same admission mechanism be used when one agent performs both roles?
  2. Do any existing ablations use the same training examples, targets, and model capacity while removing only stage information? This would help explain whether distinguishing the stages accounts for the gains.
  3. From the existing evaluation logs, how often do admitted items have negative measured utility, especially near the zero threshold? What is the total cost of collecting training labels, and how many search tasks would be needed to offset it through inference savings?
Reviewer_RhjoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work, the author propose StageAdmit, an evidence admission framework that decouples evidence storage from display. In multi-agent search, collected evidence serves two distinct purposes: guiding continued search among parallel workers and supporting final-answer synthesis by a writer. However, existing methods overlook that the same evidence item can have opposite utility across these two stages. The proposed StageAdmit retains all collected items in a global pool while using a lightweight predictor with stage-specific heads, trained on offline keep-mask counterfactual rollouts, to separately govern execution sharing and synthesis selection. On WideSearch and GISA across three backbones, StageAdmit consistently improves accuracy while reducing completion time by 13.1% and writer tokens by 36.4%.

Critical strengths
  1. Well-motivated and previously overlooked problem: The paper identifies an important yet underexplored mismatch in multi-agent search: the utility of an intermediate evidence item depends heavily on the downstream stage that consumes it (continued search vs. final-answer synthesis). Separating evidence storage (in a global pool) from stage-specific display is a clean and intuitive formulation.
  2. Empirical improvements in both quality and efficiency: Across two broad-search benchmarks (WideSearch and GISA) and three LLMs, StageAdmit consistently improves over existing multi-agent search baselines and the controlled Share-all variant, while also reducing tool calls, completion time, and writer input tokens.
Critical weaknesses
  1. The overall F1 gains over SearchOS and Share-all are modest, and it seems like no statistical significance tests are reported in Tables 1 or 2.
  2. Lack of qualitative case studies: Beyond the toy example in Figure 1, there are no real trajectory case studies from WideSearch or GISA. It remains unclear under what specific conditions extra context acts as harmful noise that causes workers to stray or the writer to fail.
  3. StageAdmit requires white-box access to the multi-agent workflow, structured intermediate evidence items, and checkpointable states for keep-mask supervision. I was wondering:

(i) If a system uses black-box sub-agents or unstructured free-form communication without explicit intermediate evidence extraction, does this method could be applied.

(ii) Moreover, all experiments rely on a single custom Coordinator–Worker–Writer harness; it is unclear whether StageAdmit can generalize to different multi-agent harnesses or topologies (e.g., hierarchical trees, peer-to-peer debate, or iterative reflection).

Questions for authors

See my weaknesses above (especially the third point)

Comments on the paper

1 comment
Area_Chair_7Jw8OpenReview ↗

Author-Reviewer Discussion

Dear Reviewers,

The authors have provided the rebuttal. Please have a look and let the authors know if you need further clarification or still have follow-up questions.

Thanks, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.