acceptodds
Under review as a conference paper at ICLR 2027

ANTMAN: Adaptive Need Tracking for Multi-Agent Navigation in Large Information Spaces

11 views

Abstract

Information-seeking agents increasingly operate over information spaces that are too large to process exhaustively. Yet many multi-agent systems organize computation around static partitions of the available space, causing coordination to grow with how information is segmented rather than with what the query still requires. We introduce ANTMAN, an adaptive coordination framework that treats evolving unresolved information needs as the unit of runtime coordination. ANTMAN maintains a revisable Need Graph that tracks unresolved requirements, accumulated evidence, prior attempts, and search progress, and uses this state to control worker selection, routing, and task-local recovery as new evidence is discovered. By separating the coordination policy from substrate-specific search interfaces, the same need-conditioned mechanism can operate across different information spaces. Experiments across multi-document question answering, controlled long-context scaling, and realistic structured navigation show that ANTMAN remains effective across settings, including when execution is delegated to substantially smaller worker models. Under a increase in searchable context, ANTMAN increases active coordination by only , compared with more than for partition-driven baselines, while preserving strong answer quality.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Systems, Information Seeking, Dynamic Coordination, Retrieval-Augmented Generation, Long-Context Reasoning, Adaptive Computation, LLM Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Sp1UOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ANTMAN, a multi-agent information-seeking framework that maintains a revisable Need Graph to control worker activation, routing, and task-local recovery. Experiments mainly focus on multi-document QA, controlled information-space scaling, repository navigation, and GAIA. The central claim is that coordination can follow unresolved information needs without growing proportionally with the information space.

Critical strengths
  1. The separation between the static substrate map and the evolving Need Graph is clearly motivated.
  2. The scaling results of ANTMAN are promising. From 32K to 512K, active coordination increases from 2.57 to 3.17 workers, while F1 increases from .775 to .840.
  3. The ablations provide evidence that explicit need tracking can help. On GAIA, Full ANTMAN improves accuracy over Graph-free Adaptive while using fewer LLM calls.
Critical weaknesses

The distinction from closely related work is insufficiently established. MindSearch [1] is absent from the related work, Table 1, and experiments. Its WebPlanner incrementally constructs a DAG using search feedback, and adding a node invokes a WebSearcher (Section 2.1). Thus, an evolving graph already controls subsequent worker execution, with workers assigned to sub-questions rather than input partitions. This directly overlaps with ANTMAN’s motivation and claimed distinction from partition-driven coordination.

ANTMAN introduces explicit resolution status, accumulated evidence, attempt history, and progress, together with territory-backed routing and task-local recovery. These mechanisms may be useful, but the paper does not clearly establish their contribution beyond MindSearch’s dynamic graph construction and worker coordination. The comparison with Graph-free Adaptive does not resolve this concern: that variant removes the explicit persistent need representation, so it does not establish an advantage over an existing graph-based method.

Moreover, the controlled information-space scaling experiment compares only against LongAgent and CoA (Table 8), both partition-driven methods. These results support an advantage over partition-driven coordination, but leave the benefit over prior adaptive coordination baselines mentioned in table 1 unclear.

[1] Chen et al. MindSearch: Mimicking Human Minds Elicits Deep AI Searcher. ICLR 2025.

Questions for authors

1.Which Need Graph operations provide capabilities beyond MindSearch’s dynamic graph construction, and can their benefit be demonstrated under comparable execution settings?

  1. Why some baselines compared in table1 are missing?
3 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for the detailed review and for pointing us to MindSearch. In response, we added MindSearch as an external baseline on seven benchmarks, ran a MindSearch-style dynamic-DAG replacement inside ANTMAN, and added KiRAG to the multi-hop QA comparison. We address each point below.

1. Baseline Selection (Q2)

Table 1 is a taxonomy of related approaches. It is not a list of methods we intended to run in every experiment. Each evaluation uses baselines chosen for the question it tests:

EvaluationQuestion TestedBaselines
Multi-hop QAAnswer quality across retrieval and coordination paradigmsSparse and dense retrieval, iterative/structured RAG (ChainRAG, S2G-RAG, KiRAG), agentic search (ReAct), multi-agent methods (LongAgent, CoA), adaptive graph-based coordination (MindSearch)
Controlled Information-Space ScalingWhether active coordination grows with the number of partitions of a large inputLongAgent and CoA, which assign workers to partitions of a long input
Repository Navigation and GAIAInformation seeking in structured repositories and heterogeneous tool environmentsReAct, ReAct+RepoGraph, RepoDistill, OWL, and MindSearch, where applicable

Controlled information-space scaling. This experiment tests whether coordination grows with how a large input is partitioned. We chose LongAgent and CoA because they are the multi-agent methods designed around that setting. We agree that this experiment shows an advantage over partition-driven coordination specifically. It does not by itself establish an advantage over adaptive coordination methods. Section 3 compares ANTMAN with adaptive methods on answer quality and efficiency.

MindSearch. Leaving MindSearch out of the related work and Table 1 was an oversight. The revision will add it to both. MindSearch was originally built on a web search engine, but its WebPlanner/WebSearcher architecture does not depend on that backend.

To compare it under controlled conditions, we made these changes:

  • We replaced its search backend with the same retriever and substrate tools that ANTMAN uses on each benchmark.
  • We ran it with the same GPT-4.1 backbone.

Its planner, graph construction, and searcher logic are otherwise unchanged. Results are in Section 3 (Q1).

KiRAG. KiRAG (ACL 2025) and S2G-RAG (ACL 2026) are both iterative, evidence-guided retrieval methods. Each acquires evidence step by step and conditions later retrieval on what has already been found. They differ in mechanism:

  • KiRAG works through knowledge triples and reasoning-chain alignment.
  • S2G-RAG judges evidence sufficiency and generates structured gap items.

Because of this overlap, we originally included only the more recent S2G-RAG as the representative of this category. We have now run KiRAG with the authors' official pipeline and trained aligner, using GPT-4.1 as the backbone and the same questions and per-question documents as the other methods.

MethodHotpotQA (100)2Wiki (100)MuSiQue (100)Average
ANTMAN.750±.043 / .830.810±.039 / .831.670±.047 / .749.743±.025 / .804
S2G-RAG.640±.048 / .782.710±.045 / .826.590±.049 / .747.647±.028 / .785
KiRAG.620±.049 / .776.570±.050 / .719.550±.050 / .722.580±.029 / .739

Each entry is EM ± SE / F1.

KiRAG's average EM (.580) is close to ChainRAG (.587) and below S2G-RAG (.647) and ANTMAN (.743).

Other methods in Table 1. These methods are related to ANTMAN through adaptive retrieval or multi-agent organization, but none targets the multi-worker information-space setting we evaluate:

  • DRAGIN decides when to retrieve during generation.
  • SelfRACG addresses retrieval-augmented code generation.
  • C-3PO optimizes retriever–LLM interaction with a multi-agent proxy.
  • MAIN-RAG uses agents to filter retrieved documents.
  • DyLAN organizes agent teams.

We list them in Table 1 to position ANTMAN, not as direct competitors in each experiment.

AuthorsOpenReview ↗

Official Comment by Authors

2. What ANTMAN Adds Beyond MindSearch's Dynamic Graph (Q1)

We agree that MindSearch already builds a DAG incrementally from search feedback and invokes a WebSearcher when a node is added. Dynamic graph construction and graph-driven worker execution are therefore not new in ANTMAN. The difference lies in what a node represents and what state is kept about it.

DimensionMindSearch: Sub-questionANTMAN: Information Need
What a node isA question to be investigated through searchAn information requirement that must be met with sufficient evidence
When it is doneWhen its searcher returns an answerWhen the evidence attached to it is judged sufficient
LifecycleExecuted as part of the search plan; its answer informs later planningKept open across search attempts, revisions, and reroutes until it is satisfied
Relation to workersA WebSearcher is invoked for each new sub-questionWorker activation and routing depend on which needs are open and what state they are in

In ANTMAN, running a search and satisfying a need are separate events. A search can finish with only partial evidence. The need then stays active and can be refined, decomposed, or rerouted.

Each need carries three kinds of state that drive coordination:

  • Sufficiency: whether the evidence gathered so far meets the requirement.
  • Readiness: whether the need can run, given its dependencies on other needs.
  • Progress: whether recent attempts on this need have made progress or stalled.

Graph edges also carry separate meanings:

  • Decomposition edges record how a need is broken into sub-needs.
  • Blocking edges determine when a need becomes executable.
  • Semantic edges link related needs without constraining execution order.

A parent need is not closed just because its children are resolved. It receives its own sufficiency check against the accumulated evidence.

These states support specific operations. MindSearch's planner can add new sub-questions after feedback, but it does not maintain the following as explicit operations. The ablations in Table 5 measure what each one contributes:

OperationWhat It DoesDrop When Removed (512K F1 / SWE-QA-Pro / GAIA)
Need revisionRefines, merges, subsumes, or drops needs as evidence changes what is known−11.41% / −7.50% / −29.17%
Adaptive reroutingReassigns a need to a different territory based on its accumulated state−2.18% / −11.42% / 0.00%
Task-local recoveryReframes, reroutes, or falls back on a stalled need without restarting the search−28.45% / −6.03% / −12.50%

GAIA uses a single territory-backed worker, so rerouting has no effect there.

These operations are what separate ANTMAN from a dynamic sub-question graph. Section 3 (Q2) tests this directly by replacing the Need Graph with a sub-question DAG.

3. Empirical Evidence

Q1. Does ANTMAN outperform an existing graph-based adaptive coordination method?

We compared ANTMAN with MindSearch on seven benchmarks, under the matched setup described in Section 1.

BenchmarkQuestionsMetricANTMANMindSearchDifference
HotpotQA100EM / F175.0 ± 4.3 / 83.055.0 ± 5.0 / 70.6+20.0 / +12.4
2WikiMultiHopQA100EM / F181.0 ± 3.9 / 83.154.0 ± 5.0 / 72.9+27.0 / +10.2
MuSiQue100EM / F167.0 ± 4.7 / 74.948.0 ± 5.0 / 69.4+19.0 / +5.5
RepoProbe108Score38.2124.94+13.27
SWE-QA-Pro40 (subset)Score81.4368.69+12.74
GAIA40 (subset)Accuracy (%)60.0 ± 7.745.0 ± 7.9+15.0
BrowseComp-Plus100Accuracy (%)56.0 ± 5.033.0 ± 4.7+23.0

± denotes standard error for EM and accuracy. The SWE-QA-Pro and GAIA comparisons use the 40-question subsets from our ablation study. Full-set ANTMAN results in the main paper are 79.78 and 58.3.

ANTMAN scores higher than MindSearch on all seven benchmarks. MindSearch already builds a dynamic graph and assigns workers to sub-questions, so this is a comparison against an adaptive graph-based method rather than a partition-driven one.

The benchmarks cover different substrates:

  • HotpotQA, 2WikiMultiHopQA, and MuSiQue test multi-hop retrieval and reasoning over documents.
  • RepoProbe and SWE-QA-Pro require navigation across code repositories.
  • GAIA tests multi-step, tool-using information seeking.
  • BrowseComp-Plus contains harder deep-search tasks.
AuthorsOpenReview ↗

Official Comment by Authors

Q2. Do the gains come from need-centric coordination rather than from having a dynamic graph?

Our original Graph-free Adaptive ablation removes the graph entirely. As you note, this cannot show an advantage over an existing graph-based approach. We therefore added a MindSearch-style Dynamic DAG Replacement.

This variant keeps ANTMAN's backbone, tools, workers, and evaluation setup. It replaces the Need Graph with a dynamic sub-question DAG that has three capabilities:

  • It generates sub-questions.
  • It expands dependencies based on execution feedback.
  • It passes the answers of earlier sub-questions to later ones.

This variant is separate from the external MindSearch implementation in Q1.

BenchmarkFull ANTMANDynamic DAG ReplacementDifference
SWE-QA-Pro (40)81.4374.98+6.45
GAIA (40)60.0%55.0%+5.0 points
EfficiencyFull ANTMANDynamic DAG Replacement
SWE-QA-Pro LLM Calls / Query54.383.8
GAIA Inference Cost / Query$0.206$0.294

Both systems build and revise a graph adaptively. What differs is the unit of coordination: the replacement organizes work around search tasks, while ANTMAN tracks information needs and their satisfaction state.

Full ANTMAN is more accurate on both benchmarks. It also uses about 35% fewer calls on SWE-QA-Pro and costs about 30% less per question on GAIA. These subsets are small, so we treat the accuracy differences as supporting evidence; the efficiency differences are larger.

On SWE-QA-Pro, Graph-free Adaptive (78.88) scores between the Dynamic DAG Replacement (74.98) and full ANTMAN (81.43). One plausible reading follows:

  • Graph-free Adaptive replans from accumulated evidence every round, so it cannot get locked into an early, unproductive decomposition.
  • The sub-question DAG can keep directing work along dependencies it committed to early on.
  • ANTMAN keeps a persistent structure but can revise, merge, subsume, or drop needs, which avoids that lock-in.

This ordering suggests that the gain depends on the structure being revisable and need-aware, not on the presence of a graph.

Q3. Do the gains hold against adaptive and iterative methods, not only partition-driven ones?

MethodCategoryAvg. EM ± SE (%)Avg. F1 (%)
ANTMANNeed-driven coordination74.3 ± 2.580.4
S2G-RAGIterative / structured RAG64.7 ± 2.878.5
ChainRAGIterative / structured RAG58.7 ± 2.871.4
KiRAGIterative / knowledge-guided RAG58.0 ± 2.973.9
CoAPartition-driven multi-agent55.3 ± 2.973.0
MindSearchAdaptive graph-based coordination52.3 ± 2.970.9
LongAgentPartition-driven multi-agent35.3 ± 2.857.4

300 questions per method (100 each from HotpotQA, 2WikiMultiHopQA, and MuSiQue).

ANTMAN has the highest average EM and F1. It exceeds MindSearch by 22.0 points in EM and 9.5 in F1, and it also scores above the three iterative retrieval methods (S2G-RAG, ChainRAG, KiRAG). The advantage is therefore not limited to partition-driven baselines.

Thank you again for the careful review. We hope the added comparisons address your concerns, and we would appreciate it if you would reconsider your rating in light of them.

Reviewer_pPRTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors propose a multi-agent coordination systems that work over large information spaces. The system maintains a list of unresolved sub-questions (“Need Graph”), route them to specialized workers, and retry on failure. The information space is partitioned into independent “territories”, each with a “scope-specialized worker”.

The authors claim that the number of active workers scales with the query’s information demand rather than with the size of the information space. This is evaluated on multi-hop QA, a synthetic needle-in-a-haystack scaling setup, two repository benchmarks (RepoProbe, SWE-QA-Pro), and a text-only GAIA subset.

Critical strengths
  • The idea of decoupling “active coordination” (i.e., the number of distinct workers that get involved when solving a question) from the partitioning of the input is well motivated against LongAgent/CoA.

  • The experimental design (fixed demand with growing space and fixed space with growing demand) is sound for testing the claim, even if under-sampled.

  • Small-worker variant (even if keeping a proprietary orchestrator) is a useful ablation.

Critical weaknesses

W1: The authors use MHQA benchmarks such as 2WIKIMHQA and hotpotQA, which are generally solved in a couple of search calls, to evaluate an heavily engineered multi-agent pipeline. BrowseCompPlus, WideSearch, or other DeepResearch benchmarks would be more appropriate. 30 questions per benchmarks is also a thin sample of most of them, and no statistic significance tests are performed. This thin sampling is present in the remaining of the paper, not just the MHQA section.

W2: The OWL paper the authors cite reports 69.70 pass@1 on GAIA under their best configuration, which is much stronger than the value reported in the paper. The authors should better establish their reproduction settings. Several baselines appear weaker than expected, e.g. on GAIA a three-round BM25 loop (45.6) outperforms both ReAct (34.0) and OWL (40.8), and on RepoProbe, ReAct (25.9), ReAct+RepoGraph (25.7) and S2G-RAG (22.9) all score below the Direct baseline (31.1), i.e., below a single LLM call with no repository access. Agentic baselines that lose to inference without tools suggest poor baseline selection or baseline mistuning.

W3: The NeedGraph seems close to the mechanism described in [1], the authors should better differentiate against this and other related graph-based approaches for state tracking.

W4: Latency is not addressed in the paper. Furthermore, the paper assumes full sequential resolution of needs, but if the workers are independent, parallelism on “ready” needs could be explored to lower said latency.

W5: Personally found the paper hard to parse, with heavy notation and verbose descriptions for a relatively simple method.

For example, “Box 1” contains a true-by-construction argument (if each need can be dispatched to a worker only a bounded number of times, then the number of workers you activate can never exceed the number of needs times that bound, nor the total number of workers that exist), yet Appendix A devotes a definitions section, a proposition with a line-by-line proof, and a corollary to deriving it. In contrast, the experiments that support the main claim ( Section 4.2.1) are thin (10 questions, 3 positions per context length, without reporting how many needs the orchestrator generates).


[1] “MINDSEARCH: MIMICKING HUMAN MINDS ELICIT DEEP AI SEARCHER”, Chen et al, ICLR 2025

Questions for authors

The authors coordination metric measures how many agents are used to solve the question. From Table 7 it seems that the cost grows 6.90x from 32K to 512K while calls grow only 1.21x, implying work is now on larger per-worker contexts rather than being decoupled from the space. Since workers use a simple retrieval system, I'd assume the LLM call should see the same number of tokens irrespective of territory size, i.e., as it would be for a plain retrieve-then-read baseline. The per-call growth of the baselines does not show this pattern (their per-call cost stays flat). Could the auhors clarify/justify the the cost growth?

6 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for the careful review. We are glad you found the motivation for separating active coordination from information-space partitioning convincing, and that you noted the controlled experimental design and the small-worker ablation. Your comments led us to extend the evaluation and to clarify ANTMAN's implementation, cost, and presentation. We respond to each point below.

W1. Evaluation Scale, Statistical Reliability, and Task Difficulty

We agree that the evaluation needed more questions and harder tasks. We made three changes: we enlarged the multi-hop QA evaluation, added question-level statistical tests for controlled scaling, and ran BrowseComp-Plus as you suggested.

Larger multi-hop QA evaluation. HotpotQA, 2WikiMultiHopQA, and MuSiQue now have 100 questions each, for 300 questions per method. Each entry is EM ± SE / F1 ± SE.

MethodHotpotQA (100)2Wiki (100)MuSiQue (100)Average
ANTMAN.750±.043 / .830±.035.810±.039 / .831±.037.670±.047 / .749±.041.743±.025 / .804±.022
ANTMAN-H.760±.043 / .838±.035.750±.043 / .783±.041.650±.046 / .744±.039.720±.026 / .788±.022
S2G-RAG.640±.048 / .782±.036.710±.045 / .826±.032.590±.049 / .747±.036.647±.028 / .785±.020
ChainRAG.610±.049 / .737±.039.660±.047 / .766±.036.490±.050 / .639±.043.587±.028 / .714±.023
KiRAG.620±.049 / .776±.035.570±.050 / .719±.038.550±.050 / .722±.037.580±.029 / .739±.021
CoA.580±.050 / .769±.035.610±.049 / .757±.037.470±.050 / .664±.040.553±.029 / .730±.022
MindSearch.550±.050 / .706±.040.540±.050 / .729±.036.480±.050 / .694±.037.523±.029 / .709±.022
ReAct.470±.050 / .641±.043.590±.049 / .727±.042.500±.050 / .680±.038.520±.029 / .683±.024
Sparse.480±.050 / .675±.037.390±.049 / .536±.043.360±.048 / .528±.044.410±.028 / .580±.024
Dense.510±.050 / .678±.040.340±.047 / .481±.043.220±.042 / .355±.043.357±.027 / .505±.024
LongAgent.400±.049 / .604±.040.370±.048 / .611±.037.290±.045 / .507±.038.353±.027 / .574±.022

KiRAG was run with the authors' official pipeline and trained aligner, using GPT-4.1 as the shared backbone and the same questions and per-question documents as the other methods.

ANTMAN has the best average EM (0.743 ± 0.025) and F1 (0.804 ± 0.022). ANTMAN-H also beats S2G-RAG, the strongest baseline on these averages, in both average EM and F1.

BrowseComp-Plus. Multi-hop QA alone does not capture how hard complex information seeking can be. Following your recommendation, we evaluated 100 BrowseComp-Plus questions, with the same question set for all six methods.

MethodAccuracy ± SE (%)Accuracy 95% CI (%)Evidence Recall ± SEGold Recall ± SE
ANTMAN56.0 ± 5.0[46.2, 65.3]0.547 ± 0.0380.647 ± 0.046
Matched ReAct41.0 ± 4.9[31.9, 50.8]0.263 ± 0.0320.330 ± 0.041
MindSearch33.0 ± 4.7[24.6, 42.7]0.324 ± 0.0390.356 ± 0.046
S2G-RAG31.0 ± 4.6[22.8, 40.6]0.212 ± 0.0290.296 ± 0.041
Dense Retrieval16.0 ± 3.7[10.1, 24.4]0.160 ± 0.0220.228 ± 0.035
Direct1.0 ± 1.0[0.2, 5.4]0.000 ± 0.0000.000 ± 0.000

ANTMAN's accuracy is 15 points higher than Matched ReAct, 23 points higher than MindSearch, and 25 points higher than S2G-RAG. All three differences are significant in paired comparisons (p ≤ 0.014). ANTMAN also has the highest evidence recall and gold recall. Against the same three methods, the paired 95% confidence intervals for the gains in both metrics do not include zero.

The two evaluations test different things. The MHQA results measure answer quality on established multi-hop benchmarks. BrowseComp-Plus extends the comparison to harder information-seeking tasks.

Statistical reliability of controlled scaling. We ran paired tests at the question level. Questions are the independent units, and the repeated evidence-position and context-length conditions within each question are accounted for. All four comparisons stay significant after Holm correction:

Scaling ComparisonAgainst LongAgentAgainst CoA
Inference-cost Growthadjusted p = 0.031adjusted p = 0.031
Active-worker Growthadjusted p = 0.012adjusted p = 0.012

These tests support the observed differences in inference-cost growth and active-worker growth under matched conditions. W5 gives more detail on the controlled design and reports the number of needs generated.

The small-worker configuration. Thank you for the positive comment. ANTMAN-H keeps the GPT-4.1 orchestrator and coordination policy fixed and replaces only the worker models with Qwen3-8B. It tests whether performance holds up when weaker workers execute scoped subtasks under the same coordination framework.

AuthorsOpenReview ↗

Official Comment by Authors

W2. Baseline Configurations and Performance

Thank you for raising baseline comparability. All methods run on the same GPT-4.1 backbone in a single experimental environment, with the same tool interface and the same per-query LLM-call budget within each benchmark. Our numbers can therefore differ from those in the original papers, which often used other models, tools, or evaluation settings. We address baseline selection first, then each case you raised.

Baseline selection. For the structured-navigation benchmarks, we chose baselines to cover each way of accessing the information space:

  • No access: Direct, a single LLM call without tools, which shows how much the model can answer from the question alone.
  • Retrieval: iterative BM25 and dense retrieval, which read retrieved content without multi-step exploration.
  • Iterative, evidence-guided retrieval: S2G-RAG, the most recent method that judges evidence sufficiency and retrieves for the remaining gaps.
  • General tool-using agents: ReAct, and OWL on GAIA as a multi-agent framework with hierarchical planning and specialized workers.
  • Repository-specific agents: ReAct+RepoGraph and RepoDistill, which add code-structure navigation or learned context compression.

We include Direct and iterative BM25 deliberately as simple reference points. That they remain competitive on some benchmarks is informative, showing that these tasks are not solved by more tool use alone.

OWL. The 69.70% you cite is WORKFORCE with Claude-3.7-Sonnet on the full GAIA validation set, measured with pass@1. The same paper reports 60.61% for WORKFORCE with GPT-4o under pass@3. The original framework also includes document-processing and multimodal toolkits in addition to web and coding tools.

We used the official OWL release with its default role configuration and prompts, changing only the backbone and toolkits. Our 40.8% comes from running it with the shared GPT-4.1 backbone on GAIA-Text-103, using the same text-oriented tool interface as all other methods. In the revision, we will label this result as a matched-backbone, text-only adaptation and document every difference from the original setup.

GAIA. On GAIA-Text-103, the three-round BM25 baseline reaches 45.6%, ReAct 34.0%, and OWL 40.8%. All three use the shared GPT-4.1 backbone and the same tool interface. The revised paper will state this configuration explicitly.

RepoProbe: why repository-reading baselines score below Direct. The same ReAct implementation, backbone, and repository tools beat Direct by 9.18 points on SWE-QA-Pro (64.92 vs. 55.74), so the implementation itself works. To understand why the order flips on RepoProbe, we took ReAct and ReAct+RepoGraph as representative examples, split each method's score into RepoProbe's Knowledge (/9) and Clarity (/1) components, and identified answers flagged for citing file paths that do not exist in the repository.

MethodFlaggedKnowledgeClarityScoreScore (flagged / unflagged)
Direct2%2.440.6731.085.0 / 31.6
ReAct57%2.190.3425.8621.3 / 32.0
ReAct + RepoGraph56%2.220.3325.7118.4 / 35.2
MindSearch39%2.140.3324.9415.3 / 31.1
ANTMAN39%3.400.4038.2132.1 / 42.1

Direct answers in general terms, rarely names files, and is flagged on only 2 of 108 questions. Methods that read the repository cite specific files and sometimes cite ones that do not exist, which matches the Hallucination failure mode in RepoProbe's own taxonomy. RepoProbe's Clarity item penalizes such hallucinations.

This accounts for most of the gap. ReAct and ReAct+RepoGraph trail Direct by about 5 points, mainly in Clarity (0.34 and 0.33 vs. 0.67); their Knowledge scores are close to Direct's (2.19 and 2.22 vs. 2.44). On the questions where they are not flagged, they score 32.0 and 35.2, compared with 31.6 for Direct. These baselines work as intended; they fall below Direct because they hallucinate file paths more often, not because they are mistuned.

Among the methods that read the repository, ANTMAN is flagged least often (39%, tied with MindSearch). We will still report path hallucination as a limitation. ANTMAN's advantage comes from Knowledge: 3.40, compared with at most 2.44 for every other method. It also scores highest on both flagged questions (32.1, vs. 21.3 for the next best) and unflagged questions (42.1, vs. 35.2). We will include this decomposition for all RepoProbe baselines in the revision.

AuthorsOpenReview ↗

Official Comment by Authors

W3. Distinction from MindSearch and Graph-based State Tracking

Thank you for pointing out the connection to MindSearch. We agree that MindSearch already builds DAG and uses search feedback to steer later execution. ANTMAN's contribution is therefore not DAG construction itself. It is a need-conditioned coordination mechanism: ANTMAN keeps information requirements as persistent objects and uses their changing states to decide evidence acquisition, worker activation, routing, and recovery.

MindSearch's WebPlanner grows a DAG of search subquestions step by step, guided by search feedback. ANTMAN instead tracks what is already known, what evidence is still missing, whether a need is resolved well enough, whether its dependencies allow it to run, and whether earlier attempts made progress.

These states drive runtime behavior:

  • Evidence sufficiency and revision: A need that is only partly supported stays open for refinement, further investigation, or decomposition.
  • Dependency-aware execution: Resolution, execution, and progress are tracked separately, so the system can tell apart needs that are partly answered, blocked, or ready but stuck.
  • State-conditioned routing and recovery: The accumulated need state determines worker selection, territory routing, rerouting, and task-local recovery.
  • Hierarchical resolution: After child needs are resolved, the parent need goes through its own sufficiency check before it is closed.

We added two comparisons: MindSearch as an external baseline, and an internal ablation that replaces the Need Graph with a dynamic subquestion DAG while keeping the same workers and tool environment. The replacement DAG can expand based on search feedback and pass parent answers to later subquestions.

EvaluationFull ANTMANDynamic DAG ReplacementMindSearch
SWE-QA-Pro (40 questions)81.4374.9868.69
GAIA (40 questions)60.0%55.0%45.0%

Full ANTMAN beats the dynamic DAG replacement by 6.45 points on SWE-QA-Pro and 5 percentage points on GAIA.

It is also more efficient to run:

MetricFull ANTMANDynamic DAG Replacement
SWE-QA-Pro Calls / Query54.383.8
GAIA Inference Cost / Query$0.206$0.294

That is about 35% fewer calls on SWE-QA-Pro and 30% lower inference cost on GAIA.

The external comparison holds in other settings as well. On RepoProbe, ANTMAN scores 38.21 and MindSearch 24.94. On MHQA, average EM/F1 is 0.743/0.804 for ANTMAN and 0.523/0.709 for MindSearch.

These results suggest that the gains come from the persistent need representation and the operations built on it, not from using a dynamic graph alone.

W4. Latency and Parallelism

Thank you for raising this; the original submission did not report latency. We now give mean wall-clock latency per execution in the controlled scaling setting, with 30 runs per context length.

Method32K64K128K512K32K → 512K
ANTMAN38.5s41.7s41.6s55.8s1.45×
LongAgent9.7s15.0s25.2s86.1s8.91×
CoA12.1s20.8s39.2s177.0s14.67×

Under these configurations, ANTMAN is slower at 32K–128K and faster at 512K. Much of its latency is sequential coordination: about 3.5–4 rounds of planning, resolution checking, and graph consolidation, followed by final synthesis.

This overhead changes little across the space sizes we tested, while the baselines' execution grows with the number of partitions. Our LongAgent runs execute members in parallel with concurrency 4, whereas ANTMAN and CoA run sequentially.

As you suggested, we analyzed parallel scheduling of ready needs offline. We computed critical-path latency from existing trajectories and calibrated the estimates with nine reruns that record exact per-call timing. Rounds cannot overlap, because each round's plan depends on the previous round's results.

The comparison below uses the matched q0–4 subset with 15 runs per context length:

Mean Seconds / Run32K64K128K512K
ANTMAN, Sequential, Measured45.351.444.753.3
Parallel Needs and Workers, Optimistic Estimate35.8–37.440.0–41.937.5–38.842.1–45.6
Coordinator Chain Only, All Worker Time = 022.8–25.425.0–28.422.9–26.225.1–32.2
LongAgent, Measured11.215.625.688.5

Parallelizing ready needs and workers would cut latency by only 13–22%, since most rounds have one executable need and most routing decisions dispatch one worker. Even with zero worker time, the coordinator alone is 2× slower than LongAgent at 32K.

Closing the small-space gap requires fewer sequential coordinator rounds, not more parallelism. We will add these results and keep measured and estimated latency separate.

Reviewer_VE7wOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose ANTMAN, which is a multi-agent framework for searching large information space efficiently. The primary statement in this paper is that the multi-agent coordination should depend on what information a query still requires, rather than how many partitions exist in the underlying document collection. Traditional partition-based method such as LongAgent and Chain of Agents may activate increasing workers when the information space becomes larger. ANTMAN maintains a runtime Need Graph that records query progress, histories, etc. Through experiments, it is proved to improve efficiency and performance on multi-document question answering.

Critical strengths
  1. A practical approach to scalable multi-agent coordination. This paper studies an important inefficiency in large-scale information-seeking system. This paper separates the static information space organization and the runtime Need Graph. This allows the system to adjust worker participation without requiring all workers to be active.
  2. Comprehensive experiments. This paper evaluates ANTMAN on multiple settings, including HotpotQA, 2WikiMultiHopQA, MuSiQue, repository navigation, and GAIA. It also evaluate smaller worker models and test the effects of disabling need revision, adaptive rerouting, and recovery.
Critical weaknesses
  1. The end-to-end computational efficiency is not fully established. although ANTMAN's active worker and LLM call increase by 1.2x, the cost growth 6.9x when the information space increases from 32k to 512k (Table 7). This suggests the computational cost is not independent of information-space size. It would be benificial to show a more complete end-to-end analysis to understand WorkerCard construction, retrieval overhead, token consumption, and latency, rather than relying primarily on active worker counts.
  2. Limited evaluation scale. The experimental settings are diverse, but some evaluations are relatively small. The multi-document QA evaluation uses only 30 questions per benchmark, while the controlled information-space scaling experiment uses 10 underlying questions at three evidence positions. Larger evaluation sets and more challenging multi-worker information spaces would strengthen the paper's generalization claims.
Questions for authors
  1. End-to-end computational efficiency: Although ANTMAN's active worker count and LLM calls increase by only around 1.2×, inference cost increases by 6.9× as the information space grows from 32K to 512K. Could the authors provide a more detailed breakdown to better demonstrate the scalability benefits?
  2. Evaluation scale and statistical reliability: The multi-document QA experiments use only 30 questions per benchmark, while the controlled scaling experiments involve just 10 underlying questions. Is it possible to construct larger datasets and report statistical uncertainty to verify the robustness of the observed improvements?
2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We sincerely thank the reviewer for the thoughtful, insightful, and constructive comments, and for recognizing the motivation, practical value, and experimental breadth of ANTMAN. We especially appreciate the reviewer's positive assessment of our separation between static information-space organization and dynamic need-driven coordination. The reviewer's suggestions regarding end-to-end efficiency and statistical robustness have been particularly valuable in guiding our additional analyses and expanded experiments. We are grateful for this opportunity to further strengthen our empirical evidence and clarify the scalability benefits of ANTMAN.

1. End-to-End Computational Efficiency

We thank the reviewer for emphasizing the importance of end-to-end computational efficiency. We therefore extend our controlled scaling analysis to include inference cost and wall-clock latency.

When the information space expands by 16× (32K → 512K), we observe:

Metric (32K → 512K)ANTMANLongAgentCoA
Active Worker Growth1.23×15.29×15.33×
LLM Call Growth1.21×12.81×11.30×
Inference Cost Growth6.90×14.22×16.11×
Wall-clock Latency Growth1.45×8.91×14.67×

These results show that ANTMAN's lower coordination growth also translates into more favorable inference-cost and latency scaling under the evaluated settings.

To explain the difference between LLM-call growth (1.21×) and inference-cost growth (6.90×), we further examine token consumption. Median tokens per LLM call remain stable (722 → 715), as do median tokens per execution (16.5K → 14.3K). The larger increase in average cost is associated with a minority of expensive fallback recovery executions.

Importantly, fallback is a configurable recovery mechanism, not a prerequisite for need-driven coordination. To isolate its impact, we additionally evaluate ANTMAN with fallback disabled before execution under the same controlled conditions.

ANTMAN Without Fallback32K512KGrowth
Active Coordination2.572.771.08×
LLM Calls27.526.40.96×
Inference Cost / Query$0.077$0.0650.84×
EM0.6000.667—
F10.7750.818—

Without fallback, coordination and inference cost remain stable across the 16× information-space expansion, while answer quality remains strong. This demonstrates that ANTMAN's core need-driven coordination can operate efficiently without fallback escalation.

The full configuration retains broader recovery capabilities, while the no-fallback configuration offers tighter computational control. Both preserve the same underlying coordination mechanism.

2. Expanded Evaluation and Statistical Robustness

We sincerely appreciate the reviewer's suggestion to strengthen our evaluation through larger sample sizes and statistical uncertainty estimates. In response, we expanded HotpotQA, 2WikiMultiHopQA, and MuSiQue from 30 to 100 questions each, resulting in 300 evaluation questions per method.

Expanded multi-hop QA results. The following table reports the expanded results, including uncertainty estimates. Each entry presents EM ± uncertainty / F1 ± uncertainty.

MethodHotpotQA (100)2Wiki (100)MuSiQue (100)Average
ANTMAN.750±.043 / .830±.035.810±.039 / .831±.037.670±.047 / .749±.041.743±.025 / .804±.022
ANTMAN-H.760±.043 / .838±.035.750±.043 / .783±.041.650±.046 / .744±.039.720±.026 / .788±.022
S2G-RAG.640±.048 / .782±.036.710±.045 / .826±.032.590±.049 / .747±.036.647±.028 / .785±.020
ChainRAG.610±.049 / .737±.039.660±.047 / .766±.036.490±.050 / .639±.043.587±.028 / .714±.023
CoA.580±.050 / .769±.035.610±.049 / .757±.037.470±.050 / .664±.040.553±.029 / .730±.022
MindSearch.550±.050 / .706±.040.540±.050 / .729±.036.480±.050 / .694±.037.523±.029 / .709±.022
ReAct.470±.050 / .641±.043.590±.049 / .727±.042.500±.050 / .680±.038.520±.029 / .683±.024
Sparse.480±.050 / .675±.037.390±.049 / .536±.043.360±.048 / .528±.044.410±.028 / .580±.024
Dense.510±.050 / .678±.040.340±.047 / .481±.043.220±.042 / .355±.043.357±.027 / .505±.024
LongAgent.400±.049 / .604±.040.370±.048 / .611±.037.290±.045 / .507±.038.353±.027 / .574±.022

The expanded evaluation shows that ANTMAN achieves the highest average EM (0.743 ± 0.025) and F1 (0.804 ± 0.022), while ANTMAN-H achieves 0.720 ± 0.026 EM and 0.788 ± 0.022 F1. Both outperform S2G-RAG, the strongest competing baseline in average EM and F1. These results further support the effectiveness of ANTMAN across multiple multi-hop reasoning benchmarks.

AuthorsOpenReview ↗

Official Comment by Authors

Statistical robustness of controlled scaling. Our controlled scaling evaluation comprises 120 distinct experimental runs per method, covering 10 independent questions × 3 evidence positions × 4 information-space sizes. Evidence positions and space sizes are repeated conditions for the same questions, rather than independent samples. This design allows us to evaluate coordination scalability across different information-space sizes and evidence placements.

To further establish statistical reliability, we conducted question-level paired statistical tests with Holm correction, treating the 10 questions as the independent units. Importantly, all four key comparisons are statistically significant (adjusted p < 0.05):

  • Inference cost growth: Holm-adjusted p = 0.031 against both LongAgent and CoA.
  • Active worker growth: Holm-adjusted p = 0.012 against both LongAgent and CoA.

These results provide statistically significant evidence that ANTMAN exhibits substantially slower growth in inference cost and active worker activation than both partition-driven baselines. Thus, our scalability advantages are also supported by question-level paired comparisons, rather than aggregate averages alone.

3. Further Evaluation on BrowseComp-Plus

To further assess generalization to challenging information-seeking tasks, we further provide results on 100 BrowseComp-Plus questions, evaluated using the same question set across all six methods.

MethodAccuracy ± SE (%)Accuracy 95% CI (%)Evidence Recall ± SEGold Recall ± SE
ANTMAN56.0 ± 5.0[46.2, 65.3]0.547 ± 0.0380.647 ± 0.046
Matched ReAct41.0 ± 4.9[31.9, 50.8]0.263 ± 0.0320.330 ± 0.041
MindSearch33.0 ± 4.7[24.6, 42.7]0.324 ± 0.0390.356 ± 0.046
S2G-RAG31.0 ± 4.6[22.8, 40.6]0.212 ± 0.0290.296 ± 0.041
Dense Retrieval16.0 ± 3.7[10.1, 24.4]0.160 ± 0.0220.228 ± 0.035
Direct1.0 ± 1.0[0.2, 5.4]0.000 ± 0.0000.000 ± 0.000

ANTMAN achieves the highest accuracy, evidence recall, and gold recall across all six methods. Its accuracy improvements over Matched ReAct (+15.0 percentage points), MindSearch (+23.0 points), and S2G-RAG (+25.0 points) are all statistically significant in paired comparisons (p ≤ 0.014). The paired 95% confidence intervals for improvements in both evidence recall and gold recall also consistently exclude zero.

These additional results further demonstrate that ANTMAN's need-driven coordination improves both answer accuracy and evidence recovery on challenging information-seeking tasks, extending the empirical evidence beyond conventional multi-hop QA.

We sincerely thank the reviewer for their support and constructive suggestions. Your feedback has helped us strengthen the evaluation and clarify ANTMAN's efficiency and scalability.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Official Response by Authors

Overall response

We thank for their careful, constructive feedback. We summarize our main responses below; details are in each reviewer's thread.

ConcernRaised byResponse
Overlap with MindSearchSp1U, pPRTMindSearch on 7 benchmarks; dynamic-DAG ablation
Small samples, no significance testsVE7w, pPRTMHQA 30→100/benchmark; paired tests with Holm correction
MHQA too easypPRTBrowseComp-Plus (100 q), significant gains
Baselines weaker than expectedpPRTOWL setup documented; RepoProbe gap traced to path hallucination
Cost grows 6.9× while calls grow 1.2×VE7w, pPRTTraced to fallback; no-fallback config stays flat
Latency, presentationpPRTLatency reported; notation and Appendix A condensed

1. Novelty: MindSearch closes a node when its search returns; ANTMAN closes a need only when its evidence is judged sufficient, and uses the need's state to decide what to revise, where to reroute, and when to recover.

Dynamic graph construction is not our contribution (Sp1U, pPRT). Our contribution is making the information need, rather than the search task or input partition, the unit of coordination. Each need keeps persistent state (sufficiency, readiness, progress), and three runtime operations act on it: need revision, adaptive rerouting, and task-local recovery. Removing any one lowers performance (Table 5). We will add MindSearch to the related work and Table 1.

2. Direct evidence against graph-based adaptive coordination.

  • External comparison: With a matched backbone and retriever, ANTMAN outperforms MindSearch on all seven benchmarks, by +12.7 to +27.0 points.
  • Internal ablation: Replacing the Need Graph with a MindSearch-style sub-question DAG lowers accuracy (−6.45 on SWE-QA-Pro, −5.0 points on GAIA) and raises compute (35% more calls, 30% higher cost).

3. Larger and harder evaluation, with reliable baselines.

  • MHQA: 100 questions per benchmark, with SEs; KiRAG and MindSearch added. ANTMAN has the best average EM/F1 (.743/.804), vs. .647/.785 for S2G-RAG.
  • BrowseComp-Plus (100 questions, per pPRT): ANTMAN reaches 56.0%, vs. 41.0% for ReAct, 33.0% for MindSearch, and 31.0% for S2G-RAG; all differences are significant (p ≤ 0.014).
  • Controlled scaling: ANTMAN's slower growth in active workers and cost, relative to LongAgent and CoA, is significant after Holm correction (adjusted p = 0.012 and 0.031).
  • Baseline reliability: On RepoProbe, ReAct and ReAct+RepoGraph fall below Direct because they cite nonexistent file paths on over half of the questions. On unflagged questions they match or exceed Direct, and the same ReAct beats Direct by 9.18 points on SWE-QA-Pro. ANTMAN leads on Knowledge (3.40 vs. at most 2.44).

4. End-to-end efficiency.

  • Cost: The 6.90× cost growth comes from a minority of fallback executions; median tokens per call stay flat (722 → 715). Without fallback, calls (0.96×) and cost (0.84×) stay flat over the 16× expansion.
  • Need counts: Realized needs grow only from 2.23 to 2.47, supporting the bound in Box 1.
  • Latency: Latency grows 1.45×, vs. 8.91× for LongAgent and 14.67× for CoA.

5. ANTMAN leads the strongest baseline on every benchmark.

BenchmarkMetricANTMANStrongest baselineMargin
HotpotQAEM75.0S2G-RAG (64.0)+11.0
2WikiMultiHopQAEM81.0S2G-RAG (71.0)+10.0
MuSiQueEM67.0S2G-RAG (59.0)+8.0
BrowseComp-PlusAccuracy56.0ReAct (41.0)+15.0
Controlled scaling (512K)F184.0LongAgent (75.4)+8.6
RepoProbeScore38.21Iterative BM25 (33.15)+5.06
SWE-QA-ProScore79.78SWE-QA-Pro Agent† (77.58)+2.20
GAIA-Text-103Accuracy58.3Iterative BM25 (45.6)+12.7

† Benchmark-specific agent. Against the strongest general baseline, ReAct+RepoGraph (67.38), the margin is +12.40.

With Qwen3-8B workers and the same orchestrator, ANTMAN-H retains 89–98% of full ANTMAN's performance on RepoProbe, SWE-QA-Pro, and GAIA.

Our contributions. ANTMAN makes the unresolved information need the unit of runtime coordination, and the need's state decides which workers activate, where search goes, and how stalled needs recover. As a result:

  • Coordination follows demand, not space: active coordination grows 1.23× under a 16× expansion, vs. over 15× for partition-driven baselines.
  • Need state, not graph structure, drives the gains: ANTMAN beats MindSearch on seven benchmarks and an internal sub-question DAG, at lower cost than the DAG.
  • The mechanism transfers across documents, repositories, web tools, and deep search, leads the strongest baseline on all eight evaluations, and holds up with 8B workers.

We thank the AC, SACs, PCs, and reviewers again for their time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.