- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces ANTMAN, a multi-agent information-seeking framework that maintains a revisable Need Graph to control worker activation, routing, and task-local recovery. Experiments mainly focus on multi-document QA, controlled information-space scaling, repository navigation, and GAIA. The central claim is that coordination can follow unresolved information needs without growing proportionally with the information space.
Critical strengths
- The separation between the static substrate map and the evolving Need Graph is clearly motivated.
- The scaling results of ANTMAN are promising. From 32K to 512K, active coordination increases from 2.57 to 3.17 workers, while F1 increases from .775 to .840.
- The ablations provide evidence that explicit need tracking can help. On GAIA, Full ANTMAN improves accuracy over Graph-free Adaptive while using fewer LLM calls.
Critical weaknesses
The distinction from closely related work is insufficiently established. MindSearch [1] is absent from the related work, Table 1, and experiments. Its WebPlanner incrementally constructs a DAG using search feedback, and adding a node invokes a WebSearcher (Section 2.1). Thus, an evolving graph already controls subsequent worker execution, with workers assigned to sub-questions rather than input partitions. This directly overlaps with ANTMAN’s motivation and claimed distinction from partition-driven coordination.
ANTMAN introduces explicit resolution status, accumulated evidence, attempt history, and progress, together with territory-backed routing and task-local recovery. These mechanisms may be useful, but the paper does not clearly establish their contribution beyond MindSearch’s dynamic graph construction and worker coordination. The comparison with Graph-free Adaptive does not resolve this concern: that variant removes the explicit persistent need representation, so it does not establish an advantage over an existing graph-based method.
Moreover, the controlled information-space scaling experiment compares only against LongAgent and CoA (Table 8), both partition-driven methods. These results support an advantage over partition-driven coordination, but leave the benefit over prior adaptive coordination baselines mentioned in table 1 unclear.
[1] Chen et al. MindSearch: Mimicking Human Minds Elicits Deep AI Searcher. ICLR 2025.
Questions for authors
1.Which Need Graph operations provide capabilities beyond MindSearch’s dynamic graph construction, and can their benefit be demonstrated under comparable execution settings?
- Why some baselines compared in table1 are missing?
3 comments
Official Comment by Authors
2. What ANTMAN Adds Beyond MindSearch's Dynamic Graph (Q1)
We agree that MindSearch already builds a DAG incrementally from search feedback and invokes a WebSearcher when a node is added. Dynamic graph construction and graph-driven worker execution are therefore not new in ANTMAN. The difference lies in what a node represents and what state is kept about it.
| Dimension | MindSearch: Sub-question | ANTMAN: Information Need |
|---|---|---|
| What a node is | A question to be investigated through search | An information requirement that must be met with sufficient evidence |
| When it is done | When its searcher returns an answer | When the evidence attached to it is judged sufficient |
| Lifecycle | Executed as part of the search plan; its answer informs later planning | Kept open across search attempts, revisions, and reroutes until it is satisfied |
| Relation to workers | A WebSearcher is invoked for each new sub-question | Worker activation and routing depend on which needs are open and what state they are in |
In ANTMAN, running a search and satisfying a need are separate events. A search can finish with only partial evidence. The need then stays active and can be refined, decomposed, or rerouted.
Each need carries three kinds of state that drive coordination:
- Sufficiency: whether the evidence gathered so far meets the requirement.
- Readiness: whether the need can run, given its dependencies on other needs.
- Progress: whether recent attempts on this need have made progress or stalled.
Graph edges also carry separate meanings:
- Decomposition edges record how a need is broken into sub-needs.
- Blocking edges determine when a need becomes executable.
- Semantic edges link related needs without constraining execution order.
A parent need is not closed just because its children are resolved. It receives its own sufficiency check against the accumulated evidence.
These states support specific operations. MindSearch's planner can add new sub-questions after feedback, but it does not maintain the following as explicit operations. The ablations in Table 5 measure what each one contributes:
| Operation | What It Does | Drop When Removed (512K F1 / SWE-QA-Pro / GAIA) |
|---|---|---|
| Need revision | Refines, merges, subsumes, or drops needs as evidence changes what is known | −11.41% / −7.50% / −29.17% |
| Adaptive rerouting | Reassigns a need to a different territory based on its accumulated state | −2.18% / −11.42% / 0.00% |
| Task-local recovery | Reframes, reroutes, or falls back on a stalled need without restarting the search | −28.45% / −6.03% / −12.50% |
GAIA uses a single territory-backed worker, so rerouting has no effect there.
These operations are what separate ANTMAN from a dynamic sub-question graph. Section 3 (Q2) tests this directly by replacing the Need Graph with a sub-question DAG.
3. Empirical Evidence
Q1. Does ANTMAN outperform an existing graph-based adaptive coordination method?
We compared ANTMAN with MindSearch on seven benchmarks, under the matched setup described in Section 1.
| Benchmark | Questions | Metric | ANTMAN | MindSearch | Difference |
|---|---|---|---|---|---|
| HotpotQA | 100 | EM / F1 | 75.0 ± 4.3 / 83.0 | 55.0 ± 5.0 / 70.6 | +20.0 / +12.4 |
| 2WikiMultiHopQA | 100 | EM / F1 | 81.0 ± 3.9 / 83.1 | 54.0 ± 5.0 / 72.9 | +27.0 / +10.2 |
| MuSiQue | 100 | EM / F1 | 67.0 ± 4.7 / 74.9 | 48.0 ± 5.0 / 69.4 | +19.0 / +5.5 |
| RepoProbe | 108 | Score | 38.21 | 24.94 | +13.27 |
| SWE-QA-Pro | 40 (subset) | Score | 81.43 | 68.69 | +12.74 |
| GAIA | 40 (subset) | Accuracy (%) | 60.0 ± 7.7 | 45.0 ± 7.9 | +15.0 |
| BrowseComp-Plus | 100 | Accuracy (%) | 56.0 ± 5.0 | 33.0 ± 4.7 | +23.0 |
± denotes standard error for EM and accuracy. The SWE-QA-Pro and GAIA comparisons use the 40-question subsets from our ablation study. Full-set ANTMAN results in the main paper are 79.78 and 58.3.
ANTMAN scores higher than MindSearch on all seven benchmarks. MindSearch already builds a dynamic graph and assigns workers to sub-questions, so this is a comparison against an adaptive graph-based method rather than a partition-driven one.
The benchmarks cover different substrates:
- HotpotQA, 2WikiMultiHopQA, and MuSiQue test multi-hop retrieval and reasoning over documents.
- RepoProbe and SWE-QA-Pro require navigation across code repositories.
- GAIA tests multi-step, tool-using information seeking.
- BrowseComp-Plus contains harder deep-search tasks.
Official Comment by Authors
Q2. Do the gains come from need-centric coordination rather than from having a dynamic graph?
Our original Graph-free Adaptive ablation removes the graph entirely. As you note, this cannot show an advantage over an existing graph-based approach. We therefore added a MindSearch-style Dynamic DAG Replacement.
This variant keeps ANTMAN's backbone, tools, workers, and evaluation setup. It replaces the Need Graph with a dynamic sub-question DAG that has three capabilities:
- It generates sub-questions.
- It expands dependencies based on execution feedback.
- It passes the answers of earlier sub-questions to later ones.
This variant is separate from the external MindSearch implementation in Q1.
| Benchmark | Full ANTMAN | Dynamic DAG Replacement | Difference |
|---|---|---|---|
| SWE-QA-Pro (40) | 81.43 | 74.98 | +6.45 |
| GAIA (40) | 60.0% | 55.0% | +5.0 points |
| Efficiency | Full ANTMAN | Dynamic DAG Replacement |
|---|---|---|
| SWE-QA-Pro LLM Calls / Query | 54.3 | 83.8 |
| GAIA Inference Cost / Query | $0.206 | $0.294 |
Both systems build and revise a graph adaptively. What differs is the unit of coordination: the replacement organizes work around search tasks, while ANTMAN tracks information needs and their satisfaction state.
Full ANTMAN is more accurate on both benchmarks. It also uses about 35% fewer calls on SWE-QA-Pro and costs about 30% less per question on GAIA. These subsets are small, so we treat the accuracy differences as supporting evidence; the efficiency differences are larger.
On SWE-QA-Pro, Graph-free Adaptive (78.88) scores between the Dynamic DAG Replacement (74.98) and full ANTMAN (81.43). One plausible reading follows:
- Graph-free Adaptive replans from accumulated evidence every round, so it cannot get locked into an early, unproductive decomposition.
- The sub-question DAG can keep directing work along dependencies it committed to early on.
- ANTMAN keeps a persistent structure but can revise, merge, subsume, or drop needs, which avoids that lock-in.
This ordering suggests that the gain depends on the structure being revisable and need-aware, not on the presence of a graph.
Q3. Do the gains hold against adaptive and iterative methods, not only partition-driven ones?
| Method | Category | Avg. EM ± SE (%) | Avg. F1 (%) |
|---|---|---|---|
| ANTMAN | Need-driven coordination | 74.3 ± 2.5 | 80.4 |
| S2G-RAG | Iterative / structured RAG | 64.7 ± 2.8 | 78.5 |
| ChainRAG | Iterative / structured RAG | 58.7 ± 2.8 | 71.4 |
| KiRAG | Iterative / knowledge-guided RAG | 58.0 ± 2.9 | 73.9 |
| CoA | Partition-driven multi-agent | 55.3 ± 2.9 | 73.0 |
| MindSearch | Adaptive graph-based coordination | 52.3 ± 2.9 | 70.9 |
| LongAgent | Partition-driven multi-agent | 35.3 ± 2.8 | 57.4 |
300 questions per method (100 each from HotpotQA, 2WikiMultiHopQA, and MuSiQue).
ANTMAN has the highest average EM and F1. It exceeds MindSearch by 22.0 points in EM and 9.5 in F1, and it also scores above the three iterative retrieval methods (S2G-RAG, ChainRAG, KiRAG). The advantage is therefore not limited to partition-driven baselines.
Thank you again for the careful review. We hope the added comparisons address your concerns, and we would appreciate it if you would reconsider your rating in light of them.
Official Comment by Authors
Thank you for the detailed review and for pointing us to MindSearch. In response, we added MindSearch as an external baseline on seven benchmarks, ran a MindSearch-style dynamic-DAG replacement inside ANTMAN, and added KiRAG to the multi-hop QA comparison. We address each point below.
1. Baseline Selection (Q2)
Table 1 is a taxonomy of related approaches. It is not a list of methods we intended to run in every experiment. Each evaluation uses baselines chosen for the question it tests:
Controlled information-space scaling. This experiment tests whether coordination grows with how a large input is partitioned. We chose LongAgent and CoA because they are the multi-agent methods designed around that setting. We agree that this experiment shows an advantage over partition-driven coordination specifically. It does not by itself establish an advantage over adaptive coordination methods. Section 3 compares ANTMAN with adaptive methods on answer quality and efficiency.
MindSearch. Leaving MindSearch out of the related work and Table 1 was an oversight. The revision will add it to both. MindSearch was originally built on a web search engine, but its WebPlanner/WebSearcher architecture does not depend on that backend.
To compare it under controlled conditions, we made these changes:
Its planner, graph construction, and searcher logic are otherwise unchanged. Results are in Section 3 (Q1).
KiRAG. KiRAG (ACL 2025) and S2G-RAG (ACL 2026) are both iterative, evidence-guided retrieval methods. Each acquires evidence step by step and conditions later retrieval on what has already been found. They differ in mechanism:
Because of this overlap, we originally included only the more recent S2G-RAG as the representative of this category. We have now run KiRAG with the authors' official pipeline and trained aligner, using GPT-4.1 as the backbone and the same questions and per-question documents as the other methods.
Each entry is EM ± SE / F1.
KiRAG's average EM (.580) is close to ChainRAG (.587) and below S2G-RAG (.647) and ANTMAN (.743).
Other methods in Table 1. These methods are related to ANTMAN through adaptive retrieval or multi-agent organization, but none targets the multi-worker information-space setting we evaluate:
We list them in Table 1 to position ANTMAN, not as direct competitors in each experiment.