acceptodds
Under review as a conference paper at ICLR 2027

Multi-Agent World: Scaling Multi-Agent Orchestration Training via Recursive Environment Improvement

Abstract

Multi-agent systems assign complementary subtasks to different agents, and their effectiveness depends on an orchestrator that partitions a request into subtasks, schedules them against their dependencies, and integrates the evidence that workers return. Training such an orchestrator requires multi-agent environments, in which a task provides subtasks that can be distributed across agents and a verifier that reports how they were distributed. Existing agentic environments are synthesized for a single agent, and they yield mostly linear call chains under outcome-only verification. An orchestrator trained inside them therefore has no scheduling decision to learn and no feedback on the decomposition it chose. We present Multi-Agent World (MAW), an end-to-end framework that develops multi-agent orchestration by coupling multi-agent environment synthesis with multi-agent reinforcement learning. MAW introduces recursive environment improvement, which recursively composes reference task graphs through serial and parallel operators inside single-agent environments, and turns each composed graph into one coherent request through semantic reconstruction and execution-grounded admission. The same composition record identifies which subtasks are verified as independent, which is what a schedule-aware process reward needs in order to score the trajectory of the whole team. Trained under these process rewards, the orchestrator learns to decompose a request, run independent work concurrently, and integrate what the workers return. Experiments on five agentic benchmarks show that our trained orchestrator, MAW-8B, improves its backbone by 8.9 points on average. Further ablations on the challenging DeepPlanning benchmark show that MAW-8B can surpass Claude Sonnet 4.6 as an orchestrator.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-agent systems, LLM agents, Multi-agent orchestration, Reinforcement learning, Task synthesis

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_pAmSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose Multi-Agent World (MAW), an end-to-end framework for training multi-agent orchestrators. The primary motivation is that existing agent environment synthesis methods are inherently designed for single agents. Consequently, the generated tasks lack the branching structures necessary for an orchestrator to practice on, and they provide no supervisory signals to evaluate scheduling quality. MAW addresses this issue through two core mechanisms:Recursive Environment Improvement: It uses sequential and parallel operators to compose the reference execution graphs of existing single-agent seed tasks. Through semantic reconstruction and execution verification, these are transformed into a coherent user request. Concurrently, the framework records Π, the set of verified independent subtask pairs. Schedule-Aware Process Reward: During training, MAW uses Π to explicitly score the orchestrator's scheduling decisions, rewarding the behavior of assigning independent subtasks to different workers for concurrent execution. Empirically, MAW-8B (trained on Qwen3-8B) yields an average improvement of 8.9 points over the base model across five agent benchmarks. Notably, it also surpasses Claude Sonnet 4.6 (48.90 vs. 45.96) acting as an orchestrator on the DeepPlanning-Travel benchmark.

Critical strengths
  1. Precise problem definition targeting a real gap The paper clearly pinpoints a structural flaw in current single-agent environment synthesis: the independence record Π in generated tasks is always empty. As a result, orchestrators lack both schedulable branches and supervisory signals during training. The formalization in Section 3.1 is crisp and directly motivates the proposed design, showing that the framework is driven by rigorous logic rather than mere intuition.

  2. Unified design of synthesis and supervision What makes the recursive environment improvement stand out is not merely its ability to "generate parallel tasks." Rather, a single construction pass simultaneously yields two critical outputs: the branching structures for the orchestrator to act upon, and the ground-truth scheduling signal Π consumed directly during training. This "synthesis-as-annotation" paradigm avoids the overhead of separate labeling and inherently aligns the reward signal with the training data. I consider this the most valuable methodological contribution of the paper.

  3. Clean ablation study with credible conclusions The two-step ablation in Table 4 (removing the process reward, then removing MAW-Data) effectively isolates the two core components. Crucially, their impact manifests in different subtasks (travel planning vs. shopping), which successfully rules out the possibility of metrics masking one another. This represents a refreshingly rigorous approach to ablation design.

  4. Transparent and convincing attribution experiments (Table 3) By freezing one component and swapping the other to separate the respective contributions of the orchestrator and the workers—while simultaneously reporting accuracy, wall-clock time, and token consumption—the authors provide solid empirical backing for their claim that MAW-8B's gains stem specifically from its orchestration capabilities rather than raw worker quality.

Critical weaknesses
  1. Small scale and opaque admission statistics of MAW-Data The training set consists of only 2,581 tasks across 539 environments. Crucially, the paper omits the total volume of generated candidate tasks, the rejection counts at each filtering stage, and the distribution of the independent pair count |Π| among the admitted tasks. Absent this information, one cannot ascertain whether the training distribution covers sufficiently diverse branching structures, nor estimate how frequently the scheduling reward is actually triggered during training. Given this limited data scale and lack of transparency, the paper's core claim—that the schedule-aware process reward yields a substantial improvement—lacks adequate empirical backing.

  2. Lack of trained orchestrator baselines obscures the novelty boundary The authors cite Team et al. (2026), Yuan et al. (2026), and Dang et al. (2026) in the related work section. All three utilize RL to train orchestrators and share highly overlapping goals with MAW, yet none appear in the evaluation tables. The current baselines (AWM-8B, EnvScaler-8B) merely apply single-agent synthesis pipelines directly to orchestrator training, effectively testing a rather weak comparative hypothesis. Without evaluating against purpose-built, trained orchestrator baselines, the paper fails to answer a fundamental question: what is MAW's exact incremental contribution over existing peer methods?

Questions for authors
  1. Admission statistics and |Π| distribution for MAW-Data While you report 2,581 admitted tasks, could you provide the missing statistics: the total number of initially generated candidates, the rejection breakdown across each filtering stage (execution, replay, semantic review), and the distribution of |Π| among the final admitted tasks? In particular, do any tasks in the training set have |Π| = 0? If so, does the scheduling reward realistically provide any meaningful signal for these specific instances?

  2. Relationship with existing trained orchestrator baselines As noted above, Team et al. (2026) and Yuan et al. (2026) share highly similar objectives (training orchestrators via RL) but are absent from the empirical results. If their checkpoints are publicly available, can you include an empirical comparison in the rebuttal? If a direct comparison is practically infeasible at this stage, could you explicitly detail the core technical differences between MAW and these works, specifically regarding training data construction or reward design? The goal here is to clearly define your novelty boundary, even in the absence of head-to-head numerical comparisons.

    If my issue is effectively resolved, I will consider increasing the points.

5 comments
AuthorsOpenReview ↗

Response to Reviewer pAmS

W1 & Q1: Is MAW-Data large and transparent enough to support the scheduling reward, and what happens on tasks with |Π| = 0?

A: Thank you for this question. The admission funnel is reported in Appendix A.2 and Figure 3: starting from 10,860 seed tasks across 598 environments, the synthesis evaluates 5,537 two-parent reference unions after rule-based pruning and admits 2,581 tasks across 539 environments. Figure 3 gives the final rejection counts of every gate (execution 947, semantic review 861, plan validity 510, DAG consistency 290, checker and replay 220), together with the attempt on which each admitted task passed (59.5% first, 27.8% second, 12.7% third). We add two further statistics below: the distribution of |Π| over admitted tasks, and how often the scheduling reward is active during training. As the reviewer points out, MAW is synthesis-as-annotation: Π comes from the construction process itself, and these statistics complete the evidence for the scheduling reward.

On scale, MAW-Data is comparable to open environment-synthesis training sets. Agent-World [1] uses 1,978 environments and about 5K RL training samples, while MAW-Data has 2,581 tasks across 539 environments, drawn from a seed pool of 10,860 tasks across 598 environments. Each task is a multi-agent task with several subtasks, so it carries more structure than a single-agent task. More importantly, our contribution is a scalable synthesis method rather than a fixed dataset, and structure matters more than quantity: recursive tasks train a better orchestrator than one-step tasks (Table A3 in our response to Reviewer wpqT).

On the branch structure itself, Π is built by permutation execution: for each pair of subgoals, we execute them in both orders and record the pair as independent only if the answer and the database effect are unchanged. Write operations and stateful calls are never recorded, because reordering them cannot be guaranteed to be safe. Table C1 shows that the resulting structures are diverse: 72.3% of admitted tasks have at least one independent pair, and 24.6% have seven or more.

Table C1: Distribution of verified |Π| over admitted tasks

Verified |Π|#TasksShare
071627.7%
150419.5%
2–341916.2%
4–630611.9%
≥ 763624.6%
All2,581100%

We also tracked how Π is used during training. The scheduling term f_sch is active in 34,136 of 61,824 rollouts (55.2%; Table C2). Regarding Q1, 716 admitted tasks have |Π| = 0, and f_sch is 0 on them by design: without a verified independent pair, there is no concurrency to reward. Where it is active, the signal clearly shapes behaviour. Without f_sch, the concurrency rate falls from 95.2% to 63.3%, and DP-Shopping drops by 3.48 points (Table A4 in our response to Reviewer wpqT).

Table C2: Whether the scheduling term is active during training

Process-reward branchRolloutsShare
Eligible34,13655.2%
Stateful / write operations19,72831.9%
No independent goal7,57612.3%
Ambiguous reference calls2880.5%
Order proof missing or failed960.2%
All61,824100%

Since Π covers only read-only subgoals, the 31.9% of rollouts that involve writes or state receive no scheduling signal. On the 55.2% where it is active, the signal is dense enough that removing it clearly changes orchestration behaviour. Together with the ablation, these statistics support the role of the scheduling reward.

Upon acceptance, we will release the data, environments and synthesis code, so that these statistics can be reproduced and the data extended. In the revision, we will describe how Π is built in Section 3, including the permutation check and the exclusion of write and stateful calls, and promote the admission funnel of Appendix A.2 and Figure 3 into the main paper alongside Tables C1 and C2. We will also state the scope of f_sch in Section 5: it supervises read-only parallelism, which covers 55.2% of training rollouts, and is 0 by design elsewhere, so that the evidence for the scheduling reward is complete.

W2 & Q2: Why are there no trained-orchestrator baselines, and how does MAW differ from existing RL orchestrator work?

A: Thank you for this question. To our knowledge, MAW is the first environment and task synthesis method for multi-agent orchestration. Existing RL orchestrator work trains on existing tasks and contributes mainly to the RL method. This is orthogonal to MAW: MAW-Data could also be used to train their orchestrators. MAW differs from this work in two ways (Table C4).

The first is the training tasks. Both ParaManager [2] and Puppeteer [3] train on existing datasets: ParaManager on DAPO-Math, and Puppeteer on reasoning and generation tasks such as MMLU-Pro, GSM-Hard and SRDD. PARL in Kimi K2.5 [4] trains on synthetic wide- and deep-search prompts. Their tools are general-purpose (search, a code sandbox, a browser), and none of them records which subtasks can run in parallel. MAW instead synthesizes long-horizon tool-calling tasks in executable, stateful environments. It builds their branch structure with serial and parallel operators, verifies the structure by execution and permutation, and records it as Π.

The second is the training objective. Existing parallel rewards count parallel calls or sub-agent instantiations without checking whether the subtasks are independent. ParaManager, for example, rewards a rollout whose average number of calls per round exceeds a threshold. PARL itself reports spurious parallelism, where the orchestrator spawns many sub-agents without meaningful decomposition, as reward hacking, and it anneals its auxiliary reward coefficients to zero during training. MAW's f_sch scores only verified pairs in Π, with a duplicate-dispatch discount δ and an integration term f_int, so dependent or redundant parallelism earns no scheduling reward.

Table C4: Comparison with RL orchestrator work

Training tasksTools / environmentsParallel structure recordedParallel reward
ParaManager [2]DAPO-MathPython sandbox, Wikipedia retrieval, agents-as-toolsNoAverage calls per round ≥ threshold
Puppeteer [3]MMLU-Pro, GSM-Hard, SRDD, CommonGen-HardLLM agents, web searchNoNone (accuracy − cost)
Kimi K2.5 PARL [4]Synthetic wide- and deep-search promptsSearch, browser, code sandboxNoSub-agent instantiation, annealed to 0
MAWSynthesized long-horizon tool-calling tasksExecutable, stateful MCP environmentsYes (Π, verified)Only on verified pairs in Π

We agree that a comparison with these orchestrators would be the most direct evidence, but a controlled one is not feasible. ParaManager has released neither code nor weights. Puppeteer has released code but no trained orchestrator, and its policy is built on a 70B reward model, which does not match our 8B setting. Kimi K2.5 releases the final model but not its PARL training data or pipeline, so the method cannot be reproduced at 8B.

In the revision, we will add Table C4 and this discussion to Related Work, positioning MAW as a source of training environments and tasks that is orthogonal to the RL method and could also be used to train prior orchestrators.


Thank you again for your careful and constructive comments, especially for pointing out that the construction statistics are what make the evidence for the scheduling reward complete. We hope our response addresses your concerns, and we would appreciate it if you could reassess our work in light of these results.

References

[1] Guanting Dong et al. Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence. arXiv preprint arXiv:2604.18292, 2026.

[2] Wenzhen Yuan et al. Small Model as Master Orchestrator: Learning Unified Agent-Tool Orchestration with Parallel Subtask Decomposition. arXiv preprint arXiv:2604.17009, 2026.

[3] Yufan Dang et al. Multi-Agent Collaboration via Evolving Orchestration. In NeurIPS, 2025.

[4] Kimi Team. Kimi K2.5: Visual Agentic Intelligence. arXiv preprint arXiv:2602.02276, 2026.

Thank you again for your careful and constructive review. We have uploaded a revised PDF; all new content is shown in blue. In response to your comments:

W1/Q1 (admission statistics and the |Π| distribution): We complete the admission funnel with per-gate rejection counts in Appendix A.2, and add the |Π| distribution and the activation of the schedule term in Appendix A.5 (Tables 11/12). 72.3% of admitted tasks carry a verified independent pair, 716 (27.7%) have |Π| = 0 and earn no schedule credit by construction, and f_sch is active on 34,136 of 61,824 rollouts (55.2%); Sec. 3.2 and 3.3 state the permutation check and this scope. The per-term evidence is the new leave-one-out ablation (Table 5).

W2/Q2 (trained orchestrator baselines): We add a technical comparison with ParaManager, Puppeteer and PARL along the training tasks and the parallel reward, where none records which subtasks may run in parallel and MAW scores only verified pairs in Π (Sec. 2, Appendix G, Table 19).

If these revisions address your concerns, we would be grateful if you would consider updating your score. We would be glad to discuss any remaining questions.

Reviewer_ji9tOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Multi-Agent World (MAW), a framework for synthesizing training tasks for multi-agent orchestration. It combines starts with a seed pool of single-agent tasks, then composes them into candidate serial and parallel paths, then filters. A Qwen3-8B orchestrator is trained on the high quality filtered data, improving base model performance by a significant margin across numerous benchmarks, beating existing single-agent orchestrators.

Critical strengths
  • Clean multi-agent data pipeline: The dataset construction methodology appears sound and practically useful for training an orchestrator to natually learn parallel and serial decomposition.

  • Orchestration-specific rewards: MAW rewards appropriate orchestration delegation rather than only final-answer supervision, which likely leads to more stable training and reinforces local desirable behavior.

Critical weaknesses
  • The gains are not consistently isolated to orchestration: The main results compare the full MAW with the base backbone. MAW's workers have been trained on the seed tasks. While Table 3 shows the isolation on one dataset, it is unclear if the orchestration gains generally contribute in other settings, or if the gains are driven by the improved workers. This should be more strongly evaluated to support the improved orchestration claim.

  • No efficiency analysis: One of the major benefits of learning parallel decomposition should be inference time efficiency. An orchestrator which usually delegates serial paths to single agents should be slower than one which differentiates between task requirements and parallelizes calls when it can. This work should directly measure inference time on a long-form dataset.

Questions for authors
  1. Where does the seed data come from? Does it come from the same distribution as the evaluation sets, or something different? What sorts of tasks are there? The data composition/sources should be detailed.

  2. How does SA-8B perform as a single agent on all five benchmarks, before multi-agent orchestration training? Also, how does an untrained/base orchestrator perform with these trained workers?

  3. Can robust efficency analysis be performed to better motivate the benefit of parallel orchestration?

6 comments
AuthorsOpenReview ↗

Response to Reviewer ji9t

W1 & Q2: Do the gains come from orchestration or from stronger workers?

A: Thank you for this important question. We agree that the gain from orchestration must be separated from the gain from the workers. We now extend the isolation in Table 3 to all benchmarks through three comparisons.

MAW targets orchestration in two places. In its training tasks, the branch structure is known and the independent subgoal pairs Π are verified. In its reward, the scheduling term f_sch scores only pairs in Π. The workers, by contrast, are trained only on single-agent seed tasks and never see an orchestration signal. As the reviewer notes, Table 3 already isolates this on DP-Travel: with Claude workers fixed, the MAW-8B orchestrator scores 48.90 against 45.96 for Sonnet 4.6. Since we did not train these workers, this gain can only come from orchestration.

Table B1: Main results with matched baselines. Avg follows the paper: the unweighted mean over the five benchmarks, with DeepPlanning entered as the mean of the travel composite and the shopping match rate.

ModelBFCLτ²-BenchMCPMarkDP-TravelDP-ShoppingWideSearchAvg
Untrained Qwen3-8B28.8827.8712.507.7324.7320.5121.20
Seed-trained SA-8B28.5028.4410.0016.4826.9221.5022.03
Untrained Qwen3-8B orch. + SA-8B workers35.1337.1110.0018.9943.0419.4526.54
MAW-8B orch. + SA-8B workers37.1340.4415.0019.8447.0724.4030.09
Δ MAW − SA-8B+8.63+12.00+5.00+3.36+20.15+2.90+8.06
Δ MAW − untrained orch.+2.00+3.33+5.00+0.85+4.03+4.95+3.54

Table B1 places the trained orchestrator next to three matched baselines on all five benchmarks.

First, stronger workers do not explain MAW's gain. Seed training raises the single-agent average by only 0.83 points (21.20 → 22.03), and this comes almost entirely from DeepPlanning. On BFCL, τ²-Bench, MCPMark and WideSearch, SA-8B stays within one point of the backbone or is lower, yet MAW improves over SA-8B on all five benchmarks, by 8.06 points on average.

Second, with the SA-8B workers fixed, the trained orchestrator outperforms the untrained one on all five benchmarks, by 3.54 points on average. The largest gains are on WideSearch (+4.95) and DP-Shopping (+4.03). On MCPMark and WideSearch, the untrained orchestrator is no better than the SA-8B single agent (10.00 vs. 10.00 and 19.45 vs. 21.50), so multi-agent execution alone does not help there; the gain appears only after orchestrator training.

Overall, MAW improves the average by 8.89 points over the backbone. Seed training of the workers contributes 0.83, multi-agent execution with an untrained orchestrator adds 4.51, and orchestrator training adds 3.54. DP-Travel is the exception: of its 12.11-point gain, 8.75 come from worker training, 2.51 from multi-agent execution and 0.85 from orchestrator training.

Third, trajectory analysis shows that orchestrator training changes how the orchestrator delegates. With the same workers and the same data, the process rewards raise the concurrency rate from 56.4% to 95.2% and cut duplicated dispatch from 28.8% to 0 (Table A4 in our response to Reviewer wpqT).

In the revision, we will add Table B1 to the main results, keeping the averaging convention of the paper, and the three-part decomposition of the gain to Section 5. The abstract and introduction will report gains against the matched baselines, so that every claim about orchestration rests on a comparison with fixed workers. We hope these results make the gains from orchestration itself clear.

W2 & Q3: Does learned parallelism actually improve efficiency?

A: Yes. Efficiency is one of the main benefits of parallel orchestration, and the paper already measures it directly. Table 3 reports per-episode wall-clock time and team-wide tokens on DeepPlanning-Travel: with the same SA-8B workers, MAW-8B takes 135.1 s against 421.6 s for the Claude orchestrator, and with the same Claude workers 108.9 s against 469.0 s. Table 11 of the appendix reports the behaviour behind this, where delegation rounds fall from 4.38 to 1.88 and 68% of episodes finish delegation in a single batch. We now extend the measurement into orchestrators that differ only in training data and reward.

Because f_sch rewards dispatching independent branches concurrently, end-to-end latency depends on the critical path, i.e., the longest dependency chain, rather than on the sum of all subtask times. Serial delegation pays for every subtask in turn. Following the reviewer's framing, we compare MAW with orchestrators that delegate more serially, all using the same workers (SA-8B trained on the seed pool). Seed pool + O is an orchestrator trained only on the seed pool, without MAW-Data. MAW-Data + O is trained on MAW-Data with the outcome reward only (the same setting as in Table A4 of our response to Reviewer wpqT).

Table B2: End-to-end latency of MAW relative to orchestrators with the same workers (bold: best score, and lowest latency among multi-agent settings)

SettingTravel time (s)Travel CompShopping time (s)Shopping Match
SA-8B single agent36.0416.4826.4126.92
Seed pool + O225.4812.47236.4932.05
MAW-Data + O213.9312.14226.7046.89
MAW-8B135.1019.8484.6247.07

We measured per-run latency on DeepPlanning. On DP-Travel, MAW is 36.8% faster than MAW-Data + O and 40.1% faster than Seed pool + O, while also scoring higher. On DP-Shopping, the reductions are 62.7% and 64.2%. This matches the change in behaviour reported in Table 11: MAW launches 95.2% of its assignments concurrently against 56.4% for MAW-Data + O, over 1.88 delegation rounds against 4.38. Note that MAW and MAW-Data + O differ in all four process terms, not only in f_sch.

A single agent is faster in absolute terms because it delegates nothing (36.04 s vs. 135.10 s on Travel, 26.41 s vs. 84.62 s on Shopping), but it is far less accurate (16.48 vs. 19.84 on Travel, 26.92 vs. 47.07 on Shopping).

In the revision, we will add Table B2 to Section 5, with a token column and the definition of each setting, which better supports our claim that learned parallelism reduces the overhead of delegation.

Q1: Where does the seed data come from, and does it overlap with the evaluation sets?

A: All environments, data and tasks are synthesized by us and are isolated from the evaluation benchmarks. We use none of their tasks, environment instances or trajectories.

We take MCP server and tool specifications from Toucan [1], a large-scale training dataset that collects existing MCP servers from Smithery and GitHub, and re-implement each environment ourselves. We synthesize seed tasks by composing tool-call chains as DAGs over a given server and then writing a question for each chain, following Agent-World [2] and related works. No step of this process involves the evaluation benchmarks.

Some services overlap with the benchmarks at the category level. For example, GitHub, Notion and PostgreSQL also appear in MCPMark, and travel and shopping are DeepPlanning domains. Our versions are re-implemented from public tool specifications with our own states and tasks, and we use no benchmark task, initial state, checker or trajectory.

Table B3 gives the domain distribution. The seed pool covers 598 environments and 10,860 tasks in 10 domain categories. MAW-Data's 2,581 tasks span 539 environments, and its domain shares stay within 2.6 points of the seed pool's. All tasks are long-horizon tool-calling tasks in executable environments. Seed tasks are single-agent tool-call chains, and MAW-Data tasks are composed from them.

Table B3: Domain distribution of the seed pool and MAW-Data

Domain category#Environments#Seed tasks (share)#MAW-Data tasks (share)Example environments
Developer tools & cloud ops1492,652 (24.4%)604 (23.4%)GitHub, Sentry, Terraform, filesystem
Project management & productivity971,772 (16.3%)450 (17.5%)Jira, Linear, Notion, Todoist
Databases & data platforms651,182 (10.9%)215 (8.3%)MySQL, PostgreSQL, Snowflake, Metabase
Finance, crypto & payments611,101 (10.1%)282 (10.9%)Alpaca, Yahoo Finance, CoinGecko, YNAB
Knowledge, research & reference531,023 (9.4%)248 (9.6%)arXiv, PubMed, Semantic Scholar, Wikipedia
Communication & social media45846 (7.8%)181 (7.0%)Slack, Discord, Reddit, Bluesky
Media, games & entertainment37649 (6.0%)137 (5.3%)Spotify, YouTube, chess, Formula 1
Business, CRM & e-commerce33590 (5.4%)161 (6.2%)Shopify, HubSpot, Salesforce, Zendesk
Security & threat intel26476 (4.4%)146 (5.7%)VirusTotal, Shodan, Keycloak, Okta
Travel, maps, weather & local services26461 (4.2%)127 (4.9%)Weather APIs, flight/stay search, Google Maps
Other6108 (1.0%)30 (1.1%)–
Total59810,8602,581 (539 environments)

In the revision, we will describe the data sources and the synthesis pipeline in Section 3 and add Table B3 to the appendix. We will also state the category-level overlap explicitly and list what is and is not shared with each benchmark, so that readers can verify that no benchmark task, state or checker enters training.


Thank you again for your careful and valuable comments. We hope the new comparisons show that the gains come from orchestration itself, and we would appreciate it if you could reassess our work in light of these results.

References

[1] Zhangchen Xu et al. TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environments. arXiv preprint arXiv:2510.01179, 2025.

[2] Guanting Dong et al. Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence. arXiv preprint arXiv:2604.18292, 2026.

Reviewer_wpqTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed a method to train an orchestrator, the model that splits a request into subtasks, assigns them to workers, and merges the results. Its argument is that existing synthetic agent environments cannot teach this. The proposed MAW can generate rich data for training this orchestrator by providing valuable rewards. MAW starts from single-agent tasks and compose two tasks into a graph with a serial operator or a parallel operator. Repeat the process to get larger graphs. A strong LLM rewrites each graph into one coherent user request. The output is MAW-Data: 2,581 tasks over 539 environments. In terms of training, this work proposed to use schedule-aware process reward. The process reward adds four terms: tool coverage, answer coverage, delivery (did workers return the needed evidence), and a schedule term. The trained orchestrator, MAW-8B, is built on Qwen3-8B. It beats the backbone by 8.9 points on average across five benchmarks (BFCL Multi-Turn, τ²-Bench, MCPMark, DeepPlanning, WideSearch). It outperforms or is competitive against AWM-8B and EnvScaler-8B, two single-agent environment-synthesis baselines. Ablations on DeepPlanning show that both the process reward or the composed data are helpful.

[After Rebuttal] The author response addressed most of my concerns and I believe I can raise my score from ~2.5 to >3 but it is not a clear 4 paper yet. I will be my rating unchanged.

Critical strengths
  • The paper connects between task construction and supervision. The generator creates branching tasks, and the reward uses those construction records to evaluate orchestration. This is a compelling way to obtain supervision without manually annotating every decomposition.
  • Comprehensive evaluation strengths the paper. The evaluation covers five benchmarks spanning tool use, interactive tasks, planning, and information gathering, with reported improvements across all five. The worker-swap experiment is particularly interesting: the trained orchestrator can use Claude workers without retraining, reaching 48.90 on travel. This suggests the learned coordination behavior is usable beyond its original worker model.
  • This work provides a valuable data construction pipeline for practitioners. Tasks undergo schema and graph checks, execution, fresh-state replay, checker tests using corrupted outputs, and semantic review. The paper also reports rejection and repair statistics. These checks make the dataset construction more credible and inspectable.
Critical weaknesses
  • MAW uses workers trained on seed tasks, while the headline comparison is against the original Qwen3-8B backbone. On travel, the backbone scores 7.73, but the seed-trained single agent already reaches 16.48; MAW reaches 19.84. Thus, the gain over that closer baseline is 3.36 points, rather than 12.11. The paper needs comparisons across all five benchmarks using the trained worker to how the over story (Trained under these process rewards, the orchestrator learns to decompose a request, run independent work concurrently, and integrate what the workers return). I acknowledge there are extra ablation study shown in Table 4 showing the rewards are helpful, under the condition that the worker are the same. But I still want to see the updated baseline in the main table to understand the real improvement from proposed method. [addressed]
  • The paper reports 2,581 synthesized tasks, but no performance comparison between one-step composition and repeated recursive composition. If I did not miss something, the text does not report the recursion depth or number of composition rounds used in the experiments. Matched experiments varying recursion depth and branch structure would better support the central claim, especially when it is highlighted in the title (recursive environment improvement). [The recursion-depth distribution is provided. But branch structure part is still unclear to me.]
Questions for authors
  • What recursion settings were actually used to generate MAW-Data? Any stop criteria? [addressed]
  • How much of the improvement comes specifically from the scheduling reward? Table 4 removes all four process-reward components simultaneously, leaving their individual contributions unclear. While a full factorial study may be impractical, could you provide leave-one-out ablations that remove each component in turn while retaining the other three? [Addressed. Authors' results clarify that its benefits vary across benchmarks and are especially evident in scheduling behavior.]
5 comments
AuthorsOpenReview ↗

Response to Reviewer wpqT

W1: How much of the gain comes from MAW itself rather than from worker training?

A: Thank you for this suggestion. We agree that the current main table mixes the gains from worker training and orchestrator training. We have added two matched baselines that use the same SA-8B workers, and will include them in the main table.

The workers are trained only on single-agent seed tasks, which teaches them to solve one subtask. How to split a request, which parts to run concurrently, and how to integrate what the workers return can only be learned from MAW-Data, whose branch structure is known, together with the process rewards. We therefore expect orchestrator training to help even when the workers are fixed.

The paper already isolates this once. Table 3 fixes Claude workers and changes only the orchestrator: on DP-Travel, the MAW-8B orchestrator scores 48.90 and the Sonnet 4.6 orchestrator scores 45.96. We did not train these workers, so this is the cleanest isolation. We now extend it with two new rows: the seed-trained SA-8B single agent, and an untrained Qwen3-8B orchestrator with SA-8B workers.

Table A1: Main results with matched baselines. Avg follows the paper: the unweighted mean over the five benchmarks, with DeepPlanning entered as the mean of the travel composite and the shopping match rate.

ModelBFCLτ²-BenchMCPMarkDP-TravelDP-ShoppingWideSearchAvg
Untrained Qwen3-8B28.8827.8712.507.7324.7320.5121.20
Seed-trained SA-8B28.5028.4410.0016.4826.9221.5022.03
Untrained Qwen3-8B orch. + SA-8B workers35.1337.1110.0018.9943.0419.4526.54
MAW-8B orch. + SA-8B workers37.1340.4415.0019.8447.0724.4030.09
Δ MAW − SA-8B+8.63+12.00+5.00+3.36+20.15+2.90+8.06
Δ MAW − untrained orch.+2.00+3.33+5.00+0.85+4.03+4.95+3.54

Against the SA-8B single agent, MAW improves on all five benchmarks, by 8.06 points on average. On DP-Travel, the gain is 3.36 points over the matched SA-8B baseline and 12.11 points over the untrained Qwen3-8B single agent.

With the SA-8B workers fixed, the trained orchestrator outperforms the untrained one on all five benchmarks, by 3.54 points on average. The gains are mainly from WideSearch (+4.95) and DP-Shopping (+4.03). MCPMark also rises by 5.00 points. On DP-Travel, the trained orchestrator scores 19.84 versus 18.99, a gain of 0.85 points.

The average gain over the untrained Qwen3-8B single agent is 8.89 points. Across the configurations in Table A1, single-agent seed training raises the average by 0.83 points, using SA-8B workers with an untrained orchestrator adds 4.51 points, and training the orchestrator with those workers fixed adds a further 3.54 points. Trajectory analysis shows that the trained orchestrator also learns the decomposition and concurrency described in the reviewer's summary: MAW launches 95.2% of its assignments concurrently and never duplicates a subtask to the same worker within a batch (Table A4, and Table 11 of the paper).

In the revision, we will add both matched baselines to the main table on all five benchmarks. The abstract and introduction will report gains against the matched SA-8B baseline rather than only against the backbone. Section 5 will split the gain over the backbone into three parts, namely worker training, multi-agent execution and orchestrator training, so that the contribution of orchestrator training is stated on its own.

W2 & Q1: What does recursive synthesis add over one-step synthesis, and how is recursion configured and stopped?

A: Recursion produces structures that one-step synthesis cannot, and these structures give the scheduling reward more supervision. We show this on the data and on training results, and then give the settings asked in Q1.

One-step synthesis can only combine two seed tasks into one serial chain or one set of parallel branches. Recursive synthesis reuses admitted composite tasks as parents. Only then do multi-level dependencies appear, such as a serial chain nested inside a parallel branch. Every task is executed and verified before admission, so these structures are executable. Every parallel composition also adds verified independent pairs to Π.

Table A2 shows that the average number of executed calls rises from 6.86 at depth 1 to 13.75 for tasks at depth 5 or greater, and the average DAG width rises from 3.83 to 8.28.

Table A2: Admitted tasks by depth

Depth#TasksAvg. executed callsAvg. DAG width
11,7096.863.83
24688.795.12
323510.696.36
411512.657.43
≥55413.758.28
Total / weighted mean2,5817.964.55

To isolate the contribution of recursion, we train orchestrators on subsets of MAW-Data that contain the same number of tasks but differ in synthesis depth (Table A3). Training on depth ≥ 2 tasks gives a better orchestrator than training on one-step tasks: DP-Travel rises from 15.83 to 18.77 (+2.94) and DP-Shopping from 37.73 to 41.03 (+3.30). Training on the full MAW-Data, which mixes all depths, gives the best results on both benchmarks (19.84 and 47.07). Simple and complex structures therefore play complementary roles: one-step tasks teach basic serial and parallel composition, and deeper tasks teach the nested dependencies that one-step synthesis cannot produce.

Table A3: Training on depth subsets of MAW-Data

Training subsetDP-Travel CompDP-Shopping Match
One-step (depth = 1)15.8337.73
Recursive (depth ≥ 2)18.7741.03
Full MAW-Data19.8447.07

Regarding Q1, the recursion works as follows:

  • Each round samples two parent tasks from an environment's candidate pool, applies a serial or parallel operator, rewrites the pair into one coherent request, and runs the admission checks (execution and replay). Admitted tasks return to the pool and can serve as parents again. Each candidate gets up to three attempts.
  • Each parent pair is used only once, and the two parents must have disjoint parent chains, so no seed task appears twice in one composite task.
  • Synthesis stops when a round admits no task, or when no environment has a valid parent pair left.
  • There is no depth cap. In practice, synthesis reached depth 8 (included in the ≥5 row of Table A2).

In the revision, we will add the recursion settings and stopping criteria to Section 3, so that the synthesis procedure is fully specified and reproducible. We will add Table A2 to the appendix to show that recursion produces deeper and wider task structures, and Table A3, with the size of each subset, to show that these structures train a better orchestrator than one-step tasks alone.

Q2: What does each process reward term contribute in a leave-one-out ablation?

A: Thank you for this question. Each term targets a different failure mode, and removing a term worsens the matching behaviour.

f_tool targets missing or wrong tool calls. f_ans targets answers that do not cover the request. f_del targets workers that do not return evidence to the orchestrator. f_sch targets independent subtasks that are not dispatched concurrently.

Table A4: Leave-one-out ablation of the process reward terms (on DeepPlanning)

SettingTravel CompShopping MatchConc. (%) ↑Same (%) ↓
MAW (O + all four terms)19.8447.0795.20.0
w/o f_tool19.5645.2496.217.6
w/o f_ans17.9645.6068.835.6
w/o f_del19.4346.1590.80.6
w/o f_sch19.8243.5963.34.5
MAW-Data + O (w/o all four)12.1446.8956.428.8

Conc. and Same follow Table 11 of the paper: Conc. is the share of assignments launched inside a batch call that starts at least two different workers, and Same is the share of episodes with duplicate task assignments to the same worker within a batch. MAW-Data + O is trained on the same MAW-Data with the outcome reward O only.

As shown in Table A4, without f_sch the concurrency rate falls from 95.2% to 63.3%, and DP-Shopping falls from 47.07 to 43.59, the largest drop caused by any single term. Without f_ans or f_tool, duplicated dispatch rises to 35.6% and 17.6% respectively, while the full model has none.

The terms interact. f_ans also lowers concurrency (68.8%), and removing all four terms gives a Shopping score close to the full model (46.89 vs. 47.07). On DP-Travel, each single term has little effect, but removing all four drops the score from 19.84 to 12.14, so the terms partly substitute for one another. Their effect on behaviour is consistent: compared with MAW-Data + O, the full model raises concurrency from 56.4% to 95.2% and cuts duplicated dispatch from 28.8% to 0. f_sch thus does what it was designed for, which is to make the orchestrator dispatch verified independent subtasks concurrently.

In the revision, we will add Table A4, including the concurrency and duplicated-dispatch columns, to Section 5. Since single terms change scores only modestly but change delegation clearly, we will describe each term by the failure mode it removes, and present the role of f_sch through delegation behaviour rather than score alone.


Thank you again for your careful and constructive comments. The matched baselines in particular make the contribution of the method clearer. We hope our response addresses your concerns, and we would appreciate it if you could reassess our work in light of these results.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

As the discussion period draws to a close, we would like to express our sincere gratitude to the Area Chair and all three reviewers (wpqT, ji9t, and pAmS) for your time, thoughtful engagement, and constructive feedback throughout the review process. We deeply appreciate the active discussion and are truly encouraged by your recognition of our work. We look forward to working alongside the community to push this frontier further and create broader impact.

Thank you once again for your dedicated effort and valuable insights.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.