acceptodds
Under review as a conference paper at ICLR 2027

The Wealth of Agents: Scaling Agentic AI Across Developers

Abstract

Today's agentic systems are typically designed, built, and deployed by a single developer with full control over all participating agents. While this simplifies development, it forces developers to rebuild existing capabilities. We study _multi-developer agentic systems_ (MDAs), which coordinate agents from independent developers with different capabilities and resources. Such systems offer a new scaling axis: developers can build on existing agents and contribute specialized capabilities beyond what they could achieve alone. We construct empirical and theoretical models of developers to compare MDAs with single-developer agentic systems (SDAs). On GAIA, a general assistant benchmark, a population of specialized agents whose individual accuracy does not exceed 35% reaches 70% when their capabilities are combined in an MDA. This is 12 percentage points above the median general-purpose SDAs built with several times the development budget. Our theoretical models show how MDAs can incentivize specialization and encourage more efficient use of developer resources. Put together, our results make a preliminary case for MDAs as a new scaling axis and way for developers with diverse capabilities and resources to advance agentic AI collaboratively.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Agents, Multi-agent systems

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_XMSuOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies multi-developer agent systems, which they call MDAs. Unfortunately, I don't understand the novelty in this paper since multi-agent systems have been studied for years at this point.

Critical strengths

Unfortunately, I do not see how this paper is novel.

Critical weaknesses

Multi-agent systems are accessible to users today in every major harness to the best of my knowledge (e.g., Codex Ultra, Claude code, ...). Furthermore, there is an enormous literature on multi-agent systems that the related work does not cover. I do not understand the novelty in this paper nor what the contributions are in light of this reality.

Questions for authors

How does this different from the thousands of papers written on multi-agent systems?

2 comments

Response to Reviewer XMSu

Thank you for the review. The distinction is who builds the system and decides which agents exist. Our paper studies how to scale agentic AI through contributions from independent developers, each with their own resources and incentives.

Why this is more than a larger MAS. The systems named in the review fall within our single-developer setting: one developer designs the agents and their orchestration. Such systems can grow very large, but remain bounded by the resources, expertise, data access, and integrations available to that developer. An MDA opens a different scaling axis by letting independent developers contribute capabilities to a shared system.

An MDA is an emergent system: its agents depend on what developers choose to contribute, and participation depends on whether doing so benefits them. This introduces second-order effects into system design. Alongside how existing agents work together, we must study how routing, evaluation, and compensation change what developers build in the first place.

What the paper contributes. Our experiments and theory provide arguments for this scaling direction and identify obstacles to realizing it. On GAIA, agents from 54 specialist developers reach 70% together although none exceeds 35% alone. The best single-developer MAS we evaluate reaches 67%, at more than twice the inference cost. An agent built to complement an existing MDA scores only 19% alone, yet raises the MDA from 40% to 53% (Fig. 4c). These results show how developers with limited individual capabilities can advance a stronger shared system. Please also see the general comment.

Related work. §2 discusses MAS frameworks, routing, interoperability protocols such as A2A, and open ecosystems such as AgentNet and IoA. These efforts reflect strong interest in extending MAS across developers. Our contribution is to explicitly study the developer population: why combining its contributions is useful, what developers have reason to contribute, and why a large-scale MDA may fail to emerge despite its potential. We would welcome references that address these questions and should be discussed alongside our work.

Soundness. The review raises novelty concerns but does not identify a specific issue with the experiments or theory. We would appreciate clarification of the soundness concerns so that we can address them directly.

I disagree with this assessment, as techniques such as swarmed from OpenAI are exactly this kind of MAS.

Reviewer_Y7oXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-developer agentic systems (MDAs), in which a coordinator composes capabilities supplied by multiple developers. It constructs simulated developer populations using a meta-agent pipeline and compares specialist, generalist, and multi-agent pools under subtask-level coordination and whole-task routing. On GAIA, the specialist MDA reaches 70.1% accuracy, compared with 62.3% for whole-task routing over the same specialist pool and 67.9% for routed generalists. A complementary agent with 19.1% standalone accuracy improves a transferred pool from 40.4% to 53.2%. The paper also reports negative results on SealQA and with generalist pools. A stylized game-theoretic model relates routing and compensation to generalist and specialist equilibria. The intended contribution is a developer-level perspective on capability reuse, specialization, and resource allocation, supported by preliminary empirical and theoretical models.

Critical strengths
  1. The experiments demonstrate useful complementarity and identify when it matters. On the same GAIA specialist pool, subtask-level coordination improves accuracy from 62.3% to 70.1% (Figure 4a). The transfer experiment is particularly relevant to the proposed application: an agent with only 19.1% standalone accuracy raises the existing pool from 40.4% to 53.2% (Figure 4c). This supports evaluating agents by their marginal contribution to an ecosystem, which could help developers decide which capabilities are worth adding. The negative results on SealQA and generalist pools make the empirical picture more informative than an unconditional claim that more agents always help.

  2. The paper makes developer incentives an explicit part of agent-system design. The model distinguishes compensation for delivered capability from compensation for traffic and formalizes how these objectives can support different allocations of development resources. The finite-population approximation bounds and explicit discussion of cost geometry, routing assumptions, and coordination overhead help delineate the conclusions. This perspective is useful for designing shared agent services, where participation and specialization cannot simply be assumed.

Critical weaknesses
  1. Clarification and existing control results: Where are the A_MCP results described in Appendix C.2 reported? How does the proposed MDA compare with a centralized system that has comparable access to the aggregate tool or agent pool and a matched orchestration budget? Which observed benefit can be attributed specifically to independently developed capabilities?

  2. Resource accounting: What are the actual development token/API costs and router-training costs for each pool, beyond the refinement-round counts? Please also clarify the normalization of the dollar costs in Figures 3-5 and the reason the specialist MDA is shown at $14.5 in Figure 3 and $13.5 in Figure 4a. These details matter for interpreting resource efficiency.

  3. Statistical scope: What is regenerated across the four runs: execution randomness, the router, the developer population, or all of these? Please provide the exact evaluation counts and, if available, paired uncertainty estimates for the 70.1% versus 67.9% comparison and variation across independently generated agent pools.

  4. Interpretation of incentives: Given the generalist MDA equilibrium in Appendix B.3 and the sensitivity to the routing exponent in Appendix B.4, what supports expecting specialization to emerge under the implemented LLM router? Is the intended conclusion that MDAs permit a high-welfare specialist equilibrium, or that a particular implementable compensation and adaptation mechanism selects it? Please distinguish evidence available in this submission from future work.

Questions for authors

See Weaknesses

2 comments

Thank you for the precise questions and for recognizing the connection between coordination and developer incentives.

W1. AMCPA_{MCP}, a centralized system, and attribution.

The omitted AMCPA_{MCP} control merges each specialist bundle into one agent. On GAIA, it reaches 66.2% under routed-SDAs and 70.8% under the MDA. The latter improves on AMASA_{MAS}'s 63.3%, illustrating the cost of its extra internal orchestration layer.

Our MDA already has a centralized coordinator with access to the aggregate agent pool. The multi-developer distinction concerns who builds and controls the agents within the pool. The advantage of independent development is in exploiting the structure of the existing developer ecosystem, not because independent development itself is necessarily better. Currently there are countless independent, heterogeneous developers who are unlikely to coalesce into one unified entity. Under this assumption, our question is how we can enable and incentivize independent developers to contribute to collective capability under a shared system.

Combining the same specialists within an MDA gains 7.8 points over a routed-SDA. A developer building a complementary agent adds 12.8 points to an existing MDA (Fig. 4c). Medium-tier developers receive 56% of calls in the MDA, compared with 33% under whole-request routing (Fig. 4b). These results illustrate the promise and challenges of using third-party contributions.

W2. Resource accounting. Dollar figures are total LLM API spend for one pass over the evaluation split, including every agent, router, and orchestrator call. The 14.5 dollars in Fig. 3 is a single run. The 13.5 dollars in Fig. 4a is the four-run mean. Development budget is matched at 270 refinement rounds per pool on GAIA, and both mechanisms train routers on a held-out split. We chose to track budget based on iteration rounds rather than meta-agent API costs for a few reasons: API costs reflect the number of input/output tokens which do not necessarily reflect difficulty/capability. For example, some tools may be simple to write but require thousands of lines of code. Instead, we found that test iterations reveal concrete information about agent weaknesses that require patching, and each iteration therefore correlates more strongly with downstream agent capability. Router-training API costs were comparable to the actual execution costs, since we try to divide the training and evaluation splits evenly. In a real MDA deployment, learning can happen dynamically as requests are served, so these costs can be amortized.

W3. Statistical scope. The developer population, its agents, and the learned routing policy are fixed across the four runs. Each run re-executes 77 tasks for GAIA and 79 for SealQA, with variation from stochastic model outputs. A separate routing policy is learned for each pool and coordination mechanism. The specialist MDA scores 70.1 ± 1.6% and routed generalists score 67.9 ± 0.3% (SEM, n = 4). Agents that reach at most 35% alone become competitive with generalists built with six times the per-developer budget.

W4. Incentives. We intend the first of your two readings: MDAs permit a high-welfare specialist equilibrium, while routed-SDAs exclude it. This addresses the second-order effect of system design on what developers choose to contribute.

Prop. 1 establishes a unique generalist equilibrium under whole-request routing. Prop. 2 establishes a specialist equilibrium under an MDA that pays for delivered capability, and Prop. 4 shows that this equilibrium has higher welfare. A generalist equilibrium can also persist in the MDA. These results explain why enabling communication between agents alone does not ensure that developers will build complementary capabilities. We use a common, well-established Tullock contest with exponent 1. The results can change under different routing exponents, as discussed in App. B.4. The main goal is to provide a framework for reasoning about MDAs and to identify game-theoretic barriers to realizing them in the wild, where prior work has largely focused on protocols or technical issues. Reaching a specialist equilibrium through an implementable compensation and adaptation mechanism remains an open question for future work, but identifying and highlighting that problem is part of the paper's contribution to understanding why large-scale MDAs have yet to emerge.

Thank you for your response. In my assessment, the manuscript still requires substantial revisions and does not yet meet the criteria for acceptance in its current form. I wish you success in further developing this work.

Reviewer_ZRjcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multi-developer agentic systems, in which agents from independent developers collaborate through shared task decomposition and routing. The authors use meta-agents to construct generalist and specialist systems under matched development-round budgets, and compare different coordination strategies on GAIA and SealQA. They also introduce a theoretical model to study how routing and compensation rules affect developers’ incentives to specialize. The results suggest that combining complementary specialists can improve performance, while coordination costs and the composition of the agent pool determine when these benefits arise.

Critical strengths
  1. Relevant research question.
    The paper considers how independently developed agents can contribute to a shared system. Evaluating an agent by its contribution to the overall system, rather than only its standalone performance.

  2. Systematic empirical comparisons.
    The experiments compare generalist agents, specialist agents, and single-developer multi-agent systems under two coordination strategies. This helps distinguish the effects of agent-pool composition and coordination.

  3. Concrete evidence of complementary capabilities.
    The agent-addition experiment shows that an agent with limited standalone performance can substantially improve an existing system. This provides an informative example of the difference between standalone capability and contribution to a larger system.

  4. Useful ablations and negative results.
    The paper examines the effects of routing experience, contextual rankings, and coordinator strength. It also reports cases where collaboration increases costs without improving accuracy, helping clarify the limitations of the approach.

Critical weaknesses

This is written by myself

  1. The core idea seems obvious, and the novelty is unclear.

    I understand that the paper is not simply comparing single-agent and multi-agent systems. Still, the central claim seems to be that, under the same budget, multiple specialist agents coordinated by a router can outperform a single-agent system. This seems fairly obvious, and I struggle to identify the novel contribution.

  2. I question the practicality of LLM-based routing.

    Using a LLM as the router could introduce substantial latency and context-window limitations, particularly for complex real-world tasks. It is unclear whether this approach remains practical as the system grows.(Maybe Use Jev?hhhh, considering putting this into real task and i cant imagine its routing efficiency even though there is a test time evolving for router)

  3. The claim that current agent development focuses on generalists is unconvincing.

    My impression is that agents today are mostly developed for specific business use cases. I therefore disagree with the paper’s characterization of the current agent ecosystem and would like to see stronger evidence supporting it.

  4. The evaluation seems too limited.

    Results on only two benchmarks seem insufficient to support the paper’s broader claims. Additional benchmarks and evidence from real-world applications would strengthen the evaluation.

  5. The purpose of the very detailed/complex(not that complex but takes much more time to read your paper!) mathematical formalization is unclear.

    Mathematical modeling can be valuable, but the main text does not clearly explain why this level of formalization is necessary or what additional insights it provides.

  6. I have concerns about venue fit.

    In its current form, I feel this paper may be better suited to CHI than ICLR.

Questions for authors

This part is also written by myself

  1. What specific new insight should a reader take away from this paper?

    I am usually very reluctant to judge a paper simply as “lacking novelty,” but I did not come away with much new understanding. If the authors can clearly explain what is new and why it matters, I would be willing to raise my score.

  2. Are task decomposition and routing the main technical contributions?

    If so, this should be stated clearly in the introduction, and more space should be devoted to explaining these methods and how they differ from existing approaches.

  3. Can the authors provide a broader evaluation?

    Results on additional benchmarks and data from real-world applications would help establish the usefulness and generalizability of the approach.

2 comments

Thank you for asking us to clarify the central insight.

Q1, W1. The core idea seems obvious. We agree that combining contributions from independent developers is a natural scaling direction beyond the resources, access, and capabilities of a single developer. There is considerable interest in making MDAs a reality, as evidenced by work such as A2A. What is less obvious is why, despite such conceptual advantages, we do not see broader MDA development. Answering this question requires both empirical study of how MDAs can be beneficial and an understanding of developer incentives and game-theoretic issues of participation under different coordination schemes. For example, our analysis exposes a chicken-and-egg problem: specialists need an MDA to make their contributions useful, while an MDA needs complementary specialists to justify its coordination costs. This is also precisely why we need the theoretical contributions (W5).

Q2. Main technical contribution. The contribution is the empirical and theoretical analysis of scaling across developers. Decomposition and routing provide a concrete implementation for studying that question. Please see the general comment above.

W2. LLM routing. Latency and context limits are important challenges for scaling MDAs. Our implementation lets us study MDAs as a scaling direction; it is not a deployment proposal. All reported execution costs include routing and decomposition. Learning which agents are useful improves accuracy from 62.3% with description-based selection to 70.1% (Tab. 1). The theory treats routing abstractly, and retrieval to shortlist candidate agents is one direction for scaling beyond the current implementation.

W3. Generalists. We agree that many agents serve specific business uses. Our distinction concerns whether a developer's system handles requests independently or contributes capabilities to tasks shared across developers. A standalone business agent provides all capabilities required by its requests on its own. In an MDA, its developer could focus on proprietary data or integrations and rely on other developers for the rest. These businesses are potential contributors that motivate our work, but currently their agents are standalone systems and are not integrated into an MDA. Prop. 1 and App. D.3 examine why whole-request competition discourages this form of specialization.

W4, Q3. Benchmarks. GAIA combines diverse skills, while SealQA focuses on web search. The MDA's advantage disappears on SealQA, showing why more collaboration does not automatically improve capability. A real-world MDA requires participation from many independent organizations. Simulated developers let us investigate this setting now and provide a basis for broader evaluations.

W5. Formalization. The theory addresses the second-order question that experiments on fixed agent pools cannot answer: how does system design change what developers are incentivized to build? The equilibrium analysis explains why there are game-theoretic barriers to MDA adoption despite the clear advantages to building MDAs.

W6. Venue. The paper studies a scaling direction for agentic AI through LLM experiments and a game-theoretic model. Its questions about capability scaling, agent evaluation, and system design are directly relevant to the ICLR community.

Thanks for reply.

Reviewer_VAzaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-developer agentic systems, where independently developed agents are coordinated to solve tasks, versus routed systems that select one developer’s system. It proposes a δ\delta-MDA decomposition, routing, and composition framework, alongside a stylized game-theoretic model of developer incentives. Experiments on GAIA and SealQA evaluate specialist and generalist agent pools, routing behavior, costs, and transfer.

Critical strengths
  1. A useful formalization that connects coordination to incentives. Definitions 1–2 distinguish δ\delta-MDAs from routed-SDAs at the system level, while Section 6 and Appendix B make the economic argument mathematically explicit. In particular, Lemma 3 proves, under the additive skill and capability-proportional routing model, that Wδ≥WrSDAW^\delta \geq W^{\mathrm{rSDA}} on a fixed pool, and Propositions 1–5 clearly delimit how equilibrium behavior depends on compensation and production geometry.

  2. The empirical analysis identifies genuine composition effects within a fixed specialist pool. Figure 4a reports that the δ\delta-MDA on AS\mathcal{A}_S reaches about 70.1%70.1\% accuracy versus 62.3%62.3\% for routed-SDAs on the same pool, and Table 1 shows that learned contextual routing and a strong decomposition/composition agent materially drive this result. The paper also reports failure modes on narrow tasks in Figure 5, which makes the empirical story more credible than an unconditional claim that decomposition always helps.

Critical weaknesses
  1. The headline comparison does not isolate the claimed benefit of scaling across developers from pooling substantially more specialized development and execution capacity. Section 4 gives each specialist in AS\mathcal{A}_S five refinement rounds, while each generalist or multi-agent SDA corresponding to a bundle of k=6k=6 specialists receives 5k=305k=30 rounds. Figure 3 then compares a δ\delta-MDA that can decompose tasks across the entire pool of 54 specialists, totaling 270 refinement rounds, against individual SDAs, including a best AMAS\mathcal{A}_{\mathrm{MAS}} system that contains only one six-specialist bundle. This is consistent with the authors’ fixed global budget construction, but it is not a control for a single developer receiving the same total BB and being permitted to build an equally large internally orchestrated system or agent pool.

    This matters because the abstract’s central interpretation is that multi-developer composition creates a distinct scaling axis. The reported 70%70\% versus 67%67\% comparison to the strongest individual AMAS\mathcal{A}_{\mathrm{MAS}} SDA, and the 12-point comparison to a median generalist, could partly reflect that the MDA accesses all 54 heterogeneous components at inference while any routed-SDA is definitionally restricted to one smaller system. Indeed, Figure 4a shows that the matched-pool result against routed AG\mathcal{A}_G is much narrower than the headline comparison, and AMAS\mathcal{A}_{\mathrm{MAS}} routed-SDA performance is already close to the MDA result. A decisive control would allocate the full BB to one developer-controlled generalist/MAS, with comparable specialist count, orchestration budget, and inference-time access, then compare it to the δ\delta-MDA under matched execution budgets. At minimum, the abstract should avoid implying that the compared generalist systems have larger total development budgets: Section 4 instead states that total global budget is held fixed across pools.

Questions for authors
  1. [Clarification] Can the authors evaluate a single-developer system given the full aggregate refinement budget of the AS\mathcal{A}_S coalition, namely 270 rounds in the GAIA setting, with access to an equally large internal specialist/MAS structure? If such a matched control remains materially below the 70.1%70.1\% δ\delta-MDA result, it would substantially increase my confidence in the claimed multi-developer scaling effect.

  2. [Clarification] Figure 4 reports n=4n=4 runs and SEM bars, but Figure 3 presents the central 70%70\% MDA result and the 67%67\% strongest AMAS\mathcal{A}_{\mathrm{MAS}} point without an explicit run count or uncertainty. Are these values evaluated over the same repeated runs as Figure 4, and what are their corresponding variances? This would clarify whether the reported 3-point gap to the best individual multi-agent SDA is robust.

1 comment

Thank you for the careful review. Your reading of the budget construction is correct. The distinction between resources held by one developer and resources contributed by many developers is central to the scaling axis we study.

W1, Q1. A single developer with the full budget. Our central question starts from resources, access, and expertise held by independent parties: how can their contributions be combined to advance collective capability? The real-world analogy of giving one developer the entire aggregate budget would be that all developers, together with their resources, expertise, and goals, would coalesce into a single entity that can build one unified agentic system. This would be extremely unlikely. A key assumption throughout our work is that no single entity decides what all other developers build, and heterogeneous developers participate according to their own interests and deploy systems independently from other developers.

So the reviewer is correct that the MDA can draw on 270 rounds of development, while an individual generalist draws on 30. The total budget is matched across developer populations, not between the MDA and each individual contributor. However, this is exactly the scaling across developers we aim to study: by combining complementary contributions, an MDA can draw on development resources beyond those of the single largest developer, even when each contributor is much smaller. We thank the reviewer for pointing out the wording in the abstract that could be further tightened, and we will do so in revisions. Note that we evaluate the routed-SDA baselines in Fig. 4a, which select from the whole population and represent the same 270-round total budget.

Q2. Uncertainty in Fig. 3. The δ\delta-MDA in Fig. 3 is the same system as in Fig. 4a and scores 70.1 ± 1.6% over four runs (SEM). The best individual AMAS\mathcal{A}_\mathsf{MAS} system scores 66.9%, averaged over two runs. These results support the section's claim that small specialist developers become competitive with the strongest individual system, at under half its inference cost.

Ethics flag. All experiments run in a closed simulation on public benchmarks with agents that we generated. The δ\delta-MDA is an experimental system for studying this scaling direction. The concerns you identify about sensitive context, capability claims, and manipulation of routing or payment are important consequences of independent participation. §7 identifies security, privacy, and accountability as open problems. Permissions, data isolation, provenance, auditing, authentication, and handling malicious agents are requirements for real deployment and are part of why MDAs warrant further research.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their feedback. We reiterate our central question and contributions.

Scaling across developers. Multi-agent systems are growing larger, but remain bounded by the resources, expertise, data access, and integrations available to a single developer. Contributions from independent developers offer a natural next scaling axis. In a multi-developer agentic system (MDA), developers can build on existing capabilities and focus on what they can add, bringing resources into the system that no single developer could assemble alone.

Protocols such as A2A, AgentNet, or IoA already reflect strong interest in this direction. But interoperability standards, orchestration methods, or protocols alone do not establish why we should build MDAs, what challenges arise in doing so, or how we might overcome them. Our paper investigates these questions empirically and theoretically.

Why this is more than a larger MAS. A key challenge of MDAs is that, unlike in a MAS, no single developer can unilaterally design and build the system. An MDA is an emergent agentic system that no single entity controls. The kinds of agents that exist in an MDA, and indeed whether developers participate in an MDA at all, completely depend on the interests and incentives of heterogeneous developers who choose which agents to deploy. This completely changes the design problem: we must study both first-order effects such as how different agents work together in a system and second-order effects such as how a system design changes what kinds of agents developers are incentivized to contribute in the first place.

These questions are critical to answer because, despite significant interest in extending MAS beyond single developers, no such paradigm shift towards large-scale MDA deployment has occurred to date.

What our analysis contributes. Our experiments provide preliminary evidence for the promise of this scaling axis. On GAIA, simulated developers whose specialists score at most 35% alone reach 70% together. An agent built to complement an existing MDA scores only 19% alone, yet raises the MDA from 40% to 53%. A developer's contribution can therefore be much greater than standalone accuracy suggests. The experiments and theory also expose a chicken-and-egg problem: MDAs are competitive only with specialized agents. However, specialized systems without an MDA perform worse than general-purpose agents. Therefore, there is no incentive to build an MDA because of the lack of specialists, and no incentive to build specialist agents because a deployed MDA does not exist. To the best of our knowledge, no prior work has tried to provide theoretical or empirical arguments for the lack of MDA deployment.

Our work cannot perfectly simulate a real-world MDA. However, we believe it is important to use theory and experiments (even if they are stylized) to more explicitly illustrate the promise, challenges, and potential of this scaling direction. Doing so would, at the very least, serve to ground future work and conversations in a new and promising direction of agentic research.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.