acceptodds
Under review as a conference paper at ICLR 2027

Depth Over Breadth: Energy-Efficient Scaling of Multi-Agent LLM Systems

Abstract

Deploying a multi-agent LLM system (MAS) means choosing how far to scale three axes: per-agent tool-calling depth , agent count , and communication rounds . Each choice buys accuracy and burns energy, yet prior scaling studies price compute in tokens or dollars on closed API models where energy cannot be measured, and tokens misprice it: a decode token costs 335 a prefill token at short context under our setup. We measure hardware-level GPU energy at every LLM call across four topologies, four agentic benchmarks, and three locally served models (9B to 35B, the larger two on SWE-bench Lite only). We find that test-time scaling axes are not interchangeable: tool-calling depth () converts energy into accuracy most efficiently; agent breadth () helps less and costs energy linear in the number of agents; and communication rounds () buy no measurable accuracy once per-agent iterative depth is held constant (for homogeneous agents), while having steep energy cost. Based on our findings of this depth-over-breadth ordering, we develop a label-free configuration search-and-deploy policy: binary-search on a single agent to the deepest budget at which agents still utilize at least half of it, then deploy the target MAS topology at that budget with three agents. Against a two-phase depth-then-breadth scaling policy, this cuts search-and-deploy energy by 59% on average while giving up at most half a point of accuracy on three of four benchmarks; and against running the same label-free rule on the MAS topology instead of a single agent, our policy saves 51% at comparable accuracy.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, LLM agents, energy efficiency, inference, test-time scaling, tool use

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PRZvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to scale tool-using multi-agent LLM systems efficiently, comparing three choices: increasing each agent’s tool-use depth, adding agents, and adding communication rounds. It measures GPU energy directly across single-agent, independent, centralized, and decentralized systems. Its main finding is that increasing depth generally gives larger accuracy gains per unit of energy than adding agents, while additional communication rounds show no consistent average benefit when the total per-agent tool-step budget is held fixed.

Based on these results, the authors propose a configuration search policy that uses a single agent’s budget utilization to select a depth budget before deploying a multi-agent system. They evaluate the approach on FanOutQA, WorkBench, BrowseComp+, and SWE-bench Lite with Qwen3.5-9B, and repeat the scaling experiments on SWE-bench Lite with two additional models. The proposed policy reportedly reduces combined search and initial deployment energy by 51% relative to searching directly on the target multi-agent system, and by 59% relative to searching both depth and breadth, with accuracy losses of at most 0.5 percentage points on three of the four benchmarks.

Critical strengths

There are some critical stengths in the paper:

  1. Direct energy measurements give the scaling study practical value. The paper measures GPU energy across depth, agent count, and communication rounds, rather than treating token counts as interchangeable units of cost. The supporting energy model is fitted on 17,250 task records and achieves an R2R^2 of approximately 0.961, with a median absolute percentage error of 7.6% under the measured serving setup. This gives the analysis a substantial empirical foundation. I find this consequential because practitioners need to know which scaling choices improve accuracy efficiently, and token-based comparisons can obscure the different costs of processing inputs and generating outputs.

  2. The findings lead to a simple configuration policy with meaningful measured savings. Using a single agent’s budget utilization to select the depth of a multi-agent system is inexpensive and requires no task labels during configuration selection once the threshold is fixed. Table 1 reports an average 51% reduction in combined search and initial deployment energy relative to applying the same utilization rule directly on the target multi-agent topology, with an average accuracy difference of +0.4 percentage points. Compared with searching both depth and breadth, the reported saving is 59%, with accuracy losses no greater than 0.5 percentage points on three of four benchmarks. These results make the paper useful beyond its descriptive scaling analysis, although the benefit remains dependent on the benchmark and deployment setting.

Critical weaknesses

There are some critical weaknesses that I would like to mention in this paper:

  1. The evaluation does not consistently measure successful task completion. On WorkBench, a task is counted as correct when the gold action set is contained in the union of all agents’ recorded calls, regardless of order or additional incorrect actions (Appendix B.3). This can reward broader exploration even when the resulting database state is wrong. BrowseComp+ similarly accepts a response containing the gold answer anywhere, including responses listing several competing candidates. These choices matter because increasing depth or agent count can increase both action coverage and candidate coverage without improving actual correctness. On SWE-bench Lite, “Independent” agents also share a worktree and run sequentially, so the measured breadth gain includes agents building on earlier agents’ edits. Although these choices are disclosed, stronger evaluation checks are needed before the results can support conclusions about task accuracy and independent ensembling.

  2. The experiments support a narrower conclusion about communication than the paper’s explanation suggests. Fixing RK=30RK=30 controls the maximum number of tool steps per agent, but increasing RR also changes execution segmentation, intermediate answers, review calls, and accumulated context. Without a comparison that keeps these factors fixed while removing or replacing peer messages, the experiment cannot isolate the contribution of communication content. Context limits further complicate the interpretation: Appendix C.1 reports that WorkBench Decentralized achieves 80–86% accuracy with 0–2% of tasks experiencing truncation at R=1–10R=1–10, compared with 56% accuracy and 57% truncation at R=30R=30. The authors acknowledge this limitation, but Figure 5 still includes settings up to R=30R=30. Moreover, the argument in Section 4.3 does not establish that another homogeneous agent’s message contains no useful additional evidence. The results therefore support limited average benefits under the tested protocol, rather than a general explanation that homogeneous communication is merely self-conditioning.

  3. The configuration-search comparisons contain unresolved differences in coverage and evaluation protocol. Appendix E.1 states that the required breadth ladders were completed for depths selected at τ=0.5\tau=0.5, including FanOutQA K=7–20K=7–20, and that Table 1 averages all three topologies over all 200 splits. However, the Table A16 caption states that FanOutQA Centralized has a full breadth ladder only at K=10K=10, while also reporting that 47% of depth probes select K=7K=7. These statements appear inconsistent and need clarification. Table A15 separately acknowledges that D retains only splits with complete breadth ladders, fewer than half the splits in several Centralized cells, and that its A and D rows are not paired. This does not establish that Table 1 is incorrect, but it makes the supporting comparisons difficult to audit. In addition, Section 5.1 selects τ=0.5\tau=0.5 using held-out accuracy from the same evaluation pool, without a separate final test of that threshold. These issues leave uncertainty about the reported policy advantage.

  4. The depth-over-breadth conclusion is sensitive to the operating range, and its deployment scope remains insufficiently validated. Appendix C.2 explicitly reports that, when the depth fit is restricted to K≥3K\geq3, BrowseComp+ has a topology-mean depth elasticity of 0.09, below its breadth elasticity of 0.19. Thus, the headline ordering partly depends on including the earliest depth increments and does not hold uniformly across benchmarks and budget ranges. Comparisons on the same task set and within comparable energy ranges would help establish when prioritizing depth is the better choice. The additional-model experiments cover only SWE-bench Lite (Section 4.5), while all actual agent runs use serial batch-1 serving (Appendix B.2). The batch-64 analysis re-prices existing trajectories using adjusted energy coefficients rather than evaluating agent execution under batching (Appendix A.3). These are useful checks, but they do not establish that the recommended scaling order remains preferable under realistic concurrent deployment.

Questions for authors

The following are my questions for the authors:

  1. Do the main scaling results hold under stricter measures of task correctness? Appendix B.3 states that WorkBench accepts the gold action set within all recorded calls without penalizing additional incorrect actions, while BrowseComp+ accepts the gold answer appearing anywhere in the response. Have you checked whether the depth-versus-breadth ordering and policy comparisons remain consistent under WorkBench’s state-based evaluator and the official BrowseComp+ evaluation? Please also clarify whether SWE-bench accuracy uses the complete harness “resolved” criterion.

  2. Can you reconcile the breadth coverage statements and clarify how D is evaluated? Appendix E.1 says the required FanOutQA ladders cover K=7–20K=7–20 and Table 1 averages all 200 splits, whereas Table A16 says Centralized has a complete breadth ladder only at K=10K=10, despite 47% of depth probes selecting K=7K=7. What happens when a split selects a depth without a complete ladder? Please specify the included splits and any fallback rules for Table 1, and explain how these differ from the explicitly unpaired comparisons in Table A15.

  3. What supports attributing the communication results to a lack of useful peer information? Fixing RK=30RK=30 still changes execution segmentation, intermediate responses, and review calls. Is there a comparison that preserves these factors while removing or replacing peer messages? Separately, how does the distributional identity in Section 4.3 establish that a peer’s independently retrieved evidence cannot improve the receiving agent’s answer? Please distinguish the empirical finding under your protocol from the stronger theoretical interpretation, particularly given the truncation effects reported in Appendix C.1.

  4. Was the utilization threshold evaluated independently of its selection? Section 5.1 selects τ=0.5\tau=0.5 from held-out accuracy across the evaluated settings. Were those same task outcomes subsequently used to report the policy’s final accuracy? Please clarify the separation between threshold selection and final evaluation, and whether the reported advantage persists with a threshold fixed before evaluating the test tasks. This would help establish how well the policy transfers beyond the tasks used to choose its setpoint.

These requests concern the validity and interpretation of the current results. Clear answers, with supporting evidence where available, could move my assessment toward weak acceptance.

Reviewer_eNQhOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the energy efficiency of multi-agent LLM scaling along three axes: tool-calling depth, number of agents, and communication rounds. It finds that scaling depth provides the best accuracy-energy trade-off. Based on these observations, the paper proposes a label-free policy that uses single-agent tool-budget for the deployment configuration..

Critical strengths
  1. The paper provides an interesting perspective on multi-agent LLM scaling by considering 3 dimensions: hardware-level energy consumption and systematically studying the accuracy-energy trade-off.
  2. The evaluation for the given benchmark (SWE-bench Lite) is very detailed.
Critical weaknesses
  1. Evaluation is my main concern. As an evaluation-driven paper, the simple datasets are insufficient to support the strong conclusion that "depth is more energy-efficient than breadth". The hardest task is SWE-bench Lite. More challenging and diverse multi-agent workloads, such as complex coding [1], deep research [2], and multi-agent social simulation [3], are needed to demonstrate whether the claim generalizes.

[1] Swe-bench pro: Can ai agents solve long-horizon software engineering tasks?

[2] Deepresearch bench: A comprehensive benchmark for deep research agents

[3] Agentsociety: Large-scale simulation of llm-driven generative agents advances understanding of human behaviors and society

  1. The batch-size-1 evaluation substantially weakens the energy conclusions. All main experiments use batch size 1 without sufficient justification. Decoding at batch size 1 is inherently resource-inefficient, while continuous batching is essential for improving GPU utilization in modern LLM serving [4]. This is particularly important for multi-agent systems, where multiple agents naturally generate concurrent requests that can be batched with limited additional energy cost. By enforcing batch-size = 1, the evaluation excludes this important advantage of breadth and therefore substantially bias the claimed "depth-over-breadth" cnclusion.

[4] Efficient memory management for large language model serving with pagedattention

  1. Missing related work and unclear novelty. The paper does not sufficiently discuss existing work on energy-efficient LLM serving systems [5], making it unclear whether energy optimizations would affect the paper’s conclusions. Moreover, evaluating agent configurations under resource or dollar cost has already been widely studied [6, 7]. It is unclear what the main novelty of this evaluation is beyond replacing existing cost metrics with measured GPU energy.

[5] Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms OSDI 26

[6] AI Agents That Matter

[7] The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective, HPCA 2026

  1. Poor writing. The paper lacks a clear technique section that summarizes its technical contributions.

  2. The appendix contains many additional analyses that appear to weaken the main conclusions, but these are not clearly reflected in the main text. or example, the depth-over-breadth advantage becomes much smaller when K >=3. I suggest that the authors reorganize the evaluation more systematically and carefully reassess whether the existing evidence supports the paper’s major claims.

Questions for authors

I believe the paper requires substantial improvement and restructuring, and I am unlikely to change my opinion. I would suggest the authors to carefully address the concerns raised in the weaknesses in a future revision.

Moreover, I suggest explicitly reporting key serving configurations, such as the use of SGLang, tensor parallelism (TP), model precision, and relevant serving parameters, in the main paper rather than only in the appendix. These configurations can substantially affect GPU utilization and energy consumption and are therefore critical for the paper’s conclusions.

Reviewer_MbtqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper mainly tests how three multi-agent parameters—per-agent tool-calling depth K, agent count N, and communication rounds R—affect system energy efficiency and accuracy. The paper finds that among the three parameters tested above, tool-calling depth (K) converts energy into accuracy most efficiently. Based on this finding, the paper develops a label-free multi-agent system configuration search-and-deploy policy.

Critical strengths

The paper identifies a very practical empirical configuration guideline for saving energy in agent systems, namely allocating computational resources as much as possible to the agents' computational depth, followed by the number of agents. The paper also provides a corresponding heuristic strategy to apply this guideline to multi-agent system configuration.

Critical weaknesses

The paper's analysis of how scaling the number of agents affects energy consumption and performance may underestimate the efficiency and effectiveness of agent-count scaling. Agent-count scaling is expected to have advantages when agents and models with different roles and capabilities collaborate. In addition, for very long contexts, when a single agent's memory space and the LLM cannot hold all historical context, agent-count scaling may also have advantages. The paper does not provide much systematic evaluation of these situations and experiments.

The authors' experiments are all conducted on relatively small models. It is unclear whether the patterns claimed by the authors still hold for larger and more capable models.

Questions for authors

We hope for a more systematic and comprehensive discussion of the work's limitations. The guideline provided by the paper is relatively general, but the scope of applicability of this guideline has not been systematically and clearly analyzed and discussed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.