acceptodds
Under review as a conference paper at ICLR 2027

Cost- and Quality-Aware Pruning of Multi-Agent LLM Systems

Abstract

Multi-agent large language model systems often incur substantial inference costs, even when additional agents contribute little to the final answer. We study which agents can be removed from a deployed system while maintaining a specified quality requirement. We apply a novel extension of the classical random-walk model in which link traversals carry costs and rewards, using closed-form first-passage calculations to evaluate sequential execution. The process ends when the current answer is returned by a reporting manager. Traversal costs represent the accounted model-call cost, while quality increments represent changes in answer correctness whose sum equals final correctness. We first calibrate agent behavior, then evaluate candidate subsets, and construct a pruning sequence using predicted quality and cost changes. We evaluate candidate subsets using closed-form matrix calculations, thereby avoiding expensive trajectory simulations and additional model calls. Finally, we deploy the least costly candidate design whose bootstrap screening percentile meets the quality target. In controlled over-provisioned systems, the pruning method reduces accounted API cost by 73% on competition mathematics, 15% on code repair, and 10% on heterogeneous code repair. Additional mathematics experiments with Qwen, Kimi, and GLM yield cost reductions of 34.8%–73.5% at a quality target of .80. The selected designs achieve held-out accuracies at least as high as the corresponding full systems.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
multi-agent LLM systems, inference cost, agent pruning, weighted first passage times, efficient AI

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_JQMMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the problem of removing unnecessary agents in multi-agent systems to reduce cost, and proposes a method based on Markov chains, weighted first-passage time, and greedy pruning for agent reduction.

Critical strengths

I suppose that this paper has almost no strengths that would warrant acceptance at ICLR. As I discuss in the weaknesses below, the paper has significant problems in both motivation and experiments, which are difficult to resolve during the rebuttal period. This is why I give my rating.

Critical weaknesses
  1. The weakness of "redundancy of agents" and "evaluation of pruning combinations" mentioned by the authors are inherent challenges of the multi-agent optimization task itself. There is a lack of analysis regarding the shortcomings of existing prior work, making it uncertain that what is the contribution lies relative to prior work. This not only lacks inspiration for related research but also casts doubt on the value of the paper itself.
  2. Although many related works are discussed, the paper only compares the proposed method with simple pruning strategies, lacking comparison with the current mainstream multi-agent methods, which calls into question the effectiveness of the method.
Questions for authors
  1. There are many overly defensive statements in the paper. For example, Lines 239–240 claim that "It does not establish optimality of a particular pruning ranking or identify the cause of an empirical difference between retained subsets". Lines 425–426 note that "These comparisons do not establish superiority of marginal efficiency under the specified quality-constrained cost objective". While this is not strong evidence, such self-negation of the proposed method appears odd and I hopes that this could bring the AC's attention.
  2. Sections 3 and 4 contain many confusing or undefined symbols, such as MqM_q, McM_c.
Reviewer_1idrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for pruning sequential multi-agent LLM systems under a quality target. It uses a calibrated Markov-chain model to predict the cost and accuracy of agent subsets without additional LLM calls, while using greedy removal and bootstrap screening select a reduced design. Experiments on controlled mathematics and code-repair systems, including additional model providers for mathematics, show lower accounted API costs while matching or exceeding full-system held-out accuracy.

Critical strengths
  1. The studied problem is well motivated and valuable. Removing unnecessary agents under an explicit accuracy requirement is a practical way to reduce the inference cost of deployed multi-agent systems.
  2. The shared cost-and-quality formulation is useful and mathematically interpretable. The correctness increments sum exactly to final correctness, while the first-passage calculation accounts for repeated calls and downstream execution within the assumed model.
  3. The evaluation separates calibration from held-out deployment and includes different worker configurations and additional model providers. The paper also explicitly reports cases where simpler removal rules perform better and acknowledges the limitations of its bootstrap screen.
Critical weaknesses
  1. The collaboration workflow is restrictive and insufficiently motivated, limiting the method's generality. Workers revise a single answer sequentially, with uniform forwarding and a fixed stopping probability of 0.5; the manager returns the answer unchanged. This policy ignores task requirements and answer quality, making it unclear whether the savings reflect effective pruning of useful collaboration or removal of overhead introduced by the initial workflow. Task-dependent routing, parallel aggregation, branching task decomposition, and adaptive stopping are not evaluated. Although the introduction briefly acknowledges the sequential scope, the authors are suggested to better justify this design and clarify what additional modeling is needed to support more general collaboration patterns. Also, the authors are suggested to clarify the current scope limitation more prominently.

  2. I feel the behavioral model and its transfer after pruning are not sufficiently validated. Each worker's repair and damage rates depend only on incoming correctness, omitting problem difficulty, error type, and the preceding worker. These rates and mean costs are then held fixed after pruning, although the remaining agents may produce different kinds of inputs. Calibration is also sparse: Appendix E.5 reports only 6, 10, and 15 revisions of incorrect inputs across the three systems. The bootstrap does not capture dependence between episodes of the same problem or provide a guarantee for the selected design, as the authors acknowledge. More extensive validation across retained multi-agent subsets is needed to establish reliable quality prediction.

  3. It seems that the proposed greedy rule is not fully supported under the stated objective. Equation (9) minimizes cost subject to a quality target, but the code-repair random order selects a design costing 0.906 usd per 100 problems, compared with 2.001 usd for marginal efficiency, while both pass every tested target. In the heterogeneous system, random removal achieves the same observed accuracy at lower cost. These results leave the proposed ranking's benefit unclear, since accuracy beyond the target receives no additional reward in the objective. With only seven agents, exhaustive feasible-subset selection within the fitted model would provide a useful reference for assessing the greedy search.

  4. The experiments and comparisons provide relatively limited evidence of practical effectiveness. Every tested roster includes a deliberately harmful reviser, all main proposed selections reduce to single agents, and each system uses only 40 held-out problems. This provides limited evidence that the method can preserve useful collaboration among complementary agents. Although simple removal baselines and additional singleton controls are included, there are no experimental comparisons with the related pruning and removal-attribution methods discussed in Section 2. Evaluation on teams without deliberately harmful workers, tasks that benefit from collaboration, and applicable prior methods under matched calibration budgets would strengthen the contribution.

Questions for authors

Most of my questions are already discussed in the Weaknesses section. I briefly restate some of them here for clarity:

  1. What supports the assumption that calibrated repair and damage rates remain stable after pruning?
  2. What advantage does the proposed greedy rule offer when simpler baselines achieve lower cost while meeting the same quality targets?
Reviewer_hAHfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a cost and quality-aware method for pruning sequential multi-agent LLM systems. It models execution as a Markov chain whose transitions carry call costs and correctness rewards, estimates agent behavior from calibration episodes, and uses closed-form first-passage calculations to evaluate reduced agent sets. A greedy removal sequence and bootstrap quality screen are then used to select a lower-cost design. The evaluation covers three seven-agent systems on mathematics and code tasks, with additional mathematics experiments using Qwen, Kimi, and GLM configurations.

Critical strengths
  1. The paper addresses a practical problem. Reducing the inference cost of an existing multi-agent system while meeting a specified quality target is a useful deployment objective. The paper also correctly recognizes that an agent’s cost depends on how often the system routes to it, not just on its individual call price.

  2. The evaluation includes real model executions and held-out testing. The authors test reduced designs on held-out problems and report both accuracy and accounted API cost. The provider experiments also offer some evidence that the analysis is not tied to a single model configuration, although this evidence is limited to mathematics tasks.

Critical weaknesses
  1. The methodological contribution is incremental, and its novelty is not clearly established. The core analysis applies weighted first-passage calculations to a Markov model of agent execution, with correctness changes represented as rewards that sum to final correctness. The pruning procedure then uses a greedy, one-agent-at-a-time removal heuristic. These are reasonable design choices, but the paper does not make clear what fundamentally new methodological insight they provide.

  2. The paper omits comparisons with the most relevant existing methods, and its own results do not establish an advantage over simple alternatives. The related-work section discusses AgentPrune, AgentDropout, and removal-based attribution, but the experiments compare only against random, direct-cost, and individual-accuracy removal orders. These baselines do not show whether the proposed method improves on the closest prior approaches.

  3. The empirical evidence is too narrow and small to validate the method’s practical effectiveness. The main evaluation uses three deliberately over-provisioned seven-agent systems, with duplicates, a deliberately harmful agent, and higher-priced model tiers. It includes only 60 calibration problems and 40 held-out problems per system, while the cross-provider experiments cover mathematics only. The reported savings, especially the largest reduction, may depend heavily on the constructed redundancy and harmful-agent setup. In addition, the bootstrap screen is pointwise rather than selection-adjusted, and the authors note that its resampling does not account for dependence between episodes from the same problem. The held-out results are useful as an initial demonstration, but they are not strong enough to establish reliable quality-constrained pruning across realistic systems.

  4. The paper is difficult to follow and does not meet publication-level presentation standards. The paper introduces a substantial amount of notation and moves between the execution model, reward construction, first-passage analysis, and pruning heuristic without making their conceptual relationships sufficiently clear. The motivation for the specific modeling assumptions and the distinction between the analytical contribution and the greedy selection rule are also difficult to track.

Questions for authors

See strengths and weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.