acceptodds
Under review as a conference paper at ICLR 2027

Not Every Agent Needs the Strongest Model: Unified Prompt-Model Optimization for Heterogeneous LLM-based Multi-Agent Systems

Abstract

Large language model (LLM)-based multi-agent systems (MAS) rely on role-specific prompts and execution models to solve complex tasks collaboratively. However, prompt optimization typically keeps model assignments fixed, leaving differences in role requirements and their associated inference costs outside the search. We present UniMASO, a Unified framework for heterogeneous Multi-Agent System Optimization through the joint adaptation of prompts and model assignments. UniMASO evaluates candidate updates through local, downstream, and global feedback, using cost-aware Pareto search to retain useful quality–cost trade-offs. Topology-aware co-evolution then coordinates these updates according to agent roles and dependencies. Cross-level trajectory memory further supplies historical successes and failures to guide subsequent optimization. Experiments on five benchmarks and five execution topologies show that UniMASO achieves competitive task performance at substantially lower deployment cost than the strongest homogeneous configurations. Under centralized execution, it improves the average score from 69.79 to 70.40 while reducing token-weighted deployment cost by 85.2% relative to DeepSeek-V4-Pro with MASPO. These results support joint prompt–model adaptation as an effective approach to efficient multi-agent collaboration. The code is available at https://anonymous.4open.science/r/UniMASO-864D.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Systems, Large Language Models, Prompt Optimization, Heterogeneous Agents, Cost-Aware Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YWfxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the joint optimization of role prompts and execution models in heterogeneous LLM-based multi-agent systems. The authors propose UniMASO, a framework that jointly optimizes task quality and deployment cost through local prompt–model search, multi-level feedback, cost-aware Pareto search, topology-aware global coordination, and trajectory memory. The empirical study covers five benchmarks and five multi-agent execution topologies, and includes ablation studies of each major component as well as several parameter analyses.

In the centralized setting, compared with DeepSeek-V4-Pro + MASPO, UniMASO improves the average task score from 69.79 to 70.40 while reducing the weighted token deployment cost of task agents by approximately 85.2%. This combination of a modest gain in average performance and a substantial reduction in deployment cost constitutes a practically meaningful quality–cost trade-off.

Critical strengths
  1. The problem addressed is of practical importance, and the motivation for joint optimization is well founded. An agent's behavior is determined by its role prompt but is also bounded by the capabilities of the underlying execution model. By treating these two variables jointly rather than in isolation, the paper more fully captures the coupling among role requirements, model capabilities, and inference cost, which is directly relevant to the deployment of heterogeneous multi-agent systems.

  2. The method is internally coherent and explicitly accounts for inter-agent interactions. Integrating local, downstream, and global feedback avoids evaluating a modification solely on the basis of a single role's performance. Trajectory memory allows prior optimization experience to be reused, while system-level coordination manages the mutual compatibility of local updates. These design choices align well with the dependency structure of multi-agent systems and with the stated quality–cost objective.

  3. The empirical study is substantial and examines the method along multiple dimensions. The evaluation spans five diverse benchmarks and five execution topologies, and is complemented by ablations of the main components, hyperparameter analyses, memory transfer experiments, and an analysis of model assignment behavior. The evaluation is reasonably comprehensive, and the reported results indicate a consistent deployment cost advantage across settings.

Critical weaknesses
  1. The presentation of the method could be further strengthened. Algorithm 1 outlines the overall optimization procedure, and the appendix documents the optimizer and evaluator configurations as well as the invocation templates. However, the rules governing candidate retention and configuration updates, namely those embodied by the operators SelectB\mathrm{Select}_B, BEST(Bi)\mathrm{BEST}(B_i), and Ωbal(M)\Omega_{\mathrm{bal}}(M), are insufficiently specified (see Question 1). Since these rules determine both the search trajectory and the final configuration, the current presentation makes the algorithm difficult to understand precisely and to reproduce faithfully.

  2. The experimental comparison with model routing methods remains insufficient. MasRouter is discussed in the related work, but no direct empirical comparison is reported. Because all existing baselines are prompt optimization methods operating with a fixed, homogeneous model, the current evidence does not establish that joint optimization outperforms optimizing the two components separately.

  3. The applicability of the method to broader model pools remains to be established (minor limitation). The five execution models do span different capability and cost tiers, but they are drawn exclusively from the Qwen and DeepSeek model families. Consequently, the existing evidence characterizes the method's behavior only within this particular model pool. Its sensitivity to model pool composition, and its effectiveness with other model families or with mixed pools that include commercial API models, lack direct empirical support.

Questions for authors
  1. Could the authors provide definitions for the main operators in Algorithm 1? Specifically, (i) how does SelectB\mathrm{Select}_B (line 15) truncate the non-dominated candidate set when its size exceeds the beam width; (ii) what criterion does BEST(Bi)\mathrm{BEST}(B_i) (line 18) use to determine the locally optimal configuration; and (iii) what is the explicit form of Ωbal(M)\Omega_{\mathrm{bal}}(M) in Eq. (6), and how is the corresponding optimization problem solved?

  2. The paper does not appear to state whether the experiments were independently repeated. How many independent runs were conducted for each setting, and are the reported values averages over these runs? It would be helpful if the authors could report the mean together with the standard deviation or standard error across independent runs to substantiate the stability of the reported results.

  3. Could the authors provide a comparison with model routing methods such as MasRouter under comparable model pools, topologies, datasets, and search budgets? If such a comparison is infeasible, which specific differences in experimental setup preclude it?

  4. How sensitive is the method to the composition of the model pool? Are there results on other model families or commercial API models, or any further analysis that delineates the method's scope of applicability?

6 comments
AuthorsOpenReview ↗

Official Comment to Reviewer YWfx (1/3)

Q1. Could the authors define the main operators in Algorithm 1? Specifically, (i) how does line 15 truncate the nondominated set when it exceeds the beam width, (ii) what criterion does line 18 use to determine the local best configuration, and (iii) what is the explicit balance term in Eq. (6), and how is the resulting optimization solved?

Thank you for highlighting the need to clarify these operators. We have added formal definitions of these operators in Appendix H.

(i) Candidate truncation. Candidate aa dominates bb if Ui(a)≥Ui(b)U_i(a) \geq U_i(b) and Ci(a)≤Ci(b)C_i(a) \leq C_i(b), with at least one strict inequality. We remove dominated candidates first. If more than BB nondominated candidates remain, Select_B retains the BB candidates with the highest cumulative search scores; otherwise, it retains all nondominated candidates. Candidates from subsequent Pareto fronts are not added to fill the beam.

(ii) Local-best selection. For a candidate configuration zi′z_i', its utility and cost-adjusted score are

Ui(zi′)=αqiloc(zi′)+βqilook(zi′)+γqiglob(zi′),si(zi′)=Ui(zi′)ηi(zi′)−λ−12,U_i(z_i')=\alpha q_i^{\mathrm{loc}}(z_i')+\beta q_i^{\mathrm{look}}(z_i')+\gamma q_i^{\mathrm{glob}}(z_i'), \qquad s_i(z_i')=U_i(z_i')\eta_i(z_i')^{-\lambda}-\frac{1}{2},

where ηi(zi′)=Ci(zi′)/Ci(zi)\eta_i(z_i')=C_i(z_i')/C_i(z_i) is the measured cost relative to the current configuration. A candidate is eligible for expansion when si(zi′)>0s_i(z_i')>0. Its score contributes to the cumulative search-path score, and BEST selects the retained state with the highest cumulative path score.

(iii) Balance term and consensus assignment. Let NN be the number of roles, KK the size of the global model pool, and nin_i the number of models feasible for role ii. For an assignment M=(m1,…,mN)\mathbf M=(m_1,\ldots,m_N), let jij_i be the rank of mim_i in the predefined ascending capability order of feasible models, starting at one. Let LiL_i and HiH_i count feasible models with costs strictly lower and strictly higher than that of mim_i, respectively. The balance term is

Ωbal(M)=(1−1K)∑i=1N[ji−1max⁡{ni−1,1}+12(1−(Li−Hini)2)].\Omega_{\mathrm{bal}}(\mathbf M) = \left(1-\frac{1}{K}\right) \sum_{i=1}^{N} \left[ \frac{j_i-1}{\max\{n_i-1,1\}} + \frac{1}{2} \left(1-\left(\frac{L_i-H_i}{n_i}\right)^2\right) \right].

The proposal support is Ai(m)=πi(m)(1−Hˉi)A_i(m)=\pi_i(m)(1-\bar H_i), where πi(m)\pi_i(m) is the fraction of focal views proposing mm for role ii. The Shannon entropy Hˉi\bar H_i is normalized by log⁡∣Mi∣\log|\mathcal M_i|, where Mi\mathcal M_i is the set of feasible models for role ii; it is set to zero when ∣Mi∣=1|\mathcal M_i|=1. Let F\mathcal F contain complete assignments satisfying model availability, role-sharing, and routing constraints. The consensus assignment maximizes

Mcon=arg⁡max⁡M∈F[∑i=1NAi(mi)+Ωbal(M)].\mathbf M_{\mathrm{con}} = \arg\max_{\mathbf M\in\mathcal F} \left[ \sum_{i=1}^{N}A_i(m_i)+\Omega_{\mathrm{bal}}(\mathbf M) \right].

AuthorsOpenReview ↗

Official Comment to Reviewer YWfx (2/3)

Q2. Were the experiments independently repeated? How many independent runs were conducted for each setting? Are the reported values means over these runs? Reporting means and standard deviations or standard errors would help establish result stability.

The main tables in the original manuscript report one independent optimization run for each setting. In the supplementary experiments, we conducted three independent runs for both UniMASO and the DeepSeek-V4-Pro + MASPO baseline under the centralized topology, covering all five benchmarks. Both methods used random seeds 42, 1024, and 2048. Other topologies retain the single-run results from the original manuscript. Each run starts from the same initial configuration, reinitializes the beam and trajectory memory, and independently executes the complete optimization process. We then evaluate the final configuration selected in that run using the same test set and evaluation protocol. The table lists the task scores and deployment costs for each run, with the final row reporting the mean ± sample standard deviation across the three seeds.

MethodSettingAGIEval Acc.CostGPQA Acc.CostBFCL Acc.CostLiveCodeBench pass@1CostHotpotQA EMCostAverage ScoreCost
MASPOSeed 4278.12249.680.30115.292.60216.635.11124.062.80336.069.79208.28
MASPOSeed 102477.34251.481.31111.893.00222.134.35123.563.00330.669.80207.88
MASPOSeed 204878.91243.081.31120.792.60213.535.88119.963.00327.770.34204.95
MASPOMean78.12 ± 0.79248.0 ± 4.480.97 ± 0.58115.9 ± 4.592.73 ± 0.23217.4 ± 4.435.11 ± 0.77122.5 ± 2.262.93 ± 0.12331.4 ± 4.269.98 ± 0.31207.04 ± 1.82
UniMASOSeed 4282.4256.978.2833.092.4012.235.1113.363.8039.370.4030.92
UniMASOSeed 102483.2059.477.7832.892.2012.735.8813.963.6040.170.5331.79
UniMASOSeed 204881.6458.577.2733.692.2011.835.8812.863.8039.170.1631.16
UniMASOMean82.42 ± 0.7858.3 ± 1.377.78 ± 0.5133.1 ± 0.492.27 ± 0.1212.2 ± 0.535.62 ± 0.4413.3 ± 0.663.73 ± 0.1239.5 ± 0.570.36 ± 0.1931.29 ± 0.45

Taking the macro-average of the five task scores within each run and then computing statistics across the three seeds gives a mean task score of 70.36 ± 0.19 and a mean deployment cost of 31.29 ± 0.45M for UniMASO. These results show that, under the centralized topology evaluated, the overall task performance and deployment cost obtained with the three random seeds are relatively close. We have added the three-seed stability results to Appendix G.1.

AuthorsOpenReview ↗

Official Comment to Reviewer YWfx (3/3)

Q3. Can the authors compare UniMASO with model-routing methods such as MasRouter under comparable model pools, topologies, datasets, and search budgets? If comparison is infeasible, which experimental differences prevent it?

Thank you for suggesting the addition of a model-routing baseline. The main experiments in this paper focus on system-configuration optimization before MAS deployment. We therefore prioritized prompt-optimization methods that also optimize before deployment and reuse the resulting configuration on subsequent samples. These methods optimize role prompts under a given model assignment, whereas UniMASO jointly determines each role’s prompt and execution model at the same stage. MAS model-routing methods, represented by MasRouter, focus on per-sample model selection at runtime: the system selects an execution model for a sample only after receiving its specific input. Given this task difference, the original manuscript discussed model-routing methods in related work. Following the reviewer’s suggestion, we added a direct comparison with MasRouter to examine the performance of pre-deployment configurations relative to per-sample routing under the same topology, model pool, and test data.

We use the setting in MasRouter’s original paper, in which its router is attached to an existing multi-agent system. Both methods use the same centralized topology, role definitions, initial prompts, execution-model pool, and data splits, as well as the same task metrics and deployment-cost coefficients, to ensure a fair comparison. During evaluation, MasRouter routes each input, whereas UniMASO uses the prompts and model assignment selected before deployment for all evaluation samples.

MethodAGIEval Acc.CostGPQA Acc.CostBFCL Acc.CostLiveCodeBench pass@1CostHotpotQA EMCostAverage ScoreCost
MasRouter74.2258.271.7229.492.0020.538.1721.958.4037.166.9033.42
UniMASO82.4256.978.2833.092.4012.235.1113.363.8039.370.4030.92

As shown in the table, UniMASO achieves a higher average score than MasRouter under the fixed topology and model pool described above. This indicates that, on the evaluated tasks, a configuration obtained through pre-deployment joint optimization can be reused across samples while maintaining a quality advantage over per-sample routing. We have added this comparison to Appendix G.2.

Q4. How sensitive is the method to model-pool composition? Are there results with other model families or commercial API models, or further analysis delineating the method's scope?

Thank you for raising the question about model-pool composition and the scope of the method. UniMASO jointly optimizes each role’s prompt and execution model at the system level, screening candidates based on task performance, their effects on downstream collaboration, and call cost. The process requires neither access to model weights or gradients nor changes to model architecture. Therefore, any model can be included in the same search procedure as long as it supports the task interface of the corresponding role, its outputs can be evaluated, and its call cost can be accounted for. This design also applies to models available only through commercial APIs.

SettingAGIEval Acc.CostGPQA Acc.CostBFCL Acc.CostLiveCodeBench pass@1CostHotpotQA EMCostAverage ScoreCost
Default model pool82.4256.978.2833.092.4012.235.1113.363.8039.370.4030.92
Expanded model pool83.2060.778.7937.492.0012.935.8817.164.0040.970.7733.80

We added GLM-5 and Grok-4.7 to the original model pool, which consists of Qwen3.5-4B, Qwen3.5-9B, Qwen3.5-27B, DeepSeek-V4-Flash, and DeepSeek-V4-Pro, and reran UniMASO on all five benchmarks under the centralized topology. As shown in the table, the task scores across the five benchmarks and the average deployment cost remain stable under this expansion. With the two model families included, UniMASO continues to use the same search and evaluation rules, assessing the match between each model and role prompt based on actual execution feedback without changing the optimization method for the internal implementation of new models. We have added the model-pool analysis to Appendix G.3.

Reviewer_p8o7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigate whether multi-agent LLM systems can retain strong task performance while reducing inference cost by jointly optimizing each agent’s prompt and model assignment? In doing so, the authors proposed the UniMASO framework, which searches over prompt–model configurations without updating model weights. It evaluates candidate changes using local role fulfillment, downstream effects, and final response quality. Cost-aware Pareto search retains promising quality–cost tradeoffs. Topology-aware coordination reconciles updates across interacting agents, while trajectory memory retrieves previous successes, failures, and cost observations to guide subsequent optimization. This allows simpler roles to use cheaper models while reserving greater capacity for demanding reasoning or verification. Experiments results shows that the proposed framework can achieve similar performance while greatly reducing the deployment cost.

Critical strengths

The main strength of the paper is that the authors recoginze that an agent’s prompt and model choice affect each other and downstream agents. Their controlled comparison supports optimizing them together: joint optimize both achieves better quality and lower cost.

Critical weaknesses

In the experiment, the reported 85.2% saving excludes offline optimization, evaluator calls, and other control calls. If I understand correctly, UniMASO needs configuration for every benchmark. Therefore, the optimization cost needs to be counted in the final cost. I do not see this reported in the paper.

Questions for authors

What is the optimization cost of the UniMASO across every benchmark?

4 comments
AuthorsOpenReview ↗

Official Comment to Reviewer p8o7 (1/2)

Q1. What is the optimization cost of UniMASO on every benchmark?

Thank you for pointing out that offline optimization costs should be included in the final cost. Under the centralized setting, we optimized each of the five benchmarks separately. Table 7 itemizes the input tokens, output tokens, and weighted cost consumed by one optimization run for each benchmark. The accounting covers the entire process from the start of search to selection of the final configuration, including execution of candidate multi-agent systems during search, optimizer and evaluator calls, and UniMASO’s Co-Evolver calls. The complete optimization cost across the five benchmarks is 206.3M for UniMASO and 4,710.6M for MASPO under the same accounting. Input and Output are raw token counts in millions (M), while Cost is measured in millions (M) of relative weighted-token units.

MethodAGIEval-MATHGPQA-DiamondBFCL-V3HotpotQALiveCodeBench
InputOutputCostInputOutputCostInputOutputCostInputOutputCostInputOutputCost
UniMASO5.920.08551.94.140.07237.01.930.06519.04.890.0742.86.450.07755.6
MASPO7.480.1031188.45.210.086843.12.370.074421.26.190.082979.98.170.0941278.0

After including these costs, UniMASO retains an overall cost advantage in this experiment. The 85.2% reported in the original paper measures the deployment-cost reduction after configuration selection. To answer the question about total cost, we sum two actual expenses for each benchmark: one configuration optimization for that benchmark and one run of the selected configuration over its complete evaluation set. Across the five benchmarks, UniMASO’s total cost is 360.9M (206.3M for optimization + 154.6M for evaluation-set execution), while DeepSeek-V4-Pro + MASPO costs 5,752.0M (4,710.6M for optimization + 1,041.4M for evaluation-set execution). The UniMASO execution sum uses the more precise per-benchmark costs reported in the manuscript. This comparison includes one complete optimization for every benchmark and does not assume future requests to amortize the cost. Both methods use their respective reported optimizer models, so the difference in weighted cost reflects both token usage and model cost coefficients. We have added the per-benchmark optimization-cost comparison to Section 5.5 (Table 7).

AuthorsOpenReview ↗

Official Comment to Reviewer p8o7 (2/2)

Q2. Can one UniMASO configuration be reused across benchmarks to reduce optimization costs?

Thank you for raising this important question about cross-benchmark transfer. After deployment, the same configuration can continue to handle subsequent questions. During optimization, prompts and execution models are selected for each role. Once the configuration is fixed, new questions are handled directly by these task agents, with no further calls to the optimizer, evaluator, or Co-Evolver. Thus, the cost in the table counts one configuration search; additional requests under the same configuration incur only the corresponding task-execution cost.

The reviewer also raises an important practical question: must a separate optimization cost be paid for every benchmark? The main experiments did optimize each of the five benchmarks separately. To test whether a configuration can be reused across benchmarks in the same domain, we conducted bidirectional transfer between AGIEval-MATH and MATH-500: we optimized using the inputs of one benchmark, then directly applied the complete set of selected role prompts and model assignments to the other benchmark. Neither direction used inputs from the target benchmark for optimization; therefore, the additional configuration-search cost on the target benchmark is zero.

Target datasetMethodOptimization poolAccuracy (%)Deployment cost (M)
AGIEval-MATHVanilla MASNone76.17217.3
AGIEval-MATHMASPOAGIEval-MATH78.12249.6
AGIEval-MATHUniMASO (in-domain)AGIEval-MATH82.4256.9
AGIEval-MATHUniMASO (cross-dataset)MATH-50081.2558.7
MATH-500Vanilla MASNone79.2262.9
MATH-500MASPOMATH-50084.8339.5
MATH-500UniMASO (in-domain)MATH-50085.880.8
MATH-500UniMASO (cross-dataset)AGIEval-MATH86.083.3

On MATH-500, direct reuse of the AGIEval-MATH configuration achieves 86.00% accuracy, compared with 85.80% when optimized specifically on MATH-500. In the reverse direction, direct reuse achieves 81.25% on AGIEval-MATH, compared with 82.42% after in-domain optimization. In both directions, deployment cost is only about 3% higher than that of a configuration optimized specifically for the target benchmark. These results show that, on the two mathematical-reasoning benchmarks tested, a configuration obtained from one optimization can handle subsequent questions on the same benchmark and can also be directly applied to a related benchmark, allowing multiple mathematical-reasoning tasks to share the offline optimization cost. We have added the cross-dataset transfer results to Section 5.4.

Dear Reviewer p8o7,

We sincerely thank you for taking the time to review our work. Your thoughtful suggestions have greatly contributed to improving the quality of our paper, and we look forward to receiving any further feedback you may have. As the ICLR author–reviewer discussion period concludes on Nov 18 (AoE), we would greatly appreciate your feedback on our rebuttal responses to ensure that all your concerns have been adequately addressed. Moreover, we would be happy to engage in further discussions and address any additional questions or considerations you may have.

Best regards, Authors

Dear Reviewer p8o7,

As the discussion period closes today, we would be very grateful for a brief note on whether our response resolves your concerns. And if you have any further questions, we would be glad to answer them. Thank you again for your time.

Best regards, The authors

Reviewer_7SfAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes UniMASO, a framework for jointly optimizing agent prompts and LLM backbone assignments in heterogeneous LLM-based multi-agent systems (MAS). The authors argue that existing prompt optimization methods typically assume fixed model assignments, whereas existing model-routing approaches generally do not jointly optimize role-specific prompts.

Critical strengths

1.Jointly considering prompts and model capabilities is a reasonable research direction. 2.The framework considers inter-agent dependencies.

Critical weaknesses

1.My main concern is the technical novelty of this work. Some components of UniMASO directly build upon existing methods, e.g., MASPO.

2.Although UniMASO is claimed to jointly optimize agent prompts and model assignment, their operations seem to be conducted sequentially, where the prompt optimization results are used for model adaptation. This makes the proposed method more like an extension work, i.e., considering additional facts, and its primary contribution focuses on using different models for different tasks. To me, this kind of contribution is limited for an ICLR paper.

3.The paper's main claim is that prompts and model assignments should be optimized jointly. However, the primary comparison mainly considers prompt optimization methods using homogeneous model assignments. This makes the comparison somewhat asymmetric: UniMASO is allowed to optimize both prompts and model assignments, while its competitors optimize only prompts. The authors should compare against methods that also support heterogeneous model assignments, especially MasRouter and Hetero-Designer, or clearly explain why these methods cannot be adapted to the same task.

Questions for authors
  1. UniMASO builds on MASPO for multi-granularity evaluation and beam search. Could the authors clarify which technical components are fundamentally new, beyond extending the search space to include model assignments?

  2. The authors claim that prompts and model assignments are jointly optimized. However, the optimizer prompts in Appendix F explicitly instruct the model to optimize the prompt first and then decide whether to change the model. How is this different from sequential optimization in practice? What makes the proposed optimization genuinely joint?

  3. The main experiments compare UniMASO against prompt optimization methods with homogeneous model assignments. Why are heterogeneous MAS methods such as MasRouter and Hetero-Designer not included as baselines? Would UniMASO still outperform these methods under comparable model pools and optimization budgets?

  4. The reported 85.2% cost reduction excludes optimization overhead, evaluator calls, and Co-Evolver calls. How expensive is the optimization process itself? Would UniMASO still provide a meaningful cost advantage when these additional costs are considered?

3 comments
AuthorsOpenReview ↗

Official Comment to Reviewer 7SfA (1/3)

Q1. UniMASO builds on MASPO for multi-granularity evaluation and beam search. Beyond extending the search space to include model assignments, which technical components are fundamentally novel?

We further clarify UniMASO’s technical contributions beyond expanding the search space. Cost-aware Pareto search uses the measured task performance and execution cost of each prompt–model candidate to retain nondominated options with different quality–cost trade-offs. Topology-aware co-evolution allows each role to propose a model assignment for the entire system based on its upstream and downstream dependencies. The algorithm coordinates the model choice for each node using the proposals from all roles and their level of agreement. Cross-round trajectory memory links each configuration change to its local, downstream, and global execution results. When generating candidates in the next round, the optimizer can retrieve earlier successful changes and cases in which local performance improved while overall performance deteriorated.

UniMASO draws on MASPO’s multi-granularity evaluation and per-agent beam search only in the local-search stage. We further design a quality–cost screening criterion for local candidates and use co-evolution and trajectory memory to incorporate each role’s local-search results into system-level configuration updates and optimization in subsequent rounds.

Q2. The optimizer prompt in Appendix F instructs the model to optimize the prompt first and then decide whether to change the model. How does this differ from sequential optimization, and what makes the proposed optimization genuinely joint?

This is a misunderstanding of the optimizer prompt in Appendix F. The instruction to “optimize the prompt first, then decide whether to change the model” describes how a candidate is formed within a single optimizer call; it is completely different from sequential optimization in practice. Before the call begins, the optimizer has already received the current model, the available models, and execution feedback, and it returns a candidate prompt together with its corresponding model in the same response.

Sequential optimization first determines and freezes the prompt, and only then assigns models in a separate stage. UniMASO instead treats the prompt and model as one complete candidate: it executes the prompt with the model specified by that candidate and selects the pair based on actual performance and cost. The selected prompt and model enter the next search round together, and feedback after a model change also affects subsequent prompt edits. Thus, joint optimization is realized through paired evaluation, paired selection, and joint updates across rounds. We will clarify this point further in Appendix F.

AuthorsOpenReview ↗

Official Comment to Reviewer 7SfA (2/3)

Q3. Why were heterogeneous multi-agent methods such as MasRouter and Hetero-Designer absent from the main baselines? Does UniMASO remain superior with comparable model pools and optimization budgets?

Thank you for suggesting a model-routing baseline. The main experiments focus on system-configuration optimization before MAS deployment. We therefore prioritized prompt-optimization methods that also optimize before deployment and reuse the resulting configurations on subsequent samples. These methods optimize role prompts under a given model assignment, whereas UniMASO jointly determines each role’s prompt and execution model in the same stage. MAS model-routing methods, represented by MasRouter, focus on per-sample model selection at runtime: only after the system receives a specific input does it select execution models for that sample. Given this task difference, the original paper discussed model-routing methods in related work. Following the reviewer’s suggestion, we added a direct comparison with MasRouter to examine the performance of pre-deployment configurations relative to per-sample routing under the same topology, model pool, and test data. Hetero-Designer’s official implementation has not been released. We cannot verify key details of its designer training, role generation, and collaboration-structure generation, making it difficult to ensure a comparable reproduction; therefore, we did not include it in the experiments.

We use the setting in MasRouter’s original paper, in which its router is attached to an existing multi-agent system. Both methods use the same centralized topology, role definitions, initial prompts, execution-model pool, and data splits, as well as the same task metrics and deployment-cost coefficients, to ensure a fair comparison. During evaluation, MasRouter routes each input, whereas UniMASO uses the prompts and model assignment selected before deployment for all evaluation samples.

MethodAGIEval-MATHGPQA-DiamondBFCL-V3LiveCodeBenchHotpotQAAverage
Acc.CostAcc.CostAcc.CostAcc.CostAcc.CostAcc.Cost
MasRouter74.2258.271.7229.492.020.538.1721.958.437.166.933.42
UniMASO82.4256.978.2833.092.412.235.1113.363.839.370.430.92

As shown in the table, UniMASO achieves a higher average score than MasRouter under the fixed topology and model pool described above. This indicates that, on the evaluated tasks, a configuration obtained through pre-deployment joint optimization can be reused across samples while maintaining a quality advantage over per-sample routing. We have added the MasRouter comparison to Appendix G.2.

AuthorsOpenReview ↗

Official Comment to Reviewer 7SfA (3/3)

Q4. The reported 85.2% cost reduction excludes optimization, evaluator, and Co-Evolver calls. How expensive is optimization, and does UniMASO retain a substantial cost advantage after accounting for these costs?

UniMASO’s optimization cost is modest, and each benchmark incurs it only once before deployment. The table lists the complete search cost for each benchmark, including execution of candidate multi-agent systems, optimizer and evaluator calls, and UniMASO’s Co-Evolver calls. Both methods retain the model configurations used in their respective main experiments, and costs are uniformly calculated from actual input and output tokens and the corresponding model cost coefficients. Across five benchmarks, with one optimization per benchmark, UniMASO consumes 206.3M, approximately 4.4% of MASPO’s cost. This difference has two sources: UniMASO uses 20.7% fewer input tokens and 15.9% fewer output tokens, and the models used by its optimization components have lower cost coefficients. In the table, Input and Output denote raw input and output token counts in millions. Cost denotes the cost after applying each model’s cost coefficient, in millions of relative weighted-token units.

MethodAGIEval-MATHGPQA-DiamondBFCL-V3HotpotQALiveCodeBench
InputOutputCostInputOutputCostInputOutputCostInputOutputCostInputOutputCost
UniMASO5.920.08551.94.140.07237.01.930.06519.04.890.0742.86.450.07755.6
MASPO7.480.1031188.45.210.086843.12.370.074421.26.190.082979.98.170.0941278.0

After including all optimization costs, UniMASO’s total cost remains approximately 93.7% lower. Under the centralized topology, optimizing each of the five benchmarks once and then running the selected configuration once over each complete evaluation set costs UniMASO 360.9M in total: pre-deployment optimization accounts for 57.2%, and evaluation-set execution for 42.8%, corresponding to 206.3M and 154.6M, respectively. The execution sum uses the more precise per-benchmark costs reported in the manuscript. DeepSeek-V4-Pro + MASPO costs 5,752.0M in total, including 4,710.6M for optimization and 1,041.4M for execution. The original 85.2% reduction compares execution costs only. The 93.7% figure includes one complete configuration search for every benchmark and does not assume additional deployment volume. After deployment, new requests for the same task directly reuse the selected role prompts and execution models, incurring execution cost only. Per-benchmark optimization-side token usage and cost are now reported in Section 5.5 (Table 7).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.