acceptodds
Under review as a conference paper at ICLR 2027

MeterRouter: Budget-Aware Dynamic Routing for Multi-Agent Systems

16 views

Abstract

Multi-agent systems (MAS) built on large language models (LLMs) outperform single-agent systems, but inter-agent communication incurs substantial token overhead. While assigning a large LLM to every agent results in inefficient resource utilization, leaving the system far from the Pareto frontier. The cost of answering a query is set jointly by how many agents are recruited, the role and LLM assigned to each, and communication topology among them. Most existing routers treat cost only implicitly rather than as an explicit optimization constraint. A fixed accuracy–cost preference also cannot express how much a caller is willing to pay for a particular query. We introduce MeterRouter, a budget-aware dynamic router that incrementally synthesizes a collaboration graph by selecting each agent's role, LLM, incoming connections, and termination conditioned on the query and a per-query monetary budget. We initialize the router by imitating successful execution graphs, then jointly refine its decisions with a variant of Group Relative Policy Optimization (GRPO) whose reward favors correct answers and penalizes spending and budget overruns. Across five benchmarks and a three-tier LLM pool, MeterRouter is (1) accurate, achieving performance comparable to the strongest MAS; (2) economical, reaching that accuracy at 82% lower cost than the strongest MAS router and extending the accuracy–cost frontier beyond the ceiling of single-agent systems; and (3) controllable, staying within the specified budget on at least 99.9% of queries.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, LLM routing, cost-efficient inference, budget aware, agent orchestration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vAA4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MeterRouter, a budget-aware dynamic routing framework for LLM-based multi-agent systems. The method jointly determines agent roles, backbone LLMs, communication edges, and termination decisions under a user-specified monetary budget.

Unlike approaches that select from predefined multi-agent configurations, MeterRouter incrementally constructs a directed acyclic collaboration graph while tracking the remaining budget and previously observed execution costs. Its training procedure consists of two stages: supervised imitation from successful execution graphs, followed by group-relative reinforcement learning to improve the accuracy–cost trade-off.

Experiments are conducted on five benchmarks covering general reasoning, mathematical reasoning, and code generation, using a heterogeneous pool of three LLMs. The reported results show that MeterRouter achieves 66.72% overall accuracy at an average cost of $0.00256 per query, compared with 66.80% accuracy and $0.01415 per query for MasRouter. The paper also reports high budget-compliance rates and investigates the effects of reinforcement learning, budget constraints, and maximum agent count.

Critical strengths
  1. A practically relevant formulation of budget-aware multi-agent orchestration.

The paper addresses an important limitation of existing multi-agent routing methods: the need to jointly optimize communication structure and heterogeneous model allocation under explicit per-query budget constraints.

Allowing different queries to receive different computational budgets is practically meaningful, and the incremental graph-construction procedure provides a flexible alternative to selecting from fixed collaboration templates.

The combination of role selection, backbone assignment, edge generation, and termination decisions is a potentially useful direction for resource-efficient multi-agent systems.

  1. Strong reported accuracy–cost trade-offs across multiple benchmarks.

MeterRouter achieves competitive accuracy while substantially reducing the estimated inference cost compared with several multi-agent baselines. In particular, the approximately 82% cost reduction relative to MasRouter, with nearly identical aggregate accuracy, is practically appealing.

The experiments cover five benchmarks and include comparisons with single-agent methods, fixed multi-agent architectures, dynamic multi-agent systems, and model routers. The budget sweeps also demonstrate that the generated system adapts its model allocation and agent count according to the available budget.

The results suggest that joint routing can improve the practical efficiency of multi-agent inference.

Critical weaknesses
  1. The independent contributions of the proposed routing components are not sufficiently established, and stronger budget-matched comparisons are needed.

Although the paper emphasizes the joint optimization of roles, backbone LLMs, communication topology, and termination, the current ablation studies do not adequately isolate these components.

Table 2 primarily examines the effect of removing the reinforcement learning stage, varying the budget, and changing the maximum agent count. Removing RL reduces overall accuracy from 66.72% to 57.34%, but this comparison does not reveal whether the improvement arises from better backbone selection, topology generation, role assignment, or termination decisions.

The lack of component-level ablations makes it difficult to assess which parts of MeterRouter are essential to its performance.

In addition, the main comparison with MasRouter and other multi-agent baselines is conducted at substantially different realized inference costs. Although this supports the reported cost savings, it does not establish how the baselines would perform under the same monetary constraints.

A stronger evaluation would compare methods under matched realized cost budgets, including simpler query-conditioned backbone routing combined with fixed or pruned multi-agent topologies.

This is particularly important because the heterogeneous model pool exhibits considerable differences in accuracy across tasks, and selecting an appropriate backbone may already account for a substantial fraction of the reported improvement.

Without these controlled comparisons, the necessity of the full joint optimization framework remains insufficiently demonstrated.

  1. The claims regarding strict budget guarantees and execution-aware routing require further qualification and validation.

MeterRouter relies on conservative byte-based estimates of token consumption to construct affordable model sets and determine completion limits. However, the paper does not establish that these estimates are guaranteed upper bounds under all tokenization patterns and supported model families.

The reported budget-compliance rates are high, but are not equivalent to a formally established per-query cost guarantee. The use of fixed accounting rates derived from an external pricing source also requires clarification, since these may not exactly correspond to actual provider billing.

Furthermore, the routing policy's observations of previous agents mainly consist of token counts, message lengths, and realized costs. The state representation does not explicitly include the semantic content, correctness, or reasoning quality of intermediate agent outputs.

Consequently, the method appears to adapt primarily to resource consumption and previously constructed graph structure, rather than directly adapting to the substantive reasoning outcomes of earlier agents. This limits the strength of the claim that routing decisions are informed by execution feedback.

The evaluation also provides insufficient information about run-to-run variability. The main performance table reports point estimates without confidence intervals, making it difficult to determine the statistical reliability of small accuracy differences between the strongest methods.

Finally, several numerical statements in the results section do not agree with Tables 1 and 2, including the claimed BBEH accuracy gap and the reported accuracy change when reducing the maximum agent count. These inconsistencies should be corrected.

Questions for authors

Q1. Component-level contribution of joint routing.

Can the authors provide separate ablations for role assignment, backbone selection, communication-edge generation, and learned termination?

In particular, how does MeterRouter compare against a simpler approach that uses a fixed sparse topology and optimizes only the backbone assigned to each agent?

Similarly, what is the contribution of allowing topology adaptation when model selection and agent count are held fixed?

These experiments would clarify whether the observed improvement genuinely requires joint optimization.

Q2. Budget-matched comparisons with existing routers.

Can the authors compare MeterRouter with MasRouter, BAMAS, and other relevant routing baselines at matched realized monetary costs?

For example, could the baselines be evaluated across comparable budget limits or tuned cost preferences, so that their accuracy–cost frontiers can be compared under similar resource constraints?

The current comparison demonstrates substantial cost savings, but does not fully establish superiority over competing methods at equivalent spending levels.

Q3. Validity of the budget guarantee.

Can the authors explain whether the byte-based token estimator is a mathematically guaranteed upper bound for all three model tokenizers, or an empirically conservative approximation?

How frequently do budget overruns, budget-blocked queries, or truncated completions occur at each budget level, and how are these cases handled in the reported accuracy?

Additionally, do the accounting prices match the actual billing rates of the LLM provider used in the experiments?

These details are important for evaluating the strength and practical scope of the claimed budget guarantee.

Q4. Adaptation to intermediate reasoning outcomes.

The routing state includes observations of previous agents' resource usage and output lengths, but does not explicitly incorporate their semantic outputs.

How can the router distinguish between a successful intermediate solution and an unsuccessful one if their token statistics are similar?

Could the authors compare the current resource-aware state representation with a content-aware routing variant, or clarify that the intended execution feedback refers primarily to realized cost and message length?

This would help establish the specific type of adaptivity provided by MeterRouter.

Q5. Statistical reliability and computational overhead.

Can the authors report results across multiple random seeds, together with paired confidence intervals for the main accuracy comparisons?

It would also be valuable to report the training-stage computational cost and end-to-end latency, rather than only the monetary cost of the executed agent calls.

Since the method requires supervised demonstration generation followed by reinforcement learning, how many inference queries are needed to amortize the training cost relative to simpler routing alternatives?

Such analyses would provide a more complete assessment of the practical efficiency of the proposed framework.

6 comments

Thanks for your questions. Here are our replies to your questions and concerns.

Q1&W1

To address your concerns, we ran two representative experiments: (1) fixed topology and (2) single LLM. Below are the results.

Fixed topology

In this experiment, MeterRouter is trained with its topology fixed to a chain or a complete graph, while the rest of the policy is unchanged, so it no longer constructs the graph but still assigns heterogeneous backbones under the same budget. We compare it with the Table 1 baselines of the same topology under the budget B=5×10−2B=5\times10^{-2}, as shown in Fig. R1.

https://github.com/Anon-Submission-1024/Figure/blob/main/3/Fig.%20R1.png

Fig. R1. Accuracy versus real average cost per query for the fixed-topology ablation and the Table 1 baselines of the same topology, Chain (top) and Complete Graph (bottom).

We can conclude that with a fixed topology, part of the gain remains: heterogeneous model selection alone places the fixed-topology variant on the Pareto front of its same-topology baselines in all 12 panels. The full MeterRouter is still more accurate than the fixed-topology variant in every panel, and over all test questions it is 7.03 points higher than the chain and 7.51 points higher than the complete graph, at a lower average cost.

Single LLM

In this experiment, MeterRouter is trained with a single shared backbone, gpt-oss-120b, while the rest of the policy is unchanged, so it still constructs the graph but can no longer choose models. We compare it with the full MeterRouter and with all Table 1 baselines that use the same backbone under the budget B=5×10−2B=5\times10^{-2}, as shown in Fig. R2.

https://github.com/Anon-Submission-1024/Figure/blob/main/3/Fig.%20R2.png

Fig. R2. Accuracy versus real average cost per query for MeterRouter with a single LLM, the full MeterRouter, and the Table 1 baselines with gpt-oss-120b.

With one backbone, MeterRouter is only 3.19 points below the full model, so most of the gain survives. Learning the topology alone makes the single-LLM variant more accurate and cheaper than the Table 1 Chain, Tree, and Complete Graph baselines with the same backbone on every benchmark; over all test questions it scores 3.92 to 5.31 points higher at half their cost or less. It also lies on the Pareto front of the gpt-oss-120b methods in all 6 panels. Four multi-agent baselines with the same backbone are more accurate overall (AutoGen 64.18%, GPTSwarm 64.56%, AgentPrune 65.90%, LLM-Debate 66.70%), but they cost 2.3 to 6.3 times more per query. The full MeterRouter is more accurate than the single-LLM variant on every benchmark. Comparing the two ablations, fixing the topology lowers overall accuracy by 7.03 to 7.51 points, while using a single backbone lowers it by only 3.19 points. Graph construction therefore accounts for more of the gain, and model selection adds to it.

We will update this in our revised paper.

Q2&W1

Thank you for the question. We understand your concern about "matched realized monetary costs" and "comparable budget limit".

A comparable budget and a comparable cost mean different things. A comparable budget gives every method the same budget. A comparable cost means every method actually spends the same amount, which depends on how each router runs and cannot be fixed in advance. Even with a shared budget, methods with soft constraints (e.g., MasRouter) or no constraints cannot keep their cost within it. BAMAS is the baseline that most closely resembles a hard budget constraint, but it applies the constraint only once, before execution: an integer linear program selects the strongest LLMs whose estimated costs fit within BB, and nothing tracks the actual cost during execution. As a result, BAMAS can exceed the budget, and its paper reports out-of-budget queries. In Table 1, we ran BAMAS at B=0.05B=0.05, the same budget as MeterRouter. Over all test questions, BAMAS reaches 56.15% at 4.93×10−34.93\times10^{-3} USD per query, while MeterRouter reaches 66.72% at 2.56×10−32.56\times10^{-3} USD, so MeterRouter is both more accurate and cheaper under the same budget.

MasRouter and BAMAS also compare methods on the accuracy–cost trade-off using a Pareto front (Figure 3 of the MasRouter paper and Figure 2 of the BAMAS paper). We will explain this point more clearly in the revised paper.

Q3&W2

Thank you for this question.

Input side: an estimate. The provider does not return token counts before a call, so input tokens are estimated from bytes, T^iin=⌈bytesi/2⌉+768\hat T_i^{\mathrm{in}}=\lceil\mathrm{bytes}_i/2\rceil+768. Here bytesi\mathrm{bytes}_i is the UTF-8 length of the full request of agent viv_i (system and role prompts, query, and the messages it reads from its predecessors), and 768 tokens cover the chat-template overhead. Typical tokenizers use about four bytes per token on English text, so assuming two bytes per token roughly doubles the count. The input cost c^iin=cmiinT^iin\hat c_i^{\mathrm{in}}=c_{m_i}^{\mathrm{in}}\hat T_i^{\mathrm{in}} is therefore also an estimate. The byte-based estimate is an empirically conservative approximation, not a guaranteed upper bound: densely tokenized text such as long digit strings can be underestimated. It is the only source of overruns, hence the in-budget rate of at least 99.98% rather than 100%.

Output side: a hard cap. Before each call, the remaining budget bib_i is converted into a token cap passed to the provider as max_tokens: Ticap=min⁡(Tmax⁡, ⌊bi−c^iin−ricmiout⌋),T_i^{\mathrm{cap}}=\min\Big(T_{\max},\ \Big\lfloor\frac{b_i-\hat c_i^{\mathrm{in}}-r_i}{c_{m_i}^{\mathrm{out}}}\Big\rfloor\Big), where rir_i reserves the cost of agents still needed to close the graph. No agent, including the last, can exceed the budget through a long output. Backbones that cannot fund 256 output tokens are masked, and the graph terminates when none remains.

In deployment, the input estimator, a single function, can be replaced by exact counts from the model's tokenizer or the provider, without changing the framework. Then ci≤ciin+cmioutTicap≤bic_i\le c_i^{\mathrm{in}}+c_{m_i}^{\mathrm{out}}T_i^{\mathrm{cap}}\le b_i for every call, so ∑ici≤B\sum_i c_i\le B holds for every query.

Handling in the reported accuracy. No query is dropped. Each query ends by a learned stop, the agent cap, or a budget stop, and is graded normally. A query counts as incorrect if its final answer is truncated at its token cap or if no agent is affordable at all. An overrun is graded normally and counted as a violation.

How often each mechanism acts. To show how often the input estimate and the output cap matter in practice, we recorded for every test query how its graph ended, whether its final answer was cut at its token cap, and whether its cost exceeded BB. Table R1 reports these shares at each test budget.

Table R1. How queries end, and how often truncation and overruns occur, at each budget BB: share of queries (%), weighted by the number of test questions over the five benchmarks, mean ±\pm std over 3 seeds. As in Table 2, B∈{1,4}×10−3B\in\lbrace 1,4\rbrace\times10^{-3} means B=4×10−3B=4\times10^{-3} for Omni-MATH-2 and BBEH and 1×10−31\times10^{-3} for the other three benchmarks. Learned, agent-cap and budget stops are the three ways a query ends and sum to 100%.

BudgetLearned stopAgent-cap stopBudget stopTruncationOverrun
B∈{1,4}×10−3B\in\lbrace 1,4\rbrace\times10^{-3}86.77 ±\pm 0.570.53 ±\pm 0.2012.70 ±\pm 0.4316.93 ±\pm 0.340.02 ±\pm 0.02
B=5×10−3B=5\times10^{-3}91.06 ±\pm 0.243.20 ±\pm 0.125.74 ±\pm 0.127.71 ±\pm 0.240.01 ±\pm 0.02
B=5×10−2B=5\times10^{-2}88.25 ±\pm 0.4811.72 ±\pm 0.460.03 ±\pm 0.030.08 ±\pm 0.120.00 ±\pm 0.00
  • Learned stop (from learned policy): the policy's stop decision ends the graph. A stop counts as learned only if at least one backbone could still afford another agent.
  • Budget stop (from hard affordability filter): no backbone can afford another agent, meaning its estimated input plus a minimum response of 256 output tokens exceeds the remaining budget, so the affordability filter ends the graph.
  • Agent-cap stop (neither; it's the fixed limit on the number of agents): the graph reaches the maximum number of agents and ends.
  • Truncation: the final answer is cut at its token cap.
  • Overrun: the query's actual cost exceeds BB.

Pricing. We use the llm prices from https://artificialanalysis.ai/ as accounting rates. Different providers charge different prices for the same model and their bills are not directly comparable, especially for open-source llms. Each call's cost is the provider-reported input and output token counts times these rates, so costs reflect actual usage, and all methods in Table 1 use the same rates.

We will update to our revised paper.

Reviewer_Yo3VOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MeterRouter, a budget-aware router for LLM-based multi-agent systems. Given a query and a monetary budget, MeterRouter incrementally constructs a collaboration graph by deciding the role and LLM backbone of each agent, its incoming communication edges, and when to stop adding agents. The method uses a two-stage training procedure. It first learns topology and role priors from successful template-based execution graphs through supervised learning. It is then refined with a GRPO-style reinforcement learning objective that rewards correctness while penalizing monetary cost. A budget meter removes unaffordable models from the action space and limits output length to reduce budget violations. Experiments are conducted on five reasoning and code-generation benchmarks using a three-tier LLM pool. MeterRouter obtains accuracy close to the strongest multi-agent baselines while using substantially less money per query. It also adjusts team size and model allocation according to the user-specified budget.

Critical strengths
  1. Making the user's monetary budget part of the routing input is a practically useful choice. MeterRouter decides not just which backbone to call, but also whether another agent is worth adding and which messages it should receive. The remaining-budget meter and the affordable-model filter give the system an explicit constraint rather than relying only on a soft accuracy-versus-cost preference. The reported in-budget rates suggest that this part of the design is working in the evaluated settings.
  2. The cost numbers caught my attention. Table 1 reports 66.72% mean accuracy, very close to MasRouter's 66.80%, but at 2.56 rather than 14.15 in the table's cost units. The method also performs particularly well on MuSR and MBPP+. Table 2 adds a useful observation: removing the RL stage drops average accuracy to 57.34% while leaving cost almost unchanged. So the reported result is not just a trivial consequence of picking the cheapest model every time.
Critical weaknesses
  1. I could not tell which part of the joint routing decision is actually responsible for the improvement. The paper emphasizes dynamic topology together with heterogeneous backbone allocation, yet Table 2 mainly removes RL or changes the budget and maximum agent count. Those tests establish that the training stage matters, but they do not separate the benefit of choosing different models from the benefit of constructing a different communication graph. A fixed-topology heterogeneous router and a dynamic-topology homogeneous router would be particularly useful controls.
  2. The near-tie in average accuracy with the strongest baselines makes variability important. The paper reports single values for its main accuracy and cost comparison, while the training includes policy optimization and the execution graph is sampled. Without multiple independently trained router runs, it is hard to know how stable the claimed accuracy-cost point is. I would also like to understand how much of the impressive budget compliance comes from the hard affordability filter as opposed to the learned policy; those are different contributions.
Questions for authors
  1. Can you isolate heterogeneous model selection from graph construction? For example, how much of the reported gain remains with a fixed topology, or with a single shared LLM backbone but a learned topology, under the same budget?
  2. What is the variation across independently trained router seeds, not just repeated inference from one checkpoint? Given that the average accuracies of MeterRouter, MasRouter, and LLM-Debate are so close, mean and standard deviation for both accuracy and cost would be helpful.
  3. How would the router behave if the model prices changed after training, or if it received budget values outside those emphasized in the evaluation? The meter can rule out unaffordable models, but it is less clear whether the learned choices would still be good.
5 comments

Thanks for your positive feedback. Here are our replies to your concerns.

W1&Q1

We ran both suggested experiments: (1) fixed topology and (2) single LLM. Below are the results.

Fixed topology

In this experiment, MeterRouter is trained with its topology fixed to a chain or a complete graph, while the rest of the policy is unchanged, so it no longer constructs the graph but still assigns heterogeneous backbones under the same budget. We compare it with the Table 1 baselines of the same topology under the budget B=5×10−2B=5\times10^{-2}, as shown in Fig. R1.

https://github.com/Anon-Submission-1024/Figure/blob/main/2/Fig.%20R1.png

Fig. R1. Accuracy versus real average cost per query for the fixed-topology ablation and the Table 1 baselines of the same topology, Chain (top) and Complete Graph (bottom). Stars are MeterRouter with its topology fixed.

We can conclude that with a fixed topology, part of the gain remains: heterogeneous model selection alone places the fixed-topology variant on the Pareto front of its same-topology baselines in all 12 panels. The full MeterRouter is still more accurate than the fixed-topology variant in every panel, and over all test questions it is 7.03 points higher than the chain and 7.51 points higher than the complete graph, at a lower average cost.

Single LLM

In this experiment, MeterRouter is trained with a single shared backbone, gpt-oss-120b, while the rest of the policy is unchanged, so it still constructs the graph but can no longer choose models. We compare it with the full MeterRouter and with all Table 1 baselines that use the same backbone under the budget B=5×10−2B=5\times10^{-2}, as shown in Fig. R2.

https://github.com/Anon-Submission-1024/Figure/blob/main/2/Fig.%20R2.png

Fig. R2. Accuracy versus real average cost per query for MeterRouter with a single LLM, the full MeterRouter, and the Table 1 baselines with gpt-oss-120b.

With one backbone, MeterRouter is only 3.19 points below the full model, so most of the gain survives. Learning the topology alone makes the single-LLM variant more accurate and cheaper than the Table 1 Chain, Tree, and Complete Graph baselines with the same backbone on every benchmark; over all test questions it scores 3.92 to 5.31 points higher at half their cost or less. It also lies on the Pareto front of the gpt-oss-120b methods in all 6 panels. Four multi-agent baselines with the same backbone are more accurate overall (AutoGen 64.18%, GPTSwarm 64.56%, AgentPrune 65.90%, LLM-Debate 66.70%), but they cost 2.3 to 6.3 times more per query. The full MeterRouter is more accurate than the single-LLM variant on every benchmark. Comparing the two ablations, fixing the topology lowers overall accuracy by 7.03 to 7.51 points, while using a single backbone lowers it by only 3.19 points. Graph construction therefore accounts for more of the gain, and model selection adds to it.

We will update this in our revised paper.

W2&Q2

Thank you for the question and we understand your concern.

Stability across runs. We trained and evaluated MeterRouter with three independent seeds on each benchmark (Table R1). Because MasRouter has the highest average accuracy among the baselines in Table 1, we also trained and evaluated it with three seeds (Table R2).

Table R1. MeterRouter: accuracy (%) and cost per query (×10−3\times10^{-3} USD) at B=5×10−2B=5\times10^{-2}, mean ±\pm std over 3 seeds.

DatasetAccuracy (%)Cost (×10−3\times10^{-3} USD)
Omni-MATH-264.7 ±\pm 1.44.12 ±\pm 0.45
MMLU-Pro83.4 ±\pm 2.51.68 ±\pm 0.21
MuSR71.6 ±\pm 1.90.61 ±\pm 0.08
BBEH45.4 ±\pm 1.82.95 ±\pm 0.35
MBPP+83.8 ±\pm 2.90.30 ±\pm 0.02

Table R2. MasRouter: accuracy (%) and cost per query (×10−3\times10^{-3} USD), mean ±\pm std over 3 seeds.

DatasetAccuracy (%)Cost (×10−3\times10^{-3} USD)
Omni-MATH-267.5 ±\pm 1.316.75 ±\pm 2.83
MMLU-Pro80.3 ±\pm 2.43.92 ±\pm 0.23
MuSR67.9 ±\pm 2.23.85 ±\pm 0.36
BBEH48.9 ±\pm 1.925.63 ±\pm 3.02
MBPP+80.8 ±\pm 2.72.65 ±\pm 0.27

For both methods, the Table 1 results lie within one standard deviation of the three-seed mean on every benchmark, for accuracy and cost alike, so the single runs in Table 1 are representative. Averaged over all test questions, the three-seed means are 66.2% at 2.49 ×10−3\times10^{-3} USD per query for MeterRouter and 66.5% at 13.31 ×10−3\times10^{-3} USD for MasRouter. The two remain close in accuracy: MeterRouter is ahead on MMLU-Pro, MuSR, and MBPP+, and MasRouter on Omni-MATH-2 and BBEH. The cost gap is much larger than the run-to-run variation, with MasRouter costing 2.3 to 8.8 times more on every benchmark. MeterRouter therefore reaches the accuracy of the strongest baseline at a fraction of its cost, and this holds across independently trained runs.

Sources of budget compliance for MeterRouter.

We agree that the hard affordability filter and the learned policy contribute to budget compliance in different ways. To separate them, we recorded how the graph of every test query ends. A graph ends in exactly one of three ways: the learned policy decides to stop, the graph reaches the agent cap, or the hard affordability filter finds that no backbone can afford another agent. Only the last case is enforced by the filter. Table R3 reports the share of each case for the three runs in Table R1.

Table R3. How queries end at B=5×10−2B=5\times10^{-2}: share of queries (%), weighted by the number of test questions over the five benchmarks, mean ±\pm std over 3 seeds. The three stop types sum to 100%.

BudgetLearned stopAgent-cap stopBudget stop
B=5×10−2B=5\times10^{-2}88.25 ±\pm 0.4811.72 ±\pm 0.460.03 ±\pm 0.03
  • Learned stop (from learned policy): the policy's stop decision ends the graph. A stop counts as learned only if at least one backbone could still afford another agent.
  • Budget stop (from hard affordability filter): no backbone can afford another agent, meaning its estimated input plus a minimum response of 256 output tokens exceeds the remaining budget, so the affordability filter ends the graph.
  • Agent-cap stop (neither; it's the fixed limit on the number of agents): the graph reaches the maximum number of agents and ends.

At B=5×10−2B=5\times10^{-2}, the affordability filter ends only 0.03% of graphs. The learned policy ends 88.25% of them while at least one backbone could still afford another agent, and the remaining 11.72% stop at the agent cap. In almost every query, the graph therefore ends before the filter has to intervene, so the budget compliance at this budget comes mainly from the learned policy, with the filter acting as a safeguard that is rarely needed.

We will update this in our revised paper.

Q3

Thank you for the question.

We agree with this distinction. The meter keeps every query within budget, but whether the routing choices stay good depends on how the router saw prices during training. The router in the paper was trained with one fixed price table and treats price as part of each model's identity. When one LLM becomes 10× more expensive but still fits within the budget BB, the router keeps choosing it, so cost rises while accuracy stays flat or even drops.

We therefore tested price-paired training, which teaches the router to adapt its LLM choice to LLM prices. The architecture and the meter are unchanged; only training differs. In every training round, the same questions and budgets are run at ×0.1, ×1 and ×10 prices with the same policy weights, and the GRPO-style update learns from all three, with advantages computed within each (question, budget, price) group. We ran this on MMLU-Pro, changing the price of gpt-oss-120b. The resulting router picks gpt-oss-120b first in 92% of queries at ×0.1, 23% at ×1 and 0% at ×10 (Table R4). Accuracy follows the price of this strong model: 75.0%, 68.2% and 67.1% at B=5×10−2B=5\times10^{-2}. Cost stays under control at 0.00009, 0.00021 and 0.00015 USD per query, so a 10× price increase does not raise spending, because the router moves away from the more expensive model.

Table R4. Price-paired MeterRouter on MMLU-Pro at B=5×10−2B=5\times10^{-2}, with the price of gpt-oss-120b changed at test time.

gpt-oss-120b price×0.1×1×10
gpt-oss-120b as first choice92.1%23.2%0.0%
Accuracy75.0%68.2%67.1%
Cost per query (×10−3\times10^{-3} USD)0.090.210.15

Price-paired training thus gives a router that responds to price in the right direction. We will add this experiment to the appendix.

Reviewer_5xJvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Traditional multi-agent systems rely on static topology and a fixed LLM backbone, which is far from an optimal configuration. This work introduces MeterRouter, a budget-aware dynamic router that optimizes the accuracy-cost tradeoff by jointly optimizing topology generation and LLM backbone assignments. The framework includes autonomous MAS orchestration with heterogeneous LLMs with budget-aware and dynamic adjustment. The experiments were conducted over a wide selection of baselines and a three-tier LLM pool, where the eval tasks ranged from general reasoning, mathematical reasoning, to code generation. Experimental results show that the MeterRouter achieves better Pareto-optimal MAS.

Critical strengths
  1. The remaining budget-driven router is a novel framework.

  2. The experiments have good coverage of tasks and model families.

  3. MeterRouter achieves a Pareto-optimal point in the experiment, and the analysis is sound.

Critical weaknesses
  1. My major concern is that, though the MeterRouter achieves comparable performance at a fraction of the cost, it does not represent every Pareto-optimal point. Ideally, the claimed framework should discover a Pareto-optimal set of MAS rather than a single MAS.

  2. The model architecture is highly engineered, but there is no discussion of how the architecture is decided and ablated.

  3. Given the argument that MeterRouter can better assign the LLM backbone, I find that only 3 available candidates are insufficient in this work to fully demonstrate the strength of the heterogeneous LLM backbone assignment.

Questions for authors
  1. Can you provide the plot of MeterRouter across different cost budgets?

  2. Can you include more LLM backbones?

4 comments

Thanks for your suggestions and questions. Here are our replies to your concerns.

Q1&W1

MeterRouter takes the budget BB as an input, so sweeping BB yields a set of MAS along a cost–accuracy curve from one trained model. Due to time constraints, we evaluated additional budgets only on the two hard benchmarks, Omni-MATH-2 and BBEH (Table R1). We chose these two because, in our original experiments, they were the benchmarks on which MeterRouter performed worst and most constrained by budget: accuracy dropped most sharply as BB decreased (Table 2). Raising the budget should therefore help most on these two. We evaluated MeterRouter at B∈{8×10−2,1×10−1}B\in\{8\times10^{-2}, 1\times10^{-1}\}. The B=5×10−2B=5\times10^{-2} column is the original result from Table 1.

Table R1. Accuracy (%) / cost per query (×10−3\times10^{-3} USD) at each budget BB.

DatasetB=5×10−2B=5\times10^{-2}B=8×10−2B=8\times10^{-2}B=1×10−1B=1\times10^{-1}
Omni-MATH-264.2 / 3.9565.4 / 6.4665.8 / 6.64
BBEH46.8 / 3.2047.7 / 7.3047.3 / 7.64

From B=5×10−2B=5\times10^{-2} to 8×10−28\times10^{-2}, MeterRouter uses the extra budget: cost rises from 3.95 to 6.46 on Omni-MATH-2 and from 3.20 to 7.30 on BBEH, and accuracy improves by 1.2 and 0.9 points. From B=8×10−2B=8\times10^{-2} to 1×10−11\times10^{-1}, accuracy no longer rises because it has largely saturated. Actual cost grows only slightly (6.46 to 6.64 on Omni-MATH-2, 7.30 to 7.64 on BBEH). As Fig. R1 shows, nearly all MeterRouter points lie on the Pareto front against all baselines.

https://github.com/Anon-Submission-1024/Figure/blob/main/1/Fig.%20R1.png

Fig. R1. Accuracy versus real average cost per query on Omni-MATH-2 and BBEH. Stars are MeterRouter at each budget (light to dark: B=5×10−2B=5\times10^{-2}, 8×10−28\times10^{-2}, 1×10−11\times10^{-1}). The dashed line is the Pareto front.

We will update this in our revised paper.


W2

Thank you for this question. The architecture follows from three requirements of the decision problem. Let cic_i be the billed cost of agent viv_i and bi=B−∑j<icjb_i=B-\sum_{j<i}c_j the remaining budget.

R1: Observe realized costs. cic_i depends on viv_i's output length, so it is known only after execution. Policies that condition on (c1,…,ci−1)(c_1,\dots,c_{i-1}) include those that fix the graph in advance, so they are at least as good. Under the hard constraint ∑ici≤B\sum_i c_i\le B, the feasible backbone set Mim={m:c^i(m)≤bi}\mathcal{M}^m_i=\lbrace m:\hat c_i(m)\le b_i\rbrace depends on bib_i, so the optimal policy must condition on (si,bi)(s_i,b_i). MeterRouter therefore generates and executes agents one at a time, summarizes the history with a node GRU, and feeds bib_i to the heads through ϕi\phi_i and ψ(B,bi)\psi(B,b_i).

R2: Order decisions by how cost arises. Since ci=cmiinTiin+cmioutTioutc_i=c_{m_i}^{\mathrm{in}}T_i^{\mathrm{in}}+c_{m_i}^{\mathrm{out}}T_i^{\mathrm{out}} with Tiin≈T0+∑jeji∣yj∣T_i^{\mathrm{in}}\approx T_0+\sum_j e_{ji}\lvert y_j\rvert, reading vjv_j costs cmiin∣yj∣c_{m_i}^{\mathrm{in}}\lvert y_j\rvert. Edges are therefore chosen after the backbone, and the edge head receives ℓj→i\boldsymbol\ell_{j\to i}, which contains ∣yj∣\lvert y_j\rvert and cmiinc_{m_i}^{\mathrm{in}}. The edge GRU computes the exact factorization π(e⋅i)=∏jπ(eji∣e>j,i)\pi(\mathbf e_{\cdot i})=\prod_j\pi(e_{ji}\mid\mathbf e_{>j,i}). Independent edge heads cannot express budget-driven either-or choices such as "read v1v_1 or v2v_2, not both": p1p2=0p_1p_2=0 and (1−p1)(1−p2)=0(1-p_1)(1-p_2)=0 force a deterministic choice.

R3: Generalize across budgets and LLM pools. The spending shares in ϕi\phi_i are invariant under (B,bi)↦(λB,λbi)(B,b_i)\mapsto(\lambda B,\lambda b_i), and BB itself enters on a log scale, since budgets span 10−310^{-3} to 10−110^{-1} USD. The role and backbone heads score each candidate with a shared function and normalize over the feasible set, so they work for any pool size: masking removes unaffordable LLMs, and a new LLM needs only its embedding (as in W3&Q2). The trainable parts are small GRUs and MLPs on a frozen encoder, matching the 200 training queries per benchmark.

Ablation. To ablate the topology itself, MeterRouter is trained with its topology fixed to a chain or a complete graph, while the rest of the policy is unchanged. We compare it with the Table 1 baselines of the same topology under the budget B=5×10−2B=5\times10^{-2}, as shown in Fig. R2.

https://github.com/Anon-Submission-1024/Figure/blob/main/1/Fig.%20R2.png

Fig. R2. Accuracy versus real average cost per query for the fixed-topology ablation and the Table 1 baselines of the same topology, Chain (top) and Complete Graph (bottom). Stars are MeterRouter with its topology fixed.

The results show that budget-aware backbone assignment alone already improves a fixed topology MAS. The fixed-topology variant lies on the Pareto front in all 12 panels. The full MeterRouter is still more accurate than the fixed-topology variant in every panel, and over all test questions it is 7.03 points higher than the chain and 7.51 points higher than the complete graph, at a lower average cost.

We will update this in our revised paper.

W3&Q2

Thank you for the question and we understand your concern. We added a fourth backbone, Qwen3.8-27B, as a tier priced above gpt-oss-120b and below minimax-m3 (0.20 and 1.00 USD per 1M input and output tokens, respectively), and retrained MeterRouter end to end (collection, SFT, RL) on the 4-LLM pool. For the same reason as in Q1&W1, we report the two hard benchmarks, Omni-MATH-2 and BBEH. The 3-LLM row is the main result from Table 1.

Table R2. Accuracy (%) / cost per query (×10−3\times10^{-3} USD) at B=5×10−2B=5\times10^{-2}.

PoolOmni-MATH-2BBEH
3 LLMs64.2 / 3.9546.8 / 3.20
4 LLMs66.3 / 4.2849.3 / 3.26

On both hard benchmarks, the router trained on the larger pool spends more per query, by 8% on Omni-MATH-2 (3.95 to 4.28) and 2% on BBEH (3.20 to 3.26), and its accuracy moves closer to the best baseline in Table 1. On Omni-MATH-2, accuracy rises by 2.1 points to 66.3%. On BBEH, it rises by 2.5 points to 49.3%.

https://github.com/Anon-Submission-1024/Figure/blob/main/1/Fig.%20R3.png

Fig. R3. Accuracy versus real average cost per query on Omni-MATH-2 and BBEH for MeterRouter with the 3-LLM and 4-LLM pools (stars) and all baselines. The dashed line is the Pareto front.

As Fig. R3 shows, the 4-LLM MeterRouter remains on the Pareto front on both benchmarks. MeterRouter thus extends to a larger, heterogeneous pool without changes to the architecture.

We will update this in our revised paper.

Thank you for your time and feedback. We hope our response addresses your concerns, and we would be glad to discuss any remaining questions. If it resolves them, we would appreciate it if you would consider raising your score.

Dear Reviewer,

Thank you for your review and for recognizing our novel budget-driven router. We have responded to each of your questions in the rebuttal above.

As the discussion deadline is approaching, we would be grateful to know whether our responses resolve your remaining concerns. If they do, we hope you will consider updating your rating.

Thank you for your time.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.