acceptodds
Withdrawn from ICLR 2027

DEALS: Decentralized Expertise-Aware Load Serving for Multi-Agent LLM Systems

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-agent systems (MAS) have recently emerged as an effective approach for coordinating large language model (LLM)-based agents to solve complex tasks through structured interactions. In practice, MASs often handle a stream of heterogeneous and complex tasks, requiring agents to decompose each task and then self-organize and self-evolve to adapt to incoming tasks while sharing execution resources. However, most early approaches to MASs rely on centralized controllers or fixed coordination patterns, which can limit scalability or adaptability. In contrast, existing decentralized and dynamic MASs often require training dedicated routers or invoking LLMs for agent selection, resulting in substantial computational costs and coordination overhead. To address these challenges and enable efficient task-level self-organization and self-evolution for task- and workload-level collaboration, we propose Decentralized Expertise-Aware Load Serving (DEALS), a decentralized and low-complexity framework that enables agents to self-organize and dynamically route concurrent tasks for processing. Specifically, each agent maintains local queues of incoming tasks, and its router decides whether to process a task locally or forward it to a neighbor based on differences in backlog and success rate. Meanwhile, executors process independent tasks concurrently within and across agents, and partially solved tasks can be resumed by other agents. Experiments show that DEALS not only improves performance along multiple dimensions (e.g., answer accuracy and task throughput) in both homogeneous and heterogeneous agent pools, but also balances agent expertise and workload in a self-organized manner, enabling effective decentralized coordination.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM-based multi-agent systems, decentralized coordination, task routing, load balancing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vVJsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DEALS, a decentralized framework for concurrent task processing in LLM-based multi-agent systems. It uses an expertise-aware routing score that combines task backlogs and historical success rates to guide task allocation, along with memory-guided execution and concurrency support. Experiments on MATH, BBH, and MMLU-Pro report improvements in accuracy and throughput over existing baselines.

Critical strengths
  1. The problem is practical and relevant. Coordinating concurrent requests across agents with different capabilities and workloads is an important problem for multi-agent serving systems.
  2. The routing mechanism is lightweight. DEALS uses only local and neighbor state, with no extra LLM calls or trained router for routing. The V=0 vs. V>0 comparison in the heterogeneous pool clearly shows that success-rate information helps steer tasks away from weaker agents.
Critical weaknesses
  1. The evidence for expertise-load balancing is limited. At (V=80), Qwen produces nearly all final answers on MATH (138/140) and most on BBH (88/100), while the heterogeneous pool achieves similar accuracy to the Qwen-only pool. These results suggest that the router mainly favors the stronger agent, rather than clearly demonstrating the benefits of heterogeneous collaboration or workload balancing. The choice of (V) also lacks a clear justification or validation procedure.
  2. The throughput gains need stronger experimental support. DEALS reports 4.7×–22.7× higher throughput than AgentNet, but DEALS supports concurrent execution while AgentNet processes tasks sequentially. It is therefore difficult to isolate the contribution of the proposed routing strategy. Comparisons with simpler routing methods under the same concurrency settings would be helpful. Moreover, although low latency is emphasized in the Introduction, no end-to-end latency results are reported.
Questions for authors
  1. Could the authors clarify how task arrivals are modeled in the experiments? The introduction emphasizes continuous task streams, but the experimental setup mainly describes how tasks are distributed across agents. Are tasks injected over time, or are they all queued at the beginning of evaluation?
  2. Could the authors provide more intuition or justification for the routing design in Eq. (3)? The current formulation seems somewhat heuristic, as it directly combines backlog and success-rate differences through a weighted sum. A more detailed explanation of the design rationale would help better understand the proposed method.
  3. Could the authors clarify what is meant by self-evolution in this framework? Since the success-rate estimates and private memories are frozen during testing, it is unclear to what extent the system continues to evolve beyond dynamic task allocation.
1 comment

I did not see an author response in the discussion available to me. Based on the evidence available to me, I will maintain my original rating.

Reviewer_bZWUOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes DEALS, a decentralized framework for coordinating concurrent tasks in multi-agent LLM systems. DEALS introduces a lightweight routing mechanism that balances agent workloads and task-specific expertise. Experiments on MATH, BBH, and MMLU-Pro with homogeneous and heterogeneous agent pools show improvements in average accuracy and throughput over existing baselines.

Critical strengths

The paper

Critical weaknesses
  1. With identical base models and similar success rates, DEALS largely reduces to agent-level load balancing. The paper lacks comparisons with multi-replica LLM serving schedulers, which can exploit finer-grained load and KV-cache information.

  2. Figure 4 shows that increasing the quality weight primarily shifts final task completion toward Qwen. Table 1 also shows that the Qwen single agent outperforms both the Mistral and Llama single agents on all three benchmarks. The current evaluation does not establish whether DEALS improves upon always selecting Qwen, selecting the best agent per task type using warm-up statistics, or routing by estimated quality alone.

Questions for authors
  1. In Table 1, why does DEALS improve average accuracy over the single-agent baseline in homogeneous pools? Is this due to random variation, or does DEALS introduce additional capabilities?

  2. Figure 3 compares concurrent DEALS against sequential AgentNet, confounding routing gains with concurrency gains. How does DEALS compare with concurrent AgentNet and standard load balancing under identical hardware, model replicas, and concurrency limits? Please report both throughput and latency.

Reviewer_kxqROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a framework for multi-agent systems to process concurrent tasks. Each agent is assigned with queues, a router, and a react executor with several parallel slots and a private memory system. The router forwards tasks to the neighbor with the highest score, or executes / splits it locally. On benchmarks including math and mmlu, the paper reports the best avg. accuracy across baselines, and up to 22.7x throughput compared to AgentNet.

Critical strengths
  • [S1] A simple, training-free router that works as intended.

Forwarding uses one closed-form score over local backlog and success-rate differences, making it easy to understand and implement.

  • [S2] A relevant and under-studied setting.

Most MAS work studies how agents collaborate on a single task, while this paper treats the MAS as a serving system with concurrent heterogeneous tasks, queues, and resumable partial work.

Critical weaknesses
  • [W1] The throughput and load-serving lack a fair evaluation.

The 22.7× speedup compares DEALS, running up to 27 tasks at once, with an AgentNet that runs one task at a time, so it mostly reflects concurrency rather than the router. Latency instead is never measured, and neither are token cost or throughput as V changes. Also, Algorithm 1 enqueues all tasks at the start, so the continuous-stream setting that motivates the paper is never tested.

  • [W2] Accuracy gains are within noise.

Results seem to be single runs on 100–140 test questions, and even in the homogeneous V sweep where V should barely matter, accuracy swings by up to 7 points, as large as most of DEALS’s gains over the best baseline in Table 1.

Questions for authors
  • [Q1] Concurrency-matched comparison. Please report throughput and latency for AgentNet and a batched single agent under the same concurrency budget and hardware, and DEALS with random or JSQ routing.
  • [Q2] Variance and choice of V. Can you report mean ± std over several ingress/request seeds for Tables 1 and 2, and explain how V=20 and V=80 were chosen without a validation split? For example, why does M5, which swaps two Qwen agents for weaker ones, beat the all-Qwen H5 by 4.3 points at V=80?
  • [Q3] Best-agent baseline. How does DEALS compare with simply sending each task type to the best warm-up agent? How do throughput and latency change in the mixed pool as V goes up, and the one agent ends up handling almost all MATH tasks?
  • [Q4] End-to-end overhead. How many LLM calls and tokens per task do DEALS and AgentNet use? The DEALS executor still calls the LLM for action selection, preliminary thoughts, and completion checks, so the end-to-end saving from the training-free router is unclear.
  • [Q5] Baseline sanity. Several baselines are far below the single agent. This is abnormal.

Comments on the paper

1 comment

Dear Reviewers,

As the AC–reviewer discussion phase is about to begin, please highlight any disagreements or inconsistencies across the reviews, as well as any concerns that remain unresolved after the author rebuttal.

Please share your perspectives on these points and engage with one another to help us reach a well-supported assessment of the paper.

Thank you for your time and contributions.

Best regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.