- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DEALS, a decentralized framework for concurrent task processing in LLM-based multi-agent systems. It uses an expertise-aware routing score that combines task backlogs and historical success rates to guide task allocation, along with memory-guided execution and concurrency support. Experiments on MATH, BBH, and MMLU-Pro report improvements in accuracy and throughput over existing baselines.
Critical strengths
- The problem is practical and relevant. Coordinating concurrent requests across agents with different capabilities and workloads is an important problem for multi-agent serving systems.
- The routing mechanism is lightweight. DEALS uses only local and neighbor state, with no extra LLM calls or trained router for routing. The V=0 vs. V>0 comparison in the heterogeneous pool clearly shows that success-rate information helps steer tasks away from weaker agents.
Critical weaknesses
- The evidence for expertise-load balancing is limited. At (V=80), Qwen produces nearly all final answers on MATH (138/140) and most on BBH (88/100), while the heterogeneous pool achieves similar accuracy to the Qwen-only pool. These results suggest that the router mainly favors the stronger agent, rather than clearly demonstrating the benefits of heterogeneous collaboration or workload balancing. The choice of (V) also lacks a clear justification or validation procedure.
- The throughput gains need stronger experimental support. DEALS reports 4.7×–22.7× higher throughput than AgentNet, but DEALS supports concurrent execution while AgentNet processes tasks sequentially. It is therefore difficult to isolate the contribution of the proposed routing strategy. Comparisons with simpler routing methods under the same concurrency settings would be helpful. Moreover, although low latency is emphasized in the Introduction, no end-to-end latency results are reported.
Questions for authors
- Could the authors clarify how task arrivals are modeled in the experiments? The introduction emphasizes continuous task streams, but the experimental setup mainly describes how tasks are distributed across agents. Are tasks injected over time, or are they all queued at the beginning of evaluation?
- Could the authors provide more intuition or justification for the routing design in Eq. (3)? The current formulation seems somewhat heuristic, as it directly combines backlog and success-rate differences through a weighted sum. A more detailed explanation of the design rationale would help better understand the proposed method.
- Could the authors clarify what is meant by self-evolution in this framework? Since the success-rate estimates and private memories are frozen during testing, it is unclear to what extent the system continues to evolve beyond dynamic task allocation.
I did not see an author response in the discussion available to me. Based on the evidence available to me, I will maintain my original rating.