- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The work presents PANDA, a decentralized architecture for large-scale LLM multi-agent systems. PANDA maintains distributed agent registries, separates collective-scale communication from reliable within-team communication, supports load balancing and concurrent scheduling, and decouples the architecture from star, chain, and mesh orchestration strategies. It also provides infrastructure/orchestration failure detection and replanning, together with a web-of-trust mechanism for decentralized governance. Evaluation on HotPotQA studies team-assembly scalability, runtime and accuracy, behavior under injected faults, and trust propagation under agent churn.
Critical strengths
PANDA is a coherent end-to-end systems design. Scalability, orchestration, fault recovery, and governance are considered within a single decentralized architecture, and the implementation demonstrates substantial engineering effort.
Critical weaknesses
-
The technical novelty appears to lie primarily in systems integration. Several central ingredients of PANDA have close precedents in recent LLM multi-agent systems. For example, Internet of Agents already studies heterogeneous third-party agents in a distributed setting with dynamic inter-agent communication [1]; Symphony proposes decentralized capability discovery and task allocation together with scalable and fault-tolerant orchestration [2]; and MultiAgentBench explicitly evaluates multiple communication topologies, including star, chain, tree, and graph structures [3]. Beyond the LLM-agent literature, PANDA's gossip-based dissemination, load balancing, failure detection, web-of-trust, and DAG-based recovery also largely instantiate established distributed-systems ideas. I therefore view the main contribution as a coherent integration of these mechanisms into one LLM-MAS architecture.
-
The empirical scope is too narrow for several broad claims. Task-level evaluation is almost entirely based on HotPotQA, which does not establish that flexible orchestration is beneficial across structurally different agent tasks. Similarly, the fault experiment compares against systems without equivalent recovery mechanisms, while the governance experiment mainly tests liveness/trust propagation under churn rather than adversarial or malicious participants.
[1] Chen, Weize, et al. "Internet of agents: Weaving a web of heterogeneous agents for collaborative intelligence." International Conference on Learning Representations. Vol. 2025. 2025.
[2] Wang, Ji, et al. "Symphony: A decentralized multi-agent framework for scalable collective intelligence." arXiv preprint arXiv:2508.20019 (2025).
[3] Zhu, Kunlun, et al. "Multiagentbench: Evaluating the collaboration and competition of llm agents." Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
Questions for authors
-
Can the authors demonstrate the benefit of flexible topology selection across multiple qualitatively different tasks, ideally including an automatic mechanism for selecting star/chain/mesh rather than configuring the topology externally?
-
How does PANDA compare against a baseline equipped with generic checkpointing/retry/failover mechanisms, rather than baselines that have no mechanism to recover from the injected crash?
-
How does the web-of-trust mechanism behave under malicious certificates, Sybil participants, compromised trusted nodes, or strategically misadvertised capabilities?
2 comments
Dear reviewer X7wE,
Thank you again for your review. With the discussion period closing soon, we wanted to make sure our responses reached you and check if our clarifications address your concerns. We are happy to clarify further or run additional experiments if that would help your reviewing.
Best, The Authors
We thank the reviewer for their careful review and constructive feedback, and address their comments below. The submission has been updated accordingly, with all changes shown in blue.
Technical novelty Like other multi-agent systems, PANDA draws on established distributed-systems techniques. We deliberately leverage well-understood primitives for scalability and fault tolerance rather than inventing new ones, allowing us to focus on the architectural challenges specific to LLM agents. PANDA's contribution is the adaptation and coherent integration of these mechanisms into an architecture that jointly supports scalability, fault tolerance, flexible orchestration, and decentralized governance. As discussed in the Introduction and summarized in Table 1, existing systems address subsets of these requirements but do not satisfy all of them jointly. PANDA addresses the interactions among these requirements through architectural decisions specific to LLM-based multi-agent systems. To our knowledge, PANDA is the first system to satisfy all four requirements jointly.
Importance of flexible orchestration PANDA does not propose flexible topology selection as a performance optimization. Rather, different topologies accommodate different requirements concerning privacy, trust, coordination, and control, which may vary across users and applications. For example, mesh supports collaborative execution among agents, star centralizes coordination in a single orchestrator, and chain distributes decision-making along the execution path. These differences reflect architectural tradeoffs rather than a universally optimal topology. We support all three to accommodate diverse application requirements, not to demonstrate that selecting among them improves task performance. Accordingly, topology selection is driven primarily by external requirements and user preferences, which an automated selection mechanism would also need to consider alongside task characteristics. We add this discussion in Section 2.4.
Malicious participants PANDA focuses on scalability and fault tolerance under crash failures, rather than Byzantine resilience. We assume that participants in the web of trust are honest and that their identities and advertised capabilities have been verified at admission. Under these assumptions, certificates provide authenticated attestations that can be propagated through the trust graph. We acknowledge that malicious certificates, Sybil identities, compromised trusted nodes, and strategically misadvertised capabilities can undermine these guarantees if the assumptions are violated. Designing a Byzantine-resilient decentralized governance mechanism is outside the scope of this work and is the subject of ongoing complementary research. We clarify the threat model and its limitations in Section 2.3.
Comparison with a checkpointing baseline We conducted a semi-synthetic ablation measuring the effect of worker failures on task completion time and display the results in the paper in Figure 5. Over 5000 simulated trials across the three topologies, we sample the number of subtasks, and the duration of each subtask, uniformly from 1000 recorded runs of each topology. We then define the overhead added upon a failure in a centralized checkpointing case vs PANDA. With checkpointing the two main costs incurred are (1) the time it takes to recover and (2) how much work was lost reverting to the checkpoint. In PANDA, once we detect a failure, we immediately swap in a healthy agent, avoiding recovery latency and losing no completed work -- only the in-flight subtask is retried. Given PANDA’s failure detector, we detect a failure with probability 1−e1 every T seconds [1] and we model this timing in our experiment. We then evaluate how task completion time increases as the probability of a subtask failing increases and demonstrate that PANDA with a reasonable T=3, performs significantly better than any recovery baseline even assuming a relatively fast 15 second recovery and immediate checkpointing. We present the results of this experiment in Appendix C.
[1] Gupta, I., Chandra, T. D., and Goldszmidt, G. S. "On scalable and efficient distributed failure detectors." PODC 2001.