Federated Multi-Agent Orchestration via Consensus-guided Interaction Dynamics

In large language model (LLM)-based multi-agent systems (MAS), the orchestration of inter-agent communication plays a crucial role in improving reasoning quality and collaboration efficiency. However, existing methods typically either learn a dedicated orchestrator for each domain or train a single orchestrator on centrally pooled multi-domain data. These paradigms limit knowledge integration across heterogeneous clients with siloed data, hindering both the learning of a unified orchestrator that accommodates diverse agent configurations and generalization to unseen clients. To address these limitations, we propose FedGLV, a federated MAS collaboration framework that learns transferable interaction dynamics for heterogeneous multi-agent orchestration. Specifically, to characterize heterogeneous inter-agent interactions across clients within a common dynamical formulation, we introduce an amortized relational dynamics model inspired by generalized Lotka–Volterra dynamics. Conditioned on queries and agent roles, this model couples latent relational-state evolution with adaptive topology generation, supporting varying numbers of agents and communication steps. At the server, data-free structural consensus distillation is introduced to integrate orchestration knowledge across heterogeneous clients. Concretely, guided by leave-one-out (LOO) valuation, FedGLV aggregates client soft topologies obtained using the server-generated probe bank into a dynamic structural consensus. This consensus provides an explicit structural supervision signal for the global orchestrator to learn transferable interaction patterns. Experiments on six source datasets for federated training and three unseen-client benchmarks demonstrate the effectiveness and generalizability of FedGLV.