Learning to Communicate: Toward End-to-End Optimization of Multi-Agent Language Systems

Multi-agent systems built on large language models have shown strong performance on complex reasoning tasks, yet most work focuses on agent roles and orchestration while treating inter-agent communication as a fixed interface. Latent communication through internal representations such as key–value caches offers a promising alternative to text-based protocols, but existing approaches do not jointly optimize communication with multi-agent reasoning. Therefore we propose DiffMAS, a training framework that treats latent communication as a learnable component of multi-agent systems. DiffMAS performs parameter-efficient supervised training over multi-agent latent trajectories, enabling agents to jointly learn how information should be encoded and interpreted across interactions. Across five backbones and six mathematical reasoning, scientific QA, code generation, and commonsense benchmarks, DiffMAS matches or exceeds single-agent inference, text-based multi-agent systems, and prior latent communication methods in every setting, achieving +26.7% on AIME24 and +20.2% on GPQA-Diamond, and it also improves decoding stability. Against single-agent and text-based pipelines fine-tuned on identical data with the same recipe, DiffMAS is better in 26 of 30 settings and never worse, while fine-tuning through the text channel lowers accuracy in 13 of 30 settings, consistent with our analysis that text communication lets training adapt only how messages are read, not how they are written.