ReLay: Are We Ready for Latent Routing for Multi-Agent Systems?

Large language model (LLM)-based multi-agent systems rely on communication and orchestration for collaborative reasoning. Existing methods often exchange reasoning traces in a fixed textual or latent form. Text preserves precise details but can be verbose, whereas latent representations offer semantic richness at the risk of obscuring critical information. Beyond representation choice, complete traces often contain essential information alongside erroneous, irrelevant, or redundant evidence. These limitations motivate segment-wise routing that combines complementary representations and filters low-utility content. Our empirical analysis further shows that three internal trace signals—entropy, maximum probability, and hidden-state norm—carry predictive information about downstream correctness, suggesting their utility for adaptive routing among TEXT, LATENT, and SKIP. We introduce ReLay, a two-level routing framework for effective multi-agent collaboration. For intra-agent routing, ReLay employs a Signal-Conditioned Utility Router (SCUR) to make segment-wise transmission decisions. SCUR aggregates internal trace signals to estimate the downstream utility of each communication choice. The router is trained on counterfactual receiver outcomes, and its predictions guide the construction of mixed-format messages. For inter-agent routing, an Adaptive Representation Orchestrator (ARO) conditions on the routed message and task state to select the next compatible agent or terminate the interaction. Experiments across reasoning and long-horizon decision-making tasks demonstrate that ReLay achieves the strongest overall performance–efficiency trade-off, outperforming baselines while substantially reducing communication cost.