PartiRoute: Decentralized Self-Participation for Multi-LLM Routing

Large language model (LLM) routing is commonly formulated as a centralized selection problem, often through a routing decision that selects a single model or determines a routing path. Such early commitment can discard useful alternative responses and limit recovery from suboptimal routing decisions. We argue that multi-LLM routing should instead separate candidate construction from final-answer selection, reformulating the question from which model should be selected to whether each model should participate. Based on this view, we introduce PartiRoute, a decentralized routing framework in which each frozen LLM is paired with an independently parameterized policy that makes a model-wise participation decision. Their joint decisions form a query-dependent candidate pool optimized under a shared quality–cost utility. To capture the set-dependent contribution of each model, PartiRoute uses an action-flip difference reward for model-specific credit assignment and trains the routing policies directly with PPO, without fine-tuning the candidate LLMs or requiring supervised router warm-up. Experiments across five benchmarks and 13 heterogeneous LLMs show that PartiRoute forms dynamic candidate pools while achieving competitive accuracy–cost performance. Response-aware aggregation further improves the utilization of retained responses, highlighting the value of decentralized self-participation and the separation between candidate construction and final-answer aggregation.