MedRoute: Dynamic Specialist Routing for Multi-Agent Medical Diagnosis

Large Multimodal Models (LMMs) have shown promising diagnostic ability in medical reasoning, yet they are typically used as monolithic generalists or as agents assigned to static clinical roles. Existing multi-agent medical systems improve specialization through fixed expert panels, role-based discussion, or external knowledge augmentation, but their coordination is often weakly adaptive and not directly optimized for final diagnostic correctness. We propose MedRoute, a reinforcement-learning-based framework that formulates multi-agent medical diagnosis as a learned consultation policy. For each case, MedRoute constructs a compact, case-conditioned specialist pool and instantiates specialists with targeted clinical guidance. A General Practitioner (GP) then dynamically selects specialists conditioned on the input and accumulated diagnostic history, allowing each consultation to inform the next rather than producing individual expert opinions. To address the lack of step-wise supervision signals, we train the router with an outcome-level policy-gradient objective using group-relative advantage normalization, assigning credit to consultation trajectories based on final diagnostic correctness. A Moderator synthesizes the resulting diagnostic trajectory into the final prediction. Across three text-only and four image-text medical QA benchmarks, MedRoute consistently outperforms both single-model and multi-agent baselines, with gains of up to +22.5% on text-only and +30.4% on image-text over the single-model baseline. Our training code will be made publicly available.