TurnRoute: Learning Modular Agent Capabilities for Dynamic Composition

Language-model agents coordinate multiple capabilities over long interactions, yet typically learn them jointly within a monolithic set of adapted parameters. Can these capabilities instead be decoupled into separate parameter modules and dynamically recomposed during execution? We introduce TurnRoute, a framework that trains capability-specific LoRA experts through turn-level loss masking of labeled trajectories over a shared frozen backbone. During execution, a router selects one adapter to generate each turn while maintaining a shared interaction history, with an unmasked adapter providing a fallback. On software engineering tasks, capability probes show that masked experts retain comparable competence to monolithic adaptation on their assigned capability while exhibiting behavioral separation across adapters. Although individual specialists underperform as standalone agents, TurnRoute achieves strict pass@1 of 15.8% at 7B and 28.4% at 14B on 500 SWE-bench Verified instances, outperforming monolithic LoRA by 1.6 and 2.0 pp. It reaches 98% and 92% of full fine-tuning performance with roughly 9.4–11.7× fewer trainable parameters than full fine-tuning, while activating only one expert adapter per turn for response generation. In addition, at 7B, TurnRoute increases non-empty patch production from 77.8% to 94.4%, reducing empty-patch failures by 74.8%. Together, these results demonstrate the feasibility of decoupling agent capabilities into separately trainable parameter modules and dynamically recomposing them while retaining end-to-end task competence, providing a practical approach to modular adaptation of language-model agents.