Granule-R1: Self-Adaptive Granularity Management for Agentic QA via Boundary-Aware Multi-LLM Routing

Current Large Language Model (LLM) routers predominantly operate at a coarse, query-level granularity or rely on static decomposition-then-allocation pipelines. These approaches often overlook the self-adaptive decision-making potential of the router itself—specifically, the ability to discern whether a task requires holistic reasoning or granular divide-and-conquer. We identify a critical opportunity to optimize this process by empowering the router to understand its own capability boundaries and those of a heterogeneous model pool. To address this, we present Granule-R1, an agentic reinforcement learning framework that redefines the router as a self-adaptive decision-making agent. Unlike prior methods that force a fixed decomposition structure, Granule-R1 autonomously evaluates the inherent difficulty of a query to decide between direct solving and strategic decomposition. By mastering the "capability-cost" trade-off, Granule-R1 identifies the most suitable model—not necessarily the strongest or cheapest—to address specific task granules. The system operates through a refined set of logic primitives—Think, Decompose, Search, Commit, and Answer—and is trained via Reinforcement Learning with a multi-objective reward that simultaneously optimizes for answer accuracy and inference efficiency. Evaluations on seven multi-hop and general QA benchmarks show that Granule-R1 significantly outperforms state-of-the-art methods. It achieves a 3.4-point absolute improvement in average Exact Match (EM) while reducing operational costs by 17.8 %. Most importantly, Granule-R1 enables a system of 7B-tier models to surpass the performance of whole-query routing using 27B-tier models at less than half the cost. These findings demonstrate that router-level self-adaptivity and granularity management are critical dimensions for building the next generation of cost-effective, high-performance LLM systems.