acceptodds
Under review as a conference paper at ICLR 2027

Granule-R1: Self-Adaptive Granularity Management for Agentic QA via Boundary-Aware Multi-LLM Routing

Abstract

Current Large Language Model (LLM) routers predominantly operate at a coarse, query-level granularity or rely on static decomposition-then-allocation pipelines. These approaches often overlook the self-adaptive decision-making potential of the router itself—specifically, the ability to discern whether a task requires holistic reasoning or granular divide-and-conquer. We identify a critical opportunity to optimize this process by empowering the router to understand its own capability boundaries and those of a heterogeneous model pool. To address this, we present Granule-R1, an agentic reinforcement learning framework that redefines the router as a self-adaptive decision-making agent. Unlike prior methods that force a fixed decomposition structure, Granule-R1 autonomously evaluates the inherent difficulty of a query to decide between direct solving and strategic decomposition. By mastering the "capability-cost" trade-off, Granule-R1 identifies the most suitable model—not necessarily the strongest or cheapest—to address specific task granules. The system operates through a refined set of logic primitives—Think, Decompose, Search, Commit, and Answer—and is trained via Reinforcement Learning with a multi-objective reward that simultaneously optimizes for answer accuracy and inference efficiency. Evaluations on seven multi-hop and general QA benchmarks show that Granule-R1 significantly outperforms state-of-the-art methods. It achieves a 3.4-point absolute improvement in average Exact Match (EM) while reducing operational costs by 17.8 %. Most importantly, Granule-R1 enables a system of 7B-tier models to surpass the performance of whole-query routing using 27B-tier models at less than half the cost. These findings demonstrate that router-level self-adaptivity and granularity management are critical dimensions for building the next generation of cost-effective, high-performance LLM systems.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM Routing, Agentic Reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AMKvOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the problem that most existing LLM routers treat the question as a whole, without judging whether the question needs to be decomposed. To this end, the paper proposes Granule-R1, which trains a small model as the router with RL. Through the actions Think, Decompose, Search, Commit, and Answer in a multi-turn interaction, the router decides (i) whether to answer directly, send the whole question to a model, or decompose it into subquestions, (ii) which model to send each subquestion to, and (iii) how to commit subanswers and integrate them into the final answer. The paper trains the router with PPO, using a reward composed of answer correctness, format, decomposition quality, efficiency based on model prices, and adjustments after failures. The paper follows the setting of Router-R1, including its models, datasets, hyperparameters, and results, and compares on several general and multi-hop QA benchmarks. It claims to improve accuracy while reducing routing cost, and that a pool of only small models can match whole-question routing with larger models.

Critical strengths

None. I cannot identify any strength of this paper. Please see the Weaknesses below.

Critical weaknesses
  1. The proposed method is not very novel. (i) The paper builds directly on Router-R1, with the same training data, models, and evaluation setting. Router-R1 can already send subquestions to different models during multi-turn interaction, and the authors acknowledge in the appendix that Router-R1 performs implicit decomposition. This paper only turns decomposition into an explicit action and adds some reward shaping. (ii) Decomposing a question into subtasks and assigning them to different models has already been done by R2-Reasoner and RouteGoT. R2-Reasoner is only compared in the appendix through a zero-shot transfer that the authors themselves acknowledge is not a fully matched comparison. (iii) The paper is framed with terms such as self-adaptive, introspection, and capability boundaries, but the actual mechanism is just a decompose tag and a few reward terms.

  2. The credibility of the experimental results is questionable. (i) The EM and F1 of all baselines in Table 1, as well as the costs of all baselines in Table 7, are all the same as the numbers published in the Router-R1 paper. However, the appendix says that "GraphRouter is independently trained" and "RouterDC uses its public checkpoint", which is inconsistent with the numbers being exactly the same. (ii) Some conclusions are inaccurate and overstated. For example, the paper states that "it surpasses the strongest 27B-access multi-round router by 0.02 EM points (0.303 vs. 0.301)", but the difference is 0.002 rather than 0.02, and such a small difference does not support the word "surpasses". (iii) Apart from Appendix A.8, the main results, the small-pool comparison, and the ablations are all single runs without standard deviations or confidence intervals. Many improvements over the baselines are very small and may not be significant.

  3. The introduction raises two questions, how to decompose the original question and where the capability boundaries of the available models lie for different subquestions, but the experiments do not decouple these two questions and analyze them separately. (i) If only decomposition is used without model routing, e.g., with a single model in the pool, could the performance gain come from decomposition itself? The paper has no such experiment and does not compare with single-model decomposition methods such as Least-to-Most and Self-Ask. (ii) If the question is decomposed randomly or by fixed rules before routing, would there also be a large gain? The paper does not test this either. (iii) The only support for capability boundaries is statistics such as call ratio, with no evidence that the router actually selects the right model. The ablation study cannot tell whether the gain comes from decomposition, from routing, or from their combination.

Questions for authors
  1. The process is a multi-turn loop in which the router generates reasoning and actions over an ever-growing context at each turn, but the cost only counts the output tokens of external models and ignores the router's own computation, the overhead of multiple external calls, and the end-to-end latency. What is the complete cost once the router's own computation is included? Please report the distribution of the router's input and output token lengths (maximum, mean, and median) and the end-to-end latency, and compare them with Router-R1.
  2. The router has a heavy reasoning load. It judges difficulty, decomposes the question, selects models, judges the returned results, commits subanswers, and integrates the final answer, all of which are reasoning-heavy, yet the router is only a 3B model. Table 15 shows that about 94% of the sampled errors come from the router itself and only 6% from the limited capability of the candidate models, and Table 10 shows that allowing replanning increases format violations and lowers accuracy, suggesting that the router gets confused as the task chain grows longer. Could the authors analyze the router's reasoning capability, e.g., the accuracy of its decomposition, model selection, and subanswer commitment, or the results with a larger router?
  3. The reward design is complex and sensitive. There are 5 reward terms and, together with the sub-terms of the decomposition reward, about 13 weights, all tuned manually in preliminary experiments without any sensitivity analysis. How do accuracy and cost change as the reward weights varies? Could the authors provide an accuracy–cost curve and a sensitivity analysis of the reward weights?
  4. Which numbers in Tables 1 and 7 are copied from the published Router-R1 results, and which are re-run by the authors? If GraphRouter was trained independently and RouterDC was re-evaluated with its public checkpoint, why are the results digit-for-digit identical to those published by Router-R1?
  5. How much does performance improve if the pool contains only one model and only decomposition is used? What are the results if the question is decomposed randomly or by fixed rules before routing?
Reviewer_bbM2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Granule-R1, an RL-based framework that jointly learns task decomposition and multi-LLM routing. A unified router decides whether to answer directly, route the entire query, or decompose it into subquestions and allocate them to different LLMs. The router is trained using PPO with correctness, decomposition, and efficiency rewards. Experiments on seven QA benchmarks demonstrate improvements in accuracy and external-model inference cost.

Critical strengths
  1. Unified framework. The integration of adaptive task decomposition and model routing into a single RL policy is intuitive and practically useful, allowing the router to adjust its strategy based on task complexity.

  2. Promising empirical results. The method consistently outperforms routing baselines, with particularly encouraging results under the same small-model pool. The component ablations and statistical evaluations further support the effectiveness of the proposed framework.

Critical weaknesses
  1. Insufficient ablation of RL optimization and reward design. The paper adopts action-segment PPO with GAE under predominantly terminal rewards, but provides no ablation of the RL optimization algorithm. How does the proposed approach compare with standard token-level PPO, outcome-level GRPO, and step-level GRPO under the same training budget? Furthermore, the efficiency reward considers only the last invoked model for each subquestion rather than the cumulative cost of all calls, potentially misaligning the training objective with actual routing efficiency.

  2. Incomplete computational cost evaluation. The reported inference cost excludes router computation and input tokens, despite increased external calls and controller latency. Similarly, the training cost analysis omits external LLM calls during rollouts. A more comprehensive evaluation of end-to-end inference and training costs is needed to substantiate the efficiency claims.

Questions for authors
  1. Can the authors provide a controlled comparison between the proposed action-segment PPO, standard token-level PPO, and GRPO-based alternatives under comparable training budgets?

  2. How would the reported efficiency gains change when accounting for router inference, input tokens, and external LLM computation during training? Could the authors also compare the proposed efficiency reward against an objective based on cumulative routing cost?

  3. Given the one-shot decomposition strategy and limited interaction budget, how well does the framework handle tasks requiring dynamic replanning or longer-horizon reasoning? Could the authors clarify the limitations of the current decomposition mechanism?

Reviewer_G3inOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Granule-R1, an agentic reinforcement learning (RL) framework that transforms LLM routing from a static query-level classification or fixed decompose-then-allocate pipeline into a unified, self-adaptive decision-making process. Using a single 3B-parameter policy trained end-to-end with PPO (without oracle decomposition or allocation labels), Granule-R1 dynamically decides whether to answer a query directly, route the entire query to an external model, or decompose it into ordered subquestions routed to cost-appropriate models in a heterogeneous pool. Across seven general and multi-hop QA benchmarks, Granule-R1 improves average Exact Match (EM) by 3.4 points and reduces external output-token routing costs by 17.8% compared to state-of-the-art multi-round routing. Notably, when restricted to a pool of 7B-tier models, Granule-R1 outperforms whole-query routing systems equipped with 27B-tier models at less than half the cost.

Critical strengths
  1. Meaningful Shift from Static to Self-Adaptive Granularity
  2. Unifying Decomposition, Routing, and Trajectory-Aware Integration
  3. End-to-End RL with Correctness-Gated Efficiency and Action-Only Scoring
  4. Extensive Experiments
Critical weaknesses
  1. Incremental RL Novelty and Severe Over-Compression of Section 3 From a modeling perspective, the algorithmic novelty is largely an incremental extension of Search-R1 and Router-R1. The framework reuses the same paradigm of training a 3B Qwen policy via veRL and PPO. Furthermore, Section 3 is compressed into only 1.5 pages of text, making the main paper not self-contained. Critical technical definitions including the transition grammar, the mathematical formulations of three out of the five reward components, and the segment-level PPO surrogate loss are entirely pushed to the appendix.

  2. Crowded Routing Literature and Narrowly Scoped Differentiation LLM routing and task decomposition are both heavily studied areas, including query-level routers (FrugalGPT, RouteLLM, RouterDC, GraphRouter), sequential RL routers (Router-R1, Route-to-Reason), and subtask-level routing frameworks (R2-Reasoner, RouteGoT). Consequently, the broad problem studied is not novel. While the paper carves out a specific niche by unifying adaptive decomposition and model selection into a single end-to-end policy, this differentiation relies on a strict one-shot, linear execution assumption and a regex-based informativeness heuristic that narrows its conceptual leap over existing multi-step routing formulations.

  3. Weak Static Baselines and Missing Matched Decompose-then-Route Competitors While the head-to-head comparison against Router-R1-Qwen is well-controlled, the broader baseline suite in Table 1 is insufficient to fully validate the paper's core claims. In Table 1, the learned static routers (RouterDC at 0.314 EM and GraphRouter at 0.297 EM) underperform naive Prompt LLM (0.329 EM) and Largest LLM (0.338 EM), while more recent reasoning-aware routers cited in Section 2 (e.g., Route-to-Reason, R2-Router, UniRoute) are omitted from the experiments.

Questions for authors

See the comments above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.