acceptodds
Under review as a conference paper at ICLR 2027

AdaRouter: Learning to Adapt Plans for Agentic Routing via Multiple Large Language Models

14 views

Abstract

Agentic routing coordinates calls to heterogeneous large language models (LLMs) and is promising to balance the task performance and inference cost. However, contrary to human problem solving, existing methods for agentic routing lack the ability of reconsideration with feedback evidence gradually acquired during task execution. In this paper, we propose AdaRouter, an adaptive planning framework incorporating reconsideration with available evidence to bridge the gap between human problem solving and agentic routing by LLMs. Specifically, we design two mechanisms for reconsideration, i.e., REFINE that incorporates the new evidence into the query for the next task and REVISE that changes the infeasible plan by updating the remaining tasks or their execution order, while maintaining a coherent guidance across worker calls. Furthermore, we develop a two-level strategy specifically for optimizing AdaRouter with the REFINE and REVISE mechanisms. We achieve trajectory-level GRPO to optimize the overall planning-and-routing behavior and token-level OPSD to distill task-relevant problem-solving skills into routing policy. Extensive experiments across five domains ranging from mathematics to general knowledge and multi-hop reasoning demonstrate that AdaRouter achieves state-of-the-art performance with substantially reduced worker calls in comparison to the strongest routing baseline.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Agentic Routing, Adaptive Planning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HPCuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors identify flaws in existing agentic routing schemes: they either make incremental, reactive decisions without sustained global guidance—leading to shortsighted planning—or fix the task structure prior to execution, unable to re-evaluate based on feedback—resulting in rigid planning. Therefore, they propose AdaRouter, an adaptive planning framework for agentic routing designed to coordinate calls to a heterogeneous, fixed set of LLM "workers," thereby balancing task accuracy against inference costs. AdaRouter addresses this gap by maintaining an explicit solution plan and dynamically adjusting it in real-time as feedback from workers arrives.

The method was evaluated across 11 benchmarks covering mathematics (GSM8K, MATH), code (HumanEval, MBPP), general knowledge/common sense (MMLU, GPQA), multi-hop QA (HotpotQA, 2WikiMultiHopQA, MuSiQue), and world knowledge (NQ, TriviaQA), as well as on four held-out datasets for out-of-distribution (OOD) testing. AdaRouter achieved a macro-average accuracy of 65.4%, outperforming the strongest routing baseline by 9.5 percentage points (with gains of 12.5 and 8.7 percentage points in code and multi-hop tasks, respectively); meanwhile, it required fewer worker node invocations (averaging 1.5) than multi-turn interaction baselines and incurred lower total costs than GraphPlanner and Prompt-LLM. Across various \alpha settings, the method consistently maintained a position on the Pareto frontier regarding the trade-off between accuracy and cost.

Critical strengths

The proposed AdaRouter method employs a scientifically grounded, two-level training framework that effectively resolves the "credit-assignment" challenge. Because the "plan-adaptation" mechanism alters both current worker invocations and future execution trajectories—making it difficult to attribute final outcomes to specific individual decisions—AdaRouter introduces a complementary supervision strategy: trajectory-level GRPO (which holistically evaluates correctness, invocation costs, and adaptation quality) provides a global signal, while token-level OPSD offers dense, token-weight-based guidance on when and how to execute "REFINE" or "REVISE" operations. This design is significant as it enables the automated learning of adaptive behaviors rather than relying on manually defined rules; consequently, AdaRouter achieves a macro-averaged accuracy of 65.4%—surpassing the strongest baseline by 9.5 percentage points—with significantly fewer worker invocations than multi-round baselines. This simultaneous enhancement of performance and efficiency embodies the core value of the "routing" mechanism.

Critical weaknesses
  1. This evaluation relies on a single, fixed router configuration and does not examine the system's scalability. All primary variants of AdaRouter utilize Qwen3-4B-Instruct as the trainable router, paired with a fixed pool of eight frozen worker models (ranging in size from 7B to 119B parameters). Although generalization experiments covered unseen tasks and unseen worker LLMs, the router strategy itself was not tested under varying conditions; there is currently no evidence regarding how this adaptive planning paradigm performs when employing a router backbone that is larger, more capable, or architecturally different. This is a crucial point, as the study's core claims—such as state-of-the-art accuracy, superior cost-accuracy trade-offs, and robust generalization—are derived from a specific 4B-parameter strategy; if these advantages are heavily dependent on the router's capacity or the specific configuration of the reward/evaluation mechanism, the paradigm's practical applicability could be far more limited than the paper suggests.

  2. Determining the REFINE/REVISE reward involves nuanced qualitative judgments—such as whether a query is self-contained and free of unverified information, and whether the revised observation yields "useful evidence"—whereas trajectory analysis relies on aggregate counts derived from "single-model offline annotation." The report does not mention any validation regarding human inter-rater agreement or evaluator quality. This is critical because these rewards serve not merely as evaluation metrics but as training signals: if the automated evaluator harbors noise or systematic bias, then the REFINE/REVISE behaviors learned by the model—as well as key conclusions regarding the reduction of unnecessary plan revisions and the increased utility of the first worker invocation following a revision—might well be artifacts of the evaluation mechanism rather than genuine improvements in routing quality.

Questions for authors

The rewards and trajectory metrics within the REFINE/REVISE mechanism rely on nuanced automated judgments—such as determining whether a query is "self-contained" and "free of unverified information," whether the remaining plan is "inappropriate," and whether the revised observations provide "useful evidence." Did you validate these evaluators against human-annotated data (e.g., by examining agreement rates or conducting error analysis)? If the evaluators are noisy, to what extent might the learned revision behaviors—and the reported outcome of "reduced unnecessary revisions"—simply be artifacts of the evaluators' characteristics rather than genuine improvements in routing?

The reported total cost accounts solely for the invocation overhead of the worker nodes. Given that the Router itself is a 4-billion-parameter Large Language Model (LLM) responsible for generating plans, queries, and revisions across multiple rounds of interaction, why was the Router's own inference cost excluded? How would the Pareto analysis results regarding the trade-off between accuracy and cost (see Figure 3) change if this cost were included?

1 comment
AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer HPCu,

Thank you again for your careful review and constructive feedback on our paper. We have submitted our point-by-point responses to the concerns raised in your review and would greatly appreciate it if you could take a moment to look through them.

We hope our clarifications address your questions and provide a better understanding of our proposed method and experimental results. If any concerns remain or further clarification would be helpful, please do not hesitate to let us know. We would be happy to continue the discussion.

Thank you again for your time and consideration.

Best regards,
The Authors

Reviewer_puBaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a multi-agent framework that combines explicit planning with dynamic routing to address two issues: the tendency of incremental planning to lose sight of global task objectives, and the inability of pre-planned tasks to fully account for new evidence. The main contributions include the definition of REFINE and REVISE, two types of rewards and the application of joint training in multi-agent routing, achieving SOTA performance on five domain-specific datasets.

Critical strengths
  1. Modifying only the router framework ensures scalability, and the experimental results are quite clear and provide a thorough comparison.
  2. Consistent global planning and local evidence propagation align with the concepts of REFINE and REVISE, reflecting adaptive planning.
Critical weaknesses
  1. It primarily provides the results and costs of worker calls, but does not include the costs associated with the Router itself. Furthermore, the number of worker calls does not indicate efficiency.
  2. Almost all datasets used only 50 test samples; in such cases, the mean and variance from multiple seed of different test samples should be reported.
Questions for authors

The reward design relies on the powerful DeepSeekV4-Pro. Are there any experiments demonstrating that the rewards obtained are relatively stable?

Reviewer_GwhiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AdaRouter, a learned framework for coordinating heterogeneous LLM workers through an explicit, revisable plan. It includes two mechanisms: REFINE, that incorporates new evidence into the next query while preserving the plan, and REVISE, which changes the unfinished task structure. Training combines trajectory-level GRPO rewards for correctness, worker cost, and adaptation quality with skill-conditioned on-policy self-distillation. Evaluation spans five domains, with additional tests on four held-out tasks and an expanded worker pool. AdaRouter reports a macro-average score of 65.4%, exceeding the strongest routing baseline by 9.5%, while averaging 1.5 worker calls per query. Structural ablations, training-stage comparisons, and trajectory analyses support the usefulness of planning, adaptation rewards, and self-distillation.

Critical strengths

AdaRouter outperforms the evaluated routing baselines in all five domains, with particularly strong gains in code generation and multi-hop reasoning, alongwith generalization to unseen tasks and LLMs. The training-stage results show consistent improvements from GRPO and then self-distillation, with reduced unnecessary revisions and improved recovery.

Critical weaknesses

The cost doesn’t include the router’s generations and plan adaptations, so comparison with single round router whose routing cost is negligible, is unfair. Also, it is not clear whether the baselines receive the same training data, LLM backbone, and worker prompt wherever possible.

Questions for authors
  1. Can the authors also try to evaluate the generalization ability to unseen domains, like holding out code/math from the training set?

  2. Can the authors also report the cost associated with the router and end to end latency, which involves initial plan generation and multiple rounds of plan adaptation?

  3. Do baselines (R2-Reasoner and Router-R1) use the same training tasks as AdaRouter? How would those baselines perform when further optimized using OPSD?

  4. For each domain, what fraction of test trajectories contain at least one REVISE, and at least one REFINE? What is the accuracy with and without REVISE?

  5. Which data defines validation accuracy?

  6. How do the individual workers score on the different benchmarks?

1 comment
AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer Gwhi,

Thank you for your detailed questions about AdaRouter's evaluation. We posted two replies under your review, including the updated cost analysis with the router's own inference cost and a clarification of the settings used to train and evaluate the baselines.

If you have a chance to revisit the replies, we would appreciate your thoughts on whether the cost analysis and baseline setup address your concerns. We would be glad to clarify any remaining points.

Thank you for your time.

Best regards,
The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank all three reviewers for their constructive feedback, which helped us improve the manuscript. We were encouraged that puBa highlighted how REFINE and REVISE connect global planning with local evidence and that HPCu recognized the complementary roles of GRPO and OPSD in assigning credit to routing decisions. Reviewer Gwhi also highlighted the gains on code generation and multi-hop reasoning and the generalization to unseen tasks and workers.

We have revised the manuscript in response to the reviewers' suggestions.

1. Clarifications on the experimental protocol

  • Baseline implementation and evaluation. We documented the Router-R1 replication setup for Table 2, including the router backbone, training and test splits, and worker prompts shared with AdaRouter (Appendix D).
  • Checkpoint selection. We corrected the term "validation accuracy" and documented how we selected the checkpoint using test accuracy (Appendix D).

2. Additional experiments and measurements

  • Expanded in-domain evaluation. We expanded the in-domain evaluation to 1,500 questions, with 300 per domain. The revised Table 2 reports 67.3% average accuracy for AdaRouter, 6.7 percentage points ahead of the strongest evaluated routing baseline. Table 8 lists the test counts by benchmark.
  • Zero-shot generalization and backbone evaluation. We added zero-shot results on IFEval, LongBench v2, and MRCR (Appendix E.3). We also trained AdaRouter with Llama-3.2-3B-Instruct and observed gains in all five domains (Appendix H.1).
  • Standalone worker and trajectory analyses. We evaluated all eight workers individually on the same in-domain test questions and reported how often trajectories use REFINE and REVISE by domain (Appendices F.1 and I.1).

3. Inference efficiency and reward repeatability

  • Sensitivity to router cost and reference inference times. Including the estimated router cost brings the total to 0.3390 USD at the main operating point, which remains on the Pareto frontier. We also added a reference inference-time comparison (Appendices F.2 and F.3).
  • Reward consistency under repeated evaluation. We evaluated each of 60 trajectories three times with identical inputs. All three evaluations agreed on every applicable reward component for all 60 trajectories (Appendix H.2).

Thank you again for your time and feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.