- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces RouteWeaver, a unified LLM routing framework that uses a Qwen3-4B-based router to coordinate six frozen Worker models and select among single-round, multi-round, and role-based agentic execution modes for each query. To jointly learn mode selection and within-mode execution, the authors propose COMET-GRPO, which separates reinforcement learning feedback into signals for these two decision levels. They also introduce progressive exploration, allowing the router to first practice executing different modes before gradually taking control of mode selection, thereby preventing premature concentration on a single mode. Experiments across mathematics, coding, knowledge, logical reasoning, and factual recall achieve an average accuracy of 61.4%, compared with 55.3% for the strongest baseline, an improvement of 6.1 percentage points. Ablations, performance–cost analyses, and evaluations on held-out benchmarks further examine the method’s effectiveness. Its central contribution is to integrate model selection and execution organization into a single trainable routing policy.
Critical strengths
-
The authors propose a unified framework that integrates single-round, multi-round, and agentic routing, allowing the router to jointly decide which LLMs to invoke and how to organize their execution.
-
The authors introduce COMET-GRPO and progressive exploration to train the router through reinforcement learning, achieving substantial performance improvements using a training set assembled from existing benchmarks.
-
Through ablation studies and additional analyses, the authors demonstrate the effectiveness of the proposed components, examine the trade-off between accuracy and Worker inference cost, and evaluate generalization to held-out benchmarks within a fixed Worker pool. Their analysis also shows that the learned router selects different execution modes across datasets.
Critical weaknesses
-
The cost analysis does not account for the overhead of the complete system.
The paper reports only Worker inference costs, excluding the Qwen3-4B router itself. The router repeatedly processes interaction histories, generates routing instructions, and may organize workflows with multiple layers. This overhead cannot simply be assumed to be negligible. -
The generalization experiments are limited to a fixed model pool and do not establish adaptability to changes in deployment.
All training and evaluation use the same six Workers. Although the results on AIME, GPQA-D, and BBH support transfer to held-out benchmarks, the paper does not evaluate performance when models are added, removed, or upgraded, or when their prices change. -
The ablation studies do not sufficiently disentangle the contributions of different training mechanisms.
COMET-GRPO combines mode/execution advantage decomposition with separate normalization by the token count of each segment. The latter itself changes the relative training weight assigned to short mode-selection decisions and longer execution sequences. The current ablations do not clearly isolate the contributions of these two design choices.
Questions for authors
-
Can the authors provide more comprehensive generalization experiments?
Some evaluations in Table 1 use datasets also used during training, while Table 3 covers a limited set of benchmarks and baselines. Could the evaluation be extended to domains such as coding and factual recall, with additional competitive routing baselines? -
Are the data splits and baseline implementations fair?
Please clarify the sampling, splitting, and deduplication procedures, along with each baseline’s implementation, tuning, and budget settings. Several complex methods substantially underperform simple baselines such as Router-KNN. Can the authors explain why? -
Can the authors report the usage distribution of the six Workers?
Please provide each model’s share of calls across tasks and execution modes, as well as the number of distinct models used per trajectory, to clarify whether the system primarily relies on a small subset of Workers. -
Why is agentic execution selected for 99% of SATBench questions but only 12% of BBH questions?
Please explain this difference and provide complete successful and unsuccessful trajectories, together with a comparison of execution modes on SATBench. -
Does adaptive mode selection actually outperform fixed execution modes?
Could the authors train fixed single-round, multi-round, and agentic baselines using the same model pool, training data, and comparable budgets, and compare them with the adaptive policy to distinguish the benefits of mode selection from those of additional computation?
4 comments
Q3: Worker usage across tasks and execution modes
Appendix D (Table 9) reports Worker-call shares by task and execution mode. Across 1,060 evaluation questions, RouteWeaver makes 2,372 calls (2.24 per question). RouteWeaver uses an average of 1.58 distinct Workers per trajectory, with repeated calls to the same Worker counted once. W3 receives 62.39% of calls; W4, W5, and W6 receive 14.33%, 10.88%, and 11.21%, respectively, accounting for 36.42% together.
W3 accounts for 47.89% of Code calls and 76.52% of Reasoning calls; its share is 86.36%, 41.11%, and 71.68% in single-round, multi-round, and agentic execution, respectively. W4--W6 jointly receive 55.89% of multi-round calls. These shares include repeated calls to the same Worker.
Q4: SATBench versus BBH mode preferences and execution cases
RouteWeaver selects agentic execution for 99% of SATBench questions versus 12% on BBH. SATBench's interacting logical constraints provide explicit checks for verification and revision. BBH spans heterogeneous tasks, where the value of additional decomposition and verification varies by task.
We compare execution modes on SATBench using the same RouteWeaver checkpoint, clamping its mode at evaluation time without retraining:
| Mode | SATBench accuracy (%) |
|---|---|
| Clamped-single | 69.2 |
| Clamped-multi | 69.2 |
| Clamped-agentic | 73.3 |
| Adaptive RouteWeaver | 76.7 |
Clamped-agentic exceeds both clamped-single and clamped-multi by 4.1 percentage points, supporting the usefulness of agentic execution on SATBench.
Successful case (Appendix A.2.1). In satbench_ev_0111, the first executor
concludes UNSAT after the planner incorrectly fixes a variable. The verifier
identifies the exact error, and the router requests re-execution. Selected
excerpts are shown below; ellipses omit intervening text.
Executor (n2):
<answer>UNSAT</answer>
Verifier (n3):
FAIL
Location: Step 4, Sub-task D, Critical Observation
...
There is no direct contradiction here. The statement "if X(A,1,2) is false,
then Condition 24 requires the other three to be false" is incorrect.
Router -> executor (refs="n1,n2,n3"):
Re-evaluate the core logical constraints without assuming X(A,1,2) = true.
...
Final router output:
<answer>SAT</answer>
An independent substitution check confirms that a satisfying assignment exists for all 25 original constraints. Here, a specific verifier finding triggers corrective execution and a correct final answer.
Failed case (Appendix A.2.2). In satbench_ev_0067, original condition 6
is J1 OR NOT L2, but the planner adds a negation to J1. The later stages
accept an assignment for this altered representation:
Planner (n1), translated condition 6:
6. $ \neg J_1 \vee \neg L_2 $
...
Executor (n2):
<answer>SAT</answer>
Verifier (n3):
PASS
Final router output:
<answer>SAT</answer>
The accepted assignment J1=false, L1=true, L2=true violates original
condition 6. The original problem is UNSAT, as established in Appendix A.2.2.
This case shows how a shared representation error can survive verification.
Together, the cases illustrate both the corrective value of verification
and the need to check intermediate reasoning against the original question.
Appendix A.2 provides the complete call sequence and dependencies for both
trajectories, with observations summarized for readability.
Q5: Adaptive versus fixed execution modes
We train fixed-single, fixed-multi, and fixed-agentic policies using the same Qwen3-4B router architecture, base initialization, training queries, six-Worker pool, and training budget as RouteWeaver. Each policy is restricted to its designated mode from the start of training through evaluation, while learning Worker selection and within-mode execution.
Worker-call limits are 1/4/8, respectively, matching RouteWeaver's corresponding mode limits. Fixed-agentic therefore shares RouteWeaver's maximum allowance of eight calls.
Section 5.4 (Table 3) reports 61.4% average accuracy for RouteWeaver versus 56.4%, 57.1%, and 58.6% for fixed-single, fixed-multi, and fixed-agentic. RouteWeaver exceeds the strongest fixed-mode policy by 2.8 percentage points and outperforms all three in every domain, supporting the benefit of adaptive mode selection under matched training settings and mode-specific call limits.
I thank the authors for the additional experiments and clarifications. The expanded cross-benchmark evaluation, the ablation controlling for segment normalization, the Worker utilization statistics, and the newly trained fixed-mode policies have addressed several of my concerns. However, a few issues remain insufficiently clarified. First, the code evaluation includes intermediate Worker outputs in the candidate set, potentially giving different execution modes different numbers of opportunities to produce a correct answer. This may affect the fairness of the comparison. Second, the authors have added an experiment that replaces three models and reassigns Worker IDs without retraining, but it does not cover adding or removing models, or changing prices independently. Moreover, the improvement from 61.4% to 62.8% does not isolate the contribution of routing adaptability from that of stronger Worker models. Although the revision strengthens the paper’s empirical support, I have decided to keep my original score unchanged.
Thank you for the detailed assessment of the revised manuscript.
First, on the Code candidate set. We rescore the retained trajectories by each system's final submitted answer only, applying the same rule to three representative baselines (for RouteWeaver, the router's final output):
| Method | Original | Final-answer-only |
|---|---|---|
| GraphRouter | 36.1 | 36.1 |
| Router-R1 | 52.8 | 48.4 |
| GraphPlanner | 48.3 | 47.0 |
| RouteWeaver | 58.2 | 54.0 |
RouteWeaver is still the strongest, 5.6 points above Router-R1 and 7.0 above GraphPlanner, and the five-domain average is 60.6 against 61.4 under the original protocol. The ranking does not change. We make the same clarification in our reply to Reviewer unsC.
Second, on the modified Worker pool.
(i) Prices. The router never sees prices: they are used only to compute the reported cost and, when α > 0, in the efficiency reward, so a price change moves the cost figure but not the routing decisions.
(ii) Unseen model. We add Qwen3.5-35B-A3B as a seventh Worker. The policy was trained without it and is not retrained:
| Pool | Math | Code | Know. | Reason. | Recall | Avg. |
|---|---|---|---|---|---|---|
| Original (6 Workers) | 74.6 | 58.2 | 55.0 | 76.7 | 42.5 | 61.4 |
| Qwen3.5-35B-A3B added (7 Workers) | 75.4 | 62.8 | 57.5 | 75.8 | 42.5 | 62.8 |
It selects the new Worker in 8.2% of calls, guided only by its description, and the average rises from 61.4 to 62.8, showing that the policy generalizes to unseen Workers. The gain is concentrated in Code (+4.6), Math (+0.8) and Knowledge (+2.5). The other two domains fluctuate slightly: Recall is unchanged at 42.5 and Reasoning is one question lower (76.7 to 75.8).
(iii) Adaptability. The router is frozen, so a stronger model can affect accuracy only if the router selects it. The gain therefore cannot come from the new models alone: they enter the system only through the router's calls. The policy makes those calls on a pool it has never seen. That is the adaptability we claim.
We thank the reviewer again for the careful reading of the revision and for the time given to this discussion.
We sincerely thank you for your careful review and constructive feedback. Your comments have helped us strengthen the evaluation and clarify the presentation. We address your concerns below. Changes in the revised PDF are highlighted in blue.
W1: Accounting for router computation
Thank you for pointing out the router overhead. We include both Worker costs and the token costs of LLM-based routing components. For RouteWeaver, router accounting includes input and output tokens across all interaction turns, including repeated processing of interaction histories, using the accounting rule and prices in Appendix E (Table 10).
Costs are totals over the same 1,060 evaluation questions and are rounded. GraphRouter and MasRouter use non-generative routing controllers and incur no additional LLM-router token charge; their LLM execution costs are included in the Worker column. Across 1,060 questions, RouteWeaver's router accounts for approximately 8.6% of its total cost and adds about 9.4% to its Worker-only cost. The multi-alpha cost curve reports Worker-only costs; the router-inclusive total here is for the main evaluated model.
W2 / Q1: Generalization across benchmarks and Worker pools
Thank you for suggesting broader generalization tests. We extend the held-out evaluation to coding (BigCodeBench, BCB) and factual recall (NQ Open), and compare against GraphRouter, Router-R1, and MasRouter alongside the existing AIME, GPQA-D, and BBH evaluations (Section 5.5, Table 4).
RouteWeaver achieves 46.6% average accuracy and leads on all five benchmarks, including 43.3% on BCB and 28.3% on NQ Open. Evaluation sample sizes are given in Appendix B.
Section 5.5 (Table 5) evaluates a modified six-Worker pool with reassigned Worker IDs and three model replacements; the replacement models are specified in that section. We reuse the original RouteWeaver checkpoint without retraining or fine-tuning, updating only the Worker descriptions, prices, and call mappings.
RouteWeaver achieves 62.8% average accuracy with the modified pool, exceeding Router-R1 by 6.9 percentage points and its own original-pool result by 1.4 percentage points. These results demonstrate transfer to the modified Worker pool without retraining RouteWeaver.
W3: Advantage decomposition versus segment normalization
The matched control in Section 5.4 (Table 2) uses ordinary GRPO advantages for both mode and execution tokens while retaining COMET-GRPO's token masks, separate segment normalization, and segment weights. All other training settings are held fixed. COMET-GRPO achieves 61.4% average accuracy versus 57.0% for this control, a 4.4-percentage-point gain with improvements in all five domains. This comparison supports the contribution of advantage decomposition beyond separate segment normalization.
Q2: Data separation and baseline implementation
We randomly sample questions without replacement within each dataset according to the quotas in Appendix B and exclude training--test overlap using question IDs. Trainable and retrieval-based routers use the same 2,880 training queries, and all methods are evaluated on the same 1,060 test questions with the same six frozen Workers.
Starting from each baseline's original settings, we adapt its implementation to our data, Worker pool, and execution limits, and tune method-specific hyperparameters to select the best-performing configuration among those evaluated under this shared setup. Router-KNN, Router-MLP, RouterDC, GraphRouter, and Router-KNN-MR use UIUC LLMRouter; Router-R1, R2-Reasoner, MasRouter, and GraphPlanner use their official implementations. Prompt LLM follows Router-R1's prompting baseline. All generative routing components use Qwen3-4B-Instruct-2507. Encoders use released pretrained checkpoints, while added heads and controllers are randomly initialized; no baseline resumes from an intermediate checkpoint. Appendix G details initialization, training budgets, and maximum Worker-call limits (Table 11).
Table 1 shows task-dependent gains: GraphPlanner exceeds Router-KNN on Code (48.3% vs. 34.8%), and MasRouter exceeds it on Code and Knowledge. Router-R1 also achieves higher average accuracy (55.3% vs. 48.7%); the lower averages of some baselines reflect losses in other domains.