- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SEER, a framework that combines structural-entropy-driven candidate expert selection with capability-oriented retrieval of historical cases. It uses GRPO to train two separate policies—a retrieval planner and a routing selector—to select a single expert before answer generation. The framework has a clear division of responsibilities and achieves accuracy gains across multiple benchmarks. However, the current evidence does not sufficiently establish whether these gains justify the additional training and inference costs of the two language-model-based policies. Comparisons with strong contemporary models and routing methods under matched computational budgets, as well as the consistency of key ablation results, also require further investigation.
Critical strengths
-
SEER has a clear division of responsibilities: it first narrows the candidate expert pool based on historical performance and then selects an expert using relevant historical cases. This design makes the routing process more interpretable.
-
The experiments cover a broad range of settings. On ID evaluations with the open-source expert pool, SEER improves accuracy by 2.89 percentage points over the strongest reported baseline. The paper also includes multi-task evaluations, OOD evaluations, and component ablations.
Critical weaknesses
-
The paper does not sufficiently establish whether the gains justify the additional computational costs or whether the comparisons are fair. SEER introduces two independently trained 4B-parameter policies and requires two additional generation steps during inference. However, it is primarily compared against non-generative routing methods. A comprehensive cost–performance evaluation against strong contemporary single-model baselines and LLM-based routing methods under matched computational budgets is needed.
-
The necessity of the core components and their individual contributions remain insufficiently established.The results for “Formula-only K=1” and "w/o Language for Choice” have not yet been aligned, which affects the assessment of their contributions to the language phase.
Questions for authors
-
What output rules and expert-pool configurations are used for “Formula-only K=1” in Figure 4(b) and “w/o Language for Choice” in Table 3? The reported results appear inconsistent; could you clarify the differences between these settings and explain the discrepancy?
-
Given the reported gains over the best fixed expert, does SEER retain a favorable accuracy–cost trade-off when all routing and expert-generation costs are included under matched inference budgets?
3 comments
Response to Reviewer DjU5 [part 2/2: W1 & Q2]
Benefits and Costs
W1: Two separately trained 4B policies require two additional generation steps, yet comparisons are primarily against non-generative routing methods. Cost–performance evaluations against strong single models and LLM-based routing methods under the same computational budget are missing.
Q2: After accounting for all routing and expert-generation costs, does SEER still offer a favorable accuracy–cost trade-off relative to the best fixed expert?
Answer
2a · Additional generative routing baselines and end-to-end costs.
Thank you for your suggestions regarding generative routing baselines and complete inference costs. We added Arch-Router 1.5B and RouteLLM Causal-LLM 8B and report their generation performance. All cost measurements use a fixed set of 100 ID queries (seed 42), excluding warm-up and model-loading time. End-to-end latency sums "routing decision + selected expert generation" for each query, including both prefill and decoding. Accuracy for the existing methods is taken from Tables 1–2. The two added methods are evaluated on 3,543 ID and 4,479 OOD queries using historical expert-answer correctness, with equal-weight macro averaging across benchmarks. The measurements show encouraging results for SEER in both inference time and performance.
| Method | ID / OOD Macro (%)† ↑ | Routing (s) ↓ | Expert generation (s) ↓ | End-to-end (s) ↓ | Serial QPS ↑ | Final output tokens ↓ | Decoding FLOPs (TF) ↓ |
|---|---|---|---|---|---|---|---|
| Direct 7B (OpenReasoning-Nemotron-7B) | 57.37/63.71 | 0.000 | 3.015 | 3.015 | 0.332 | 512.0 | 7.17 |
| Direct 8B (II-Medical-8B) | 66.69/74.04 | 0.000 | 3.179 | 3.179 | 0.315 | 497.7 | 7.96 |
| Direct 27B (Qwen3.8-27B) | 74.18/81.14 | 0.000 | 7.576 | 7.576 | 0.132 | 383.6 | 20.72 |
| kNN Router | 62.49/68.17 | 0.090 | 7.489 | 7.579 | 0.132 | 385.0 | 20.45 |
| MLP Router | 63.75/69.15 | 0.038 | 7.030 | 7.068 | 0.141 | 384.6 | 19.12 |
| RouteLLM MF | 76.53/86.59 | 0.038 | 7.231 | 7.269 | 0.138 | 383.6 | 19.73 |
| Arch-Router 1.5B (new) | 61.09/65.37 | 0.055 | 4.235 | 4.290 | 0.233 | 455.59 | 10.27 |
| RouteLLM Causal-LLM 8B (new) | 74.21/75.78 | 0.056 | 4.829 | 4.885 | 0.205 | 489.25 | 12.50 |
| SEER | 79.70/87.03 | 1.946 | 5.972 | 7.918 | 0.126 | 406.7 | 19.61 |
↑ indicates higher is better, and ↓ indicates lower is better; ↓ for output tokens refers only to generation overhead. Final output tokens count only expert answers. Decoding FLOPs are computed by summing over calls and then averaging across queries (TF/query). For generative routers, this includes decoding by both the router and the expert, but excludes prefill, encoder computation, and non-generative routing computation. Serial QPS is the reciprocal of mean end-to-end latency.
2b · SEER's design objective and cost trade-off.
Thank you for your insightful attention to cost and for your suggestion. We would like to clarify SEER's design objective. In the Individual Experts Pool, it aims to route local experts to achieve performance comparable to commercial API models. In the inexpensive Commercial Models Pool (Flash), it aims to achieve performance comparable to top-tier commercial models (Pro) through routing. Under this accuracy-oriented objective, we consider the additional inference latency and cost of two 4B policies an acceptable trade-off. In the inference-time evaluation, SEER's end-to-end latency is 7.918 s, close to Direct 27B's 7.576 s, an increase of only 0.342 s. However, ID/OOD Macro accuracy improves from 74.18/81.14 to 79.70/87.03, a difference of 5.52/5.89 percentage points. We believe this is valuable for practical production deployment.
Dear reviewer:
We have observed that you have not commented on our reply; do you have any other questions about our response? If you have any questions, please feel free to contact us at any time, and we will respond to your concerns as soon as possible.
Thank you once again for your hard work and your contributions to our paper! Looking forward to your reply!
Response to Reviewer DjU5 [part 1/2: W2 & Q1]
Numerical Inconsistency Between Fig. 4(b) and Table 3
Answer
Thank you for pointing out this inconsistency in statistical aggregation. Table 3 uses macro accuracy, giving equal weight to each benchmark, whereas Fig. 4(b) uses micro coverage aggregated over all queries. To facilitate direct comparison, we have changed Fig. 4(b) to macro aggregation and updated the PDF. We have checked the entire manuscript to ensure that macro accuracy is used throughout. We are deeply grateful for your contribution to improving the rigor of our paper.