acceptodds
Under review as a conference paper at ICLR 2027

Structural Entropy Guided Capability-Aware Expert Routing

Abstract

Large language model services increasingly rely on heterogeneous pools of experts, but selecting the right expert requires query-level capability understanding while querying every model is prohibitively expensive. We introduce SEER (Structural Entropy Guided Capability-Aware Expert Routing), a pre-answer router that separates routing into structure for search and language for choice. Structure for search organizes train-only query–expert outcomes as a semantic graph, partitions the graph by minimizing two-dimensional structural entropy, and derives interpretable community-level expert priors. For each incoming query, soft community assignment and local memory correction export a compact Top- candidate set. Language for choice then uses a to analyze the query's required capabilities and generate capability-conditioned retrieval queries. Constrained retrieval provides historical questions with anonymous route outcomes, from which a chooses one candidate without inspecting current expert responses. The two policies are independently optimized with Group Relative Policy Optimization. SEER combines interpretable structural memory with evidence-grounded language decision making while invoking only the selected expert for the final answer. We evaluate SEER across 20 benchmarks with both open-source and closed-source expert pools.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Router, Structural Entropy

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DjU5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SEER, a framework that combines structural-entropy-driven candidate expert selection with capability-oriented retrieval of historical cases. It uses GRPO to train two separate policies—a retrieval planner and a routing selector—to select a single expert before answer generation. The framework has a clear division of responsibilities and achieves accuracy gains across multiple benchmarks. However, the current evidence does not sufficiently establish whether these gains justify the additional training and inference costs of the two language-model-based policies. Comparisons with strong contemporary models and routing methods under matched computational budgets, as well as the consistency of key ablation results, also require further investigation.

Critical strengths
  1. SEER has a clear division of responsibilities: it first narrows the candidate expert pool based on historical performance and then selects an expert using relevant historical cases. This design makes the routing process more interpretable.

  2. The experiments cover a broad range of settings. On ID evaluations with the open-source expert pool, SEER improves accuracy by 2.89 percentage points over the strongest reported baseline. The paper also includes multi-task evaluations, OOD evaluations, and component ablations.

Critical weaknesses
  1. The paper does not sufficiently establish whether the gains justify the additional computational costs or whether the comparisons are fair. SEER introduces two independently trained 4B-parameter policies and requires two additional generation steps during inference. However, it is primarily compared against non-generative routing methods. A comprehensive cost–performance evaluation against strong contemporary single-model baselines and LLM-based routing methods under matched computational budgets is needed.

  2. The necessity of the core components and their individual contributions remain insufficiently established.The results for “Formula-only K=1” and "w/o Language for Choice” have not yet been aligned, which affects the assessment of their contributions to the language phase.

Questions for authors
  1. What output rules and expert-pool configurations are used for “Formula-only K=1” in Figure 4(b) and “w/o Language for Choice” in Table 3? The reported results appear inconsistent; could you clarify the differences between these settings and explain the discrepancy?

  2. Given the reported gains over the best fixed expert, does SEER retain a favorable accuracy–cost trade-off when all routing and expert-generation costs are included under matched inference budgets?

3 comments
AuthorsOpenReview ↗

Response to Reviewer DjU5 [part 1/2: W2 & Q1]

Numerical Inconsistency Between Fig. 4(b) and Table 3

W2: The necessity and individual contributions of the core components have not been sufficiently demonstrated. "Formula-only K=1" and "w/o Language for Choice" yield inconsistent results, complicating assessment of the language stage's contribution.

Q1: What output rules and expert-pool configurations are used in Fig. 4(b) and Table 3? The results appear inconsistent; please explain the differences and their causes.


Answer

1a · Unifying the aggregation scheme.

Thank you for pointing out this inconsistency in statistical aggregation. Table 3 uses macro accuracy, giving equal weight to each benchmark, whereas Fig. 4(b) uses micro coverage aggregated over all queries. To facilitate direct comparison, we have changed Fig. 4(b) to macro aggregation and updated the PDF. We have checked the entire manuscript to ensure that macro accuracy is used throughout. We are deeply grateful for your contribution to improving the rigor of our paper.


AuthorsOpenReview ↗

Response to Reviewer DjU5 [part 2/2: W1 & Q2]

Benefits and Costs

W1: Two separately trained 4B policies require two additional generation steps, yet comparisons are primarily against non-generative routing methods. Cost–performance evaluations against strong single models and LLM-based routing methods under the same computational budget are missing.

Q2: After accounting for all routing and expert-generation costs, does SEER still offer a favorable accuracy–cost trade-off relative to the best fixed expert?


Answer

2a · Additional generative routing baselines and end-to-end costs.

Thank you for your suggestions regarding generative routing baselines and complete inference costs. We added Arch-Router 1.5B and RouteLLM Causal-LLM 8B and report their generation performance. All cost measurements use a fixed set of 100 ID queries (seed 42), excluding warm-up and model-loading time. End-to-end latency sums "routing decision + selected expert generation" for each query, including both prefill and decoding. Accuracy for the existing methods is taken from Tables 1–2. The two added methods are evaluated on 3,543 ID and 4,479 OOD queries using historical expert-answer correctness, with equal-weight macro averaging across benchmarks. The measurements show encouraging results for SEER in both inference time and performance.

MethodID / OOD Macro (%)† ↑Routing (s) ↓Expert generation (s) ↓End-to-end (s) ↓Serial QPS ↑Final output tokens ↓Decoding FLOPs (TF) ↓
Direct 7B (OpenReasoning-Nemotron-7B)57.37/63.710.0003.0153.0150.332512.07.17
Direct 8B (II-Medical-8B)66.69/74.040.0003.1793.1790.315497.77.96
Direct 27B (Qwen3.8-27B)74.18/81.140.0007.5767.5760.132383.620.72
kNN Router62.49/68.170.0907.4897.5790.132385.020.45
MLP Router63.75/69.150.0387.0307.0680.141384.619.12
RouteLLM MF76.53/86.590.0387.2317.2690.138383.619.73
Arch-Router 1.5B (new)61.09/65.370.0554.2354.2900.233455.5910.27
RouteLLM Causal-LLM 8B (new)74.21/75.780.0564.8294.8850.205489.2512.50
SEER79.70/87.031.9465.9727.9180.126406.719.61

↑ indicates higher is better, and ↓ indicates lower is better; ↓ for output tokens refers only to generation overhead. Final output tokens count only expert answers. Decoding FLOPs are computed by summing 2PTout2P T_{\mathrm{out}} over calls and then averaging across queries (TF/query). For generative routers, this includes decoding by both the router and the expert, but excludes prefill, encoder computation, and non-generative routing computation. Serial QPS is the reciprocal of mean end-to-end latency.


2b · SEER's design objective and cost trade-off.

Thank you for your insightful attention to cost and for your suggestion. We would like to clarify SEER's design objective. In the Individual Experts Pool, it aims to route local experts to achieve performance comparable to commercial API models. In the inexpensive Commercial Models Pool (Flash), it aims to achieve performance comparable to top-tier commercial models (Pro) through routing. Under this accuracy-oriented objective, we consider the additional inference latency and cost of two 4B policies an acceptable trade-off. In the inference-time evaluation, SEER's end-to-end latency is 7.918 s, close to Direct 27B's 7.576 s, an increase of only 0.342 s. However, ID/OOD Macro accuracy improves from 74.18/81.14 to 79.70/87.03, a difference of 5.52/5.89 percentage points. We believe this is valuable for practical production deployment.


Dear reviewer:

We have observed that you have not commented on our reply; do you have any other questions about our response? If you have any questions, please feel free to contact us at any time, and we will respond to your concerns as soon as possible.

Thank you once again for your hard work and your contributions to our paper! Looking forward to your reply!

Reviewer_3uRxOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

SEER is a two-stage router that picks one expert LLM per query before answering. Stage 1 partitions an offline query graph with 2D structural entropy. It then uses kNN with Bayesian smoothing to produce Top-3 candidates and a primary recommendation (the anchor). Stage 2 runs two GRPO-trained 4B models in sequence (a Planner, then a Selector) over retrieved historical examples. An ESS gate decides when Stage 2 may override the anchor.

Critical strengths
  1. The split is well motivated. Graph structure prunes candidates, and the LLM makes the fine-grained choice. This handles two problems: global averages that hide local expertise, and the high variance of pure retrieval.
  2. The ESS gate and the penalty for overriding a correct anchor (R1R_1) keep Stage 2 from over-correcting when evidence is sparse or noisy.
  3. The evaluation covers 15 datasets and several model pools, with detailed ablations.
Critical weaknesses
  1. Stage 2 is expensive, and its cost is not compared with baselines. It uses about 4.3k tokens and adds 1.955 s per query, close to the cost of calling a 7B/8B expert directly. The paper reports no latency, throughput, or FLOPs for lightweight routers (e.g., millisecond-level kNN) or for calling the best single model. Without these numbers, the trade-off between accuracy and cost cannot be judged.

  2. The setup may favor SEER. The 11-model pool was pre-screened from 25 models using SEER’s own scoring rule, and about 72% of the GRPO training distribution was modified. Please state whether the baselines used the same pool and data, and report results on the full pool with the original distribution. Comparisons with UniRoute and HybridFlow, and results on LLMRouterBench, would also help.

  3. There is little evidence that the main components are needed. Replacing structural entropy with K-Means lowers accuracy by only 0.76 (ID) and 0.54 (OOD) points, with no variance or significance test reported. Routing is also concentrated: 60.4% (ID) and 72.8% (OOD) of queries go to Qwen3.8-27B, so the paper should report the gain over always calling that model. In the commercial OOD setting, SEER (91.44%) scores slightly below GLM-5.3-Flash alone (91.50%).

  4. The scope is narrow. All tasks are single-turn QA with binary rewards. The query graph is static, and the Top-3 cutoff caps oracle accuracy at 86.5%. The paper does not address newly added models, distribution drift, or multi-turn and agentic tasks.

Questions for authors
  1. Counting Stage 2, how does SEER compare with lightweight routers and with direct 7B/8B/27B calls in latency, throughput, and FLOPs per query? An accuracy-vs-cost Pareto plot would answer this.
  2. Given the small gap to K-Means and the GLM-5.3-Flash result, what justifies structural entropy plus two RL modules? Would a single 4B selector do as well?
  3. How would SEER handle new models, shifting query distributions, and tasks without binary rewards?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 3uRx [part 1/4: W1 & Q1]

Inference Latency and Computational Overhead

W1: The second stage uses approximately 4.3k tokens and adds 1.955 s per query, close to directly calling a 7B/8B expert. Comparisons with lightweight routers or the best single model in latency, throughput, and FLOPs are not reported.

Q1: Including the second stage, how does SEER compare with lightweight routers and direct 7B/8B/27B calls in latency, throughput, and FLOPs? An accuracy–cost Pareto plot is recommended.


Answer

1a · The 4.3k-token figure does not represent 4.3k output tokens.

Thank you for pointing out this potentially misleading wording. The 4.3k figure is the sum of input and output tokens for the Planner and Selector, not the autoregressive generation length. In our measurements, the Planner generates an average of 196.2 output tokens, and the Selector generates an average of 230.3 output tokens. The two stages therefore produce 426.5 output tokens/query in total. To address your concern, we measured the costs (see 1b and Appendix B.4 of the manuscript).


1b · End-to-end inference costs under matched conditions.

We added matched measurements on a fixed set of 100 ID queries (seed 42), excluding warm-up and model-loading time. For routing methods, we sum "routing decision + selected expert generation" for each query. Accuracy is taken from the full-benchmark results in Tables 1–2.

MethodID / OOD Macro (%)† ↑Routing (s) ↓Expert generation (s) ↓End-to-end (s) ↓Serial QPS ↑Final output tokens ↓Decoding FLOPs (TF) ↓
Direct 7B (OpenReasoning-Nemotron-7B)57.37/63.710.0003.0153.0150.332512.07.17
Direct 8B (II-Medical-8B)66.69/74.040.0003.1793.1790.315497.77.96
Direct 27B (Qwen3.8-27B)74.18/81.140.0007.5767.5760.132383.620.72
kNN Router62.49/68.170.0907.4897.5790.132385.020.45
MLP Router63.75/69.150.0387.0307.0680.141384.619.12
RouteLLM MF76.53/86.590.0387.2317.2690.138383.619.73
SEER79.70/87.031.9465.9727.9180.126406.719.61

↑ indicates higher is better, and ↓ indicates lower is better; ↓ for output tokens refers only to generation overhead. Serial QPS is the reciprocal of mean end-to-end latency. Decoding FLOPs are computed by summing 2PTout2P T_{\mathrm{out}} over calls for each query and then averaging across queries (TF/query, where PP is the active parameter count).


1c · SEER's design objective and cost trade-off.

Lightweight routers do have lower decision latency, and we do not position SEER as the lowest-latency router. However, end-to-end latency also depends on the selected expert. SEER's expert generation averages 5.972 s, compared with 7.030–7.489 s for lightweight routers, a reduction of 1.06–1.52 s. Its end-to-end latency is therefore only 0.34–0.85 s higher. Compared with directly calling 27B, SEER adds only 0.342 s (4.5%). Meanwhile, Tables 1–2 show an improvement in ID/OOD Macro accuracy from 74.18/81.14 to 79.70/87.03.

We would like to clarify SEER's design objective. In the Individual Experts Pool, it aims to route local experts to achieve performance comparable to commercial API models. In the Commercial Models Pool, it aims to achieve performance comparable to top-tier commercial models through routing. In the inference-time evaluation, SEER's end-to-end latency is 7.918 s, close to Direct 27B's 7.576 s, an increase of only 0.342 s. However, ID/OOD Macro accuracy improves from 74.18/81.14 to 79.70/87.03, a difference of 5.52/5.89 percentage points. We believe this is valuable for practical production deployment.


AuthorsOpenReview ↗

Response to Reviewer 3uRx [part 2/4: W2]

Model-Pool Fairness, the Original Distribution, and Cross-Pool Generalization

W2: The 11-model pool is preselected from 25 models using SEER's scoring rule, and approximately 72% of GRPO training samples are modified. Please clarify whether the baselines use the same model pool and data, and report results for the full model pool and original distribution. Comparisons with UniRoute and HybridFlow, as well as results on LLMRouterBench, would also be helpful.


Answer

2a · All methods share the same model pool and data

The 11-model pool is constructed and frozen before routing-policy training. SEER and all baselines share 8,287 training queries, 3,543 ID queries, and 4,479 OOD queries. They also share the same query representations, frozen expert-answer matrix, and 11-model action space. Thus, the comparisons in the main tables follow exactly the same training and evaluation protocol.


2b · The 25→11 selection does not use test information

This process uses only the training set. An expert is included in the frozen pool if it appears among the structural Top-3 candidates for any training query. Neither ID/OOD labels nor test accuracy are involved in model selection. The aim is to preserve complementary capabilities across queries, rather than select the globally strongest 11 models. The Avengers uses a similar setting, selecting from 22 candidate models to construct a 10-model pool. Thus, this is not a design intended to favor SEER, but a common model-pool construction practice in this field, independent of the routing method.


2c · Control under the original candidate distribution

Thank you for this insightful comment. In our initial training, we did use natural Top-3 candidates for RL. However, when all three candidate outcomes are identical (111111 or 000000), rewards become sparse and GRPO cannot obtain a relative signal. We therefore keep R1R_1 unchanged and use the current training query's correctness labels only to replace one of R2/R3R_2/R_3, making candidate outcomes distinguishable (Appendix A.3). Regarding the original distribution, we apologize for previously omitting this control and thank you for pointing it out. We have added a control using natural Top-3 candidates throughout, retaining all 111/000111/000 samples without replacing candidates:

Training candidatesIndividual Experts Pool ID / OOD (%)Commercial Models Pool ID / OOD (%)
SEER79.70/87.0383.05/91.44
Natural Top-3 RL (original distribution)77.34/81.9382.67/90.98

On the Individual Experts Pool, Natural Top-3 RL achieves 77.34%/81.93%. On the Commercial Models Pool, its cross-pool transfer results reach 82.67%/90.98%, but both are below SEER. These results show that the natural candidate distribution can still yield an effective policy capable of cross-pool transfer. Candidate repair further improves performance by alleviating reward sparsity in 111/000111/000 samples.


2d · Cross-model-pool generalization

We directly transferred the frozen two-stage policies to LLMRouterBench without task-specific fine-tuning. This benchmark contains 20 experts:

MethodAvg. (%)
SEER72.66
Avengers71.94
Model-SAT71.88
EmbedLLM71.24
Best single (Qwen3-8B)68.01

SEER achieves a Macro Avg. of 72.66%, exceeding Avengers by 0.72 percentage points and the best single model, Qwen3-8B, by 4.65 percentage points. This evaluation changes both the data and the expert pool, validating cross-pool generalization. Our manuscript's OOD evaluation (Table 2) and inclusion of API models (Tables 1–2) also provide cross-model and cross-dataset validation.


AuthorsOpenReview ↗

Response to Reviewer 3uRx [part 3/4: W3 & Q2]

Necessity of the Components

W3: Replacing structural entropy with K-Means reduces performance by only 0.76/0.54, without variance estimates or significance tests. Routing is highly concentrated (60.4%/72.8% goes to Qwen3.8-27B), so gains over "always calling it" should be reported. On commercial-pool OOD, SEER's 91.44% is slightly below GLM-5.3-Flash's 91.50%.

Q2: Given the small gaps against K-Means and GLM, what justifies structural entropy and two RL modules? Would a single 4B selector suffice?


Answer

3a · Concentrated routing reflects the effectiveness of the design

Assigning most queries to Qwen3.8-27B is intuitive. SEER sends most samples to Qwen3.8-27B, but assigns a query to a suitable expert when it predicts that Qwen3.8-27B cannot answer it. Figure 4 of the manuscript provides the relevant results:

ReferenceID (%)OOD (%)
Always call Qwen3.8-27B74.1881.14
SEER79.7087.03
Gain+5.52+5.89

Qwen3.8-27B receives 60.4% (ID)/72.8% (OOD) of the routed queries, yet SEER still achieves gains of +5.52/+5.89 at the same single-call cost. The reassigned 39.6%/27.2% of queries are precisely those that SEER predicts Qwen3.8-27B would answer incorrectly.


3b · Is a single 4B selector sufficient?

It is not sufficient; our manuscript has already demonstrated this clearly. In w/o Evidence in Table 3, we remove the Planner and use only a single 4B Selector. ID/OOD performance falls from 79.70/87.03 to 75.17/81.25, decreases of 4.53/5.78 percentage points, respectively. This is a substantial performance drop.

If you mean using the same 4B model for both the Planner and Selector, Prompt-only in Figure 3 uses one untrained 4B model. Its ID/OOD performance is 77.40/84.60. However, GRPO is difficult in this setting because the Planner and Selector require substantially different capabilities. Designing a unified joint reward for GRPO training is therefore challenging. Training the same model sequentially for the two stages also makes it prone to catastrophic forgetting.

Figure 3 also reports the contribution of each stage. Training only the Route Selector yields 78.00/85.20, while training only the Capability Planner yields 78.30/85.80. Sequentially training the Planner and Selector further improves performance to 79.70/87.03. These results show the individual contributions of the two stages and their complementarity.


3c · Comparison with GLM-5.3-Flash on commercial-pool OOD

Thank you for this careful observation. GLM-5.3-Flash does exceed SEER's OOD average by 0.06 percentage points. However, GLM-5.3-Flash's main advantage comes from BrainTeaser, where it exceeds SEER by 5.3 percentage points; SEER leads on MathBench, StudentEval, and DailyDialog. We can therefore conclude that GLM-5.3-Flash performs particularly well on BrainTeaser, but this does not demonstrate inadequate performance by SEER. The extensive experiments in our manuscript have already demonstrated SEER's encouraging performance.


3d · The gap between structural entropy and K-Means

Thank you for raising this concern. The K-Means control in Table 3 directly uses the number of communities automatically determined by SE, Kc=∣PSE∣K_c=\lvert\mathcal{P}_{\mathrm{SE}}\rvert. This matched-KK setting favors K-Means because it avoids having to reselect KcK_c when the data or expert pool changes. SE, by contrast, determines the number of communities adaptively. To examine this factor, we further tested three fixed values of KcK_c:

MethodKcK_cID / OOD Macro (%)Δ\Delta vs. SEER (ID / OOD)
SEER (SE)Adaptive79.70/87.03—
K-Means (Table 3)matched-KK78.94/86.49−0.76/−0.54
K-Means3077.48/79.68−2.22/−7.35
K-Means6477.25/82.63−2.45/−4.40
K-Means20077.05/81.94−2.65/−5.09

With fixed KcK_c, the gap between SEER and K-Means widens further, illustrating the value of SE's adaptive determination of the community count. Moreover, SE not only partitions communities but also generates Top-3 candidates and restricts the Planner's evidence retrieval to candidate communities relevant to the current query. Removing SE Restriction reduces ID/OOD performance by 1.76/1.85pp, respectively. Thus, "historical-case retrieval + LLM selection" in the language stage is not a source of gains independent of the structural layer. It depends on the candidate and evidence spaces provided by SE.


Reviewer_MwmtOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to select the most suitable language model from a heterogeneous pool of experts without calling every model for each incoming query. The authors propose SEER, which separates the routing process into two stages. In the first stage, historical queries are organized into a semantic graph, which is partitioned using two-dimensional structural entropy. The resulting communities provide expert performance profiles, and the system combines soft community assignment with local memory correction to produce a set of three candidate experts for each new query. In the second stage, a Capability Planner analyzes the abilities required by the current query and generates capability-conditioned retrieval queries. The system then retrieves relevant historical questions and their anonymized routing outcomes. A Route Selector uses this evidence to choose one of the three candidates. The two language policies are trained independently using Group Relative Policy Optimization (GRPO), and only the selected expert is called to generate the final answer. The paper evaluates SEER on 15 in-distribution (ID) benchmarks and five out-of-distribution (OOD) benchmarks. The reported average accuracy is 79.70% on ID and 87.03% on OOD for the open-source expert pool, and 83.05% on ID and 91.44% on OOD for the commercial API pool. The paper also includes component ablations, parameter sensitivity analyses, candidate coverage evaluations, and routing latency measurements. Overall, the main idea is to combine structured organization of historical routing experience with evidence-based language-model decisions. The goal is to improve routing accuracy while retaining a single expert call for final answer generation.

Critical strengths

1.The problem is clearly defined, and the overall design is practically meaningful. Calling every expert and comparing their answers at inference time can provide richer decision signals, but it also increases serving costs. SEER instead uses historical information to narrow the candidate set, makes a decision based on retrieved historical cases, and calls only the selected expert. This addresses a concrete problem in multi-model serving. The explicit relationship between structural communities and expert performance also provides a potentially more interpretable alternative to relying entirely on a black-box routing score. 2.The evaluation is broad and includes targeted ablations of the main components. The paper reports results across multiple tasks and examines components such as structural search, evidence retrieval, candidate restriction, and training strategy. For example, removing historical evidence reduces the reported accuracy by 4.53 percentage points on ID and 5.78 percentage points on OOD. This suggests that historical evidence plays an important role in the complete system. The authors also investigate different candidate budgets, parameter settings, and OOD tasks, and report routing latency. This is informative.

Critical weaknesses

1.Human candidate repair during training may create a mismatch between training and actual inference. Appendix A.3 introduces a strategy called discriminative candidate construction: if all three candidate experts are correct on a training query (111), or all three are incorrect (000), the authors replace one of the challengers so that the current sample exhibits distinguishable correctness outcomes. Specifically, of the 5,748 repaired policy-training samples, 3,666 come from 111 repair, 471 from 000 repair, and only 1,611 are natural candidate samples. In other words, approximately 72% of the retained training samples have undergone human candidate repair. At the same time, the paper previously reports a total of 8,287 training queries, so the number of samples used for this part of policy training after repair is substantially reduced. I understand the authors' motivation for doing this: if the rewards of the three candidates are always the same, GRPO will struggle to obtain useful relative training signals from routing correctness. This is a real training difficulty. However, the current solution brings two problems. First, the replaced expert is selected based on the correctness labels of the current training query. For 111 samples, the authors deliberately introduce an expert that will answer the current question incorrectly; for 000 samples, they deliberately introduce an expert that will answer the current question correctly. Therefore, the candidate composition during training is not entirely determined by the system's normal Top-3 rule, but is partly affected by the correctness labels of the current query. Second, the test phase uses naturally generated candidate sets, whereas the policy-training phase largely uses repaired candidate sets. Even if the correctness labels of the current query do not directly appear in the policy prompts, this label-based way of constructing candidates still changes the candidate distribution during training, as well as the routing relationship corresponding to retrieved historical performance. I therefore wonder: can the selection patterns learned by the model during training reliably transfer to the natural Top-3 candidates actually encountered at deployment? The paper does not currently prove this sufficiently. I suggest that the authors at least add a key control: train and evaluate the route selector without repairing candidate sets, or adopt an alternative training approach that does not require artificially changing the identities of candidate experts. Without such verification, it is difficult to judge whether the current results reflect SEER's normal routing ability or depend to a large extent on this special training-sample construction.

2.The existing experiments do not yet sufficiently demonstrate the independent contribution of the method or how stable the reported gains are. The complete system in the paper achieves good accuracy, but there are still two issues that need to be addressed. First, the contribution of structural entropy itself does not appear to be particularly large. According to Table 3, replacing structural search with K-Means reduces ID and OOD accuracy by 0.76 and 0.54 percentage points, respectively; removing local memory correction reduces them by 0.79 and 0.64 percentage points, respectively. These results support the view that the structural modules have some value, but the gaps are relatively limited. By contrast, removing evidence retrieval leads to a larger drop in accuracy. So does SEER's main gain come from structural entropy communities, or mainly from the more general idea of "retrieving historical cases and letting a language model make a choice"? The current ablations are not yet sufficient to fully disentangle the contributions of the two. Second, the paper does not report error bars or statistical significance in the main results table, nor does it clearly state whether SEER's main results, like those of the baselines, were obtained through repeated experiments with multiple random seeds. This is especially important for the OOD experiments on the open-source model pool, where SEER's advantage over the strongest routing baseline is only 0.44 percentage points. Such a small gap requires more evidence before one can judge whether it is stable. I would like to see clearer evidence that the performance improvement not only exists but is sufficiently stable, and also a more precise explanation of its advantage over existing routing methods.

Questions for authors

1.Candidate repair and policy training. Why is it necessary to replace R2/R3 using the correctness labels of the current training query? Have the authors evaluated a version trained without candidate repair, or explored alternative ways of handling 111/000 samples? I would particularly like to see how the route selector performs when the training pipeline uses natural Top-3 candidates throughout. This experiment would help establish whether the current candidate-repair strategy is necessary for the reported performance.

Q2. Stability of the results. Are the main SEER results from a single training run or averaged across multiple random seeds? Please report repeated-run results, standard deviations, or confidence intervals. In particular, the 0.44-percentage-point improvement over the strongest routing baseline on open-source OOD benchmarks requires more evidence of stability. Please also clarify how each reported “Avg.” is computed, including whether it is an unweighted average across benchmarks or a sample-weighted average.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Mwmt [part 1/2: W1 & Q1]

Candidate Repair and Policy Training

W1: Repair makes approximately 72% of the retained training samples depend on the current query's correctness labels, whereas testing uses natural Top-3 candidates. Transferability has not been demonstrated.

Q1: Why must the current query's labels be used to replace R2/R3? Have you evaluated a version without repair? How does the Selector perform when natural Top-3 candidates are used throughout?


Answer

1a · Why repair is necessary.

Thank you for this insightful comment. In our initial training, we did use natural Top-3 candidates for RL (results in 1c). However, as you noted, when all three candidate outcomes are identical (111111 or 000000), rewards become sparse and GRPO cannot obtain a relative signal. To address this issue, we introduced a data-construction strategy that uses the current training query's correctness labels to replace R2/R3. Appendix A.3 describes this strategy, and the results show its positive effect.


1b · Anonymization facilitates cross-pool transfer.

Your concern about whether selection patterns learned during training reliably transfer to natural Top-3 candidate paths at deployment is important. We considered this issue when designing the Capability Planner and Route Selector, and therefore introduced candidate-expert anonymization. The Planner receives only the current query. The Selector receives the current query and retrieved historical evidence, with candidate experts represented by anonymous identifiers R1/R2/R3R_1/R_2/R_3. Neither policy receives specific expert identities or numerical structural scores. This design is intended to help the Planner and Selector learn transferable routing capabilities, rather than memorize the current model pool's logs and fixed selection preferences. The results in Tables 1–2 further validate this point. SEER's Planner and Selector are trained only on model logs from the Individual Experts Pool. When transferring the trained policies to the routing setting for the Commercial Models Pool, we did not retrain either the Planner or the Selector. SEER nevertheless achieved encouraging performance. This cross-pool result shows that anonymous routing supports policy transfer from the Individual Experts Pool to the Commercial Models Pool, which has a different expert composition.


1c · Natural Top-3 control.

We apologize for previously omitting these experimental results and thank you for pointing this out. We trained the routing Selector using natural Top-3 candidates throughout, retaining all 111/000111/000 rows without any candidate repair. The results are as follows:

MethodIndividual Experts Pool ID / OOD Macro (%)Commercial Models Pool ID / OOD Macro (%)
SEER79.70/87.0383.05/91.44
Natural Top-3 RL77.34/81.9382.67/90.98

On the Individual Experts Pool, training with natural Top-3 candidates achieves 77.34%/81.93%, below SEER. On the Commercial Models Pool, cross-pool transfer of Natural RL achieves 82.67%/90.98%, also below SEER. This pattern is consistent with the explanation that candidate repair helps alleviate reward sparsity in 111/000111/000 rows.


AuthorsOpenReview ↗

Response to Reviewer Mwmt [part 2/2: W2 & Q2]

Contributions of Structural Entropy, Number of Runs, and Avg.

W2: ① Structural entropy makes only a small contribution. Do the main gains come from "historical-case retrieval + LLM selection"? ② The main tables lack error bars and significance tests, and the OOD lead in the open-source pool is only 0.44pp.

Q2: Are the main results from a single run or averaged over multiple seeds? Please provide standard deviations or confidence intervals. How is "Avg." calculated?


Answer

2a · The role of structural entropy.

Thank you for raising this concern. The K-Means control in Table 3 directly uses the number of communities automatically determined by SE, Kc=∣PSE∣K_c=\lvert\mathcal{P}_{\mathrm{SE}}\rvert. This matched-KK setting favors K-Means: in practical deployment, its KcK_c must be specified manually and may vary with the data and expert pool. SE instead determines the number of communities automatically through coding cost. We further tested three fixed values of KcK_c:

MethodKcK_cID / OOD Macro (%)Δ\Delta vs. SEER (ID / OOD)
SEER (SE)Adaptive79.70/87.03—
K-Means (Table 3)matched-KK78.94/86.49−0.76/−0.54
K-Means3077.48/79.68−2.22/−7.35
K-Means [6]6477.25/82.63−2.45/−4.40
K-Means20077.05/81.94−2.65/−5.09

With fixed KcK_c, the gaps widen to 2.22–2.65pp on ID and 4.40–7.35pp on OOD. This shows that the matched-KK setting in Table 3 downplays the parameter-selection problem faced by K-Means. Moreover, the 0.76/0.54pp differences measure only the effect of replacing communities, not the full contribution of SE. SE also generates Top-3 candidates and restricts the Planner's retrieval to the current query's candidate communities. Removing SE Restriction reduces ID/OOD performance by 1.76/1.85pp. Thus, "historical-case retrieval + LLM selection" depends on the candidate and evidence spaces provided by SE, rather than being an independent source of gains. Unlike K-Means centroid distances, SE provides structural scores with information-theoretic interpretations through community coding costs, boundaries, merging entropy, and compression gains.


2b · End-to-end evaluation with a single fixed seed.

Thank you for your attention to statistical stability. The main results come from a single end-to-end training and evaluation run with one fixed seed, rather than an average over multiple seeds. Multiple seeds directly estimate training randomness, but fully repeating two-stage GRPO and large-scale commercial API evaluation is costly. We therefore prioritized broader evaluation coverage: 20 benchmarks (15 ID and 5 OOD), 8,022 held-out queries, and an 11-model Individual Experts Pool constructed from 25 candidate models. We also evaluated an independent 7-model Commercial API Pool. This breadth does not replace seed-variance estimation, but it tests consistency across tasks, distributions, and model pools. Similar high-cost multi-model studies [1–5] also do not report end-to-end means and variances across multiple training seeds. We will explicitly state the single-seed setting and its limitations in the main text, and kindly ask for your understanding of this cost constraint.


2c · How "Avg." is calculated.

"Avg." is a macro average: we first calculate accuracy on each benchmark and then average the benchmark accuracies with equal weights.


References

[1] Chen et al. FrugalGPT. 2023.
[2] Ong et al. RouteLLM. ICLR 2025.
[3] Jiang et al. LLM-Blender. ACL 2023.
[4] Zhang et al. Router-R1. 2025.
[5] Ding et al. BEST-Route. ICML 2025.
[6] Zhang et al. The Avengers. AAAI 2026.


Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns.

The Natural Top-3 experiments and cross-pool transfer results provide useful evidence for the effectiveness and generalization of the proposed approach. The additional K-Means comparisons and SE Restriction ablation also help clarify the contribution of structural entropy. I also appreciate the clarification regarding the evaluation setup and the calculation of average accuracy.

Overall, the rebuttal has helped address my concerns and improved my understanding of the paper. After considering the authors' response, I have decided to maintain my original rating.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank all three reviewers for the time devoted to reading and evaluating our paper and for their detailed, constructive feedback. We greatly appreciate your recognition of SEER's core value. Mwmt acknowledged the clear problem formulation and the practical value of using historical experience while calling only one expert. 3uRx recognized the motivation for separating structural pruning from language-based decision-making; DjU5 appreciated the interpretability of this division. All three reviewers also acknowledged the breadth of our multi-task evaluation and component ablations. Your comments helped us identify the design choices requiring further explanation and the experimental evidence that needed to be supplemented.

To address your main concerns about costs and benefits, training strategies, and component contributions, we have added targeted comparisons. The individual responses clarify the experimental settings and statistical procedures. Our main responses are summarized below.

1a · Additional end-to-end cost comparisons and generative routing baselines.

To address the concerns shared by 3uRx and DjU5, we added cost comparisons with lightweight routers and direct calls to 7B/8B/27B models. We separately report routing latency, expert generation latency, end-to-end latency, serial QPS, output tokens, and Decoding FLOPs. We also clarify that approximately 4.3k tokens include both inputs and outputs; the two stages actually generate 426.5 tokens on average. In the cost evaluation using a fixed set of 100 ID queries and a 512-token expert output limit, SEER's end-to-end latency is 7.918 s. This is close to Direct 27B's 7.576 s, with an additional latency of 0.342 s (4.5%). We also evaluated Arch-Router 1.5B and RouteLLM Causal-LLM 8B using their official weights, extending the comparisons to both non-generative and generative routing methods. Please see 3uRx part 1 and DjU5 part 2.


1b · A natural Top-3 control to explain the role of candidate repair.

To address Mwmt's and 3uRx's concerns about the training candidate distribution, we added a control trained entirely on natural Top-3 candidates. It retains all 111/000111/000 samples without replacing any candidates. We explain why GRPO lacks a relative reward signal when all candidate outcomes are identical, and how repair provides distinguishable training feedback. We further explain how candidate anonymization prevents the policy from relying directly on expert names. This control directly addresses whether candidate repair is necessary. Please see Mwmt part 1 and 3uRx part 2 for the settings and results.


1c · The value of structural entropy in parameter selection and component cooperation.

To address Mwmt's and 3uRx's questions about structural entropy, we added K-Means controls with Kc=30,64,200K_c=30,64,200. We clarify that the original K-Means control in Table 3 uses the number of communities automatically determined by SE. These additional controls distinguish the effects of fixed cluster counts from the role of adaptive structural-entropy partitioning. We further explain how structural entropy organizes both the candidate space and the evidence-retrieval space. We discuss component roles using SE Restriction, w/o Evidence, and training-strategy ablations. These additions explain how the structural and language layers cooperate through candidate pruning, evidence retrieval, and final selection. Please see Mwmt part 2 and 3uRx part 3.


1d · Clarifications of evaluation metrics, figure–table correspondence, and generalization settings.

We clarify that Avg. in the main tables gives equal weight to each benchmark, and provide the experimental run settings. Following DjU5's comment, we updated the values in Fig. 4(b) and the corresponding text to facilitate comparison with the relevant entries in Table 3. In response to 3uRx, we further explain model-pool construction, the candidate budget, and cross-pool adaptation. We also discuss the generalization capabilities examined by the OOD and cross-model-pool evaluations. Please see Mwmt part 2, 3uRx parts 2/4, and DjU5 part 1.

Thank you again for helping us improve this work. SEER aims to use structured historical experience and capability-relevant evidence to exploit complementary model strengths with a single expert call. We hope these additions clarify the rationale and practical costs of this design and provide a fuller basis for reassessing our contribution. We warmly welcome any further questions or suggestions.


AuthorsOpenReview ↗

Summary of PDF Revisions

We thank the reviewers for their helpful feedback. We have updated the manuscript PDF as follows:

  1. Tables 1 and 2: Renamed “Avg.” to “Macro Avg.” to clarify that benchmarks are equally weighted.
  2. Figure 4(b): Updated the values in Formula-only candidate coverage and synchronized the corresponding descriptions and analyses in the main text and Appendix B.1.
  3. Appendix B.4: Added end-to-end inference-cost comparisons covering SEER, lightweight routers, Arch-Router, RouteLLM Causal-LLM, and Direct 7B/8B/27B. Specified the directly called models, input/output token accounting, and the calculation of Decoding FLOPs.
  4. Appendix C: Clarified the shared model-pool and data protocol for the adapted baselines in the main tables, and documented the official weights and configurations of the generative routers. Clarified that K-Means in Table 3 uses the community count determined by SEER.
  5. Appendix E (More Experiments): Added the Natural Top-3 RL control (E.1), K-Means experiments with fixed Kc=30,64,200K_c=30,64,200 (E.2), and additional LLMRouterBench evaluation (E.3), together with analyses of the results.

To keep the main text concise, we placed the additional experiments and detailed analyses primarily in the appendices. We thank the reviewers again for helping us improve the paper.


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.