- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
E2-Filter aims to address the problem of dynamic model admission, where a model is added to a router with a small support set. E2-Filter will then determine whether this updated router is worth deploying over the old router.
Critical strengths
- The layer of deciding whether a router is deployable seems relatively novel and worth studying.
Critical weaknesses
-
I am not exactly convinced by the problem setup, where the updated router is only calibrated on a small support set to onboard a new model. Routers are generally lightweight and supervised-trained, so the old data is already collected. The authors motivate this by saying repeated retraining is “computationally prohibitive” but the cited work does not seem to support that claim well. Without showing that retraining/calibrating on more data is actually impractical, the observation risks reducing to “if we onboard a model with very few examples, the router is unreliable”, which is expected.
-
The benefit of E2-Filter is concentrated in the small support size regime. Results from Figure 1(b) and Table 6 seem to suggest with around 256 samples the updated router would beat the old one. This raises the question of why we cannot simply collect a few hundred inference-only samples from a model we are seriously considering routing to.
-
The novelty relative to broader dynamic deployment/model-selection work is unclear. StageRoute studies a broader streaming deployment problem that appears to subsume the single model admission E2 faces. It is unclear why its uncertainty-aware deployment rule, or a simplified version of it, cannot be adapted to the single-candidate setting considered here.
-
The deployable baselines are mainly Always Admit and Predicted-Gain Admit, which are too simple. Natural baselines like confidence thresholds or bootstrap intervals should be included.
-
A nitpick: cost-efficiency is treated as a secondary extension rather than a core evaluation, despite being one of the main motivations for model routing.
Overall, I am skeptical of the realism of this proposed problem setup. I will need to see substantial evidence for why there is a practical need to admit a new model with just a few data samples. The baseline and evaluation axes will also need to be more extensive. Happy to change my mind if there is strong evidence supporting the problem setup and e2's advantage.
Questions for authors
None
9 comments
Author Response to Reviewer wh9r (Part 2/2)
W2. Why not collect ~256 observations and always admit?
As noted in the response to W1, collecting hundreds of user preference labels per candidate is unrealistic in personalized routing, making small- onboarding the natural operating regime. Even when larger support sets are available, more data does not make admission safe: at , Always Admit still trails the incumbent router in 8/24 settings (and even at in several LOCUS settings), whereas -Filter beats Always Admit in 20/24 settings. Rather than replacing data collection, -Filter gates deployment safety at whatever is currently available, with admission rates naturally rising with ; we will explicitly note that gains concentrate at small .
W3. Relation to StageRoute.
The decision problem is distinct between -Filter and StageRoute. StageRoute optimizes online stream-level model routing from per-model rewards, whereas -Filter makes a one-shot gating decision on whether a specific routing update improves over a fixed incumbent—depending on which queries are rerouted and whether they fix more errors than they introduce. Furthermore, we evaluated an adapted StageRoute UCB baseline. As shown below, -Filter substantially outperforms StageRoute UCB, since optimism suits stream exploration rather than one-shot deployment. We will add this baseline and discussion to the revised paper.
W4. Stronger baselines: confidence thresholds and bootstrap intervals.
We implemented the requested confidence-threshold and bootstrap baselines—alongside the StageRoute UCB rule from W3—and evaluated every rule under a strict validation-to-test protocol (where all hyperparameters are tuned solely on validation queries) as well as the paper's original post hoc protocol:
- Threshold & UCB baselines: Including zero-threshold rules (Always Admit, Predicted-Gain Admit, Positive-Gain Admit using Eq. 3), validation-tuned threshold rules (Predicted-Gain threshold, Positive-part threshold), the sample-count rule (Budget threshold), and StageRoute UCB (which admits an update when the upper confidence bound on the cross-fitted paired gain is positive, with confidence width tuned on validation).
| Method | Val → Test AUDG | Post hoc (original protocol) AUDG |
|---|---|---|
| Always Admit | −0.999 | −0.999 |
| Positive-Gain Admit (Eq. 3) | −0.999 | −0.999 |
| Predicted-Gain Admit | −0.446 | −0.449 |
| Predicted-Gain threshold | −0.002 | −0.002 |
| Positive-part threshold (Eq. 3) | −0.002 | −0.001 |
| Budget threshold | 0.091 | 0.086 |
| StageRoute UCB | 0.107 | 0.105 |
| -Filter | 0.173 | 0.162 |
- Bootstrap baselines: We perform resamples with replacement over the labelled candidate support queries (evaluating on the fixed unlabelled validation or test queries) to compute either a bootstrap confidence interval on the predicted score differences (Predicted-Gain + Bootstrap) or a bootstrap lower bound on our dual-evidence statistic (-Filter + Bootstrap).
| Method (with Query-Level Bootstrap) | Val → Test AUDG | Post hoc (original protocol) AUDG |
|---|---|---|
| Predicted-Gain + Bootstrap | −0.001 | −0.001 |
| -Filter + Bootstrap | 0.190 | 0.182 |
Across all comparisons, -Filter outperforms every baseline under both protocols: without our cross-fitted candidate evidence, zero-threshold rules incur severe losses while ungated thresholds and bootstrap stall near zero on noisy small- predictions. Inspired by your suggestion, pairing query-level bootstrap resampling with our own evidence (-Filter + Bootstrap) pushes performance even higher—offering a strong drop-in upgrade when resampling overhead is acceptable, while standard -Filter already beats all baselines in a single closed-form pass. All of these baselines will be included in the revision.
W5. Cost should be a core evaluation.
We will move the cost-aware evaluation to the main paper.
Thank you for the reply. Unfortunately, I don't find this rebuttal particularly helpful in addressing my concerns.
In terms of whether the proposed scenario is realistic, the authors cite UniRoute, but I don't see evidence in this work that collecting a few hundred candidate observations and updating a router is prohibitive. In general, I don't believe the authors have provided strong evidence to support the idea that a model provider would be unwilling to collect a few hundred samples from a new model before deciding whether it should be deployed.
The authors then pivot to personalized routing—where users can have their own preferences for model quality—which is a much better justified scenario. However, I have several major concerns with this pivot:
- This is a substantial pivot: personalized routing is not the focus of, or even mentioned in, the original submission. Given the limited support for the original motivation, it now appears to carry much of the practical justification. This kind of pivot may be too substantial to incorporate during rebuttal.
- The added personalized routing results are very preliminary. The authors adopt an MBTI-based setting that neither of the two cited works (PersonalizedRouter and GMTRouter) uses, with almost no details on how such preferences are set up and judged. The displayed results use just one router and one training budget. This seems way too thin if personalized routing becomes the new focus. The two LLM judges also produce substantially different satisfaction rates, which warrants further explanation and validation.
- Most importantly, the displayed table seems to only show that tailoring a router to simulated user preferences improves satisfaction. It does not evaluate new-model admission or demonstrate E2-Filter’s benefit.
- I couldn't find results supporting the claim that “Applying E2-Filter in this few-shot per-user onboarding setting—without any hyperparameter re-tuning—consistently improves onboarding AUDG over naive admission across routers and pool sizes.” I couldn't locate any accompanying results or an updated PDF, so it is difficult to assess this otherwise significant claim.
My general impression here is that personalized routing is a good scenario for the proposed work, but its evaluation remains too rough and preliminary to show that the proposed method is competitive in this scenario.
Moving on to why one couldn't simply collect more samples, the authors' counterarguments are:
- “at k=256, Always Admit still trails the incumbent router in 8/24 settings”
- “E2-Filter beats Always Admit in 20/24 settings”
I acknowledge that these results qualify my initial reading of Figure 1(b) and Table 6. However, my concern is that the gaps here are very small. In the first comparison, Always Admit is worse than the old router by around 0.19 percentage points on average across those eight settings. In the second comparison, E2 beats Always Admit by around 0.21 percentage points on average across all 24 settings. I am not convinced that these gaps are practically meaningful, and without uncertainty estimates, it is difficult to tell whether they are distinguishable from noise.
Lastly, regarding StageRoute, I am not entirely sure how to interpret the discussion and results. My understanding is that StageRoute selects which models to deploy, collects feedback from using them, and then decides which models to retain at the next update. The scenario presented in this work is to collect candidate feedback from a new model, build an updated router, and then decide whether to deploy it. I am therefore a bit confused about what prevents adapting StageRoute to this setting. For example, why can't we give StageRoute the same candidate feedback used in E2, and then compare their decisions on whether to include the new model in deployment? I am not sure whether the reported StageRoute UCB baseline does this, and would appreciate more details.
For the above reasons, the rebuttal has not changed my initial impression. I will be happy to review any further updates from the authors, though I want to be honest that I don't expect my opinion to change substantially unless the authors can address the concerns about the proposed scenario highlighted above. Additionally, if further results are cited, I would appreciate accompanying table or section references. The authors response to W2 lacks these references, which makes the relevant results difficult to locate.
Author Response to Reviewer wh9r (Part 1/2)
We thank the reviewer for the comments. Below, we address each concern with new empirical evidence—including a personalized routing study demonstrating the few-shot regime—and comprehensive validation-tuned baseline comparisons.
W1. Why few-shot admission, when routers can be retrained?
Even with a lightweight router, frequent full retraining across dozens of models and candidate updates disrupts production deployment pipelines, motivating test-time onboarding without retraining [1].
At the same time, we agree that in standard task-specific routing, collecting a few hundred objective accuracy labels is often feasible. However, the few-shot regime is unavoidable in personalized routing, an increasingly active direction [2, 3]. Unlike objective correctness, personalized routing depends on each individual user's subjective preferences (e.g., preferred tone, verbosity, formatting, or reasoning style), which diverge significantly from global benchmark accuracy. Because labels must come from each user's own preference feedback—and asking a user to evaluate hundreds of responses for every newly released candidate model is impractical—per-user onboarding inherently operates in the small-k regime.
To verify the practical value of personalized routing and illustrate this setting, we constructed a personalized routing benchmark over LLMRouterBench across 16 independent user environments corresponding to the 16 MBTI personas—a standard, structured persona taxonomy in LLM user simulation that captures orthogonal preferences in communication style, detail granularity, and reasoning structure [4, 5]. In each user environment, candidate model responses are evaluated for subjective user satisfaction (like vs. dislike) using two LLM judges (
gemini-3.5-flash-liteandgemini-3.1-flash-lite) conditioned on that persona's profile. Using a kNN router with 100 initial training queries per user environment, we compare four routing policies by average preference satisfaction (% liked) across all 16 users:These results highlight two key insights that directly motivate our problem setting:
[1] Jitkrittum et al., UniRoute: Universal Model Routing for Efficient LLM Inference, ICLR 2025.
[2] Dai et al., PersonalizedRouter: Personalized LLM Routing via Graph-based User Preference Modeling, TMLR 2025.
[3] Xie et al., GMTRouter: Personalized LLM Router over Multi-turn User Interactions, 2025.
[4] Jiang et al., Evaluating and Inducing Personality in Pre-trained Language Models, NeurIPS 2023.
[5] Tseng et al., Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization, Findings of EMNLP 2024.