- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies when to replace a small language model's response with that of a larger, potentially stronger model. Rather than directly estimating signed deferral gain, it factorizes that gain into the probability of answer disagreement and the expected signed gain conditional on disagreement. Under answer-factored grading, the authors characterize what paired outputs alone identify and fit a two-stage router using labels collected on sampled disagreements. Experiments routing from Qwen2.5-1.5B-Instruct to Qwen2.5-7B-Instruct show improvements in integrated accuracy versus call rate, while advantages at particular call rates and token costs vary. My primary concern is the practical significance and scope of the contribution, including its extension beyond tasks where answer disagreement is straightforward to define.
Critical strengths
- A useful approach to efficient supervision. Under answer-factored grading, paired outputs identify where replacement can matter, while outcome labels determine whether it helps. Agreement provides zero-gain supervision, allowing grading to focus on disagreements (§§3.2–3.4). The bounds and selective-label design are useful even though the factorization is elementary and the estimator does not dominate at every call or label budget.
Critical weaknesses
-
The practical contribution remains insufficiently established. Label-budget experiments and observation-matched controls support learning a routing signal (§§5.2–5.3), but the signed-stage increment depends on the budget and operating point. The main evaluation uses one fixed model pair and readily normalized answers. It remains unclear whether simple query/confidence cues or hidden-state features drive the gains, and how the method performs when answer equivalence is harder to define. The imperfect-equivalence bound does not establish an empirical implementation. Unexplained benchmark exclusions further limit the reported scope.
-
The figures and tables provide insufficient intuitive guidance. Figure 1 is difficult to follow, while dense tables and overlapping curves obscure the central routing comparisons (Figures 2–3).
Questions for authors
-
Excluded conditions and selection rationale. Which benchmark was omitted from the primary collection after task-mixture and token-cost analyses, which family was omitted from the wider sweep, and why? What do their existing outcomes imply for the routing benefit, given the reuse of earlier test queries? Was the selection based on outcome-independent technical reasons?
-
Presentation suggestion. Figure 1 needs a concrete link between disagreement probability, signed improvement and the routing decision. The result figures and tables need more distinguishable curves and clearer router–baseline comparisons at fixed call or label budgets.
4 comments
Comment 2/3: Model-pair coverage and practical routing outcomes
A separately trained cross-family cascade
To test whether the signed-stage benefit extends beyond one model pair, we compared Qwen3.5-9B to Qwen3.5-27B with Mistral-7B-Instruct-v0.3 to Qwen3.5-27B on another common set of 10,000 Natural Questions queries. Each pair used 8,000 training queries, 1,000 gold-free calibration queries, and its own fitted representation, Support model, and supervised heads. The methods and tuning ranges were frozen, and all training and tuning finished before test evaluation.
At 20% deferral:
| Model pair | Outcome labels | Stage 2 minus Support, percentage points [simultaneous 95% interval] |
|---|---|---|
| Qwen to Qwen | 500 | +0.359 [-0.006, 0.723] |
| Qwen to Qwen | 2,000 | +1.737 [1.240, 2.234] |
| Mistral to Qwen | 500 | +2.931 [2.366, 3.495] |
| Mistral to Qwen | 2,000 | +3.819 [3.183, 4.454] |
The Mistral-to-Qwen cascade shows a signed-stage benefit at both budgets; the Qwen-to-Qwen increment is positive at 2,000 labels and inconclusive at 500. The approximate simultaneous bootstrap intervals are adjusted across 24 prespecified utility/harm comparisons. They resample shared queries and 20 label-sampling runs, conditional on the training pool, fitted representation and Support model, and fixed generations. Each pair has a separately trained router. Model size, accuracy, and answer distributions also differ, so the larger Mistral increment cannot be attributed specifically to model family.
Net improvement and harmful substitutions
We counted rescued errors and harmful substitutions to determine what produces the accuracy increment. A rescue replaces an incorrect small-model answer with a correct large-model answer; harm makes the reverse substitution. Their difference is the net number of additional correct answers. At 2,000 labels and 20% deferral, averages per 1,000 total queries were:
| Pair | Method | Rescue | Harm | Net |
|---|---|---|---|---|
| Qwen | Support | 13.800 | 0.500 | 13.300 |
| Qwen | Stage 2 | 34.875 | 4.205 | 30.670 |
| Mistral | Support | 25.300 | 0.200 | 25.100 |
| Mistral | Stage 2 | 63.910 | 0.625 | 63.285 |
The net increments arise because additional rescues exceed additional harm. Qwen's harm increase is positive under the simultaneous interval; Mistral's harm difference remains uncertain.
We also compared Stage 2 with a two-head direct estimator (twohead) that fits common correctness and large-minus-small difference components and routes using the difference. It receives the same revealed outcome labels and validation folds. For Stage 2 trained as specified in the paper versus fully tuned twohead, all four net-gain intervals include zero. A separate comparison matches coefficient counts in the margin and difference routing heads, preprocessing, final-gain validation, and ten tuning candidates per method. The matched two-stage variant improves Mistral accuracy by 0.459 points [0.282, 0.635] at 500 labels and 0.260 [0.106, 0.413] at 2,000, while reducing actual harm. The fitting losses still differ between conditional-margin and direct-gain estimation. These results concern the matched variants; the table above reports Stage 2 with the paper's training procedure.
Preparation and serving costs
To measure the resources required for these gains, we recorded paired generation, feature extraction, representation/Support tuning, head tuning, and calibration. At 2,000 labels, the preparation subtotal is 2.44 hours for Qwen and 2.66 hours for Mistral on one NVIDIA GB10. Support, Stage 2, and twohead share the paired pool and representation; the two supervised methods additionally require 2,000 outcome labels.
A separate replay of 512 common queries uses one prespecified 2,000-label fit, three timing repetitions, and batch size 1. Stage 2's mean service times are 1.004/1.432 seconds for Qwen/Mistral, versus Support's 1.189/1.570 seconds. The replay uses calibrated thresholds and separately graded outputs; actual call rates differ slightly. Shorter large-model generations among the selected queries account for much of this timing difference. Annotation labor and some acquisition/preprocessing costs remain unmeasured, and the repeated timings do not create additional independent quality samples.
Comment 3/3: Answer equivalence, excluded conditions, and presentation
Semantic equivalence beyond normalized short answers
The completed independent evaluations concern short-answer routing under normalized exact-match grading. They broaden task and model coverage, but they do not establish semantic equivalence for unrestricted answers. The imperfect-equivalence bound gives a criterion for assessing a matcher; it does not validate an implementation.
We have prepared 400 blinded answer-pair audit cases from Natural Questions and TriviaQA to assess this issue. The rubric separately asks whether the answers are semantically equivalent and whether each is correct. Stratified sampling weights are retained for estimating population rates. Human annotations are pending, so there is no completed semantic-validation result to report. These cases use previously generated short answers and do not constitute a long-form generation evaluation.
The relevant implementation criterion is grading consistency. Let be the large-model grade minus the small-model grade, indicate disagreement under the matcher, and denote router-visible features. If the matcher declares agreement despite different grades, the factorization omits . For grades in , the magnitude of this term is bounded by the conditional probability of such grade-changing false agreements. The audit must therefore assess both equivalence and grading consistency. Equal correctness scores alone do not establish semantic equivalence.
Excluded benchmarks and selection rationale
The available records do not establish the names, outcomes, or selection rationale for the excluded benchmark and task family. Appendix B documents retaining two of three benchmarks and ten of eleven families after earlier mixture/token-cost analyses, with reuse of test queries. We cannot establish that those exclusions were outcome-independent and will state this selection limitation in the main text.
The new studies directly test the questions about feature contributions, label budgets, and model-pair coverage raised in your review. For the independent evaluations, methods, tuning ranges, label budgets, and evaluation criteria were frozen before test access, and all training and tuning finished before the results were opened. The partitions exclude exact and near matches to retained historical queries. They are drawn from official training splits and held out from router development; they are not the official benchmark test sets. This protocol strengthens the additional evidence while leaving the historical selection question unresolved.
Concrete routing decisions and readable comparisons
We will revise Figure 1 around two queries. For query A, the disagreement probability and conditional margin are (0.9,-0.2); for B they are (0.4,0.8). Support ranks A first because disagreement is more likely. Their expected gains are -0.18 and +0.32, so the product ranks B first. The example connects the learned quantities to the replacement decision.
The result figures will show differences at fixed call budgets, with intervals and distinct line styles. The submitted operational tables already make these comparisons possible: Support exceeds entropy by 0.797 and 1.148 accuracy points at 10% and 20% deferral, while Stage 2 with 2,000 labels exceeds Support by 0.842 and 0.771 points. We will bring these comparisons into the main presentation instead of asking readers to estimate gaps between overlapping curves. The original split/label-fit stability analyses will remain distinct from the new independent evaluations.
Thank you for the additional experiments and clarifications. They strengthen the evidence that signed supervision can improve routing. However, two concerns remain:
- The incremental value of the factorization remains unclear. Improvements over Support establish the benefit of adding outcome labels. While the positive matched-control results are useful, the evidence for an advantage over direct gain estimation remains limited to particular settings.
- The original evaluation remains insufficiently traceable. The statement that the excluded benchmarks, outcomes, and selection rationale cannot be established does not resolve my question. These are basic facts needed to assess the reported evidence. Maintaining and providing an accurate account of the experiments underlying the reported conclusions is a basic responsibility in scientific research. Please verify these details against the experimental records. If they cannot be recovered, please explain which original conclusions depend on this evaluation and which are independently supported by the new studies.
I appreciate the additional work, but these issues remain important to my assessment of the contribution and reliability.
Comment 1/3: Feature contributions and the label budget
Thank you for asking which information drives the gain and when signed supervision is useful. We added feature ablations to separate the contribution of hidden states, followed by independent evaluations to test the label-budget dependence on new queries.
Query/confidence features versus hidden states
In these comparisons, hidden states add accuracy, and Stage 2 also improves over Support when only scalar features are available. We compared ten query/confidence summaries with the same summaries plus small-model hidden states, holding the model pair, data, revealed outcome labels, and training-only tuning procedure fixed. Labels come from benchmark gold and simulate a limited outcome-label budget. The scalar features describe query form, sequence length, token entropy, and token probabilities. Hidden states are extracted after the small model's final normalization and reduced using training data. Large-model states are never router inputs.
For Qwen3.5-9B to Qwen3.5-27B, with 1,000 outcome labels and 20% deferral:
The first two comparisons measure the additional value of hidden states. The third establishes a signed-stage increment with hidden states removed. Query and confidence summaries are grouped, so this experiment does not isolate their individual contributions. These are exploratory, pointwise query-bootstrap intervals on previously examined evaluation data, conditional on five fitted label draws and without correction for multiple comparisons.
How much signed supervision is needed?
We then tested the budget dependence independently, using Qwen3.5-9B to Qwen3.5-27B on newly held-out NQ and TriviaQA queries. Each task used 8,000 training queries and 10,000 evaluation queries. The seven label budgets were 25, 50, 100, 250, 500, 1,000, and 2,000, with 20 label-sampling runs and subsets nested across budgets. Methods, tuning ranges, and evaluation criteria were fixed before test access; all training and tuning finished before evaluation.
At 20% deferral, Stage 2's accuracy increment over Support was:
These approximate simultaneous 95% bootstrap intervals are adjusted across the study's 96 prespecified comparisons. Budgets from 25 to 500 did not establish a positive increment. Resampling covers evaluation queries and label draws, conditional on the training pool, fitted representation and Support model, and fixed generations.