- Rating
- 2
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 3(+2)
Read review
Paper summary
This work studies weak-to-strong routing for language model cascades without relying on ground-truth correctness labels. It formalises the budgeted routing problem and characterises the Bayes-optimal routing rule in terms of the gain in correctness from escalating.
Other than that, the method learns predictors for these two terms using only model-generated data and a task-specific equivalence checker, and demonstrates strong performance on both agentic and single-turn reasoning tasks.
Critical strengths
The work provides a clean, unified formulation of budgeted weak-to-strong routing that considers weak and identifies the Bayes-optimal policy via an accuracy gain score.
Other than that, the corrected disagreement score is a simple, elegant derivation from the optimal gain by replacing the unknown ground truth distribution with the strong model's predictive distribution.
At last, the decompistion intro weak strong disagreement minus stong self disagreement is intuitive and actionable; it also connects to known failure modes of raw disagreement and explains when the approximation tightens.
Critical weaknesses
There seems to be some issue with work; the key approximation is justified conceptually but lacks quantitative error bounds; no theoretical guarantees are provided on the gap between corrected disagreement and the optimal gain as a function of model miscalibration or distributional shift.
The work assumes access to a reliable disagreement indicator; when this requires an LLM judge, it may introduce its own biases and cost and could implicitly leak supervision.
The work relies on the strong model being a better proxy of ground truth; when the strong model is weak or systematically biased, corrected disagreement can mislead and mode formal robustness analysis would be valuable.
Other than the above, plots in the work are of accuracy vs escalation budget but do not present the token/latency costs for k-sample baselines or for collecting offline strong model samples during training, making Pareto comparison less complete.
At last, recent baselines that optimise for signed escalation value or provide label-free cost control are missing, so the claim of outperforming correctness predictions may be overstated.
Questions for authors
The things mentioned in the weaknesses raise some questions, such as:
-
Can you provide quantitative bounds for the approximation error under assumptions on the divergence between q and s, or at least empirical calibration/error analyses of s vs q on labelled benchmarks?
-
What are the training-time costs to collect K_w/K_s samples and train the two predictors, and how do they amortise over deployment? Could you report end-to-end cost-accuracy Pareto comparisons including live k-sampling overheads for baselines?
-
When A(x, y') requires an LLM judge, how sensitive are training targets to judge errors or style variation, and do you use any calibration of adversarial pairs to improve robustness?
-
Could you compare against Supervised Signed-Gain Routers and/or C3PO on at least overlapping benchmarks to position performance relative to recent methods with similar objectives?
-
In the agentic settings, did you measure rescue and harm rates for escalations? If so, how do corrected disagreements and raw disagreement differ on those diagnostics, especially when the teacher is imperfect?
I would or could raise the rating if the justifications seem convincing enough.
6 comments
Rebuttal Continued: Part II
Robustness to strong models of varying capability: We thank the reviewer for raising this. We study a setting where no correctness labels are available, so the strong model is our only reference for the ground truth. If the strong model is itself unreliable, there is no way to recover the ground truth from the two models alone, and this limitation is shared by any label-free method. We therefore study how the scores behave as the strong model becomes weaker.
To address this directly, we ran additional targeted analysis where we fixed the weak model (Phi-3.5-mini), the prompts and the probe configuration, and varied only the strong model across four models of varying accuracy, from 51.5% to 97.5% (Results: https://imgur.com/a/ZZFlDVQ). Over the same sequence, the shared error with the weak model, i.e. the probability that the strong model is wrong when the weak model is wrong, falls from 66.2% to 3.6%. We report results at the population level, as in Section 4.2, and with learned probes.
We report results both at the population level, using Monte-Carlo estimates of the routing scores as in Section 4.2, and in the finite-sample setting with learned probes. Our findings are as follows:
Corrected disagreement tracks optimal routing for every strong model, and the gap shrinks as the strong model becomes more accurate. This is the behavior predicted in Section 4.1. The score approximates the unknown ground-truth distribution with the strong model’s predictive distribution, so it approaches optimal routing as the strong model improves. It remains close to optimal even for the least accurate strong model, which shares a large portion of the weak model’s errors.
Weak-strong disagreement alone is less robust to strong-model errors: The raw weak-strong disagreement falls short of optimal routing relative to corrected disagreement unless the strong model is almost always accurate. A less accurate strong model also disagrees with the weak model on prompts where escalation does not help, and the self-disagreement correction can recover these cases. When the strong model is highly accurate, it rarely disagrees with itself and the two scores coincide. Results are consistent in finite samples with learned probes as well, however the gains are smaller than at the population level, since the probes are learned from finite data.
Overall, corrected disagreement remains a reliable label-free approximation of the optimal routing across a range of strong models of varying capability. We note that Appendix A.1.1 compares the routing signals across several weak–strong pairs at the population level; and we will add this controlled comparison to the revision.
W4+ Q2: Training and test-time cost: We note that we find it helpful to look at the two type of costs separately. Test-time cost. Our contribution is the signal that a router learns, and each routing score is used within the same test-time procedure. When comparing baselines we tried to keep the weak-model sampling cost matched: in the one-shot setting (N = 1) every score uses a single weak response, and in the multi-sample setting every score is given the same N samples. Within each comparison, the escalation budget on the x-axis is therefore the main cost that varies across methods. Some baselines, such as self-consistency, require more than one sample by nature, which is why they appear only in the N > 1 comparisons, while our scores can also be used with a single sample. We will state this more clearly in the revision.
Training-time cost. Here the scores do differ in the data they need, and we have added a table that reports, for each dataset, the number of weak-model samples, strong-model samples, and ground-truth labels each score uses (Results: https://imgur.com/a/bxKWVeT). Disagreement-based scores use weak and strong samples but require no ground-truth labels, and the correction reuses the same strong samples at no additional cost. Supervised scores additionally require ground-truth labels. We have further added a training-time sensitivity analysis of the probes over the number of training prompts and the number of sampled responses per model (Results: https://imgur.com/a/p2gAE2c) for completeness.
Which method to use. As a rule of thumb, when ground-truth labels are readily available, one should use them. A supervised router that learns the optimal score directly is the natural choice, and we now include it in our comparisons as supervised-baseline. Our method is intended for the many practical settings where ground truth does not exist, is not well defined, or is costly to obtain, such as multi-step agentic tasks. In those settings supervised routers cannot be trained, and on the benchmarks where both can be evaluated, corrected disagreement comes close to them without using any labels.
Rebuttal: Part III
Q4: Rescue and Harm rates for escalation in agentic setting: We thank the reviewer for this interesting analysis, and we include it in the revision: (Results: https://imgur.com/a/zGFyhCM). Since step-level correctness is not defined in these environments, we measure rescue and harm per task, relative to the weak model acting alone on the same task: a task is rescued if the weak model alone fails it and the routed system solves it, and harmed in the opposite case. Our findings are as follows:
With both disagreement scores, escalation rescues far more tasks than it harms. Harm remains low at every budget on both Webshop and BFCL, and the number of of rescued tasks grows with the budget.
Corrected disagreement outperforms raw weak-strong disagreement in rescue counts. On WebShop, corrected disagreement rescues substantially more tasks than weak–strong disagreement at the same budget, with similar harm, and the difference is well beyond the variation across runs. On BFCL, where the teacher is imperfect and solves only about half of the tasks on its own, the two scores have similar rescue and harm rates, consistent with our observation in the paper. In both cases harm stays low, so an imperfect teacher reduces how much can be rescued but does not make escalation harmful.
Q3 + W2: Reliability of the disagreement indicator: To address this, we ran an additional experiment on AceCode, our most open-ended benchmark, where we vary the disagreement indicator used to build the training targets. We compare the execution-based indicator with two LLM judges (GPT-4o-mini and Gemini 2.5 Flash-Lite), a much smaller LLM judge ( the 1.5B weak model itself), and an NLI model, and we report the cascade accuracy fo the probes trained under each. ( Results: https://imgur.com/a/TFD6LKd).
We note that determining whether two responses agree is a much easier task than determining whether a response is correct, and in many tasks it can be done deterministically (e.g. exact match, or running both programs on the same inputs). In our experiment, the cascade accuracy was similar across the different indicators, with the two larger LLM judges slightly ahead at higher routing budgets. We expect judges of similar capability to perform similarly, and a more reliable indicator to give a better cascade in general.
Rebuttal: Part I
We thank the reviewer for their thorough reading and feedback and are glad that they found our formulation clean, the proposed corrected disagreement score elegant and simple, and the decomposition of the final approximation into two terms actionable and intuitive. Below we thoroughly address each of the reviewer's concerns:
Additional Baselines ( Supervised Signed-Gain routers + C3PO ): We thank the reviewer for pointing us to these recent works. We have now implemented both and we will add the results and both references (SRR [1] and C3PO [2]) to the revised manuscript: (Results: Comparison with SRR: https://imgur.com/a/D2aK7fb, Comparison with C3PO: https://imgur.com/a/ObB7rNe )
Supervised signed-gain routing: Signed Rescue Routing (SRR) trains its router with ground-truth correctness labels for both the weak and the strong model. It's target, rescue minus harm, is closely related to the optimal routing gain of our Theorem 1. To thoroughly cover this family of supervised baselines, we also include an optimal-target probe that learns the two terms of Theorem 1 directly from ground-truth labels. Both use the same training prompts, features, and probe head as our scores but utilize ground-truth labels. (Results: https://imgur.com/a/D2aK7fb). Our label-free approximation recovers the gains of these supervised routers without requiring any supervision during training. Corrected Disagreement matches or exceeds SRR on all three benchmarks, and comes very close to optimal-target probe.
That said, our central motivation is to approximate this quantity without ground-truth supervision. Learning the optimal target directly requires correctness labels for both weak and strong responses, which are unavailable in the label-free settings that motivate our paper. However we agree this is a great comparison to measure the gap to supervised routing. In our main experiments, we used the standard correctness probe simply as a common proxy for weak-model reliability. This result strengthens our main point by showing that our approximation can recover a substantial portion of the gains from the optimal target without needing supervision.
C3PO: C3PO is complementary in focus. It is a test-time procedure that thresholds the weak model’s self-consistency, with thresholds chosen from agreement with the strong model under a cost constraint, whereas our contribution is the routing signal itself. Because C3P builds on self-consistency, it requires several weak samples per query; we follow its paper’s setting and give our scores the same samples so that the comparison is matched (Results: https://imgur.com/a/ObB7rNe). We compare at the routing budgets that C3PO reaches. Corrected Disagreement is ahead at every such budget on Spider and AceCode, and on MATH-500 it is ahead at low and moderate budgets, with C3PO slightly ahead only at high budgets.This is consistent with our observation in the paper (Figure 3) that multi-sample baselines close the gap only on closed-form tasks such as MATH-500, where clustering answers is trivial. We also note that the two approaches can be combined, since C3PO’s cost-control procedure can be applied on top of any confidence score, including ours.
[1] Zheyuan Wang, Siyu Li, Peiqiao Song, Sijia Chen, Qianqian Song, and Qian Liu. Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference. arXiv:2609.07786, 2026. https://arxiv.org/abs/2609.07786
[2] Antonios Valkanas, Soumyasundar Pal, Pavel Rumiantsev, Yingxue Zhang, and Mark Coates. C3PO: Optimized Large Language Model Cascades with Probabilistic Cost Constraints for Reasoning. arXiv:2511.07396, 2025. https://arxiv.org/abs/2511.07396
W1 + Q1: Quantitative bounds on the approximation error: We thank the reviewer for this suggestion. We have added a bound on the approximation error to the paper (Proposition 3). Due to space constraints, we keep the statement and its proof in Appendix C.
Corrected disagreement replaces the ground truth with the strong model’s answers, so its error depends on how often the strong model is wrong. We show that for any prompt, the gap between corrected disagreement and the optimal routing score is at most twice the probability that the strong model answers that prompt incorrectly. The more accurate the strong model, the closer corrected disagreement is to the optimal score, and the two coincide when the strong model is always correct.
Empirically, we also added an experiment that varies the strong model’s accuracy on the labelled benchmarks (see our response in Part II, https://imgur.com/a/ZZFlDVQ ), and the results follow the same trend.