- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies service withdrawal in multi-tier LLM routing: a request accepted at a lower tier can be rejected after an upgrade, even when aggregate coverage and budget increase. CoRoute represents each tier as a mixture of threshold policies and routing rules. Ordered threshold distributions and shared randomness ensure persistent acceptance, while joint optimization minimizes the fitted performance cost within a fixed policy library. Across 96 external cases, the method reduces withdrawal from 7.99% to zero, with a mean accuracy cost of 0.039 percentage points.
Critical strengths
-
A clear problem with a theoretically grounded solution. The paper clearly distinguishes aggregate coverage from request-level persistence. The ordered-policy construction provides a way to prevent service withdrawal without retraining the underlying router or LLMs, supported by the theoretical analysis.
-
A small accuracy cost for preventing withdrawal. The external results in Table 3 show that withdrawal can be eliminated at a small average accuracy cost. This makes the approach promising for maintaining service continuity across tiers.
Critical weaknesses
-
The empirical gain from joint fitting over tierwise fitting remains uncertain. Appendix A.7 and the known-risk diagnostics in Appendix D demonstrate that joint fitting can help. However, the external results in Table 3 give both approaches the same rounded mean maximum-tier loss of 0.38 percentage points, and the matched comparisons in Appendix G have intervals that include zero. The authors explicitly acknowledge this limitation. Thus, the evidence for the persistent-policy construction is stronger than the evidence for a reliable additional benefit from joint fitting with estimated risks. This is a limitation of the demonstrated empirical contribution, not a flaw in the persistence guarantee.
-
Repeated qualifications make the presentation harder to follow. The distinctions between fitted-risk guarantees and test accuracy, and between population diagnostics and held-out improvements, are important. However, these qualifications are scattered and revisited throughout the paper, which interrupts the main argument. A short overview of what each result establishes, with cross-references where appropriate, would make the exposition easier to follow.
Questions for authors
- Given the estimation sensitivity documented in Appendix D, is there a practical diagnostic based on fitting or validation data that would help users decide whether joint fitting is likely to improve on tierwise fitting? A discussion based on the existing results would be sufficient.