- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes GREAT, a framework for generalizing LLM routing to unseen models and candidate sets. The method combines a dynamic case-based memory of past routing decisions and a strategy for constructing diverse candidate sets during training. Experiments across routing backbones and benchmarks report improvements in accuracy and execution cost, including settings with unseen LLMs and held-out datasets.
Critical strengths
Generalization to unseen LLMs and changing candidate pools is a practically important challenge for LLM routing. The method is evaluated across five routing backbones and multiple benchmarks, and the paper reported improvements in both accuracy and execution cost.
Critical weaknesses
Theorem 2 establishes submodularity over collections of candidate LLM sets, whereas Algorithm 1 greedily adds individual LLMs to construct a single candidate set. Submodularity in the former does not imply submodularity in the latter. Thus, the claimed approximation guarantee does not appear to apply to the actual algorithm, undermining a central theoretical contribution. GraphPlanner+GREAT achieves 85.8% accuracy and 64.4 cost in Table 1, compared with 87.0% and 59.4 in Table 2, and 83.2% and 91.5 in Table 4. The paper does not clearly explain the differences in experimental settings underlying these results, making the reported improvements difficult to assess reliably.
Questions for authors
How does Theorem 2 justify Algorithm 1, given that the theorem concerns selecting sets from a collection, while the algorithm selects individual LLMs within a set? Why do the GraphPlanner+GREAT results differ across Tables 1, 2, and 4?