- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper frames budget-constrained SLM-LLM collaboration as information acquisition. A 7B SLM reasons on its own and, at most once per trajectory, sends a short self-contained query to a black-box advisor instead of routing the whole problem or resending its trace. A three-stage RLVR pipeline learns whether to call, what to ask under a response budget (through a separate query generator, QMaker), and how to use the advice. On math and coding benchmarks, the method is reported to dominate problem-level and step-level baselines on accuracy versus advisor tokens, and to transfer to unseen advisors.
Critical strengths
The problem is timely: strong models are increasingly consumed through metered black-box APIs, and framing SLM-LLM collaboration as information acquisition under a token budget fits that setting. The results on math and coding, and the transfer to unseen advisors, support the approach within the paper’s cost definition.
The paper is well written. The introduction places the work against routing, step-level co-generation, and tool-augmented LMs, and the pilot study motivates each learned capability with a measured failure of the zero-shot SLM.
The formulation in Section 3 is clean and explicit about its assumptions, and the three-stage reward design follows from it directly.
Critical weaknesses
Cost. The comparison is not compute-matched. The cost in Equation 4 counts advisor tokens only, and Appendix A excludes all local inference by design. On every calling trajectory, the method runs three 7B generations: the SLM up to the call, QMaker over the full reasoning trace so far, and the SLM again with the advice in context. A problem-level router runs a BERT-scale classifier; the cascades run the SLM once plus a verifier. QMaker is itself a second Qwen2.5-7B-Instruct. Replacing advisor output with local work is what reduces the quantity on the x-axis, so the omission affects the main result. The local side has a public price. OpenRouter lists Qwen2.5-7B-Instruct at $0.10 per million input tokens and $0.20 per million output tokens, and the paper’s advisor, Qwen3.5-35B-A3B, at $0.065 and $0.75. At those rates, one advisor reply capped at 2048 tokens costs about as much as 7,500 SLM output tokens, which is the scale of one trajectory plus a QMaker pass. Cheaper 7B models exist (Llama 3.1 8B at $0.02 and $0.04); three passes are still a nonzero fraction of each call. Appendix C.5 concedes that on math the advisor reply is often already close to the final answer, and Figure 5c shows that at budgets of 4,096 tokens and above 94 to 97% of math queries are FULL-TASK requests. The paper does not measure how often the reply contains the answer, and the results of the budget-capped LLM baseline (listed in Section 6.1, plotted in Figure 4) are never discussed in the text. At large budgets, the method may reduce to a capped LLM call with a 7B wrapper.
Oracle comparison. The AMC23 result against the oracle is explained in one sentence: the method has access to a richer collaboration space (Section 6.2). The oracle pays for an uncapped LLM solution on every routed problem while the method’s replies are capped at b, so the gain is a cost effect rather than a routing effect. An oracle allowed to choose the budget-capped LLM for each problem would be a fair comparison, and AMC23 has only 50 problems.
Questions for authors
-
Can the authors report the comparison with SLM tokens (both SLM segments and the QMaker pass) priced alongside advisor tokens, for the method and the baselines? Counting both sides would make the comparison with routing and cascading methods more balanced, since those methods spend far less locally per problem.
-
Since the advisor reply on math is often already the final answer (Appendix C.5), what accuracy and cost does the advisor reach when its output is capped at about 100 tokens (for example, by prompting it to give only the answer within 100 tokens), enough for an answer and little else? And if that capped advisor is used as the LLM side of a plain SLM/LLM router (the paper’s RouteLLM or oracle setup), does the resulting frontier match or beat the proposed method? This would show whether the gain comes from the learned queries or simply from capping the advisor’s output.
-
The method assumes that the SLM’s partial trajectory reveals impending failure and that the missing information fits in a short query; math and coding satisfy both. Do the authors expect task types where either assumption fails (long-context QA, open-ended generation, difficulty revealed late in the trajectory), such that routing or cascading the whole problem would be better on average? How would such tasks be identified in advance?
7 comments
W1 - How often the reply contains the answer?
We appreciate the reviewer for raising this concern. We acknowledge that this is an important concern that warrants further investigation.
To address this, we conducted an additional analysis measuring how frequently the advisor's response contains the student's final answer. For mathematical reasoning tasks, we measured the proportion of queries in which the student's final submitted answer also appears in the advisor's response. For coding tasks, we measured the similarity between the advisor-generated code and the student's final submitted code. Specifically, when the advisor's response contained code, we computed the string similarity using Python's difflib.SequenceMatcher.ratio(), where a score of 1.0 indicates an exact match. We consider the advisor's response to contain the answer when the similarity is above 0.9. We split the results into FULL_TASK and Non-FULL_TASK query types.
The results show that only 23% of advisor responses contain the student's final answer for Non-FULL_TASK queries, where the original question is not directly forwarded to the advisor. Interestingly, even for FULL_TASK queries, where the original question is provided in its entirety, 46% of advisor responses do not contain the student's final answer.
| Cap | FULL_TASK | Non-FULL_TASK | All |
|---|---|---|---|
| 512 | 0.38 (51%, 905) | 0.12 (49%, 859) | 0.25 (1,764) |
| 1k | 0.43 (34%, 597) | 0.16 (66%, 1,171) | 0.25 (1,768) |
| 2k | 0.53 (45%, 837) | 0.36 (55%, 1,037) | 0.43 (1,874) |
| 4k | 0.49 (86%, 1,588) | 0.24 (14%, 268) | 0.45 (1,856) |
| 8k | 0.60 (93%, 1,725) | 0.35 (7%, 122) | 0.58 (1,847) |
| 16k | 0.67 (90%, 1,544) | 0.41 (10%, 179) | 0.65 (1,723) |
| All caps | 0.54 (66%, 7,196) | 0.23 (34%, 3,636) | 0.44 (10,832) |
This is because the FULL-TASK delegation does not necessarily translate into receiving a complete, self-contained solution from the advisor. In practice, advisor responses often terminate before completing the full reasoning process, leaving substantial portions of the solution unfinished. In such cases, the SLM continues reasoning based on the advisor's response, filling in the missing steps to produce the final answer. Thus, even when FULL-TASK requests are frequent, the SLM can still play an important role in completing the solution, rather than merely reproducing the advisor's output.
We will add these results on the Appendix.
W1 - The results of the budget-capped LLM baseline are never discussed in the text.
We have newly added a brief discussion about the budget-capped LLM baseline in Appendix E. Please See the paper for further information.
W1, Q1 - Cost. The comparison is not compute-matched.
We sincerely thank the reviewer for suggesting the use of publicly available pricing from OpenRouter to account for local model inference costs. This is a valuable evaluation perspective that we had not previously considered. We report the results and analysis that our method remains competitive, on Appendix C.4 of the revised manuscript.
However, we would like to clarify the motivation behind our original cost formulation. Minimizing remote API expenditure is a practically meaningful optimization objective in its own right, distinct from minimizing the total computational cost of the system. Our work considers a hybrid inference setting in which smaller models are executed on locally available, user-controlled hardware, while a stronger black-box model is accessed through a usage-based API. In such settings, local inference consumes computational resources but does not incur the same per-request external API charges as remote inference. Consequently, the primary budget constraint can naturally be the expenditure associated with accessing the remote model. Our original metric was designed to capture this aspect of deployment cost, rather than to serve as a comprehensive measure of total computational overhead.
Prior work also adopts cost metrics centered on expensive model utilization, rather than fully compute-matched comparisons that account for all inference costs. For instance, Hybrid LLM [1] and RouteLLM [2] primarily focus on reducing the cost of expensive model invocations, without explicitly accounting for the computational cost of increased smaller-model inference in their primary cost metrics. Both use the fraction of queries routed to the large model as a proxy for cost, implicitly treating the inference cost of the smaller model as negligible. More closely related to our setting, Router-R1 [3] uses model-weighted output tokens from invoked candidate models in its cost related evaluations, while TRIM [4] measures cost through stronger-model output tokens, despite employing two additional local models for generation and step-level verification. These studies similarly treat larger model utilization as a meaningful efficiency metric without explicitly incorporating all local inference costs into their primary cost formulations. Alongside these works, our cost formulation is a step forward toward explicitly capturing real-world remote black-box API charges through a weighted sum of input- and output-token counts using their respective API prices.
We also conducted an additional cost analysis using OpenRouter's publicly listed model prices and included the results in Appendix C.4, following the reviewer’s suggestion. In summary, our method remains competitive in terms of performance–cost trade-offs even when accounting for additional local inference costs, although some baselines achieve comparable or better trade-offs at certain budget levels on the easy split.
[1] Ding, Dujian, et al. "Hybrid llm: Cost-efficient and quality-aware query routing." International Conference on Learning Representations. Vol. 2024. 2024.
[2] Ong, Isaac, et al. "Routellm: Learning to route llms from preference data." International Conference on Learning Representations. Vol. 2025. 2025.
[3] Zhang, Haozhen, Tao Feng, and Jiaxuan You. "Router-r1: Teaching llms multi-round routing and aggregation via reinforcement learning." Advances in Neural Information Processing Systems 38 (2026): 141233-141265.
[4] Kapoor, Vansh, et al. "TRIM: Hybrid inference via targeted stepwise routing in multi-step reasoning tasks." arXiv preprint arXiv:2601.10245 (2026).