acceptodds
Under review as a conference paper at ICLR 2027

SLM Reasons, LLM Advises: A Cost-Efficient Collaborative Reasoning Framework

15 views

Abstract

Deploying large language models (LLMs) for reasoning-intensive tasks presents a fundamental cost–accuracy trade-off: frontier models achieve strong performance but incur substantial inference costs, whereas small local models are inexpensive but considerably less reliable. We show that this trade-off can be substantially mitigated through selective collaboration. We introduce SRLA, a refinement framework in which a small language model (SLM) performs the primary reasoning and selectively seeks feedback from a LLM only when its reasoning is likely to contain an error. SRLA determines when and where external guidance is needed using Chunk-level Guided Scoring (CGS), a lightweight adapter trained with direct preference optimization on chunk-level correctness preferences. At inference time, CGS evaluates intermediate reasoning without access to reference answers and identifies the least reliable portion of the solution. Only when the resulting score triggers the refinement gate does SRLA invoke the LLM advisor. The advisor receives the original problem, the complete solver trajectory, and the identified problematic chunk, and returns a structured diagnosis with targeted corrective feedback. The SLM then uses this feedback to refine its solution, while otherwise reasoning entirely locally. We evaluate SRLA on five mathematical reasoning and six code generation benchmarks using Qwen3.5-9B as the local solver and GPT-5.4 as the advisor. SRLA matches or even exceeds GPT-5.4 used as a standalone solver on the majority of benchmarks while reducing per-problem API cost by up to 92% on mathematical reasoning and 94% on code generation. On several benchmarks, the advisor is invoked only rarely, yielding negligible external API cost while preserving strong accuracy. These results show that frontier LLMs need not perform the entire reasoning process: selectively using them to diagnose and advise a capable local solver can achieve frontier-level reasoning performance at substantially lower cost.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Small Language Models, Large Language Models, Cost-Efficient Reasoning, Mathematical Reasoning, Code Generation, Model Collaboration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9xxPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the cost–accuracy trade-off in reasoning-intensive tasks by proposing SRLA, a selective refinement framework that enables small language models (SLMs) to leverage frontier LLMs only when needed. SRLA introduces Chunk-level Guided Scoring (CGS), a lightweight adapter trained via direct preference optimization to identify unreliable reasoning segments without reference answers, triggering targeted LLM feedback only when necessary. Experiments on five mathematical reasoning and six code generation benchmarks show that SRLA matches or surpasses standalone GPT-5.4 on most benchmarks while reducing API costs by up to 92% and 94%, respectively.

Critical strengths
  1. Effective selective collaboration. SRLA dynamically determines when and where external guidance is needed, allowing SLMs to retain local reasoning autonomy while leveraging frontier LLMs for targeted error correction.
  2. Lightweight and targeted refinement. The proposed CGS identifies unreliable reasoning chunks without reference answers, enabling fine-grained feedback and avoiding unnecessary LLM calls.
  3. Good performance and cost efficiency. Experiments across mathematical reasoning and code generation demonstrate frontier-level performance with API cost reductions of up to 92% and 94%, respectively.
Critical weaknesses
  • Reliability of Chunk-level Guided Scoring. Is the proposed Chunk-level Guided Scoring (CGS) mechanism reliable? The authors should conduct experiments to validate its effectiveness.
  • Comparison with self-refinement using GPT-5.4. Would using GPT-5.4 to perform self-refinement based on CGS achieve better results? The authors should include this comparison to demonstrate the necessity of the proposed small-model–large-model collaboration framework.
  • Limited backbone diversity and insufficient statistical validation. The authors should evaluate more backbone models rather than relying solely on Qwen3.5-9B as the local solver. Given the limited number of mathematical problems, the reported results may be subject to considerable variance; therefore, the authors should report averages over multiple runs or provide a rigorous robustness analysis.
  • Unclear rationale for external guidance. When the local solver is likely to make an error, as indicated by a low CGS score, why should the advisor provide targeted feedback rather than solve the problem directly? The authors should justify the advantages of this design choice in terms of accuracy and cost.
  • Insufficient analysis of overall cost and computational overhead. Does training CGS incur additional costs that outweigh the savings from reduced API calls? The authors should compare the monetary costs of training the scoring model and invoking external APIs to establish the overall cost-effectiveness of the proposed method.
  • Limited novelty and insufficient comparison with prior work. Several existing studies have explored related approaches to model collaboration, uncertainty-based routing, and selective refinement. The authors should provide a more comprehensive discussion and systematic comparison with these works to clarify the novelty and advantages of the proposed method.
    • MentorCollab: Selective Large-to-Small Inference-Time Guidance for Efficient Reasoning
    • Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning
    • Selective Deferred Routing: Enabling Cost-Efficient Collaboration between Local SLMs and Remote LLMs
    • Language Model Cascades: Token-level Uncertainty and Beyond
    • A Unified Approach to Routing and Cascading for LLMs
Questions for authors

Explain the concerns in Weaknesses part.

4 comments

We sincerely thank the reviewer for their thoughtful evaluation of our work and for highlighting the strengths of SRLA, particularly its effective selective collaboration, lightweight and targeted refinement, and strong performance with reduced API costs. We appreciate the reviewer's constructive feedback and suggestions, which will help us improve the clarity and rigor of the paper. We address the reviewer's concerns in detail below.

Q1: How reliable is the proposed Chunk-level Guided Scoring (CGS) mechanism in detecting and localizing reasoning errors?**

A1: We evaluate CGS reliability in terms of error detection and discuss its use as a chunk-level localization cue for targeted advisor feedback. For error detection, we treat incorrect first-pass solutions as the positive class, with advisor intervention triggered when the average CGS score falls below the threshold of 0.005. Table 1 reports advisor invocation rates, AUROC, false-positive rates (FPR), and false-negative rates (FNR) across five mathematical reasoning benchmarks.

Table 1: CGS Reliability and Advisor Invocation Rates

BenchmarkAdvisor Called (%)CGS AUROCFPR (%)FNR (%)
AIME 202426.67 (8/30)0.857.6958.82
AIME 202520.00 (6/30)0.8611.1176.19
AIME 202630.00 (9/30)0.867.6952.94
AMC2357.50 (23/40)0.3660.6157.14
MinervaMath9.56 (26/272)0.549.0989.58

Error Detection Reliability. As shown in Table 1, CGS achieves consistently high AUROC values of 0.85–0.86 across the three AIME benchmarks, with relatively low false-positive rates (7.69–11.11%). However, false-negative rates range from 52.94% to 76.19%, indicating that a substantial proportion of incorrect solutions remain undetected. Performance is weaker on AMC23 and MinervaMath, with AUROC values of 0.36 and 0.54, respectively. AMC23 exhibits a high false-positive rate (60.61%), leading to frequent advisor interventions, while MinervaMath has a high false-negative rate (89.58%), indicating that most incorrect solutions are not flagged.

Error Localization. To evaluate whether CGS provides useful error-localization information to the advisor, we conduct an additional study on 17 randomly selected incorrect first-pass solutions from all three AIME and AMC23 benchmarks. We use an independent LLM (GPT-6.1 Sol) to identify the first erroneous reasoning step and assess whether the lowest-CGS chunk overlaps with this independently identified error. In 13 out of 17 cases (76.47%), the lowest-CGS chunk overlaps the first error identified by GPT-6.1 Sol. These findings provide preliminary evidence that CGS can identify potentially erroneous reasoning segments and provide useful localization guidance to the advisor, supporting SRLA's targeted feedback mechanism.

Overall, these results suggest that CGS provides a useful error-detection signal on complex reasoning tasks, but its reliability varies across benchmarks.

Q2: Would using GPT-5.4 for self-refinement based on CGS achieve better results than SRLA's advisor-guided SLM refinement?

A2: We thank the reviewer and would like to clarify the scope of SRLA. Our goal is to improve the performance of small language models (SLMs) while substantially reducing reliance on frontier models, at a fraction of their API cost: SRLA keeps reasoning local and queries GPT-5.4 only when the SLM's trajectory appears unreliable. A GPT-5.4 self-refinement pipeline uses the frontier model on every problem, which is precisely the dependency SRLA is designed to avoid, so it addresses a different problem; we include direct GPT-5.4 inference in Table 1 and 2 as the frontier-model reference and compare SRLA with it on both accuracy and cost.

Moreover, CGS is specific to the SLM, as it is computed from the likelihood difference between its DPO-optimized policy and frozen SFT reference, averaged over reasoning chunks. Since GPT-5.4 does not have an equivalent accessible policy–reference pair, CGS cannot be directly applied to its reasoning. Therefore, GPT-5.4 self-refinement based on CGS is not directly defined under our framework.

Q3: How does SRLA generalize across different solver backbones, and how statistically reliable are the reported improvements?

A3: > We address backbone generalization and statistical reliability separately.

Generalization Across Solver Backbones. Beyond our primary solver, Qwen3.5-9B, we have also evaluated SRLA using two domain-specific 7B models: Qwen2.5-Math-7B-Instruct for mathematical reasoning and Qwen2.5-Coder-7B-Instruct for code generation. These results were already included in Tables 9 and 10 of Appendix C.1 in our original submission. For clarity, we summarize their performance alongside Qwen3.5-9B in Tables 2 and 3 below.

Table 2: SRLA Performance Across Mathematical Reasoning Backbones

Solver BackboneAIME 2024AIME 2025AIME 2026AMC23MinervaMath
Qwen2.5-Math-7B-Instruct40.0036.6736.6765.0058.09
Qwen3.5-9B80.0063.3080.0092.5071.32

Table 3: SRLA Performance Across Code Generation Backbones

Solver BackboneHumanEvalMBPPCodeContestsLC-EasyLC-MediumLC-Hard
Qwen2.5-Coder-7B-Instruct95.7375.4928.8083.0061.0028.00
Qwen3.5-9B98.1090.0047.0083.0065.0076.60

All values represent accuracy (%). Results are taken from Tables 1, 2, 9, and 10 of our original submission.

These results demonstrate that SRLA can be applied to both general-purpose and domain-specific solver backbones. Although absolute performance varies with the underlying solver's capabilities, SRLA improves the corresponding local baselines across the evaluated mathematical reasoning and code generation benchmarks.

Statistical Reliability. To assess the robustness of our results beyond the single-run evaluations in the original submission, we conduct an additional avg@8 evaluation on the three AIME benchmarks using eight different random seeds. Table 4 reports the mean accuracy and standard deviation across these runs.

Table 4: Statistical Reliability of SRLA Across Eight Runs on AIME Benchmarks

BenchmarkSRLA Accuracy (avg@8, Mean ± Std., %)
AIME 202477.08 ± 6.53
AIME 202560.00 ± 4.71
AIME 202672.92 ± 5.76

As shown in Table 4, the standard deviation ranges from 4.71 to 6.53 percentage points, reflecting variability due to solver sampling. Compared with the single-run GPT-5.4 results (76.67%, 53.30%, and 60.00%), SRLA achieves comparable mean accuracy on AIME 2024 and higher mean accuracy on AIME 2025 and AIME 2026 by approximately 6.7 and 12.9 percentage points, respectively.

Q4: Why does SRLA use targeted LLM feedback instead of directly routing difficult problems to GPT-5.4? What are the advantages in terms of accuracy and cost?

A4: To evaluate the advantage of targeted advice over direct routing, we conducted additional experiments on five mathematical reasoning benchmarks using two alternatives: (i) CGS + Direct GPT-5.4 Routing, which uses the same CGS gate as SRLA but sends triggered problems directly to GPT-5.4 for solving, and (ii) confidence-based cascades, following FrugalGPT and COREA, which defer problems to GPT-5.4 based on low SLM token log-probability or high entropy. Table 5 presents the accuracy and external API cost of these approaches.

Table 5: Accuracy and API Cost Comparison on Mathematical Reasoning

MethodAIME 2024AIME 2025AIME 2026AMC23MinervaMath
Confidence Cascade (Log-prob)80.00 / 0.270.00 / 0.370.00 / 0.292.50 / 0.6067.28 / 0.1
Confidence Cascade (Entropy)80.00 / 0.270.00 / 0.370.00 / 0.492.50 / 0.6067.65 / 0.1
CGS + Direct GPT-5.4 Routing83.33 / 0.366.67 / 0.473.33 / 0.592.50 / 0.7067.28 / 0.1
GPT-5.4 (Direct)76.67 / 1.4453.30 / 1.8460.00 / 1.8892.50 / 0.8876.50 / 0.47
SRLA (Ours)80.00 / 0.1463.30 / 0.1480.00 / 0.1492.50 / 0.4071.32 / 0.04

Each entry reports Accuracy (%) / External API Cost (¢ per problem).

SRLA and CGS-based direct routing achieve comparable accuracy across the benchmarks. Direct routing performs better on AIME 2024 (83.33% vs. 80.00%) and AIME 2025 (66.67% vs. 63.30%), while SRLA performs better on AIME 2026 (80.00% vs. 73.33%) and MinervaMath (71.32% vs. 67.28%). Both achieve the same accuracy on AMC23 (92.50%). However, SRLA is approximately 2–4× cheaper than direct routing on AIME, costing only 0.14¢ per problem compared with 0.3–0.5¢. SRLA also has lower API costs than both confidence-based cascades across all five benchmarks. These results demonstrate that SRLA's primary advantage is achieving competitive accuracy at substantially lower external API cost.

Overall, these additional comparisons demonstrate that SRLA's main advantage over direct routing is cost efficiency rather than consistently higher accuracy. SRLA achieves competitive accuracy while substantially reducing external API expenditure.

Q5: Does the training and computational overhead of CGS outweigh the API cost savings achieved by SRLA? How does the overall cost compare with direct GPT-5.4 inference?

A5: We would like to clarify that CGS is not a separately trained scoring model; rather, it is computed using the DPO-optimized SLM and its frozen SFT reference model. Furthermore, the cost comparisons reported in Tables 1–2 of our original submission account for external API costs only, rather than total end-to-end computational costs. We will relabel these tables accordingly. To provide a more comprehensive analysis, Table 6 below reports GPT output tokens, inference latency, and one-time training overhead, averaged across the five mathematical reasoning and six code generation benchmarks.

Table 6: API Cost, Inference Latency, and Training Overhead

MetricMath: Direct GPT-5.4Math: SRLACode: Direct GPT-5.4Code: SRLA
GPT output tokens (per call)*91211219957
External API cost (¢/problem)1.300.170.210.14
Average latency (s/problem)7.75332.3190
One-time training overheadNone~10 A100-hours + $11.76 data-construction costNone~8 A100-hours + $13.10 data-construction cost

As shown in Table 6, SRLA reduces external API costs by approximately 8× on mathematical reasoning (1.30¢ to 0.17¢ per problem) and 1.5× on code generation (0.21¢ to 0.14¢ per problem) by selectively invoking the advisor and generating concise corrective feedback instead of complete solutions. However, this comes at the expense of higher inference latency due to long SLM reasoning trajectories, CGS scoring, and additional refinement steps. SFT and DPO-CGS training, together with data generation, required approximately 18 A100 GPU-hours across both domains, while GPT-5.4 training-data construction incurred a one-time API cost of $24.86. These training costs can be amortized across subsequent queries, and all local experiments were conducted on institutional A100 GPUs.

Overall, SRLA's primary advantage is substantially lower external API expenditure, with comparable or higher accuracy on most benchmarks, at the expense of increased local computation and inference latency.

Q6: How does SRLA differ from existing model collaboration, uncertainty-based routing, and selective refinement methods? What are the novel contributions of SRLA?

A6: We first describe how SRLA differs from prior work, then compare it empirically with two alternatives on five mathematical reasoning benchmarks: (i) CGS + Direct GPT-5.4 Routing, which uses the same CGS gate as SRLA but sends triggered problems directly to GPT-5.4 for solving, and (ii) confidence-based cascades, following FrugalGPT [1] and COREA [2], which defer problems to GPT-5.4 based on low SLM token log-probability or high entropy.

Differences and contributions. Routing and cascade methods (e.g., FrugalGPT [1], COREA [2]) decide whether to defer a problem to a larger model, which then solves it from scratch. SRLA takes a different approach: the SLM remains the solver, and the frontier model is consulted only when the CGS signal indicates an unreliable trajectory, and then only to provide short, localized diagnostic feedback that the SLM uses to revise its own solution. The frontier model therefore never replaces the SLM's reasoning, which is what keeps external API cost low.

Table 7 compares SRLA with confidence-based cascades, CGS-based direct GPT-5.4 routing, and direct GPT-5.4 inference across five mathematical reasoning benchmarks, reporting accuracy and external API cost.

Table 7: Accuracy and API cost comparison on mathematical reasoning

MethodAIME 2024AIME 2025AIME 2026AMC23MinervaMath
Confidence Cascade (Log-prob)80.00 / 0.2070.00 / 0.3070.00 / 0.2092.50 / 0.6067.28 / 0.10
Confidence Cascade (Entropy)80.00 / 0.2070.00 / 0.3070.00 / 0.4092.50 / 0.6067.65 / 0.10
CGS + Direct GPT-5.4 Routing83.33 / 0.3066.67 / 0.4073.33 / 0.5092.50 / 0.7067.28 / 0.10
GPT-5.4 (Direct)76.67 / 1.4453.30 / 1.8460.00 / 1.8892.50 / 0.8876.50 / 0.47
SRLA (Ours)80.00 / 0.1463.30 / 0.1480.00 / 0.1492.50 / 0.4071.32 / 0.04

*Each entry reports accuracy (%) / external API cost (¢ per problem).

Accuracy. CGS-based direct routing is better than SRLA on AIME 2024 (83.33% vs. 80.00%) and AIME 2025 (66.67% vs. 63.30%), while SRLA is better on AIME 2026 (80.00% vs. 73.33%) and MinervaMath (71.32% vs. 67.28%). The confidence cascades match SRLA on AIME 2024 and AMC23, exceed it on AIME 2025 (70.00% vs. 63.30%), and fall below it on AIME 2026 (70.00% vs. 80.00%) and MinervaMath (67.28–67.65% vs. 71.32%). On AMC23, all methods reach 92.50%.

Cost. SRLA has the lowest external API cost on every benchmark. On AIME it costs 0.14¢ per problem, about 2–4× less than direct routing (0.30–0.50¢) and lower than both cascades (0.20–0.40¢). On AMC23 and MinervaMath it costs 0.40¢ and 0.04¢, versus 0.60–0.70¢ and 0.10¢ for the alternatives.

Overall, SRLA's main advantage over routing and cascades is cost efficiency at competitive accuracy.

[1] Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv preprint arXiv:2305.05176, 2023.

[2] Chuang Zhang, Zizhen Zhu, Yihao Wei, Bing Tian, Junyi Liu, Henan Wang, Wang Xavier, and Yaxiao Liu. Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2026.

Reviewer_fEuMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces SRLA, a selective collaboration framework in which a small language model performs the primary reasoning and queries a frontier LLM only when its trajectory is judged unreliable. SRLA first trains the local model to revise failed solutions using GPT-5.4-generated error localization and corrective feedback. It then applies DPO to aligned correct and incorrect reasoning chunks. At inference time, Chunk-level Guided Scoring (CGS), defined using the likelihood difference between the DPO policy and the frozen SFT reference, is averaged to gate advisor invocation, while the lowest-scoring chunk is supplied as a localization cue.

The method is evaluated on five mathematical-reasoning and six code-generation benchmarks, primarily using Qwen3.5-9B as the local solver and GPT-5.4 as the advisor, with additional experiments using Qwen2.5 domain-specific solvers. The authors report that SRLA matches or exceeds direct GPT-5.4 accuracy on nine of eleven benchmarks while substantially reducing external API expenditure.

Critical strengths
  1. Practically relevant and coherent system design. SRLA addresses an important deployment problem by keeping reasoning local while reserving frontier-model access for targeted diagnosis. The separation between solving, reliability estimation, advising, and local revision is intuitive and potentially useful for settings constrained by API cost, privacy, or service quotas.

  2. Broad empirical coverage with useful implementation detail. The evaluation covers eleven benchmarks across mathematics and code, and the appendix extends the study to two domain-specific 7B solvers. The paper also reports training-corpus construction, SFT and DPO hyperparameters, prompts, and external data-generation expenditure. It appropriately acknowledges the two benchmarks on which direct GPT-5.4 remains stronger.

Critical weaknesses
  1. The central CGS mechanism is neither directly validated nor experimentally isolated. The paper reports no advisor invocation rates, false-positive or false-negative rates, calibration, AUROC/AUPRC, risk-coverage curves, localization accuracy, or held-out threshold-selection procedure. It also lacks essential controls such as SFT-only, DPO without selective advice, always-advice, never-advice, random or entropy-based gating at matched call rates, and advice without the lowest-CGS chunk. Consequently, the experiments show that the complete system works, but not that CGS reliably detects errors or that selective localization causes the gains.

  2. The comparative cost-performance claim is not yet sufficiently supported. Baseline prompts, decoding settings, loop budgets, training procedures, and advisor-token budgets are not reported, while the closest selective collaboration methods discussed in Section 2 are absent from the evaluation. Results are presented as single point estimates without uncertainty or paired analysis, even for small differences. Moreover, “cost” refers only to external API expenditure and excludes local generation, likelihood evaluation under both the DPO and reference policies, up to three revision loops, training compute, latency, and amortization. The evidence therefore supports reduced external API spend under a self-hosted-model assumption, rather than lower end-to-end cost.

Questions for authors
  1. How were the CGS thresholds selected, and what are the per-benchmark invocation rates, false-positive/false-negative rates, localization accuracy, and accuracy-cost curves across different thresholds?

  2. Do the reported gains persist under matched-budget ablations, including SFT-only, DPO-only, always/never-advice, alternative confidence gates, and advice without the lowest-CGS chunk?

  3. Were all baselines evaluated with matched models, prompts, decoding settings, token and iteration budgets? How does SRLA compare with the closest selective-collaboration methods discussed in Section 2?

6 comments

We sincerely thank the reviewer for the detailed and constructive review. We especially appreciate the recognition of SRLA's practical and coherent system design, where the SLM performs the primary reasoning while the frontier LLM provides targeted guidance only when needed. We also appreciate the acknowledgment of our broad evaluation across eleven benchmarks, additional experiments with two domain-specific 7B solvers, and detailed reporting of training procedures and implementation settings. We address the reviewer's concerns and questions below.

Q1: How were the CGS thresholds selected, and what are the per-benchmark invocation rates, false-positive/false-negative rates, localization accuracy, and accuracy-cost curves across different thresholds?

A1: We thank the reviewer for these questions. To better assess CGS reliability, we provide additional analyses of advisor invocation rates, error detection, and accuracy–cost trade-offs across different thresholds. We will incorporate these analyses and clarify the limitations in the revised manuscript.

(a) Threshold Selection. SRLA uses one fixed CGS threshold per domain, rather than a benchmark-specific threshold: τ = 0.005 for the five mathematical reasoning benchmarks and τ = 0.002 for code generation. To select these thresholds without using evaluation data, we conduct held-out validation experiments on 300 MATH [1] Level 4–5 problems and 300 Magicoder [2] problems, respectively, all disjoint from the evaluation benchmarks. For mathematical reasoning, maximizing Youden's J statistic (TPR − FPR) identifies τ = 0.005* (95% bootstrap CI: [0.002, 0.007]), triggering advisor intervention on 28% of problems, with a TPR of 58.6% and an FPR of 25.1%. We follow the same validation procedure for code generation and select τ = 0.002. As further examined in Section (c), Table 2, increasing the threshold does not necessarily improve accuracy, despite increasing advisor invocation rates and external API costs. We acknowledge that threshold selection depends on the task and the desired accuracy–cost trade-off, and we will clarify this procedure in the revised manuscript.

(b) Advisor Invocation and Gate Error Rates. We evaluate CGS reliability by treating incorrect first-pass solutions as the positive class. The advisor is triggered when the average CGS score falls below 0.005. Table 1 reports the advisor invocation rates, AUROC, false-positive rates (FPR), and false-negative rates (FNR).

Table 1: CGS Reliability and Advisor Invocation Rates

BenchmarkAdvisor Called (%)CGS AUROCFPR (%)FNR (%)
AIME 202426.67 (8/30)0.857.6958.82
AIME 202520.00 (6/30)0.8611.1176.19
AIME 202630.00 (9/30)0.867.6952.94
AMC2357.50 (23/40)0.3660.6157.14
MinervaMath9.56 (26/272)0.549.0989.58

As shown in Table 1, CGS achieves consistently strong error-detection performance on the three challenging AIME benchmarks, with AUROC values of 0.85–0.86 and relatively low false-positive rates (7.69–11.11%). Across these benchmarks, the pooled CGS AUROC is 0.84 [0.76, 0.92], compared with 0.61 [0.48, 0.74] for token entropy, yielding a statistically significant paired improvement of +0.23 [+0.07, +0.40]. This demonstrates that CGS provides a stronger error-detection signal than conventional token-entropy-based confidence estimation on challenging AIME-style reasoning tasks. Across all five benchmarks, the paired AUROC difference is smaller (+0.01 [−0.09, +0.11]), indicating that the relative effectiveness of the two gating approaches varies across tasks. Overall, these findings support the effectiveness of CGS for identifying potentially incorrect reasoning trajectories and guiding selective advisor intervention.

(c) Accuracy–Cost Trade-off Across CGS Thresholds. We further examine how different CGS thresholds affect advisor invocation, error detection, accuracy, and external API cost. Table 2 compares SRLA with CGS-based direct GPT-5.4 routing across the pooled AIME 2024, 2025, and 2026 benchmarks (90 problems).

Table 2: Accuracy–Cost Comparison Across CGS Thresholds

Threshold (τ)Advisor Called (%)Error Recall (%)FPR (%)SRLA Accuracy / CostDirect Routing Accuracy / Cost
0.0025.610.32.060.0 / 0.03660.0 / 0.144
0.00525.648.77.874.4 / 0.14074.4 / 0.400
0.00826.751.37.875.6 / 0.19274.4 / 0.566
0.01243.371.821.677.8 / 0.31273.3 / 0.829
0.02064.494.941.276.7 / 0.46470.0 / 1.242

Each accuracy/cost entry reports Accuracy (%) / External API Cost (¢ per problem).

At the selected threshold of τ = 0.005, SRLA achieves 74.4% accuracy at 0.14¢ per problem, matching CGS-based direct routing at approximately 2.9× lower external API cost (0.40¢). As the threshold increases, more problems are flagged for advisor intervention, improving error recall but also increasing false positives and API costs. Accuracy does not consistently improve with additional interventions, highlighting the trade-off between detection reliability, intervention frequency, and inference cost.

(d) Error Localization Accuracy. To evaluate whether CGS provides useful error-localization information to the advisor, we conduct an additional study on 17 randomly selected incorrect first-pass solutions from all three AIME and AMC23 benchmarks. We use an independent LLM (GPT-6.1 Sol) to identify the first erroneous reasoning step and assess whether the lowest-CGS chunk overlaps with this independently identified error. In 13 out of 17 cases (76.47%), the lowest-CGS chunk overlaps the first error identified by GPT-6.1 Sol. These findings provide preliminary evidence that CGS can identify potentially erroneous reasoning segments and provide useful localization guidance to the advisor, supporting SRLA's targeted feedback mechanism.

Overall, these analyses indicate that CGS provides useful error-detection signals on complex reasoning such as AIME, although its effectiveness varies across datasets. They also illustrate how threshold selection affects advisor utilization and the accuracy–cost trade-off. We will clarify these findings and discuss the limitations of CGS-based gating in the revised manuscript.

[1] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021.

[2] Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: Empowering code generation with oss-instruct. arXiv preprint arXiv:2312.02120, 2023.

Q2: Do the reported gains persist under matched-budget ablations, including SFT-only, DPO-only, always/never-advice, alternative confidence gates, and advice without the lowest-CGS chunk?

A2: To further examine the contribution of individual SRLA components, we conduct additional ablations comparing SFT-only, DPO-only, never-advice, always-advice, and entropy-based confidence gating. All variants use the same solver and advisor backbones (where applicable), decoding configurations, and maximum refinement budget. For entropy-based gating, we match the advisor invocation rate of SRLA to ensure a comparable external API budget. Table 3 reports accuracy and external API cost across five mathematical reasoning benchmarks.

Table 3: Ablation Study on Mathematical Reasoning

MethodAIME 2024AIME 2025AIME 2026AMC23MinervaMath
BASE30.00 / 0.0023.33 / 0.0030.00 / 0.0075.00 / 0.0066.18 / 0.00
SFT-only30.00 / 0.0023.33 / 0.0033.33 / 0.0075.00 / 0.0066.91 / 0.00
DPO-only46.70 / 0.0030.00 / 0.0043.33 / 0.0082.50 / 0.0064.34 / 0.00
Never-advice43.33 / 0.0030.00 / 0.0043.33 / 0.0082.50 / 0.0064.71 / 0.00
Always-advice80.00 / 0.6870.00 / 0.7670.00 / 0.7092.50 / 0.7066.50 / 0.42
Confidence Cascade (Entropy)80.00 / 0.2070.00 / 0.3070.00 / 0.4092.50 / 0.6067.65 / 0.10
SRLA (Full)80.00 / 0.1463.30 / 0.1480.00 / 0.1492.50 / 0.4071.32 / 0.04

*Each entry reports Accuracy (%) / External API Cost (¢ per problem).

The results show that SFT-only, DPO-only and Never-advice do not achieve the performance of the complete SRLA framework, highlighting the benefit of selective external feedback. Although always-advice and entropy-based gating achieve higher accuracy on AIME 2025, SRLA consistently incurs lower external API costs while maintaining competitive accuracy, demonstrating the advantage of selective advisor intervention.

Q3: Were all baselines evaluated with matched models, prompts, decoding settings, token and iteration budgets? How does SRLA compare with the closest selective-collaboration methods discussed in Section 2?

A3: > We confirm that all baselines were evaluated using matched model backbones, decoding configurations, token limits, and refinement budgets to ensure a fair comparison. We used consistent task instructions while preserving the method-specific prompts required by each baseline.

Our original experiments compare SRLA against several refinement methods, including Self-Refine, Reflexion, CRITIC, REFINER, and RL-Tango. To further strengthen the comparison with selective-collaboration approaches, we additionally evaluate two alternatives: (i) CGS + Direct GPT-5.4 Routing, which uses the same CGS gate as SRLA but sends triggered problems directly to GPT-5.4 for solving, and (ii) confidence-based cascades, following FrugalGPT and COREA, which defer problems to GPT-5.4 based on low SLM token log-probability or high entropy. For confidence-based cascades, we match SRLA's advisor invocation rate on each benchmark to ensure comparable external API usage.

All additional comparisons use Qwen3.5-9B as the local solver and GPT-5.4 as the frontier model, with matched evaluation datasets and decoding configurations. Table 4 reports accuracy and external API cost across five mathematical reasoning benchmarks.

Table 4: Comparison with Selective-Collaboration Methods on Mathematical Reasoning

MethodAIME 2024AIME 2025AIME 2026AMC23MinervaMath
Confidence Cascade (Log-prob)80.00 / 0.2070.00 / 0.3070.00 / 0.2092.50 / 0.6067.28 / 0.10
Confidence Cascade (Entropy)80.00 / 0.2070.00 / 0.3070.00 / 0.4092.50 / 0.6067.65 / 0.10
CGS + Direct GPT-5.4 Routing83.33 / 0.3066.67 / 0.4073.33 / 0.5092.50 / 0.7067.28 / 0.10
GPT-5.4 (Direct)76.67 / 1.4453.30 / 1.8460.00 / 1.8892.50 / 0.8876.50 / 0.47
SRLA (Ours)80.00 / 0.1463.30 / 0.1480.00 / 0.1492.50 / 0.4071.32 / 0.04

Each entry reports Accuracy (%) / External API Cost (¢ per problem).

Comparison. SRLA achieves competitive accuracy while maintaining lower external API costs than all evaluated routing and confidence-based alternatives. CGS-based direct routing performs better on AIME 2024 and AIME 2025, while SRLA achieves higher accuracy on AIME 2026 and MinervaMath, with identical accuracy on AMC23. On the AIME benchmarks, SRLA is approximately 2–4× cheaper than CGS-based direct routing.

Reviewer_hw8EOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SRLA, a framework in which a small local model (Qwen3.5-9B) solves problems and a frontier model (GPT-5.4) is consulted only when a learned score flags the solution as unreliable. Training has two stages. Stage 1 is SFT on GPT-5.4-annotated corrections of the SLM's own failed trajectories. Stage 2 is chunk-level DPO. The gate (CGS) is the mean per-chunk log-likelihood ratio between the DPO policy and the SFT reference. When the gate fires, GPT-5.4 receives the problem, the full trajectory and the lowest-scoring chunk, returns an error span plus feedback, and the SLM re-solves. On 5 math and 6 code benchmarks, the authors report matching or beating GPT-5.4 on 9 of 11 benchmarks at up to 92–94% lower API cost.

Critical strengths

Deciding when a cheap model should ask for help is a relevant problem, and keeping the SLM responsible for the final answer is a clean framing.

Critical weaknesses

1. Practical value is unclear, and the most obvious baseline is missing. If the goal is to save cost, the natural approach is to route by difficulty or confidence: the SLM answers easy problems and hard ones go straight to GPT-5.4. The paper cites RouteLLM [1] and COREA [2] but compares against neither, nor against any cascade (e.g., FrugalGPT [3]). Without this comparison there is no evidence that "advise + re-solve" is better than simply deferring. On hard problems I would not expect advising to be cheaper than solving. The advisor reads the full SLM trajectory (up to 20k tokens), likely has to solve the problem itself to diagnose it, and can be called up to 3 times (Table 7).

2. The cost analysis is incomplete and cannot be verified.

  • Latency is ignored. One full SLM solve, two forward passes for CGS, an API call, then another full SLM solve, repeated for up to 3 loops. This is much slower than a single API call.
  • Local compute is ignored. The GPU cost of serving a 9B model, plus the reference model needed for CGS, is not counted.
  • The absolute savings are small. Direct GPT-5.4 costs about 1–2 cents per problem, and SRLA saves roughly one cent. That would take very large query volume to justify the extra training, serving and maintenance.

[1] Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M. Waleed Kadous, and Ion Stoica. RouteLLM: Learning to Route LLMs from Preference Data. In International Conference on Learning Representations (ICLR), 2025.

[2] Chuang Zhang, Zizhen Zhu, Yihao Wei, Bing Tian, Junyi Liu, Henan Wang, Wang Xavier, and Yaxiao Liu. Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2026.

[3] Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv preprint arXiv:2305.05176, 2023.

Questions for authors
  1. Routing baseline. Please add a baseline that uses the same gate but sends triggered problems directly to GPT-5.4 to solve. On the triggered subset, how do its accuracy and API cost compare with SRLA's "advise + re-solve"? If direct solving is cheaper or more accurate there, what is the advantage of the advisor design?

  2. Cost breakdown. For each benchmark, please report: the fraction of problems on which the advisor is called and the average number of loops.

4 comments

We thank the reviewer for recognizing the relevance of deciding when a smaller model should seek help and for highlighting the clear division of responsibilities between the SLM solver and the LLM advisor.

Q1: Why use targeted advice instead of routing CGS-triggered problems directly to GPT-5.4?

A1: To directly evaluate the benefit of targeted advice over full frontier-model delegation, we conduct additional experiments on five mathematical reasoning benchmarks using two alternatives: (i) CGS + Direct GPT-5.4 Routing, which uses exactly the same CGS gate as SRLA but sends triggered problems to GPT-5.4 for complete solutions, and (ii) confidence-based cascades, following FrugalGPT and COREA, which defer problems to GPT-5.4 based on low SLM token log-probability or high entropy. Table 1 compares their accuracy and external API costs with SRLA.

Table 1: Accuracy and External API Cost Comparison on Mathematical Reasoning

MethodAIME 2024AIME 2025AIME 2026AMC23MinervaMath
Confidence Cascade (Log-prob)80.00 / 0.2070.00 / 0.3070.00 / 0.2092.50 / 0.6067.28 / 0.10
Confidence Cascade (Entropy)80.00 / 0.2070.00 / 0.3070.00 / 0.4092.50 / 0.6067.65 / 0.10
CGS + Direct GPT-5.4 Routing83.33 / 0.3066.67 / 0.4073.33 / 0.5092.50 / 0.7067.28 / 0.10
GPT-5.4 (Direct)76.67 / 1.4453.30 / 1.8460.00 / 1.8892.50 / 0.8876.50 / 0.47
SRLA (Ours)80.00 / 0.1463.30 / 0.1480.00 / 0.1492.50 / 0.4071.32 / 0.04

Each entry reports accuracy (%) / external API cost (¢ per problem).

The additional experiments directly validate the benefit of SRLA's targeted advisor design. With exactly the same CGS gate, SRLA achieves comparable or higher accuracy than full GPT-5.4 delegation on 3 out of 5 benchmarks: higher accuracy on AIME 2026 (80.00% vs. 73.33%) and MinervaMath (71.32% vs. 67.28%), and identical accuracy on AMC23 (92.50%). Although direct routing performs slightly better on AIME 2024 and AIME 2025, SRLA reduces external API costs by approximately 2–4× across the three AIME benchmarks. SRLA also consistently incurs lower external API costs than both confidence-based cascades across all five benchmarks.

Importantly, these findings show that SRLA's advantage does not arise merely from routing difficult problems to GPT-5.4. Rather, they support the effectiveness of using the frontier model as a concise advisor that provides targeted corrective feedback while retaining the SLM as the primary solver, instead of delegating the entire problem to the frontier model. This design enables SRLA to maintain competitive reasoning accuracy while substantially reducing external API expenditure.

Q2: How does SRLA account for latency, local GPU costs, and training overhead?

A2: Our original claim and evaluation metric explicitly focus on reducing external API expenditure, which is the deployment cost targeted by SRLA. Unlike approaches that rely on frontier models to generate complete solutions, SRLA uses the frontier model selectively for concise corrective feedback while retaining the SLM as the primary solver. Nevertheless, to provide a more comprehensive evaluation, we additionally analyze local computation, inference latency, and one-time training overhead. We clarify that the cost comparisons in Tables 1–2 of our original submission report external API costs rather than total end-to-end computational costs, and we will update their labels accordingly. Table 2 below summarizes the additional measurements, averaged across five mathematical reasoning and six code generation benchmarks.

Table 2: External API Cost, Inference Latency, and Training Overhead

MetricMath: Direct GPT-5.4Math: SRLACode: Direct GPT-5.4Code: SRLA
GPT output tokens (per call)*91211219957
External API cost (¢/problem)1.300.170.210.14
Average latency (s/problem)7.75332.3190
One-time training overheadNone~10 A100 GPU-hours + $11.76 data-construction costNone~8 A100 GPU-hours + $13.10 data-construction cost

External API Cost Efficiency. As shown in Table 2, SRLA reduces external API expenditure by approximately 8× on mathematical reasoning (1.30¢ to 0.17¢ per problem) and 1.5× on code generation (0.21¢ to 0.14¢ per problem). These savings are achieved through SRLA's core design: using the frontier model for targeted advice rather than full solution generation. This directly supports our primary contribution of enabling cost-efficient collaboration between SLMs and frontier LLMs.

Local Computation and Training Overhead. SRLA performs additional local reasoning, CGS scoring, and refinement, resulting in longer inference latency than a single direct GPT-5.4 call. This reflects the framework's design trade-off: reducing paid frontier-model inference through increased local computation, rather than optimizing for minimum response latency. SFT and DPO-CGS training, together with data generation, required approximately 18 A100 GPU-hours across both domains, while GPT-5.4 training-data construction incurred a one-time API cost of $24.86. All local experiments were conducted on institutional A100 GPUs, and the one-time training overhead can be amortized across subsequent queries.

Overall, these additional measurements reinforce SRLA's central contribution: substantially reducing reliance on costly frontier-model inference while maintaining competitive reasoning accuracy. SRLA is particularly relevant to settings that prioritize external API cost efficiency and can accommodate additional local computation.

Q3: For each benchmark, what fraction of problems triggers an advisor call, and what is the average number of refinement loops?

A3: To clarify advisor utilization and refinement overhead, we report the percentage of problems that trigger an advisor call and the average number of solver passes per problem across all eleven benchmarks. Table 3 summarizes these statistics across the five mathematical reasoning and six code generation benchmarks.

Table 3: Advisor Invocation and Refinement Statistics Across All Benchmarks

DomainBenchmarkAdvisor Called (%)Avg. Solver Passes
MathAIME 202426.67 (8/30)1.53
MathAIME 202520.00 (6/30)1.40
MathAIME 202630.00 (9/30)1.60
MathAMC2357.50 (23/40)1.60
MathMinervaMath9.56 (26/272)1.10
CodeHumanEval3.05 (5/164)1.07
CodeMBPP23.74 (61/257)1.54
CodeCodeContests66.67 (44/66)2.73
CodeLeetCode Easy22.00 (22/100)1.62
CodeLeetCode Medium49.00 (49/100)2.19
CodeLeetCode Hard60.00 (60/100)2.50

For mathematical reasoning, advisor invocation rates range from 9.56% on MinervaMath to 57.50% on AMC23, with an average of 1.10–1.60 solver passes per problem. For code generation, invocation rates range from 3.05% on HumanEval to 66.67% on CodeContests, with an average of 1.07–2.73 solver passes per problem. The initial solution counts as one solver pass, so the average number of additional refinement loops ranges from 0.10–0.60 for math and 0.07–1.73 for code. These results demonstrate that advisor utilization and refinement overhead vary across benchmarks. We will include these statistics and clarify the loop-counting convention in the revised manuscript.

Thank you for your thoughtful response. I will increase my score accordingly.

Thank you for your feedback and for reconsidering your score. We appreciate your thoughtful review. We would also like to ask if you have any remaining concerns that we could address. We would be happy to provide further clarification.

Comments on the paper

1 comment

Dear reviewers,

We have now received the author responses to your raised questions and concerns. Please go through them. Also, have they answered your questions appropriately? Please also look other reviewers' strengths / concerns. Have they changed your perception of the paper? In all cases, please communicate with the authors. It is important that we let them know what relevant issues still exist or have been answered. Please remember we have limited time to get any further responses from them.

Regards,

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.