- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the cost–accuracy trade-off in reasoning-intensive tasks by proposing SRLA, a selective refinement framework that enables small language models (SLMs) to leverage frontier LLMs only when needed. SRLA introduces Chunk-level Guided Scoring (CGS), a lightweight adapter trained via direct preference optimization to identify unreliable reasoning segments without reference answers, triggering targeted LLM feedback only when necessary. Experiments on five mathematical reasoning and six code generation benchmarks show that SRLA matches or surpasses standalone GPT-5.4 on most benchmarks while reducing API costs by up to 92% and 94%, respectively.
Critical strengths
- Effective selective collaboration. SRLA dynamically determines when and where external guidance is needed, allowing SLMs to retain local reasoning autonomy while leveraging frontier LLMs for targeted error correction.
- Lightweight and targeted refinement. The proposed CGS identifies unreliable reasoning chunks without reference answers, enabling fine-grained feedback and avoiding unnecessary LLM calls.
- Good performance and cost efficiency. Experiments across mathematical reasoning and code generation demonstrate frontier-level performance with API cost reductions of up to 92% and 94%, respectively.
Critical weaknesses
- Reliability of Chunk-level Guided Scoring. Is the proposed Chunk-level Guided Scoring (CGS) mechanism reliable? The authors should conduct experiments to validate its effectiveness.
- Comparison with self-refinement using GPT-5.4. Would using GPT-5.4 to perform self-refinement based on CGS achieve better results? The authors should include this comparison to demonstrate the necessity of the proposed small-model–large-model collaboration framework.
- Limited backbone diversity and insufficient statistical validation. The authors should evaluate more backbone models rather than relying solely on Qwen3.5-9B as the local solver. Given the limited number of mathematical problems, the reported results may be subject to considerable variance; therefore, the authors should report averages over multiple runs or provide a rigorous robustness analysis.
- Unclear rationale for external guidance. When the local solver is likely to make an error, as indicated by a low CGS score, why should the advisor provide targeted feedback rather than solve the problem directly? The authors should justify the advantages of this design choice in terms of accuracy and cost.
- Insufficient analysis of overall cost and computational overhead. Does training CGS incur additional costs that outweigh the savings from reduced API calls? The authors should compare the monetary costs of training the scoring model and invoking external APIs to establish the overall cost-effectiveness of the proposed method.
- Limited novelty and insufficient comparison with prior work. Several existing studies have explored related approaches to model collaboration, uncertainty-based routing, and selective refinement. The authors should provide a more comprehensive discussion and systematic comparison with these works to clarify the novelty and advantages of the proposed method.
- MentorCollab: Selective Large-to-Small Inference-Time Guidance for Efficient Reasoning
- Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning
- Selective Deferred Routing: Enabling Cost-Efficient Collaboration between Local SLMs and Remote LLMs
- Language Model Cascades: Token-level Uncertainty and Beyond
- A Unified Approach to Routing and Cascading for LLMs
Questions for authors
Explain the concerns in Weaknesses part.
4 comments
Q3: How does SRLA generalize across different solver backbones, and how statistically reliable are the reported improvements?
A3: > We address backbone generalization and statistical reliability separately.
Generalization Across Solver Backbones. Beyond our primary solver, Qwen3.5-9B, we have also evaluated SRLA using two domain-specific 7B models: Qwen2.5-Math-7B-Instruct for mathematical reasoning and Qwen2.5-Coder-7B-Instruct for code generation. These results were already included in Tables 9 and 10 of Appendix C.1 in our original submission. For clarity, we summarize their performance alongside Qwen3.5-9B in Tables 2 and 3 below.
Table 2: SRLA Performance Across Mathematical Reasoning Backbones
| Solver Backbone | AIME 2024 | AIME 2025 | AIME 2026 | AMC23 | MinervaMath |
|---|---|---|---|---|---|
| Qwen2.5-Math-7B-Instruct | 40.00 | 36.67 | 36.67 | 65.00 | 58.09 |
| Qwen3.5-9B | 80.00 | 63.30 | 80.00 | 92.50 | 71.32 |
Table 3: SRLA Performance Across Code Generation Backbones
| Solver Backbone | HumanEval | MBPP | CodeContests | LC-Easy | LC-Medium | LC-Hard |
|---|---|---|---|---|---|---|
| Qwen2.5-Coder-7B-Instruct | 95.73 | 75.49 | 28.80 | 83.00 | 61.00 | 28.00 |
| Qwen3.5-9B | 98.10 | 90.00 | 47.00 | 83.00 | 65.00 | 76.60 |
All values represent accuracy (%). Results are taken from Tables 1, 2, 9, and 10 of our original submission.
These results demonstrate that SRLA can be applied to both general-purpose and domain-specific solver backbones. Although absolute performance varies with the underlying solver's capabilities, SRLA improves the corresponding local baselines across the evaluated mathematical reasoning and code generation benchmarks.
Statistical Reliability. To assess the robustness of our results beyond the single-run evaluations in the original submission, we conduct an additional avg@8 evaluation on the three AIME benchmarks using eight different random seeds. Table 4 reports the mean accuracy and standard deviation across these runs.
Table 4: Statistical Reliability of SRLA Across Eight Runs on AIME Benchmarks
| Benchmark | SRLA Accuracy (avg@8, Mean ± Std., %) |
|---|---|
| AIME 2024 | 77.08 ± 6.53 |
| AIME 2025 | 60.00 ± 4.71 |
| AIME 2026 | 72.92 ± 5.76 |
As shown in Table 4, the standard deviation ranges from 4.71 to 6.53 percentage points, reflecting variability due to solver sampling. Compared with the single-run GPT-5.4 results (76.67%, 53.30%, and 60.00%), SRLA achieves comparable mean accuracy on AIME 2024 and higher mean accuracy on AIME 2025 and AIME 2026 by approximately 6.7 and 12.9 percentage points, respectively.
Q4: Why does SRLA use targeted LLM feedback instead of directly routing difficult problems to GPT-5.4? What are the advantages in terms of accuracy and cost?
A4: To evaluate the advantage of targeted advice over direct routing, we conducted additional experiments on five mathematical reasoning benchmarks using two alternatives: (i) CGS + Direct GPT-5.4 Routing, which uses the same CGS gate as SRLA but sends triggered problems directly to GPT-5.4 for solving, and (ii) confidence-based cascades, following FrugalGPT and COREA, which defer problems to GPT-5.4 based on low SLM token log-probability or high entropy. Table 5 presents the accuracy and external API cost of these approaches.
Table 5: Accuracy and API Cost Comparison on Mathematical Reasoning
| Method | AIME 2024 | AIME 2025 | AIME 2026 | AMC23 | MinervaMath |
|---|---|---|---|---|---|
| Confidence Cascade (Log-prob) | 80.00 / 0.2 | 70.00 / 0.3 | 70.00 / 0.2 | 92.50 / 0.60 | 67.28 / 0.1 |
| Confidence Cascade (Entropy) | 80.00 / 0.2 | 70.00 / 0.3 | 70.00 / 0.4 | 92.50 / 0.60 | 67.65 / 0.1 |
| CGS + Direct GPT-5.4 Routing | 83.33 / 0.3 | 66.67 / 0.4 | 73.33 / 0.5 | 92.50 / 0.70 | 67.28 / 0.1 |
| GPT-5.4 (Direct) | 76.67 / 1.44 | 53.30 / 1.84 | 60.00 / 1.88 | 92.50 / 0.88 | 76.50 / 0.47 |
| SRLA (Ours) | 80.00 / 0.14 | 63.30 / 0.14 | 80.00 / 0.14 | 92.50 / 0.40 | 71.32 / 0.04 |
Each entry reports Accuracy (%) / External API Cost (¢ per problem).
SRLA and CGS-based direct routing achieve comparable accuracy across the benchmarks. Direct routing performs better on AIME 2024 (83.33% vs. 80.00%) and AIME 2025 (66.67% vs. 63.30%), while SRLA performs better on AIME 2026 (80.00% vs. 73.33%) and MinervaMath (71.32% vs. 67.28%). Both achieve the same accuracy on AMC23 (92.50%). However, SRLA is approximately 2–4× cheaper than direct routing on AIME, costing only 0.14¢ per problem compared with 0.3–0.5¢. SRLA also has lower API costs than both confidence-based cascades across all five benchmarks. These results demonstrate that SRLA's primary advantage is achieving competitive accuracy at substantially lower external API cost.
Overall, these additional comparisons demonstrate that SRLA's main advantage over direct routing is cost efficiency rather than consistently higher accuracy. SRLA achieves competitive accuracy while substantially reducing external API expenditure.
Q5: Does the training and computational overhead of CGS outweigh the API cost savings achieved by SRLA? How does the overall cost compare with direct GPT-5.4 inference?
A5: We would like to clarify that CGS is not a separately trained scoring model; rather, it is computed using the DPO-optimized SLM and its frozen SFT reference model. Furthermore, the cost comparisons reported in Tables 1–2 of our original submission account for external API costs only, rather than total end-to-end computational costs. We will relabel these tables accordingly. To provide a more comprehensive analysis, Table 6 below reports GPT output tokens, inference latency, and one-time training overhead, averaged across the five mathematical reasoning and six code generation benchmarks.
Table 6: API Cost, Inference Latency, and Training Overhead
| Metric | Math: Direct GPT-5.4 | Math: SRLA | Code: Direct GPT-5.4 | Code: SRLA |
|---|---|---|---|---|
| GPT output tokens (per call)* | 912 | 112 | 199 | 57 |
| External API cost (¢/problem) | 1.30 | 0.17 | 0.21 | 0.14 |
| Average latency (s/problem) | 7.7 | 533 | 2.3 | 190 |
| One-time training overhead | None | ~10 A100-hours + $11.76 data-construction cost | None | ~8 A100-hours + $13.10 data-construction cost |
As shown in Table 6, SRLA reduces external API costs by approximately 8× on mathematical reasoning (1.30¢ to 0.17¢ per problem) and 1.5× on code generation (0.21¢ to 0.14¢ per problem) by selectively invoking the advisor and generating concise corrective feedback instead of complete solutions. However, this comes at the expense of higher inference latency due to long SLM reasoning trajectories, CGS scoring, and additional refinement steps. SFT and DPO-CGS training, together with data generation, required approximately 18 A100 GPU-hours across both domains, while GPT-5.4 training-data construction incurred a one-time API cost of $24.86. These training costs can be amortized across subsequent queries, and all local experiments were conducted on institutional A100 GPUs.
Overall, SRLA's primary advantage is substantially lower external API expenditure, with comparable or higher accuracy on most benchmarks, at the expense of increased local computation and inference latency.
Q6: How does SRLA differ from existing model collaboration, uncertainty-based routing, and selective refinement methods? What are the novel contributions of SRLA?
A6: We first describe how SRLA differs from prior work, then compare it empirically with two alternatives on five mathematical reasoning benchmarks: (i) CGS + Direct GPT-5.4 Routing, which uses the same CGS gate as SRLA but sends triggered problems directly to GPT-5.4 for solving, and (ii) confidence-based cascades, following FrugalGPT [1] and COREA [2], which defer problems to GPT-5.4 based on low SLM token log-probability or high entropy.
Differences and contributions. Routing and cascade methods (e.g., FrugalGPT [1], COREA [2]) decide whether to defer a problem to a larger model, which then solves it from scratch. SRLA takes a different approach: the SLM remains the solver, and the frontier model is consulted only when the CGS signal indicates an unreliable trajectory, and then only to provide short, localized diagnostic feedback that the SLM uses to revise its own solution. The frontier model therefore never replaces the SLM's reasoning, which is what keeps external API cost low.
Table 7 compares SRLA with confidence-based cascades, CGS-based direct GPT-5.4 routing, and direct GPT-5.4 inference across five mathematical reasoning benchmarks, reporting accuracy and external API cost.
Table 7: Accuracy and API cost comparison on mathematical reasoning
| Method | AIME 2024 | AIME 2025 | AIME 2026 | AMC23 | MinervaMath |
|---|---|---|---|---|---|
| Confidence Cascade (Log-prob) | 80.00 / 0.20 | 70.00 / 0.30 | 70.00 / 0.20 | 92.50 / 0.60 | 67.28 / 0.10 |
| Confidence Cascade (Entropy) | 80.00 / 0.20 | 70.00 / 0.30 | 70.00 / 0.40 | 92.50 / 0.60 | 67.65 / 0.10 |
| CGS + Direct GPT-5.4 Routing | 83.33 / 0.30 | 66.67 / 0.40 | 73.33 / 0.50 | 92.50 / 0.70 | 67.28 / 0.10 |
| GPT-5.4 (Direct) | 76.67 / 1.44 | 53.30 / 1.84 | 60.00 / 1.88 | 92.50 / 0.88 | 76.50 / 0.47 |
| SRLA (Ours) | 80.00 / 0.14 | 63.30 / 0.14 | 80.00 / 0.14 | 92.50 / 0.40 | 71.32 / 0.04 |
*Each entry reports accuracy (%) / external API cost (¢ per problem).
Accuracy. CGS-based direct routing is better than SRLA on AIME 2024 (83.33% vs. 80.00%) and AIME 2025 (66.67% vs. 63.30%), while SRLA is better on AIME 2026 (80.00% vs. 73.33%) and MinervaMath (71.32% vs. 67.28%). The confidence cascades match SRLA on AIME 2024 and AMC23, exceed it on AIME 2025 (70.00% vs. 63.30%), and fall below it on AIME 2026 (70.00% vs. 80.00%) and MinervaMath (67.28–67.65% vs. 71.32%). On AMC23, all methods reach 92.50%.
Cost. SRLA has the lowest external API cost on every benchmark. On AIME it costs 0.14¢ per problem, about 2–4× less than direct routing (0.30–0.50¢) and lower than both cascades (0.20–0.40¢). On AMC23 and MinervaMath it costs 0.40¢ and 0.04¢, versus 0.60–0.70¢ and 0.10¢ for the alternatives.
Overall, SRLA's main advantage over routing and cascades is cost efficiency at competitive accuracy.
[1] Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv preprint arXiv:2305.05176, 2023.
[2] Chuang Zhang, Zizhen Zhu, Yihao Wei, Bing Tian, Junyi Liu, Henan Wang, Wang Xavier, and Yaxiao Liu. Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2026.
We sincerely thank the reviewer for their thoughtful evaluation of our work and for highlighting the strengths of SRLA, particularly its effective selective collaboration, lightweight and targeted refinement, and strong performance with reduced API costs. We appreciate the reviewer's constructive feedback and suggestions, which will help us improve the clarity and rigor of the paper. We address the reviewer's concerns in detail below.
A1: We evaluate CGS reliability in terms of error detection and discuss its use as a chunk-level localization cue for targeted advisor feedback. For error detection, we treat incorrect first-pass solutions as the positive class, with advisor intervention triggered when the average CGS score falls below the threshold of 0.005. Table 1 reports advisor invocation rates, AUROC, false-positive rates (FPR), and false-negative rates (FNR) across five mathematical reasoning benchmarks.
Table 1: CGS Reliability and Advisor Invocation Rates
Error Detection Reliability. As shown in Table 1, CGS achieves consistently high AUROC values of 0.85–0.86 across the three AIME benchmarks, with relatively low false-positive rates (7.69–11.11%). However, false-negative rates range from 52.94% to 76.19%, indicating that a substantial proportion of incorrect solutions remain undetected. Performance is weaker on AMC23 and MinervaMath, with AUROC values of 0.36 and 0.54, respectively. AMC23 exhibits a high false-positive rate (60.61%), leading to frequent advisor interventions, while MinervaMath has a high false-negative rate (89.58%), indicating that most incorrect solutions are not flagged.
Error Localization. To evaluate whether CGS provides useful error-localization information to the advisor, we conduct an additional study on 17 randomly selected incorrect first-pass solutions from all three AIME and AMC23 benchmarks. We use an independent LLM (GPT-6.1 Sol) to identify the first erroneous reasoning step and assess whether the lowest-CGS chunk overlaps with this independently identified error. In 13 out of 17 cases (76.47%), the lowest-CGS chunk overlaps the first error identified by GPT-6.1 Sol. These findings provide preliminary evidence that CGS can identify potentially erroneous reasoning segments and provide useful localization guidance to the advisor, supporting SRLA's targeted feedback mechanism.
Overall, these results suggest that CGS provides a useful error-detection signal on complex reasoning tasks, but its reliability varies across benchmarks.
A2: We thank the reviewer and would like to clarify the scope of SRLA. Our goal is to improve the performance of small language models (SLMs) while substantially reducing reliance on frontier models, at a fraction of their API cost: SRLA keeps reasoning local and queries GPT-5.4 only when the SLM's trajectory appears unreliable. A GPT-5.4 self-refinement pipeline uses the frontier model on every problem, which is precisely the dependency SRLA is designed to avoid, so it addresses a different problem; we include direct GPT-5.4 inference in Table 1 and 2 as the frontier-model reference and compare SRLA with it on both accuracy and cost.
Moreover, CGS is specific to the SLM, as it is computed from the likelihood difference between its DPO-optimized policy and frozen SFT reference, averaged over reasoning chunks. Since GPT-5.4 does not have an equivalent accessible policy–reference pair, CGS cannot be directly applied to its reasoning. Therefore, GPT-5.4 self-refinement based on CGS is not directly defined under our framework.