acceptodds
Withdrawn from ICLR 2027

Understanding Thinking-Mode Switch Strategies in Hybrid-Reasoning LLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Hybrid-reasoning large language models (LLMs) expose explicit controls over reasoning effort, allowing users or systems to trade off answer quality against inference cost. However, existing methods for adaptive thinking-mode selection are typically evaluated under different models, datasets, and implementation assumptions, making it difficult to compare their practical behavior. We introduce THINKSWITCH, a unified evaluation framework for studying thinking-mode switching in hybrid-reasoning LLMs. THINKSWITCH organizes the design space along two axes: three switching strategy families, prompt-based selection, external routing, and speculative execution, and four training regimes, training-free, SFT, offline and online RL, yielding 12 controlled evaluation settings. We evaluate these settings across 6 LLMs, from Qwen3.5-2B to Kimi-K2.5-1.1T, and 5 reasoning benchmarks covering mathematics, science, and code, while reimplementing 12+ representative prior methods within the same pipeline. Our analysis characterizes how different switching strategies occupy distinct effectiveness-efficiency trade-off regions: prompt-based methods often provide favorable token-accuracy trade-offs, routing methods offer more stable cost reduction, and speculative methods tend to improve accuracy at higher token cost. We further find that training affects strategies differently, and that the preferred strategy varies with model scale and task domain. THINKSWITCH provides reference implementations and a unified evaluation platform to support more controlled research on efficient reasoning in hybrid-reasoning LLMs. Our data, code and repository are available at https://anonymous.4open.science/r/ThinkSwitch.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Hybrid reasoning LLMs, Thinking mode switch, Benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pUUfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how LLMs switch between "think" and "no-think" modes. The authors claim that prior work are unclear which strategy really works. So they build THINKSWITCH, a unified framework that crosses three switch strategies with four training regimes. They do experiments on 6 models, 5 benchmarks and 527 runs in total. They also conduct 12 existing methods in the same pipeline. The main findings are that PT gives the best accuracy-token trade-off, RT gives moderate token savings, SPEC raises accuracy but costs more tokens. Training is helpful for token cost but not accuracy. The best strategy changes with model scale and task domain.

Critical strengths

The unified setup is useful. Prior works don't do experiments under the same settings. This paper makes the comparison possible.

The scale and scope of the experiments are strong. 6 models, 5 benchmarks, 12 own configurations, and 12 external methods is a lot of work. The findings on scale and domain shifts are interesting and useful.

Critical weaknesses

Some baselines use modified implementations. These changes may affect the results. Although the paper explains these differences, it is still unclear how much they affect the main findings.

Token counts alone cannot fully reflect actual inference cost and efficiency.

Questions for authors

How much do the modified baseline implementations affect the main results?

Can the authors report inference latency alongside token cost?

Reviewer_3VWNOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents THINKSWITCH, a framework for comparing how hybrid reasoning models choose their thinking mode. It studies prompt-based selection, routing, and speculative execution, with and without training. The experiments cover six models and five math, science, and code benchmarks. The paper reports that different strategies have different accuracy and token-cost trade-offs.

Critical strengths
  1. A common evaluation pipeline is useful because previous methods often use different models and cost accounting.

  2. The paper counts router calls and discarded speculative outputs in its token cost. This makes the comparison more informative.

  3. The appendix provides per-dataset results, an equal-budget comparison on MATH500, and some sensitivity checks. The authors also state where their reproductions differ from prior methods.

Critical weaknesses
  1. The average accuracy numbers need to be checked before the main conclusions can be trusted. Section 4.2 says that AVG is accuracy pooled over questions. Table 2 gives 500, 30, 198, 167, and 366 questions for the five benchmarks. Using these counts and the five accuracies in Table 3, I obtain 53.6% for Full-Think, but the table reports 40.9%. For PT-TF, I obtain 58.2%, but the table reports 47.6%. Even the simple, unweighted means are different: 50.9% and 57.6%. Please explain the aggregation and correct the affected tables and figures. This matters because the strategy ranking can change under the stated question-level pooling.

  2. The implemented speculative strategy differs from the paper’s mid-generation definition. Section 3 describes switching after a trigger during generation. Appendix F.7 instead says that the method generates a complete no-think answer, checks it, discards it if triggered, and then generates a new think answer. This two-pass design naturally costs more tokens. Please distinguish conclusions about this implementation from conclusions about speculative switching in general, or add a true mid-generation baseline.

  3. The full training comparison does not cover all six models. Appendix F.2 explains that SFT, DPO, and GRPO are tested only on three models, up to 36B parameters. This is a reasonable compute limit, but the abstract and contribution statements could more clearly separate the six-model training-free study from the three-model trained study.

  4. Some reproduced methods have substantial changes. For example, Appendix F.8 uses a prompt approximation for Budget-Guidance, replaces the classifier in Sketch-of-Thought with a domain rule, and cannot perform DEER’s real early exit. These choices are disclosed, which is good. Still, Figure 5 should be interpreted as a comparison of these implementations, not as a direct ranking of the original methods.

  5. The uncertainty evidence is limited for a broad ranking claim. The main comparison uses one greedy output per problem. The three-seed check covers only Qwen3.5-9B on AIME and GPQA. Please give paired uncertainty estimates for the main strategy comparisons, especially where accuracy differences are small.

Questions for authors
  • What exact formula produced every AVG Acc value in Tables 3, 4, and 6 and Figures 2–5?
  • How would a true mid-generation switch compare with the complete-response, two-pass speculative method?
  • Can the authors report wall-clock latency as well as output tokens?
Reviewer_MbtEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces THINKSWITCH, a unified taxonomy and evaluation pipeline for adaptive thinking-mode selection in hybrid-reasoning LLMs. It crosses three switching strategies (prompt-based, routing and speculative) with four training regimes (training-free, SFT, DPO and GRPO). The training-free settings are evaluated on six models (Qwen3.5-2B to Kimi-K2.5-1.1T), and the trained settings on three of them. Evaluation covers five math, science and code benchmarks. The study also re-implements 12 prior methods under shared token accounting and decoding conditions.

Critical strengths
  1. The paper’s main contribution is making existing methods comparable, rather than proposing yet another router. The token accounting in Section 4.2 charges router calls and discarded speculative generations in full, and Table 11 documents the reproduction deviations for every external method. This matters because, in this area, apparent efficiency gains can easily come from inconsistent treatment of auxiliary calls, retries and multi-sample decoding.

  2. The empirical analysis is broad enough to show that behavior depends on the strategy. Table 3 shows qualitatively different operating points on Qwen3.5-9B. Table 6 supports the claim that training mainly reduces token cost: RT-GRPO, for example, lowers the average cost from 14.3k to 5.0k tokens with similar pooled accuracy. Including models with three different native control interfaces (binary, discrete effort and numeric budget) is also a useful design choice, since it goes beyond evaluating a single custom think/no-think prompt.

Critical weaknesses
  1. Finding 1 states that PT “consistently” achieves Pareto-optimal trade-offs and that the win-win pattern is unique to PT. However, in Table 4, PT is below Full-Think in accuracy on gpt-oss-20B (32.9 vs 36.4) and Seed-OSS-36B (46.3 vs 51.5), and it uses more tokens on Qwen3.5-2B (29.2k vs 26.6k). RT-TF also improves both accuracy and cost over Full-Think on Qwen3.5-2B, Qwen3.5-9B, DeepSeek-V3.1 and Kimi-K2.5, so the win-win pattern is not unique to PT.

  2. The taxonomy and implementations depart from the paper’s own definitions. The PT family includes RASC (multi-sample self-consistency), DEER (early exit) and DynaThink (a fast pass followed by escalation, which matches the paper’s own definition of SPEC). Finding 8 attributes RT’s training gains to a binary route-or-not decision, while the routing judge in Appendix A.4 chooses among three modes. As a result, some strategy-level conclusions, such as PT spanning the widest Pareto frontier (Finding 11), may partly reflect how the methods were grouped rather than intrinsic properties of the strategies.

I would be willing to raise my score if the authors address these concerns

Questions for authors
  1. Given the counterexamples in Table 4, could the authors report, for PT, RT and SPEC respectively, on how many models each improves or degrades accuracy relative to Full-Think? This would clarify whether the intended conclusion is a general strategy-level regularity or a set of model-dependent observations.

  2. The abstract and conclusion describe 12 configurations evaluated across 6 models, whereas the trained configurations cover only three models. Will the authors revise these statements to make the evaluated settings explicit?

  3. What is the rationale for placing RASC, DEER and DynaThink under PT? Does Finding 11 still hold if these methods are regrouped or excluded?

  4. How does the two-pass implementation of SPEC in Appendix F.7 relate to the mid-stream switching defined in Eq. (3)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.