- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies overthinking in reasoning language models and explores reasoning termination strategies. The authors first introduce Step-wise Truncation Probing (STP), which samples answers from different prefixes of a fixed reasoning trajectory to estimate how answer accuracy evolves with additional reasoning. In addition, this paper formulates three reasoning termination strategies, including dynamic routing, fixed budgets, and dynamic early exit, within a budget-constrained optimization framework. By optimizing stopping decisions using complete STP measurements, they construct strategy-specific token–accuracy Pareto frontiers. The experiments show that the proposed method could improve reasoning efficiency.
Critical strengths
- The proposed framework provides a perspective on examining the potential accuracy–efficiency trade-offs, which is an interesting problem for reasoning models.
- The STP provides an approach to analyzing the effects of reasoning length on accuracy by investigating multiple truncation positions along the same sampled trajectory.
- The paper organizes dynamic routing, fixed budgets, and dynamic early exit within a unified optimization framework.
Critical weaknesses
- Equation (1) selects the earliest position achieving the maximum observed accuracy across all probed positions of a trajectory. Since each accuracy estimate is based on only 16 sampled answers, the maximum could show fluctuations. As a result, it could have selection bias.
- The oracle Pareto frontiers may be optimistic because the same measurements are used for action selection and performance evaluation. The oracle may select stopping positions that happen to have favorable sampling noise.
- The oracle frontiers rely on complete reasoning trajectories and repeated correctness measurements at every candidate stopping position. Such information is unavailable to a practical termination algorithm when deciding whether to stop. The authors should include stronger online baselines or experiments evaluating attainable performance with limited observation.
- The main cost metric considers retained reasoning tokens but excludes answer-generation costs, feature extraction, stopping-signal computation, and other inference overhead.
- There is a typo in Figure 1, "acciracy" -> accuracy.
Questions for authors
Please see the weaknesses.
4 comments
W2: The same measurements are used for action selection and performance evaluation, and the oracle may select stopping positions that happen to have favorable sampling noise. Could the oracle Pareto frontiers be optimistic?
Yes: at 16 probes the original frontier is optimistic. We measured by how much, and we also tested the natural remedy of scoring the selected actions on independent answers. Independent scoring errs in the other direction, by a similar or larger amount, and its total error is higher. We therefore keep the original estimator and will report its bias.
What is being estimated. For each strategy class, the frontier is the highest accuracy the class can reach at a given thinking budget. An estimator can miss this value in two ways: by scoring the selected actions too favourably, or by selecting actions that are not the best ones.
Experiment. We used 16 trajectories from GPT-OSS-20B and DeepSeek-R1-Distill-Qwen-7B on the four benchmarks: two questions for every model and benchmark, one trajectory per question. At every position we collected about 2,048 answers; the frontier computed from them is the reference. In each of 8,192 repetitions we drew 16 answers per position without replacement and built the frontier in three ways, each with the same 16 answers per position:
- Original 16: all 16 answers select the policy and score it.
- 8+8: eight answers select the policy, the other eight score it.
- Rotating 15+1: fifteen answers select, one scores; averaged over the 16 choices of the held-out answer.
In all three, the full policy is re-selected under the constraints of the strategy class. Results average the eight model–benchmark groups. To fit the rebuttal period we used one trajectory per question; FB and DEE then allow the same actions and coincide, so the gap between strategy classes is FB/DEE accuracy minus DR accuracy.
Results at a budget of 50% of the full-thinking tokens. Accuracies in %, gaps and errors in pp. Bias is the mean estimate minus the reference; SD is the variation over repetitions.
| Quantity | Estimator | Reference | Mean estimate | Bias | SD | RMSE |
|---|---|---|---|---|---|---|
| DR accuracy | Original 16 | 58.26 | 58.26 | 0.00 | 1.39 | 1.39 |
| DR accuracy | 8+8 | 58.26 | 57.74 | −0.52 | 2.15 | 2.21 |
| DR accuracy | Rotating 15+1 | 58.26 | 58.06 | −0.20 | 1.41 | 1.42 |
| FB/DEE accuracy | Original 16 | 70.84 | 75.25 | +4.41 | 1.55 | 4.67 |
| FB/DEE accuracy | 8+8 | 70.84 | 63.46 | −7.38 | 3.91 | 8.35 |
| FB/DEE accuracy | Rotating 15+1 | 70.84 | 66.36 | −4.48 | 3.87 | 5.92 |
| Gap FB/DEE − DR | Original 16 | 12.58 | 16.99 | +4.41 | 1.93 | 4.82 |
| Gap FB/DEE − DR | 8+8 | 12.58 | 5.72 | −6.86 | 4.14 | 8.01 |
| Gap FB/DEE − DR | Rotating 15+1 | 12.58 | 8.30 | −4.28 | 3.97 | 5.84 |
The original FB/DEE frontier is 4.41 pp too high. The held-out frontiers are 7.38 and 4.48 pp too low, and their RMSE is higher (8.35 and 5.92 versus 4.67 pp). Independent scoring removes the favourable scoring, but it cannot recover a better policy that was missed during selection, and setting answers aside for scoring leaves fewer for selection. A lower estimate is therefore not a more accurate one. The DR frontier is hardly affected by the choice of estimator, because DR only chooses between the two endpoints of each question.
The gap between strategy classes behaves in the same way: 16.99 pp with the original estimator, 5.72 and 8.30 pp with the held-out estimators, 12.58 pp for the reference. The gap stays positive under every estimator, including the conservative held-out ones. For FB and DEE separately, on the full benchmark with 16 trajectories per question, see our response to Reviewer bSdV, Q1 (part 3).
Figure A (anonymous link) shows the mean estimate, SD and RMSE of the three estimators at all budgets; the pattern at 50% holds over most of the range. We will add this analysis to the paper and state the upward bias of the frontiers at 16 probes next to the frontier results.
W3: The oracle frontiers rely on complete reasoning trajectories and repeated correctness measurements at every candidate stopping position, and such information is unavailable to a practical termination algorithm when deciding whether to stop. Could you include stronger online baselines or experiments evaluating attainable performance with limited observation?
We agree that no online method has the information the oracle uses; the frontier is a reference for what a class of stopping decisions allows, not a claim about what is attainable online. What is attainable with limited observation is measured by the practical methods themselves, and the paper had only one per class. We have added six recent methods, two per class, and now compare nine methods with the frontier of their class. Some of the added methods improve on the original ones, but all nine remain below their frontier. Apart from one fixed-budget method (TALE), the gaps are largest for dynamic early exit.
Methods. Methods added in the rebuttal are marked *.
- Dynamic routing (DR): ThinkSwitcher, Scout Before You Route *, CARROT (KNN) *
- Fixed budget (FB): Predictive Scheduling, ThoughtTerminator *, TALE *
- Dynamic early exit (DEE): ThinkBrake, RCPD *, LYNX *
Information available to the methods. DR and FB methods decide from the question alone. DEE methods are replayed along the trajectory and see only the current and earlier prefixes. No method sees test correctness, later parts of the trajectory or the final thinking length. Learned components are fitted on training data, operating parameters are chosen on development data (highest accuracy, then fewest thinking tokens), and test correctness is used only to score the decisions. Predictive Scheduling uses the corrected supervision (our response to Reviewer bSdV, Q3).
Setup. GPT-OSS-20B and DeepSeek-R1-Distill-Qwen-7B on the same 16 test questions from each of AIME, MATH500, GPQA-Diamond and LiveCodeBench, with 16 trajectories per question and 16 answers per position (2,048 test trajectories). Benchmarks have equal weight; learned methods are averaged over three seeds.
Results at the development-selected operating points. Cost is the retained thinking tokens as a percentage of full thinking, computed per benchmark and then averaged. "Frontier" is the accuracy of the class frontier at the same cost as the method, matched within each benchmark and seed before averaging; the gap is frontier minus method. G: GPT-OSS-20B; D: DeepSeek-R1-Distill-Qwen-7B.
| Class | Method | G: cost (%) | G: accuracy (%) | G: frontier (%) | G: gap (pp) | D: cost (%) | D: accuracy (%) | D: frontier (%) | D: gap (pp) |
|---|---|---|---|---|---|---|---|---|---|
| DR | ThinkSwitcher | 100.0 | 79.56 | 80.04 | 0.48 | 100.0 | 59.92 | 62.04 | 2.12 |
| DR | Scout Before You Route * | 99.7 | 79.53 | 80.04 | 0.51 | 84.1 | 60.20 | 61.97 | 1.77 |
| DR | CARROT (KNN) * | 100.0 | 79.56 | 80.04 | 0.48 | 86.7 | 61.01 | 61.80 | 0.79 |
| FB | Predictive Scheduling | 98.3 | 79.52 | 81.41 | 1.90 | 98.6 | 59.92 | 64.65 | 4.74 |
| FB | ThoughtTerminator * | 95.0 | 79.13 | 81.32 | 2.20 | 95.7 | 59.84 | 64.65 | 4.82 |
| FB | TALE * | 28.3 | 56.15 | 68.16 | 12.01 | 29.3 | 48.90 | 60.39 | 11.49 |
| DEE | ThinkBrake | 99.5 | 79.45 | 86.39 | 6.94 | 100.0 | 59.92 | 71.14 | 11.22 |
| DEE | RCPD * | 99.5 | 79.47 | 86.39 | 6.92 | 97.6 | 59.89 | 71.14 | 11.25 |
| DEE | LYNX * | 92.3 | 78.35 | 86.39 | 8.04 | 84.2 | 56.82 | 71.14 | 14.32 |
On these test questions, full-thinking accuracy is 79.56% and 59.92%. Most selected settings are close to full thinking because the selection rule puts accuracy first, and for these two models full thinking is close to the most accurate setting of most methods. The methods do stop earlier at other settings: in Figure C, most methods' tested settings range from little thinking to nearly full thinking, and on DeepSeek-R1-Distill-Qwen-7B several of them exceed full-thinking accuracy.
What the comparison shows.
- Some added methods improve the trade-off. On DeepSeek-R1-Distill-Qwen-7B, CARROT reaches 61.01% at a cost of 86.7%, whereas ThinkSwitcher stays at full thinking (59.92%).
- Routing methods are close to their frontier (gaps of 0.48 to 2.12 pp), but that frontier is itself the lowest of the three.
- The largest unused potential is in dynamic early exit (gaps of 6.92 to 14.32 pp). Under rotating 15+1 held-out scoring on the full benchmark, the accuracy ceilings of the three frontiers keep the same order (see our response to Reviewer bSdV, Q1, part 3).
Figure C (anonymous link) shows all tested settings of the nine methods against the frontier of their class. The frontiers in the figure use a common relative budget cap for the four benchmarks, so distances differ slightly from the matched gaps in the table.
We will add these methods and curves to the paper.
W4: The main cost metric considers retained reasoning tokens but excludes answer-generation costs, feature extraction, stopping-signal computation, and other inference overhead. Could these costs also be included in the metric?
We have added the cost of answer generation, which can be measured independently of any particular method: in all eight settings we examined, the thinking tokens saved far exceed the additional answer tokens. The other overheads depend on how a specific method is implemented, so we keep them outside the common metric and state this scope explicitly.
Why the metric counts retained thinking tokens. OTBench compares what three classes of stopping decisions (DR, FB, DEE) allow. Retained thinking tokens are defined in the same way for every method in every class. Feature extraction, predictor inference and stopping-signal computation differ from one method and implementation to another, not from one class to another; including them would turn the comparison of decision classes into a comparison of implementations. Our efficiency statements are about retained thinking tokens, not about latency or FLOPs. The OTBench section and Appendix A.4 of the paper state that the metric counts retained thinking tokens and excludes these overheads; we will repeat it where the results are discussed.
Answer-generation cost. Answers generated after an early stop can be longer than answers after full thinking. We measured this on GPT-OSS-20B and DeepSeek-R1-Distill-Qwen-7B with 64 questions from each of the four benchmarks, one trajectory per model and question (512 trajectories), b = 256 and 16 answers per position. The answer cost at a position is the mean length of its 16 answers.
For each strategy class we consider two operating points on its frontier: the smallest budget that reaches full-thinking accuracy, and the smallest budget that reaches the highest accuracy of the class. The stopping decisions and accuracies are kept as they are, and the cost of one answer is added at each selected position. With one trajectory per question, FB and DEE allow the same actions and coincide.
The table gives the change relative to full thinking, in mean tokens per question, with equal weight for the four benchmarks. A minus sign is a saving, a plus sign an increase.
| Model | Class | Operating point | Thinking tokens | Answer tokens | Total tokens |
|---|---|---|---|---|---|
| GPT-OSS-20B | DR | Reaches full-thinking accuracy | −2549 | +82 | −2467 |
| GPT-OSS-20B | DR | Reaches class maximum | −2054 | +55 | −1999 |
| GPT-OSS-20B | FB/DEE | Reaches full-thinking accuracy | −3345 | +132 | −3213 |
| GPT-OSS-20B | FB/DEE | Reaches class maximum | −2272 | +106 | −2166 |
| R1-Distill-Qwen-7B | DR | Reaches full-thinking accuracy | −7005 | +452 | −6553 |
| R1-Distill-Qwen-7B | DR | Reaches class maximum | −5889 | +358 | −5530 |
| R1-Distill-Qwen-7B | FB/DEE | Reaches full-thinking accuracy | −7979 | +673 | −7306 |
| R1-Distill-Qwen-7B | FB/DEE | Reaches class maximum | −5766 | +520 | −5246 |
Entries are rounded independently. For reference, full thinking plus one answer costs 5,570 tokens per question for GPT-OSS-20B and 9,137 for DeepSeek-R1-Distill-Qwen-7B.
Answers do become longer after an early stop, by 55 to 673 tokens, but this offsets only 3% to 9% of the thinking tokens saved. For example, at the FB/DEE budget that reaches full-thinking accuracy, the total saving is 3,213 of 5,570 tokens for GPT-OSS-20B and 7,306 of 9,137 for DeepSeek-R1-Distill-Qwen-7B. Including the answer cost therefore does not change the conclusions drawn from the retained-token metric. We will add this analysis to the appendix.
Typo. Thank you also for spotting "acciracy" in Figure 1; it is corrected in the revision.
W1: Each accuracy estimate is based on only 16 sampled answers, and the earliest position achieving the maximum observed accuracy is selected across all probed positions of a trajectory. Do fluctuations in the maximum and the resulting selection bias affect the OR and AD estimates?
Yes, they do, and we have now measured the effect by comparing the 16-probe estimates with reference curves built from about 2,048 answers per position on 16 trajectories. The findings are:
The details are in our response to Reviewer bSdV, Q1 (five parts): the comparison of the three estimators with bias, SD and RMSE, the estimates at every doubling of the probe count from 16 to 1,024, the posterior estimator, and the recomputed main table. We will add the posterior estimator and the error analysis to the paper.