acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Overthinking: Effects of Run-to-Run Instability in Large Reasoning Models

Abstract

Large Reasoning Models solve complex tasks by generating intermediate steps before outputting a final answer. However, they often exhibit a failure mode known as *overthinking*, intuitively defined as generating excessively long reasoning chains for a problem the model could otherwise solve concisely. The literature traditionally diagnoses this by checking whether a single reasoning trace exceeds a fixed token budget. This single-trajectory view ignores the complexity of stochastic sampling. To overcome this limitation, we generate 100 independent runs per problem, revealing high variance in trace length. We reframe overthinking as *instability* and introduce a difficulty-aware, threshold-free framework to measure it. We define a model-relative measure of *empirical difficulty*, a new overthinking-quantifying metric, and we use a variance decomposition to isolate within-problem instability from between-problem difficulty. Applying this framework across three datasets (MATH500, GSM8K, DUMB500 Math) and seven models (Gemma4 and Qwen3, from B to B parameters), we find that models think more than they *could* and *should* in comparison to their ability. We show that runs longer than usual for their problem are systematically less likely to be correct, and for some models the relationship between accuracy and relative length takes an inverted-U shape. This instability does not scale monotonically with parameter count, and it concentrates on easy-to-medium problems before collapsing on the hardest ones. We suggest that future evaluation protocols report this run-to-run instability alongside mean length and accuracy.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Large Reasoning Models, Overthinking

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_BpnMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues overthinking should be studied not via accuracy and average token length of reasoning traces, but with respect to instability of decoding. Then the authors introduce several metrics to quantify overthinking.

Critical strengths

I agree with the general idea: only looking at Acc, token length is not the best way to quantify reasoning efficiency. However, this is the norm currently in the community for reasoning efficiency.

Critical weaknesses
  1. No intuition. The paper starts right away by defining several metrics. There is no motivation for why the authors choose these specific metrics over other choices.

  2. Limited insights. The paper evaluates the framework with seven models on three math benchmarks, but provides little insight. It is still unclear why one should adopt this framework over traditional accuracy and token-count metrics. The good thing about accuracy and token count is that they provide an easy way to compare checkpoints: at the same accuracy, if one checkpoint uses fewer tokens, it is clearly better. However, the several metrics introduced in the paper do not seem as interpretable or meaningful.

  3. Expensive. To reliably estimate the introduced metrics, the authors generate 100 samples per problem. That is very expensive on hard benchmarks such as AIME or LiveCodeBench. This is probably why the framework’s main experiments are limited to easier benchmarks such as MATH500 and GSM8K.

Questions for authors

I seem unable to draw any conclusions from the results in Table 1, and I would appreciate it if the authors could clarify how to interpret them. For example, in Section 5.2, the authors argue that “Instability costs accuracy, not only compute, since on the same problem a run that goes unusually long is more likely to be wrong,” based on the value of W-corr. But for many models, the values are around −0.1 or even smaller in magnitude. These negative correlations seem too weak for the authors to draw such a concrete conclusion.

Also, one suggestion that I think could make this framework more interesting: there are several early-stopping and post-training methods for reasoning efficiency ([1,2,3,4] and several others) that are evaluated mainly through accuracy and token length, but these metrics do not explain exactly how the methods differ. Applying this framework to these methods could provide more insight into how each method works differently.

[1] Yang, Chenxu, et al. "Dynamic Early Exit in Reasoning Models."

[2] Hosseini, Parsa, et al. "Early Stopping for Large Reasoning Models via Confidence Dynamics."

[3] Zhao, Anhao, et al. "On-policy supervised fine-tuning for efficient reasoning."

[4] Arora, Daman, and Andrea Zanette. "Training language models to reason efficiently."

Reviewer_2hD9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper reframes overthinking as variability in reasoning length across repeated generations of the same problem. Introduces distributional metrics and reports that unusually long runs are often less accurate, particularly on easier problems.

Critical strengths

The repeated-sampling design addresses a genuine deficiency in single-trajectory evaluations. Within-problem comparisons help separate problem-level heterogeneity from run-level variation, and the variance decomposition offers useful descriptive information.

Critical weaknesses

The foundational identification of overthinking with instability is the most questionable element. A model can be consistently wasteful without being unstable, or variable without wasting computation. The overthinking ratio compares successful runs with their shortest successful decile, making values above one largely inevitable; negative length–accuracy correlations do not establish harmful overthinking or second-guessing.

Questions for authors

Why, really, should variance in reasoning length constitute overthinking rather than simply stochastic variation? Can the authors demonstrate, through interventions or early stopping, that the supposedly excessive reasoning is genuinely unnecessary and that reducing it improves efficiency without sacrificing accuracy?

Reviewer_TBCKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies overthinking through variation in reasoning length across repeated generations of the same problem. It introduces an empirical difficulty measure, an overthinking ratio, and several instability statistics, including a decomposition of within-problem and between-problem length variance. Across seven models and three mathematical datasets, the authors examine the relationships among instability, correctness, difficulty, and model scale, and propose a repeated-sampling evaluation protocol.

Critical strengths
  1. A useful and practical evaluation perspective. Measuring the distribution of reasoning length provides information that mean length and accuracy can conceal, particularly concerning occasional extremely long generations.
  2. Substantial sampling and attention to statistical uncertainty. The study uses 100 generations per problem and includes robust dispersion measures, cluster-bootstrap confidence intervals, and corrections for multiple comparisons. These are meaningful strengths for an empirical evaluation paper.
Critical weaknesses
  1. The additional insight beyond existing observations is not sufficiently clear. As acknowledged in the related-work discussion, shorter reasoning traces being more frequently correct and continued verification after reaching an answer have already been observed in prior studies. Repeated sampling and variance decomposition provide a useful characterization, but the paper does not clearly establish how they substantially advance our understanding of overthinking. The contribution would be stronger if the analysis revealed a previously overlooked mechanism, challenged an existing explanation, or led to a consequential change in evaluation practice.
  2. The analysis is primarily descriptive, while the proposed explanation remains insufficiently tested. The paper characterizes how length variability relates to correctness and difficulty, but does not adequately distinguish competing explanations, including unnecessary second-guessing, prolonged exploration following an incorrect step, and legitimate variation across solution paths. Appendix D shows that long correct traces often mention the answer early, but this does not establish that subsequent verification is unnecessary. Consequently, the study provides substantial statistical description without comparably strong evidence explaining why the observed instability occurs.
  3. The findings are not translated into a demonstrated improvement in mitigation or evaluation. For example, the framework could reveal failures of an existing mitigation that mean length and accuracy overlook, change the ranking of competing methods, or guide a targeted intervention.
  4. The boundary between beneficial additional reasoning and overthinking is insufficiently explored. The evaluated benchmarks are largely solved by the tested models, making them suitable for observing repeated verification but less informative about situations where extended reasoning enables recovery from mistakes or solves genuinely difficult problems. The paper does not sufficiently examine such counterexamples or identify conditions under which longer, more variable reasoning is beneficial.
Questions for authors

As shown in weaknesses.

Reviewer_hcjEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that overthinking in Large Reasoning Models is better understood as run-to-run instability in reasoning length than as a single trace crossing a fixed token limit. The authors generate 100 responses per problem for seven Gemma4 and Qwen3 models on DUMB500 Math, GSM8K, and MATH500. They then introduce several metrics: a model-relative difficulty score d(p)d(p), an Overthinking Ratio (OR), standard and robust coefficients of variation, a runaway-generation rate, an instability share ρ\rho based on variance decomposition, and a within-problem correlation between response length and correctness. Their main findings are that correct runs use 1.21–2.33 times as many tokens as the shortest correct decile, unusually long runs are less accurate for most models, instability does not scale monotonically with model size, and the reported instability share is lower on the hardest group of problems. The paper ends with a practical recommendation to evaluate reasoning models using repeated stochastic sampling, typically 25–50 generations per problem depending on the metric

Critical strengths
  1. The paper focuses on a real gap in current evaluation practice. Mean accuracy and mean response length can hide large differences between repeated generations of the same prompt. Looking at conditional, within-problem variation is useful for benchmarking, deployment, and evaluating methods that claim to reduce overthinking.

  2. The sampling budget is unusually substantial. With 100 generations per problem, the authors can study tails, outliers, within-problem correlations, and estimator stability in a way that would not be possible with only a handful of runs.

  3. Separating pooled and within-problem length-correctness correlations is a good choice. A pooled correlation mixes together two effects: harder problems tend to require longer responses, and they also tend to have lower accuracy. The within-problem statistic is a much cleaner attempt to isolate what happens when the same model answers the same prompt multiple times.

  4. The statistical treatment is stronger than what is common in this area. The paper includes bootstrap confidence intervals, multiple-comparison correction, and a subsampling analysis designed to estimate how many runs are needed for each statistic.

  5. The qualitative examples are useful. The long traces do not merely wander before finding the answer; in several cases, the model states the correct answer early and then spends most of the remaining trace checking, doubting, and re-deriving it. That gives the quantitative results a concrete interpretation.

  6. The paper is generally well organized. The progression from per-run quantities to per-problem quantities and then dataset-level summaries is easy to follow. Box 1 is also a useful attempt to turn the analysis into a practical evaluation recipe.

Critical weaknesses
  1. The main Overthinking Ratio result is partly built into the definition. OR divides the mean length of all correct runs by the mean length of the shortest correct decile. That makes �2��2� almost automatic, except in the degenerate case where correct runs all have nearly identical lengths. So the fact that OR is above one for every model and dataset is not, by itself, evidence of overthinking. It follows from comparing an overall mean with a lower-tail mean. The reported magnitudes may still be useful, but the paper goes too far when it treats �3��3� as the amount of reasoning the model “could and should” use. A very short correct response could be lucky, poorly justified, or correct only at the final-answer level. The authors should either present OR more cautiously as a lower-tail efficiency or dispersion ratio, or validate the denominator against something more meaningful, such as the first point where the correct answer appears, a minimally verified derivation, or a held-out set of runs. Cross-fitting the numerator and denominator on separate samples would also reduce dependence between them.

  2. The interpretation of �4��4� as expected tokens before the first correct answer is not mathematically justified. The paper defines $

d(p)=L10(p)WRs(p).d(p)=\frac{L_{10}(p)}{\mathrm{WR}_s(p)}.

$ But the actual expected compute used before obtaining a correct answer depends on the lengths of failed attempts as well as successful ones. Under independent attempts with success probability pp, the expected cost would involve both �6��6� and �7��7�. Dividing a lower-tail successful length by the win rate does not recover that quantity.

  1. A lower value of �8��8� on hard problems does not show that absolute instability has collapsed. The statistic is a ratio: $
ρ=within-problem variancetotal variance.\rho=\frac{\text{within-problem variance}}{\text{total variance}}.

$ If hard problems differ greatly in their average lengths, the between-problem term in the denominator can grow. That alone would push �9��9� downward, even if the absolute within-problem variance stayed constant or increased. The current analysis therefore supports a change in the share of variance attributed to instability, not necessarily a decline in instability itself. This matters because the paper repeatedly describes the result as instability “collapsing” on the hardest tasks and uses it to support a second-guessing explanation. The authors should report the numerator of �10��10� directly, along with CV, rCV, and runaway rate across difficulty groups. Difficulty and instability should also be estimated from separate rollout subsets, since the current terciles are built using �11��11�, which is calculated from the same generations later used to measure instability.

  1. The paper often uses causal language for an observational result. A negative within-problem correlation shows that longer runs and failures occur together. It does not establish that longer reasoning causes the failure, or that instability “costs” accuracy. Both could be consequences of a third event: the sampled trajectory becomes confused, enters a bad search path, or starts repeatedly revisiting earlier steps. The wording should be changed from “costs accuracy” to “is associated with lower accuracy” unless the authors add an intervention.

  2. The cross-model comparisons are confounded by different decoding settings. Qwen3 and Gemma4 are evaluated using family-specific recommended parameters rather than a common decoding policy. That means differences in CV, tail behavior, or runaway frequency may reflect temperature, top-�12��12�, or top-�13��13�, not just model size or training recipe. The paper currently attributes the pattern to training recipe without directly controlling for decoding entropy. At minimum, the claim should be narrowed. The authors can discuss within-family scaling trends, but they should avoid attributing family-level differences to training unless they run a shared decoding grid or entropy-match the sampling settings

  3. The novelty is meaningful but somewhat narrower than the framing suggests. Earlier work has already used repeated generations, studied non-determinism, and reported that shorter chains are often more accurate than longer ones. The paper itself acknowledges those strands of literature. The clearest new contribution is the combined evaluation framework: the variance decomposition, the set of complementary instability metrics, and the sample-budget analysis. That is useful. Still, the paper would be stronger if it demonstrated practical added value on an actual mitigation method—for example, a case where mean length and accuracy suggest improvement but the proposed instability metrics reveal that the tail has not changed.

  4. The sample-size recommendations are not fully consistent. Box 1 assigns 25 generations to CV, rCV, and runaway rate, while the appendix says that studies interested in the magnitude of CV or rCV should use at least 50. The runaway rate was also not included in the pairwise comparison experiment used to justify reduced budgets. The recommendations should distinguish three goals: estimating an absolute value, detecting the direction of a change, and estimating the size of that change. Those tasks do not require the same number of samples. The paper should also avoid calling �14��14� the “true” value in the subsampling study; it is simply the full-sample estimate available here.

  5. The experiments cover a narrow and mostly high-accuracy regime. All three datasets are mathematical, and the evaluated models solve most of them at very high rates. That limits what can be concluded about genuinely difficult reasoning tasks. Many prompts will have little or no within-problem variation in correctness, which makes the length-correctness relationship hard to estimate.

  6. “Threshold-free” is not quite accurate. The framework avoids a single global absolute token threshold, which is a useful distinction. But OR still uses a shortest-decile cutoff, and the runaway rate uses an explicit multiplier threshold �15��15�. “Problem-relative” or “free of a global token budget” would be more precise

Questions for authors
  1. How should OR be interpreted when values above one are largely guaranteed by its definition? OR compares the mean length of all correct runs with the mean of the shortest correct decile, so �0��0� follows almost automatically. What evidence shows that �1��1� represents how much reasoning the model “could and should” use, rather than a lucky lower-tail sample? Do the conclusions hold if the denominator is estimated on a separate set of runs or replaced with the position where the correct answer first appears?

  2. Can the authors justify the interpretation of �2��2� as expected tokens before a correct answer? The current formula uses the shortest-decile correct length divided by smoothed win rate, but it does not include the lengths of failed attempts. Would the main results remain unchanged under an actual expected-compute-to-success estimator, or under other reasonable definitions of problem difficulty?

  3. Does instability really fall on hard problems, or does only its share of total variance fall? Since �3��3� is a ratio, a lower value in the hard tercile could simply reflect greater variation in average length across hard problems. Could the authors report absolute within-problem variance, CV, rCV, and runaway rate for each difficulty tercile? It would also help to estimate difficulty and instability from separate rollout subsets.

  4. How sensitive are the cross-model conclusions to decoding settings? Gemma4 and Qwen3 are sampled with different family-specific settings, so differences attributed to scale or training recipe may partly come from temperature, top-�4��4�, or top-�5��5�. Do the non-monotonic scaling results persist under a shared decoding grid or entropy-matched sampling policy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.