acceptodds
Under review as a conference paper at ICLR 2027

Conserved Verdicts, Drifting Commitments: Identifying Harmful Overthinking

Abstract

When does further reasoning systematically reduce answer quality? We give an identification framework that separates this question from realized answer flips and changing sample composition. For every fixed continuation policy, the conditional probability of terminal correctness is a conserved verdict: its increments have zero conditional mean by the tower property, even under a policy that overthinks. The model's current commitment, defined by a specified answer-if-stopped-now operator, need not be conserved. We connect its deviation from the verdict to expected remaining drift and to the accuracy advantage of stopping, and decompose survivor-conditioned accuracy changes into within-cohort and termination-induced composition terms. On MATH-500, eight fresh samples for each of 367 problems produce 100/2936 early-correct/final-wrong 1.5B trajectories (3.4%; problem-clustered 95% CI [2.4%, 4.5%]). Across R1-distilled 1.5B/7B configurations, post-2048-token marginal declines of 8.3/4.4 percentage points decompose into positive within-cohort terms (8.5/20.0) and negative composition terms (-16.8/-24.4). A protocol-matched follow-up at 12 previously wrong-ending prefixes and 12 matched controls reproduces every immediate correct answer; continuation accuracy is 50.0% versus 97.9% (192 fresh continuations). This supports local expected loss at selected states, not a prevalence estimate. Probe-refit diagnostics leave the verdict control unresolved, and no tested stopping replay improves accuracy at lower token cost. The resulting audit separates behavioral erosion, survivorship, and measurement uncertainty within a 4096-token regime.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, reasoning, overthinking, test-time computation, martingales, survivorship bias, optimal stopping

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YSLtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tries to understand when more reasoning is needed / when more reasoning flips a correct answer wrong. The paper proposes an auditing / verdict (?) framework to detect when the answer would flip under more reasoning. The paper then verified the method on one dataset (Math500) and two very old deepseek models. Overall I don't quite understand what the paper is doing, there are too many concepts I've never learned about before: verdict conversatoin / harmonic reference / martingale

Critical strengths
  • The problem studied in the paper is somewhat important: Understanding when to stop the reasoning model is a problem widely studied in 2025.

  • The experimental setting is well documented.

  • There are lots of equations and theorems; Although I don't understand them, the method seems to be very theoratically-grounded.

Critical weaknesses
  • I am not sure who the target audience is: I have trouble understanding sections 3 and 4 as well as most of the paper. The paper seems to be more suitable for journals as the paper is very theory-heavy. I would not be surprised if reviewers would be tempted to just feed the draft into LLM for assistance in understanding.

  • The evaluation is limited to a single dataset Math500 and some old DeepSeek models from early 2025.

  • Some empirical results are hard to understand, e.g. what's survivorship in Fig.2

Questions for authors

Can the author explain the paper in plainer language what they problem they are trying to resolve? Or what's the background knowledge is required to understand the paper? I have worked on reasoning models and efficient early stopping of reasoning models, but I can't even understand Figure 1.

Why is Qwen 7B model using NF4... why is precision even specified or changed?

4 comments
AuthorsOpenReview ↗

Response to Reviewer YSLt

Thank you for the clear questions. The revision is rewritten for readers who work on reasoning models and early stopping, as you do. Below we explain the problem in that language, then answer your questions about Figs. 1–2, the datasets and models, and NF4.

What problem are we solving? Many early-stopping methods train a probe that predicts whether the model's final answer will be correct, and stop when the probe is confident. Our point is that this probe predicts the wrong thing for deciding when to stop.

  • What matters is whether the answer the model would give if stopped now is more likely to be right than the answer it will finish with.
  • Two chains can both have a 50% chance of a correct final answer, while the answer available now is right 75% of the time in one and 25% in the other. Stopping helps the first and hurts the second, and the probe cannot tell them apart.
  • So two numbers are needed. We call them the commitment (stop-now correctness) and the verdict (final correctness).

The only background required is conditional probability. "Martingale" means only that an accurate forecast cannot predict its own next revision. That is why a steady forecast of final correctness says nothing about whether reasoning is helping. Terms such as "harmonic reference" are now in the appendix, and the main text uses just these two quantities.

Figure 1. The revision replaces it with a worked example for each of the two comparisons (new Fig. 1).

  • Left: the two chains above, with the same final-correctness forecast but opposite answers to "should we stop?".
  • Right: four chains, two of which finish early and correct. Accuracy among the chains still running falls from 75% to 50%, although no chain changes its answer.

Survivorship in Fig. 2 (Fig. 3 in the revision).

  • The gray line is accuracy among the chains still running at each length; it falls after about 2,000 tokens.
  • The colored lines follow one fixed group of chains, and they are flat or rising.
  • So the gray line falls because easy, already-correct chains have finished, not because chains get worse. This is the right panel of the new Fig. 1, measured on real models.

A single dataset and older models.

  • What we ran. (a) The same audit on a second benchmark, OlympiadBench: 367 text-only competition problems, harder than MATH-500. (b) The same audit on Qwen3.5-2B, released in February 2026, with its thinking mode on.
  • What we found.
    • (a) OlympiadBench is much harder for the 1.5B model (final accuracy 48.8%), and the picture is the same. Harmful overthinking occurs in 3.8% of chains (95% CI [2.3, 6.3]). Accuracy among the chains still running falls 4.4 points with length. The fall comes from which chains remain (−11.7), while the chains themselves improve (+7.3, 95% CI [+3.1, +11.8]).
    • (b) Qwen3.5-2B (final accuracy 66.2%) shows harmful overthinking in 1.9% of chains (95% CI [0.9, 3.9]). Its chains still reasoning also improve (+8.7, [+5.6, +11.9]).
  • This resolves your concern. Both findings carry over to a second, harder benchmark and to a model released in 2026. With (b), the audit covers six reasoning models from three families. Besides the two R1 distills, they are Qwen3-1.7B, R1-Distill-Llama-8B and Phi-4-mini-reasoning.
  • In the revision. Sec. 4.3, Table 1 and App. O.

Why NF4 for the 7B model, and why state the precision? We loaded the 7B model in NF4 for memory efficiency. NF4 is the standard 4-bit format introduced with QLoRA (Dettmers et al., 2023) and designed to preserve 16-bit quality. We state the precision because our two main settings differ in it, and we tested directly that it does not affect the results.

  • What we ran. The 1.5B model at the 7B's difficulty, in FP16 and in NF4. The 7B model in 8-bit, which matches 16-bit quality in published evaluations (LLM.int8; Dettmers et al., 2022).
  • What we found.
    • At the same difficulty, 1.5B FP16 shows the same 3.4% rate as 7B NF4.
    • Switching the 1.5B model from FP16 to NF4 gives 3.1% vs 3.4%, with accuracy 71.4% vs 69.8% and the same decomposition signs.
    • The 7B model in 8-bit gives 3.1% vs 3.4% in NF4, with almost the same decomposition.
  • This resolves your question. Precision, at either model size, moves neither the rate nor the decomposition.
  • In the revision. Secs. 4.1 and 4.3, and Table 1.

We hope the plain-language explanation, the new figure and the added benchmark and model make the paper's problem and evidence clear. If so, we would be grateful if you would consider raising your score.

AuthorsOpenReview ↗

Follow-up to Reviewer YSLt

Dear Reviewer YSLt,

Thank you again for your review. As the author–reviewer discussion closes on November 18, we would like to check whether our response and the revised PDF address your concerns. In brief:

  • Clarity. The revision states the problem in early-stopping terms: a probe of final correctness cannot tell whether stopping now helps, so two quantities are needed. The new Fig. 1 shows both comparisons with a worked example.
  • Second benchmark and a 2026 model. On OlympiadBench, harmful overthinking occurs in 3.8% of chains, and the chains still reasoning improve (+7.3, 95% CI [+3.1, +11.8]) while composition pulls accuracy down. Qwen3.5-2B (released February 2026) shows 1.9% and the same pattern.
  • Precision. The 7B model in 8-bit gives 3.1% against 3.4% in NF4, with almost the same decomposition; switching the 1.5B model to NF4 also leaves the results unchanged.

If any point is still unclear, we would be glad to address it before the deadline. If your concerns are resolved, we would be grateful if you would consider updating your score.

I've read the author's response and a glance over the updated draft, which seems to be a complete rewrite.

I've decided to maintain my score as there is still huge room for the presentation to be improved.

I encourage the authors to remove all unncessary math and hard-to-interpret terminology as they could be too difficult and time-consuming for human readers to understand.

AuthorsOpenReview ↗

Response on presentation

Thank you for taking another look at the revision.

We agree that the paper should be easy to follow for readers working on reasoning models, and the revision is organized around that. Each formal result in Secs. 2–3 is followed by a plain statement of what it means for the decision to stop. Fig. 1 works through both comparisons with a small example, and the main empirical findings are shown directly in Figs. 2–5. A notation glossary is in App. A.

At the same time, the paper is a statistical analysis of LLM reasoning, so its main results are formal statements: a forecast of final correctness has no predictable change (Prop. 2), the commitment–verdict gap equals the value of stopping (Cor. 4), and accuracy-versus-budget curves decompose into two terms (Prop. 5). These statements define what the experiments measure, so we keep them in the main text and place proofs in the appendix. The same tools are common in this line of work at ICLR and related venues, for example:

  • early exit with statistical guarantees: Schuster et al., Confident Adaptive Language Modeling (NeurIPS 2022);
  • statistical guarantees for LLM outputs: Quach et al., Conformal Language Modeling (ICLR 2024); Angelopoulos et al., Conformal Risk Control (ICLR 2024);
  • value estimates from Monte Carlo rollouts for reasoning: Lightman et al., Let's Verify Step by Step (ICLR 2024); Wang et al., Math-Shepherd (ACL 2024);
  • allocating test-time compute: Snell et al. (ICLR 2025).

For background, the stopping comparison follows Chow, Robbins and Siegmund, Great Expectations: The Theory of Optimal Stopping (1971), and Williams, Probability with Martingales (1991) introduces the martingale tools we use.

If a specific passage remains hard to follow, we would be glad to clarify it here or in the final version.

Reviewer_2DdzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a theoretical framework to analyze when additional computing improves or erodes the answers of LLMs. The forecast of the final outcome (named verdict) is a martingale and cannot decide when to stop: the paper's idea is to leverage the information regarding the correctness of the answer if the model stopped now (commitment). On MATH-500 with R1-Distill-Qwen, harmful overthinking is rare (~3%), and the decline of accuracy-vs-budget curves is mostly due to the composition of chains, not to within-chain deterioration.

Critical strengths
  1. The problem is timely and of interest to the community
  2. The distinction between verdict and commitment is clear and useful for the early-exit literature and it shows why a probe that predicts final correctness estimates the wrong quantity for deciding when to stop
  3. I found the survivorship decomposition simple, exact and practically relevant to show that the apparent decline of accuracy with budget is mostly due to composition
Critical weaknesses

Major

  1. Limited theoretical novelty. The results follow from the tower property, the Doob decomposition and telescoping, as it is acknowledged in the paper (“not a new martingale theorem”). The commitment-verdict gap is essentially the standard comparison between the value of stopping and the value of continuing in optimal stopping and RL. Hence, the contribution is the framing, but the framing itself does not yield new empirical leverage: where it works (matched continuations), it reduces to Monte Carlo rollout estimates, already used for instance for PRMs. Which result is not possible through standard Monte Carlo rollout estimating the stopping advantage?

  2. Weak evidence of harmful overthinking. Prevalence is about 3% and depends on the checkpoint grid, while only 50% of overthink-wrong problems repeat the label in at least one of the 8 fresh samples. The main evidence of loss comes from 12 prefixes selected post hoc because their original chain ended wrong, so they are enriched for low continuation success by construction: the 50.0% vs. 97.9% contrast describes these states, not how often they occur. Moreover, matching controls only on level and checkpoint does not remove differences between problems. Can you repeat the matched constinuations on prefixes sampled independently wrt the final oucomes? Moreover, how much of the nearly 3% prevalence is due to chain-level sampling noise rather than a problem-level propensity?

  3. The setup excludes the regime of interest. With a 4096-token cap reached by 37–41% of chains, long reasoning, where overthinking should emerge, is untested (R1 distills are designed for much larger budgets). The models are small, the 7B is NF4-quantized, and the two main configurations differ in both precision and difficulty. Would prevalence and the survivorship decomposition change with a larger (like 16k–32k) token cap? Would the 7B results stay in full precision, and with precision and difficulty fixed in all sizes?

Minor

  1. “Identifying Harmful Overthinking” promises more than the paper can deliver: identification remains theoretical and the phenomenon seems to be rare.
  2. Several components are only tested on synthetic data or not tested at all: the variance-blind training objective, Level 2 steering, and the multi-agent and self-improvement extensions in appendix M. These parts dilute the main contribution by widening the scope without supporting evidence. Their discussion in the main text should be shortened.
  3. The writing is very dense and the notation heavy, making the paper very hard to follow. The mix of internally pre-registered, post-hoc and “historical” analyses is hard to follow, and many key results are in the appendix.
  4. The null result on AIME is little informative (with only 12 eligible chains), as it is acknowledged in the paper.
Questions for authors

See weaknesses

6 comments
AuthorsOpenReview ↗

Response to Reviewer 2Ddz (1/2)

Thank you for a precise and constructive review. You identified the two contributions we consider central: the verdict/commitment distinction and the survivorship decomposition. Your concerns are novelty (Major 1), the strength of the evidence (Major 2) and the tested regime (Major 3).

  • For Majors 2 and 3, we ran every analysis you asked for, and each one confirms the conclusions.
  • For Major 1, we give a concrete case where a standard rollout estimate misses what the framework catches.

Major 1. Novelty: which result is not possible through standard Monte Carlo rollouts? We agree that the tools are classical: the tower property, the Doob decomposition and telescoping. We also agree that the commitment–verdict gap is the comparison of stopping and continuing values from optimal stopping. The revision cites that literature explicitly (Chow et al., 1971; revised Secs. 1 and 5). Monte Carlo rollouts are also the right estimator, and our continuation experiments use them. What rollouts alone do not provide is which quantity to estimate. The framework supplies three results that rollouts of final correctness do not.

  1. A rollout estimate of final correctness cannot decide when to stop (Cor. 3). Such estimates are the labels behind process reward models and value probes, and they estimate the verdict. Deciding to stop also requires the stop-now accuracy from the same prefix. The new misleading-hint experiment shows this concretely (reply to ka25, W3):
    • After the hint, the rollout estimate is 93.6% against 94.4% without it, essentially unchanged.
    • The stop-now accuracy falls to 32.2%.
    • A rollout-trained value therefore does not register the damage, and reading only the current answer would stop at the worst moment.
  2. Length curves are not rollout quantities at all (Prop. 5). The decomposition applies to the accuracy-versus-budget curves that the literature reports. It shows, in all 11 configurations of revised Table 1, that the within-chain change is positive and the composition change negative: wherever the curve declines, the decline comes from which chains remain.
  3. A ceiling before building any rule (Cor. 4). The best possible gain of a stopping rule is the expected loss at the states where it stops. Measured as an oracle, this ceiling is +1.8 points at 1.5B and +2.5 at 7B.

These three results are what the framework adds to rollout estimation: which estimand answers the overthinking question, and evidence that two standard practices estimate a different one.

Major 2. Strength of the evidence.

  • Prefixes sampled independently of the final outcome.
    • What we ran: every 1.5B chain with a correct answer at or before 2,048 tokens, at its first such checkpoint, chosen without reading how it ended. This gives 379 points, including the 12 from App. J, with four continuations each under the App. J protocol.
    • What we found: continuing loses 3.6 points on average (95% CI [2.2, 5.2]). At 348 points the model never loses the answer, and 13 points (3.4%) lose it in at least two of four continuations. With fresh draws, the 12 earlier prefixes again finish correct only 47.9% of the time (50.0% in App. J), against 98.0% for all other points.
    • This resolves your concern: this is the outcome-independent sample you asked for. It measures how often such states occur, and the matched contrast replicates.
    • In the revision: Sec. 4.2, Fig. 2 and App. O.
  • Chain-level noise versus problem-level propensity.
    • What we ran: an overdispersion analysis of the 8 fresh chains per problem.
    • What we found:
      • Per-problem counts are 2.6 times as variable as a single shared rate would allow (permutation p < 10⁻⁴), and the intraclass correlation is 0.23.
      • Five problems lose the answer in at least 5 of 8 chains, where a shared rate predicts none.
      • The transitions fall in 57 problems, against 89 expected under a shared rate, and the ten most affected problems produce 40% of them.
    • This resolves your question: the rate is not chain-level noise alone. A concentrated problem-level propensity drives a large share of it.
    • In the revision: Sec. 4.2 and App. O.
  • Dependence on the checkpoint grid. The outcome-blind sample above does not count flips. It measures the expected loss at each correct state directly, so it avoids the flip label's sensitivity to how often answers are checked.

(continued in the next comment)

AuthorsOpenReview ↗

Response to Reviewer 2Ddz (2/2)

Major 3. The tested regime.

  • Longer budgets.
    • What we ran: fresh 1.5B chains with a 16,384-token cap. The same chains give the 4,096-token result, so both budgets are compared on identical chains.
    • What we found:
      • At 4,096 tokens, 44% of chains are still running and final accuracy is 75.7%. At 16,384, only 11% are, and accuracy is 80.9%.
      • Harmful overthinking is 2.5% (95% CI [1.3, 4.6]) at 4,096 tokens and 3.3% ([1.9, 5.6]) at 16,384.
      • From 4,096 to 16,384 tokens, accuracy among running chains falls 28.6 points, entirely through which chains remain (−36.5). The chains themselves do not get worse (+7.9, 95% CI [−4.4, +21.8]).
    • This resolves your concern: at 16,384 tokens most chains finish on their own, and the rate and the decomposition are unchanged. A larger cap could affect only the 11% of chains still running.
    • In the revision: Sec. 4.3 and Table 1.
  • Precision and difficulty held fixed.
    • What we ran:
      • (a) the 1.5B audit restricted to MATH levels 4–5, the 7B problems;
      • (b) the 1.5B model in NF4 on those problems;
      • (c) the 7B model in 8-bit.
    • What we found:
      • (a) 1.5B FP16 shows 9/262 = 3.4%, the same as 7B NF4, with the same decomposition signs.
      • (b) 1.5B NF4 shows 8/262 = 3.1% (accuracy 71.4% vs 69.8% in FP16), again with the chains improving (+15.0, 95% CI [+8.4, +22.1]) and composition pulling down (−17.2).
      • (c) 7B 8-bit shows 8/262 = 3.1% (95% CI [1.6, 5.9]), against 3.4% in NF4. Its decomposition, −4.1 = +17.9 − 22.0, is almost the same as in NF4 (−4.4 = +20.0 − 24.4).
    • This resolves your concern: with difficulty fixed, changing precision at either model size moves neither the rate nor the decomposition.
    • In the revision: Sec. 4.3 and Table 1.
  • Model coverage. The audit now covers six reasoning models from three families: the two R1 distills, Qwen3-1.7B, R1-Distill-Llama-8B, Phi-4-mini-reasoning and the 2026 Qwen3.5-2B. It also covers a second benchmark, OlympiadBench. All show the same pattern (revised Table 1).

Minor.

  1. Title. We agree it can be more precise, and would retitle the final version Conserved Verdicts, Drifting Commitments: The Value of Continued Reasoning.
  2. Components without real-model experiments. The revision moves the training objective to App. F and leaves the extensions in App. M; the main text only points to them.
  3. Density and analysis types. The revision rewrites the introduction around one question, moves the formal assumption and operator form to the appendix, and brings the key new results into the main text. A new table marks each analysis as pre-registered, follow-up, or added during review (Table 8, App. O).
  4. AIME. The revision moves it to the appendix (App. H.1).

Each point under Majors 2 and 3 now has a direct experiment: an outcome-independent sample, a propensity analysis, a 16,384-token budget, fixed difficulty and precision, and further models and a benchmark. If these resolve your concerns, we would be grateful if you would consider raising your score.

AuthorsOpenReview ↗

Follow-up to Reviewer 2Ddz

Dear Reviewer 2Ddz,

Thank you again for your detailed review. As the author–reviewer discussion closes on November 18, we would like to check whether our response and the revised PDF address your concerns. In brief:

  • Novelty (Major 1). The misleading-hint experiment is a concrete case where a rollout estimate of final correctness misses what matters for stopping: it stays at 93.6% (94.4% without the hint) while the stop-now accuracy falls to 32.2%.
  • Outcome-independent evidence (Major 2). At all 379 points where the 1.5B model holds a correct answer, chosen without reading how chains end, continuing loses 3.6 points on average (95% CI [2.2, 5.2]). The matched contrast replicates with fresh draws (47.9% vs 98.0%), and per-problem counts are 2.6 times as variable as a shared rate allows (ICC 0.23), so a problem-level propensity drives much of the rate.
  • Tested regime (Major 3). At a 16,384-token cap, and with difficulty and precision held fixed at both model sizes, the rate and the decomposition are unchanged. The audit now covers six models from three families and a second benchmark.

If any point is still unclear, we would be glad to address it before the deadline. If your concerns are resolved, we would be grateful if you would consider updating your score.

Reviewer_ka25OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to identify when the extra reasoning spent on a question by an LLM helps vs. abandons an answer that was already correct. It argues that predicting whether a chain will end correct is different from knowing whether the answer would be better if the model stopped now. It also shows that accuracy-vs-length curves can fall just because chains that finish early tend to be correct, even when the chains still running are improving.

Critical strengths
  1. I like the notions of “commitment” and “verdict”. It’s a nice way to think about throwing test time compute at a problem that may or may not require reasoning. It’s a very interesting and well motivated problem, well worth the time of any academic researcher.
  2. It was an interesting result that accuracy can appear to decrease with longer reasoning even when the models that are still reasoning are improving
  3. I respect that the authors submitted a negative result / something that wasn’t yet discovered to help, and the careful experimental analysis
Critical weaknesses
  1. I appreciate the detail of the paper, but it’s just a little bit hard to read sometimes. It feels like a lot of clauses have qualifiers “X, not Y” structure. For e.g. “illustrates a training implication, not a fine-tuning gain”, “is a criterion, not an observable oracle”, “calibrates the apparatus, not real-model decision accuracy”, “our cached-checkpoint protocol, not the cited methods”
  2. While I appreciate that the authors are reporting negative results, I am not entirely convinced by the practical significance of the proposed framework. The authors evaluate several stopping methods and find that none improves accuracy at lower token cost. I think the paper would be more compelling if it went further in investigating why these methods fail in the evaluated setting. For example, is this primarily because harmful overthinking is relatively rare, or because the proposed measurements are not reliable enough to identify it? A more systematic investigation of these failure modes would strengthen the contribution.
  3. I understand that limited compute may have constrained the experiments, particularly given how rarely harmful overthinking occurs in the evaluated setting (might just require a lot more test time compute thrown at the problem). Rather than requiring larger models or longer reasoning budgets, I wonder whether the authors could construct a controlled setting where harmful overthinking is more common. For example, they could introduce misleading information during reasoning or otherwise create conditions that encourage models to abandon initially correct answers. This would allow for a more targeted evaluation of whether the proposed framework can detect harmful overthinking in actual language models, beyond the synthetic experiments. I understand this likely isn’t possible during the short rebuttal period, but it’s food for thought.
Questions for authors
  1. Is there some way you can frame the work in the paper into a way that adds to or contributes to an existing published conclusion? In particular, given that the proposed stopping method does not improve performance, I would be interested in understanding what new conclusions the framework allows us to draw.
  2. Do the authors expect these observations to hold beyond the 4096-token cap used in the experiments?
5 comments
AuthorsOpenReview ↗

Response to Reviewer ka25 (1/2)

Thank you for engaging with the commitment/verdict idea and for valuing a carefully reported negative result. Your central question is practical significance: why do the stopping methods fail, and what does the framework let us conclude? Our short answer has three parts.

  • The methods fail mainly because harm is rare: even an oracle with perfect information gains only about two points (W2).
  • In the controlled setting you proposed, the framework detects the abandonment of a correct answer exactly where it happens, and shows that the abandonment is reversible (W3).
  • The framework changes how three published conclusions should be read (Q1).

W2. Why do the stopping methods fail: rare harm, or unreliable measurement? Mainly rare harm. Two new analyses separate the two explanations.

  • What we ran. (a) An oracle on our archived replays. It knows the correct answer and stops each chain at its best checkpoint, so measurement error plays no role. (b) Four fresh continuations from each of the 379 points where the 1.5B model holds a correct answer, chosen without looking at how chains end.
  • What we found. (a) The oracle gains only +1.8 points at 1.5B (95% CI [0.0, +4.5]) and +2.5 at 7B ([0.0, +6.2]), while saving 39% of tokens. (b) Continuing loses 3.6 points on average (95% CI [2.2, 5.2]). At 348 of the 379 points the model never loses the answer, and a few problems account for most of the loss.
  • This resolves your question. No real rule can beat the oracle. In this setting early stopping is therefore worth at most about two points of accuracy, however reliable the measurement. The ceiling comes from how rare harm is, and (b) measures that rarity directly. A stopping method's value here is the tokens it saves.
  • In the revision. Sec. 4.2 with Fig. 2, and Sec. 4.4 with the oracle row of Table 2.

W3. A controlled setting with misleading information. Thank you for this suggestion; we ran it during the rebuttal.

  • What we ran. At 90 points where the 1.5B model's current answer is correct, we inserted into its reasoning: "Wait, I made an error earlier. Rechecking carefully, the correct answer should be W." Here W is a wrong answer the model itself gave on that problem in other chains (60 points), or the correct answer plus one (30 points). We then measured both quantities of the framework against unperturbed continuations from the same points:
    • the answer if the model stops right after the hint (the commitment);
    • the final answer of four continuations (the verdict).
  • What we found. The hint makes the model abandon its correct answer. Stopped right after the hint, it gives W in 64.4% of cases and is correct in only 32.2%. Continuing, it recovers: the continuations end correct in 93.6%, against 94.4% without the hint (difference 0.8 points, 95% CI [−1.7, 3.6]), and only 5.0% end on W.
  • This resolves your concern. This is the targeted test in an actual language model that you describe, and the framework detects what the hint does.
    • The commitment falls by about 68 points while the verdict stays where it was.
    • The framework therefore locates the abandonment exactly and shows it is reversible: the right decision is to keep reasoning.
    • A method that reads only the current answer would call this harmful overthinking and stop. A final-correctness forecast would not register the abandonment at all.
  • In the revision. Sec. 4.2; protocol in App. O.

(continued in the next comment)

AuthorsOpenReview ↗

Response to Reviewer ka25 (2/2)

Q1. What new conclusions does the framework allow? Three, each about a published line of work.

  1. "Accuracy falls with longer reasoning" (e.g. Zhou et al., 2026). In six models from three families, on two benchmarks and up to 16,384 tokens, wherever accuracy falls the fall comes from which chains remain, and the chains still reasoning keep or improve their answers (Prop. 5; revised Table 1).
  2. Final-correctness probes and rollout values for early exit (e.g. Zhang et al., 2025a). These estimate the verdict, which cannot decide when to stop (Cor. 3). W3 shows the consequence: after the hint the verdict is unchanged while the stop-now answer is mostly wrong.
  3. Early-exit gains. In this setting, the most any stopping rule can add is about two points of accuracy (Cor. 4 and the oracle in W2), so the gain of early exit here is the tokens it saves.

Q2. Beyond the 4,096-token cap? Yes.

  • What we ran. Fresh 1.5B chains with a 16,384-token cap. Generation runs left to right, so the same chains also give the 4,096-token result, and the two budgets are compared on identical chains.
  • What we found.
    • Raising the cap cuts the share of chains still running at the cap from 44% to 11%, and raises final accuracy from 75.7% to 80.9%.
    • Harmful overthinking stays rare: 2.5% of chains (95% CI [1.3, 4.6]) at 4,096 tokens and 3.3% ([1.9, 5.6]) at 16,384.
    • Between 4,096 and 16,384 tokens, accuracy among the chains still running falls by 28.6 points. All of the fall comes from which chains remain (−36.5). The chains themselves do not get worse (+7.9, 95% CI [−4.4, +21.8]).
  • In the revision. Sec. 4.3 and the last row of Table 1; details in App. O.

W1. Readability. We agree. The revision rewrites the "X, not Y" sentences to state the point directly. For example, "This is a criterion, not an observable oracle" now reads "A stopping rule gains accuracy only from the states where it stops and where remaining drift is negative" (revised Sec. 3).

We hope these results answer your questions, in particular through the controlled test you suggested. If they do, we would be grateful if you would consider raising your score.

AuthorsOpenReview ↗

Follow-up to Reviewer ka25

Dear Reviewer ka25,

Thank you again for your review. As the author–reviewer discussion closes on November 18, we would like to check whether our response and the revised PDF address your concerns. In brief:

  • Controlled test you suggested (W3). A misleading hint makes the model abandon a correct answer: stopped right after the hint it is correct in only 32.2% of cases, yet its continuations end correct in 93.6%, against 94.4% without the hint. The framework locates the abandonment exactly and shows that the right decision is to keep reasoning.
  • Why stopping methods fail (W2). An oracle with perfect information gains at most +1.8 points at 1.5B and +2.5 at 7B, because harm is rare: continuing from a correct state loses 3.6 points on average (95% CI [2.2, 5.2]).
  • Longer budgets (Q2). With a 16,384-token cap, the decline of accuracy with length again comes entirely from which chains remain, while the chains still reasoning do not get worse.

If any point is still unclear, we would be glad to address it before the deadline. If your concerns are resolved, we would be grateful if you would consider updating your score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: new results and revised PDF

Thank you for careful reviews. Two questions came up in all three: what the paper is asking in plain terms, and what it lets us conclude about real models. We answer both here and reply to each review point by point below. We ran the experiments you asked for; each reply says what we ran, what we found, and how it answers the question.

Revised PDF. We uploaded a revision. It rewrites the introduction around the question in plain terms, replaces Fig. 1 with a worked example, moves the formal assumption, the operator form and the training objective to the appendix, and adds all new results (Secs. 4.2–4.4, Table 1, App. O). In the revision the Experiments section is Sec. 4, and theorem and proposition numbers are as in the submission. Each "In the revision" line gives the new location.

What the paper establishes, now on six models, two benchmarks and budgets up to 16,384 tokens.

  1. The accuracy drop with longer reasoning is mostly survivorship. The chains still reasoning keep or improve their accuracy. The drop comes from easy chains finishing early, which leaves the harder ones behind.
  2. Truly harmful overthinking is real but rare. Continuing turns a correct answer into a wrong one in about 2–4% of chains, and these chains concentrate on a few problems.
  3. Early stopping can therefore add at most about two points of accuracy. Its value is the tokens it saves. To decide when to stop, a method must compare stop-now accuracy with final accuracy. Predicting final correctness alone cannot make that decision.

The paper in plain language. Partway through a chain of thought, should we stop the model and take its answer, or let it keep reasoning? That compares two numbers at the same point:

  1. the chance that the answer it would give if stopped now is correct, which we call the commitment;
  2. the chance that the answer it will finish with is correct, which we call the verdict.

Stopping helps exactly when the first is larger.

Point 1 (Secs. 3–4). A predictor of final correctness estimates only the second number, so it cannot decide when to stop. Two chains can both have a 50% chance of ending correct while the answer available now is right 75% of the time in one and 25% in the other (Cor. 3). Stopping helps the first and hurts the second. The "martingale" statement says only that an accurate forecast cannot predict its own next revision, so a steady forecast of the final outcome says nothing about whether reasoning is helping.

Point 2 (Sec. 4.3, Fig. 3). Accuracy-versus-length curves usually average over the chains still running. Easy problems finish early and correctly, so the chains left at long lengths are harder, and accuracy can fall although no chain gets worse. We split each change into "chains changing their answers" and "which chains remain".

New results.

Question (reviewer)Answer
How often does continuing lose a correct answer? (2Ddz, ka25)Rarely. At all 379 points where the 1.5B model holds a correct answer, chosen without looking at how chains end, continuing loses 3.6 points on average (95% CI [2.2, 5.2]); in 348 of the 379 points it never loses the answer.
Chain noise or problem propensity? (2Ddz)A clear problem-level propensity: 5 problems lose the answer in at least 5 of 8 fresh chains, where chance predicts none.
Why does early stopping not help? (ka25)Little to gain: even an oracle that knows the answer gains only +1.8 (1.5B) and +2.5 (7B) points.
Difficulty and precision (2Ddz, YSLt)At the same difficulty, 1.5B and 7B show the same 3.4% rate, and running the 1.5B model in NF4 instead of FP16 changes it only to 3.1%. The 7B model in 8-bit gives 3.1% vs 3.4% in NF4, with almost the same decomposition.
Beyond 4,096 tokens? (ka25, 2Ddz)Same picture at 16,384 tokens, where only 11% of chains hit the cap: harmful overthinking is 3.3% (2.5% at 4,096), and the accuracy fall with length comes from which chains remain (−36.5 points), while the chains themselves do not get worse (+7.9).
Misleading information inserted mid-reasoning? (ka25)The 1.5B model abandons its correct answer (stop-now accuracy falls from 100% to 32.2%), but continuing recovers it (93.6% vs 94.4% without the hint). The framework shows the drop and shows it is reversible.
A second dataset and a newer model? (YSLt)On OlympiadBench (1.5B accuracy 48.8%), harmful overthinking is 3.8% and the length decline again comes from which chains remain (−11.7), while the chains themselves improve (+7.3). Qwen3.5-2B (released February 2026) shows 1.9%, and its chains still reasoning also improve (+8.7).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.