acceptodds
Withdrawn from ICLR 2027

When to Stop Thinking: Answer-Movement Certificates and Conditional Regret Guarantees

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reasoning language models can improve their answers by thinking longer, but additional reasoning consumes resources and can also move an answer in the wrong direction. Efficient inference therefore requires deciding when to stop without access to the correct answer. We develop a framework that connects this decision to changes in the distribution of answers the model would give if stopped now. We derive answer-movement certificates that bound changes in any bounded answer score without correctness labels, yielding conditional regret bounds under cumulative movement budgets and sharper bounds near an optimal stopping policy. The analysis accommodates both beneficial and harmful reasoning and establishes why entropy stabilization alone is insufficient. We complement the theory with a calibrated entropy controller evaluated on 463 questions across four mathematics benchmarks using Qwen3.5-9B and Qwen3.5-27B. Across all eight model–benchmark comparisons, the controller reduces mean thought-plus-answer output tokens by 25–62% relative to horizon-limited native termination. Measured accuracy losses on competition tasks and the consistently lower cost of calibrated fixed budgets expose the limits of the entropy proxy. The framework provides explicit conditions for reliable stopping and a practical methodology for testing whether inexpensive stopping signals deliver useful accuracy–cost tradeoffs.

Primary area:
causal reasoning
Keywords:
reasoning+ thinking+llm

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_pBjzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper formalizes the distribution of answers obtained by interrupting a reasoning process and applying a fixed answering procedure. It relates changes in that distribution to changes in bounded utility, and derives stopping-regret bounds under cumulative-movement or oracle-proximity assumptions. It also explains why stable token entropy need not certify stable answer quality. An entropy-based controller is evaluated on two Qwen3.5 models and four mathematics benchmarks. It reduces output tokens relative to horizon-limited native termination, but does not establish accuracy preservation and is often worse than calibrated fixed budgets.

Critical strengths
  1. The paper makes an important distinction explicit. A distribution can change substantially without changing entropy, and a quiet recent prefix does not exclude useful later computation. Sections 5–6 explain these limitations and the mismatch between full-answer distributions and first-token probes. The question connects to established work on compute allocation (Snell et al., ICLR 2025), budget control (s1, EMNLP 2025), and stability-based stopping (Certaindex, NeurIPS 2025). The submission already cites these directions; distinguishing what their signals do and do not certify is worthwhile.

  2. The reporting is candid about unfavorable evidence. The authors include a calibrated fixed-budget baseline, distinguish output tokens from serving latency, and explicitly state that noninferiority was not established. The finding that a simple fixed budget can outperform an adaptive entropy rule could be informative if the underlying comparison were sufficiently well controlled.

Critical weaknesses
  1. The theory does not yet demonstrate sufficient insight or decision value beyond its conditional statements. Proposition 4 follows directly from bounded-utility transport and Pinsker's inequality. Theorem 5 can reduce to the trivial horizon bound unless a sufficiently small cumulative movement budget is available. Theorem 6 assumes that stopping is already near the oracle and that intervening movement is small; the window rule establishes neither premise. These are legitimate conditional results, but the paper does not demonstrate a meaningful setting where the assumptions can be checked and the guarantee informs a decision. The implemented entropy rule is expressly outside those guarantees. A comparison with Certaindex's stability assumptions should clarify what additional conclusion this framework enables. This is a concern about demonstrated contribution, not an allegation of incorrect mathematics. A finite-answer example with computed certificates and an informative numerical bound would help establish the theory's value.

  2. The empirical study cannot yet explain the negative result reliably. In Table 2, the controller loses 13 and 18.5 accuracy points to the fixed baseline on MATH while using more tokens. Yet Section 7.6 reports unresolved grading for 32% of primary MATH answers, and 233 of 378 competition answers reach their answer cap. These failures can depend on the policy, making it difficult to distinguish a poor stopping signal from answer-format, decoding, or measurement effects. The controller also uses a signed entropy-drop threshold, whereas EAT uses an entropy-variance criterion; failure of this implementation is not evidence against all entropy-based stopping. A matched answer-stability control motivated by Certaindex would help distinguish failure of this proxy from limitations of adaptive stopping more broadly. Negative results need not beat a baseline, but they should isolate what failed and why. Development reuse of the evaluation cohort further limits the conclusions.

Questions for authors
  1. Theory clarification: Is there a concrete example in which the proposed certificates are computable, their assumptions are verifiable, and Theorem 5 or 6 yields an informative numerical guarantee before seeing the correct answer?
  2. Measurement clarification: After a blinded audit of unresolved answers and a common extraction procedure, do the MATH and competition comparisons retain their direction and magnitude? Please separate unresolved grading from genuinely incorrect answers without dropping difficult cases.
  3. Prior-work and mechanism clarification: How do the stopping assumptions differ from Certaindex's, and does an answer-stability control retain the observed failure under matched decoding, calibration, grading, and probe-cost accounting? This would clarify what the present theory and experiment add beyond established stopping approaches.
Reviewer_oKk3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper is about adaptive stopping in large reasoning models. They develop a framework to achieve adaptive stopping by leveraging the changes in the distribution of answers produced at intermediate reasoning steps. Authors formalize the stop-now answer distribution and derive information-theoretic bounds that relates the distributional movement to changes in expected utility. The paper further establishes regret bounds for stopping rules under cumulative movement constraints or proximity assumptions relative to an optimal stopping policy. For practical implementation, this paper provides an entropy-based stopping controller using truncated next-token distributions.

Critical strengths
  1. This paper provides a clear formulation of stopping decisions in terms of answer-distribution movement and bounded utility. It appropriately distinguishes distributional stability from correctness and does not assume that additional reasoning always improves performance. Some of the theoretical results presented in the article have certain significance.
Critical weaknesses
  1. The theoretical results presented in the paper seem to be quite disconnected from the actual algorithm, which raises doubts not only about the significance of the theory but also about the reliability of the algorithm. For example, the main theories depend on KL or total-variation distances between complete answer distributions, whereas the proposed method applies entropy changes from truncated next-token distributions. Authors also have acknowledged that these quantities do not provide the required movement certificates. Furthermore, the theoretical results of the article are based on some assumptions that cannot be verified. Theorem 5 requires a bounded cumulative movement budget, and Theorem 6 assumes proximity to the optimal stopping time and bounded intervening movement, neither of which is verified for the proposed method.

  2. Please see Section 7.4, Table 2, p. 8 and Appendix C.1, Table 4, p. 14. The proposed method is outperformed by the calibrated fixed-budget baseline on MATH-500, with 13.0/18.5 percentage points lower accuracy and 41.4%/44.5% higher token costs for the 27B/9B models, respectively. The other experiments in the article also revealed similar issues, which made me doubt the validity of the method.

Questions for authors
  1. Could the author make a more detailed connection between the theoretical results and the actual strategies employed? I would like to know how these theoretical results contribute to the method of developing the article.

  2. I also have some doubts about the rationality of the method. The stopping rule uses signed entropy drops, so an increase in entropy can also satisfy this stopping condition. What motivates this choice over absolute entropy changes or distributional distances?

Reviewer_6zsmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper explores a new method for reasoning in Large Reasoning Models that optimises the amount of thinking performed before providing a final answer. This is an important problem in inference-time scaling for reasoning models, and extensive prior work on it is not mentioned. After reading it, I still struggle to understand the exact contribution.

Critical strengths

The problem is important, and the path to a solution building on entropy and filtering (which seems to be the solution favoured here) is promising. I would be interested in understanding this more clearly and how it differentiates from other, unmentioned previous work.

Critical weaknesses

Many of the experiments and statistical analyses do not make sense to me. The majority of the critical terms and concepts are not explained at all. E.g. what is an "answer-movement certificates" and what is a "conditional regret guarantee"? These are central concepts that should be conveyed clearly.

Questions for authors

What is an "answer-movement certificates" and what is a "conditional regret guarantee"?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.