acceptodds
Under review as a conference paper at ICLR 2027

Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models

Abstract

Large Reasoning Models (LRMs) achieve strong performance by generating long chains of thought (CoT), but often overthink, continuing to reason after a solution has already stabilized and thereby wasting tokens and increasing latency. Existing inference-time early-exit methods rely primarily on answer-level signals, such as confidence or trial-answer consistency, to decide when to stop. However, these signals mainly reflect answer readiness rather than reasoning convergence: they may trigger before the model has finished exploring or self-correcting, causing premature exits that can degrade final-answer accuracy and leave the retained reasoning chain semantically incomplete. We identify reasoning-level semantic redundancy as a complementary signal for semantic-preserving early exit: when successive steps no longer add novel progress and instead revisit established conclusions, the reasoning trajectory has likely converged. Building on this insight, we propose PUMA, a plug-and-play framework that combines a lightweight Redundancy Detector with answer-level verification. The detector flags semantically redundant candidate exits, while verification confirms whether stopping is safe, allowing PUMA to remove redundant continuation while preserving both answer accuracy and a coherent reasoning prefix. Across five LRMs and five challenging reasoning benchmarks, PUMA achieves 20.6% average token reduction while preserving accuracy and retained CoT quality. Additional experiments on code generation, zero-shot vision-language reasoning, and learned stopping-policy internalization further demonstrate that reasoning-level redundancy is a robust, transferable, and learnable signal for efficient reasoning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Reasoning Models, Reasoning Efficiency, Semantic Redundancy, Reasoning Convergence, Test-Time Inference, Overthinking

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_q3sYOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to develop an early stopping for reasoning models. Previously, early stopping methods mostly used confidence or entropy as the signal for early stopping since these signals indicate that model has reached the final solution and is just wasting tokens. However, these methods don't account for the case where the model is confident but still exploring. To this end, the authors propose PUMA, a method that stops when reasoning has converged and the model is confident. To verify whether the reasoning has converged, PUMA uses a fine-tuned embedding model (with contrastive loss) to measure the similarity of each steps with its previous steps. PUMA obtains better token reduction across different models while achieving the same accuracy in comparison with baselines.

Critical strengths
  1. I think the experimental results are the main strength of the paper. The authors have compared their method with baselines across many domains including vision language reasoning. In many cases their method beats existing method because PUMA can use a more conservative threshold than the existing confidence-based methods.
Critical weaknesses
  1. I have concerns regarding the novelty of the method for this venue. The main intuition of the method has already been discussed in the RCPD paper [1]. Although RCPD method doesn't use an embedding model, but the authors of that paper have already showed the convergence of the reasoning using an embedding model.

  2. Some baselines are missing. For example [2] and [3] are early stopping methods considering two cases: i) stopping when the model is too confident and wasting budget ii) stopping when the model shows incompetency in its problem solving. I think the authors need to compare their method with these methods as well.

[1] Wei, Zihao, et al. "The evolution of thought: Tracking llm overthinking via reasoning dynamics analysis." Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

[2] Hosseini, Parsa, et al. "Early Stopping for Large Reasoning Models via Confidence Dynamics." arXiv preprint arXiv:2604.04930 (2026).

[3] Wang, Xi, et al. "Conformal thinking: Risk control for reasoning on a compute budget." arXiv preprint arXiv:2602.03814 (2026).

Questions for authors
  1. Please look at the critical weaknesses.

  2. I have some concerns regarding hyperparams. In particular, choice of tau seems to have a significant effect on the final acc and token reduction. Can authors elaborate on this?

  3. it seems the authors are using a different notion of step in comparison to DEER (which uses "wait" to separate steps). Could the authors apply DEER stopping rule to their steps and report the results. I do wonder how much step definition impacts the results.

Reviewer_MsoSOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PUMA, a method that stops a reasoning model when further thinking appears unnecessary. It uses a trained embedding model to detect repetition between consecutive reasoning steps. When repetition is detected, PUMA checks whether the model produces a confident and stable answer before stopping. It also includes a separate rule for breaking prolonged repetitive loops. Across five models and five reasoning benchmarks, the authors report an average token reduction of 20.6% while broadly maintaining answer accuracy. Additional experiments examine reasoning quality, actual inference speed, transfer to other tasks, and training models to stop earlier. 18653_Stop_When_Reasoning_Conv …

Critical strengths
  1. The method combines two useful sources of evidence. PUMA checks both whether reasoning has become repetitive and whether the answer is stable. This is a sensible design because repetition alone does not mean that stopping is safe. The ablations show that both components contribute to performance.

  2. The evaluation considers more than answer accuracy. The authors examine whether the shortened reasoning remains complete, coherent, and understandable. They also count the tokens spent on answer checks and measure actual inference speed, making the efficiency results more useful.

  3. The experiments cover a broad range of settings. The paper evaluates several model families and reasoning benchmarks, with additional experiments on code, vision-language tasks, and agents. It also tests alternative detectors and ways of splitting reasoning into steps.

Critical weaknesses
  1. The novelty over existing early-stopping methods is not sufficiently established. Several parts of PUMA have already been explored, including detecting redundant reasoning, checking intermediate answers, and learning when to stop. Five relevant works are:

[1] Sun, Renliang, et al. "Stop when enough: Adaptive early-stopping for chain-of-thought reasoning." Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

[2] Liu, Xin, and Lu Wang. "Answer convergence as a signal for early stopping in reasoning." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

[3] Wang, Junda, et al. "ESTAR: Early-Stopping Token-Aware Reasoning For Efficient Inference." arXiv preprint arXiv:2602.10004

[4] Shrivastava, Sahil. "Semantic Early-Stopping for Iterative LLM Agent Loops." arXiv preprint arXiv:2606.27009 (2026).

PUMA already compares against DEER and answer consistency, but direct comparisons with REFRAIN and ESTAR-LITE are missing. These comparisons would help show whether its trained semantic detector offers a meaningful improvement over existing approaches.

  1. Repetition does not necessarily mean that useful reasoning has finished. A model may repeat an idea before finding an error or switching to a better solution. The detector’s reported accuracy shows that it can recognize labeled repetition, but not that later reasoning would be unnecessary. The paper should directly measure how often apparently repetitive steps are followed by useful corrections or answer changes.

  2. The ablation does not clearly show whether semantic detection is better than simply checking less often. Removing the redundancy detector makes the system check answers at every step. This changes both the frequency and timing of checks. Some benefits may therefore come from fewer checks rather than better stopping decisions. Comparisons with periodic or random checks using the same probing budget would clarify this point.

  3. The claim of preserving reasoning meaning is stronger than the evidence supports. The reasoning-quality evaluation mainly asks an LLM judge whether shortened explanations appear complete, coherent, concise, and justified. The human agreement check is helpful, but these assessments do not establish that all necessary logical steps have been preserved. The results support better judged explanation quality more directly than preservation of the original reasoning.

  4. The detector evaluation does not directly measure its most harmful mistakes. The main detector metric measures whether a redundant step receives a higher similarity score than a novel step. In practice, the important question is how often a necessary step is incorrectly flagged as redundant at the chosen threshold. Precision and recall on natural reasoning traces, especially those containing subtle corrections, would provide more relevant evidence.

  5. The practical speed and training results have limited coverage. Actual speedups are measured on two models under one hardware setup, while the experiments that train models to stop earlier use one base model and math-focused data. These results are promising, but do not yet show how well the approach works under concurrent serving or how reliably the learned stopping behavior transfers to other models and domains.

Questions for authors

Please see weakness

Reviewer_vxCwOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper targets overthinking in large reasoning models and proposes PUMA, an inference-time early-exit method. PUMA uses a small redundancy detector (Qwen3-Embedding-0.6B with LoRA, trained contrastively on step pairs labeled by GPT-5-mini plus redundant steps synthesized by GPT-4o-mini) to flag a step as a candidate exit when it is too similar to the previous one. At each flagged point, it runs answer verification: the trial answer must have confidence above 0.98 and match the answer at the next candidate. A Loop Breaker fallback handles long stretches of redundant steps. Across 5 LRMs and 5 benchmarks, PUMA cuts about 20.6% of tokens with roughly unchanged accuracy and gives 1.15–1.40x wall-clock speedup. The paper also reports results on code, vision-language, and τ-bench, and uses PUMA's exit points to train models (SFT/DPO/GRPO) to stop on their own.

Critical strengths
  1. It's practical. One detector works for all target models, model weights stay untouched, it runs on standard vLLM, and the speedups are real wall-clock numbers with small overhead.
  2. The experiments are broad: ablations, sensitivity sweeps, segmentation robustness, significance tests, and latency breakdowns. The authors are also upfront about cases where baselines do better.
Critical weaknesses

W1. The ablations don't really back up the main claim. The paper's pitch is that semantic redundancy signals reasoning convergence. But in Table 22, verification alone already gets close to full PUMA on DS-7B (58.3 acc / 28.5 TR vs. 60.2 / 29.2), while removing confidence and consistency drops accuracy by 16.2 points. Table 7 shows the same pattern. Most of the work seems to come from answer verification, and the detector mainly cuts down how often you probe. The "reasoning convergence" framing feels bigger than what the results show.

W2. It's unclear what the detector actually learns. About 80% of the positives (560K of 701K) are GPT-4o-mini rewrites built from a fixed set of patterns: restating, hesitation, filler. The detector only compares each step with the one right before it. The average similarity gap is small (0.333 vs. 0.184), and the 91% accuracy comes from a held-out split of the same distribution. My concern is that it picks up phrasing like "Wait, let me double-check" rather than whether a step makes progress. There's no human-labeled evaluation on the target models' own traces, and no comparison against a simple keyword trigger.

W3. Limited novelty, and the closest baseline is missing. Confidence probing (DEER), answer consistency (Dynasor, Answer Convergence), and step-to-step embedding similarity (REFRAIN; Sun et al., "Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoning", ACL 2026) all exist already. PUMA mostly combines them with a trained similarity model. REFRAIN is the closest prior work, but it's only discussed in text and never compared experimentally.

W4. Too many knobs for a "plug-and-play" method. The method needs τ_sim, λ, ε, L, the Loop Breaker start step, the weak confidence gate, a per-model m, and a step-length range for segmentation. Code tasks need a different τ_sim (0.50), and m is tuned per model on AMC23. The sweeps are only run on DS-7B, and using one m for all models costs up to 2.7 accuracy points.

Questions for authors
  1. Can you compare gating strategies at a matched probe budget: random, fixed interval, a keyword rule ("Wait", "verify", "double-check"), and the base embedding? If keyword + verification comes close, what's left of the redundancy claim?
  2. How does the detector do on human-labeled steps from the target LRMs (DS, Nemotron, Qwen3)? How well do the GPT-5-mini labels agree with human labels?
  3. What happens if you strip or replace reflection keywords before scoring? Can you show results on adversarial cases ("Wait..." steps that add new content, and pure repeats with no cue words), and compare seed-only vs. seed + synthetic training on all models, not just DS-7B / MATH-500?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.