acceptodds
Under review as a conference paper at ICLR 2027

Settle: Learning When to Stop Reasoning

Abstract

Reasoning models often continue generating after their answers have settled. Settle learns when to stop from answer stability in completed traces. It trains the existing end-of-reasoning token while keeping other predictions close to the base model, and requires only ordinary decoding at inference. On MATH-500 with Qwen3-4B, Settle reduces token count by 40% with a 0.5-percentage-point decrease in accuracy. It gains 6.16 percentage points over supervised fine-tuning on the same traces shortened at their first stable answer, at nearly identical token counts. Its stopping score predicts whether a correct answer will remain correct. Settle reaches the accuracy–token-count Pareto frontier among stopping methods.

Primary area:
foundation or frontier models, including LLMs
Keywords:
reasoning models, efficient inference, early stopping, answer stability, supervised fine-tuning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_p6MNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces a new method to make the reasoning model more efficient by getting the same accuracy with fewer tokens. The main idea here is that after a model reaches an answer and becomes stable in its reasoning process, the model should close the thinking part of its reasoning and start outputting the final answer. So, this motivates the authors to fine-tune the model to increase its probability of outputting </think> and closing its thought process in stable boundaries. However, this leads the model to repeatedly outputting </think> without producing an answer, and this motivates the authors to add a regularizer term to follow the distribution of the base model in non-boundary tokens. Across several reasoning models and benchmarks, Settle cuts reasoning tokens by roughly 13–49% with small accuracy changes, outperforming baselines.

Critical strengths
  1. The paper clearly explains its observation about the stability and properly motivates its training objective. It also has done a rigorous study on the effect of the KL and stopping weights.
Critical weaknesses
  1. There are some problems with the experiments. The first one is some missing baselines that have already outperformed DEER [1, 2]. Also, I am not sure how much improvements the method gains over DEER. For example, I am not sure that Settle is Pareto dominant over DEER from Figure 2. The other problem is also the evaluation benchmarks. More benchmarks lie towards the easy spectrum of benchmarks, and papers miss harder benchmarks like AIME or even harder.

  2. The model has several hyperparameters that might be challenging for tuning. The authors provided a complete sensitivity of these hyperparameters that in most cases seems to be less sensitive. However, there are cases as in Table 24 that performance is sensitive w.r.t. ww.

[1] Hosseini, Parsa, et al. "Early Stopping for Large Reasoning Models via Confidence Dynamics." arXiv preprint arXiv:2604.04930 (2026).

[2] Wang, Xi, et al. "Conformal thinking: Risk control for reasoning on a compute budget." arXiv preprint arXiv:2602.03814 (2026).

Questions for authors
  1. Please refer to weaknesses.

  2. I do have some concerns about what the training objective is actually teaching; it pushes the model to stop earlier and shorten its reasoning overall, while trying to be close to the base model. In particular, I am not sure how much of the improvement comes from the label signal itself. Can the authors run some controlled experiments with, for example, random labels, and be tuned to match Settle’s token count?

1 comment

Dear Reviewer,

As the author–reviewer discussion deadline approaches, I would like to remind you to review the authors’ rebuttal and indicate whether your concerns have been adequately addressed.

If you have any remaining concerns or questions, please share them with the authors before the discussion period ends.

Thank you again for your time and valuable feedback.

Reviewer_9UzxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes to finetune LLMs to encourage/discourage the end-of-reasoning token at certain positions. The training target is decided by forcing an answer there; if it agrees with all future forced answers, an end-of-reasoning token should be generated. The paper conducted experiments on mathematical datasets and compared accuracy and token counts against baselines and variants.

Critical strengths

The proposed method demonstrates favorable accuracy-token balance with the standard inference algorithm.

Critical weaknesses
  • The candidate boundaries depend on keywords like "Wait", "Alternatively", which can be sensitive to the model's language style.

  • Table 1 mentions SFT uses mean losses while SETTLE uses sums, but why SETTLE uses sum? Using sum over positions instead of mean creates another problem, the optimization objective in equation 2, the relative strength of KL regularization compared to the EOR token learning depends not only on lambda, but also on ratio between the length of the trajectory and the number of sampled boundaries.

  • The introduction section gives little background information and overview of the current development on this problem. A more comprehensive overview of relevant methods would help better understand the motivation and novelty of the proposed method.

  • I find the presentation of section 4 hard to follow, the comparison settings keep changing, and the evaluation metrics are changing as well. I would suggest avoid too much mentions of the numbers in the main text, and spend more effort on describing the settings in each subsection, and leave the exact numbers in the tables or figures.

Questions for authors
  • The training data is only 50 unique math problems, each has 20 sampled trajectories; why make such a choice? This is potentially a strength, but more investigation would be favorable, e.g. how would the proposed method perform if there were even fewer training trajectories?
  • Following the above question, what's the method actually learning? (because with only 50 unique problems, there is likely no new knowledge to be learned at all, especially the trajectories come from the model itself) Is the model simply learning to not do any superficial reflections? One possible experiment to do would be to test the resulting model on non-mathematical domains, and compare accuracy and token cost.
Reviewer_6VkEOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Settle, a method for teaching reasoning models when to stop thinking. It trains the native </think> token with a binary stop/continue loss on future-agreement labels, plus KL to the base model elsewhere. On Qwen3-4B and MATH-500 experiment, it cuts tokens by about 40% at a 0.5-point accuracy cost, with no inference overhead.

Critical strengths

The method is simple, needs no inference overhead, and the main comparison is statistically careful.

Critical weaknesses

Incremental novelty. The labels and stability criterion come from another paper. The main change is training the native token instead of an external controller.

Limited experiments. All comparisons with other methods use one model (Qwen3-4B) on one benchmark (MATH-500). That benchmark is nearly saturated, with the base model at about 95%, which leaves little room to show accuracy differences between methods. This scope falls short of the standard for a top-tier venue.

Selective reporting of comparisons. The appendix evaluates GSM8K, AMC23, Minerva, OlympiadBench, and GPQA-Diamond, but only against the base model. No baseline (DEER, SFT, Halt Vector, PUMA) is run on them. This matters because Settle loses accuracy on AMC23 (−2.6) and Minerva (−3.7), so its advantage over other methods on harder tasks is unsupported.

Questions for authors

NA

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.