acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Stability for Reasoning Paths in Large Language Models

Abstract

Large language models (LLMs) demonstrate remarkable capabilities in complex reasoning tasks, where test-time compute has emerged as a crucial means for performance enhancement. Existing methods mainly favor outputs supported by answer consistency or high token confidence, implicitly treating stable reasoning paths as more reliable. However, these approaches provide limited guidance on when stability is a reliable selection signal. In this study, we analyze how the relationship between token stability and correctness varies with the model's confidence in its reasoning path. Our analysis shows that stability is informative when the model is confident, whereas unstable paths should not be uniformly discarded when the model is uncertain. Motivated by this finding, we introduce the (), which combines macro-level verbalized confidence with micro-level token confidence fluctuations to filter reasoning paths across confidence regimes. Comprehensive evaluations across eight diverse benchmarks spanning mathematical, multidisciplinary, code, and multimodal reasoning tasks on three model families (Qwen, Llama, and InternLM) demonstrate that VeTo achieves the highest average accuracy within each model family among all compared strategies. Crucially, our method scales robustly with inference budgets from to and across model sizes from 4B to 14B, while also yielding competitive calibration performance in mathematical reasoning. Overall, our findings provide a new perspective on reasoning path selection by interpreting stability and instability according to model confidence.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM reasoning, verbalized confidence

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dDuEOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how the relationship between token-level stability and answer correctness varies with a model's verbalized confidence. It proposes VeTo, a training-free method for filtering sampled reasoning paths before majority voting. Stability is measured using jitter, the mean absolute difference between average token-confidence values in consecutive non-overlapping windows. If any paths receive high verbalized confidence, VeTo selects from that pool and favors lower jitter; otherwise, it favors higher-jitter paths from the full pool. Experiments span eight benchmarks across mathematical, multidisciplinary, code, and multimodal reasoning using three model families. The authors report improvements in average accuracy over the compared baselines, together with component ablations and analyses across sampling budgets and model sizes.

Critical strengths
  1. Informative controlled observations of confidence dynamics. The misleading-prefix experiment starts from reasoning paths with correct final answers and injects erroneous intermediate steps, reducing ambiguity over whether failures simply reflect an inability to solve the original problems. The resulting observations illustrate that incorrect continuations can remain confident and stable, while correct continuations can exhibit substantial fluctuations. This provides a useful illustration of the limitations of token confidence and stability as proxies for correctness, motivating a more nuanced investigation of these signals.

  2. Training-free and transparent path selection. VeTo does not require fine-tuning the generator or training a dedicated verifier or reward model. It operates on sampled reasoning paths using verbalized confidence and token-level scores, with an explicit filtering-and-voting procedure. This avoids additional model-training costs and makes the selection decisions straightforward to inspect.

Critical weaknesses
  1. The presentation makes the main argument unnecessarily difficult to follow. Several issues obscure the meaning of the central signals and the interpretation of the supporting evidence. For example, the Introduction and Figure 1 discuss confidence and stability before providing sufficiently clear explanations, and Section 3.1 presents analyses using jitter and confidence groupings before their definitions and operational details appear in Sections 3.2-3.3. Token confidence itself also remains underspecified: Section 3.2 names pip_i as a token-confidence value without explicitly stating how it is computed. Readers must therefore infer definitions, consult later sections, and reconstruct the context of key results rather than follow a self-contained account of the motivating observations.

  2. The selection rules are insufficiently motivated. It is unclear why verbalized confidence should be trusted enough to discard all low-confidence paths whenever a high-confidence path exists. Likewise, stability becoming less informative under low confidence does not imply that higher jitter should be preferred. Table 7 provides only modest support for this reversal, with a net gain of one correct answer over no filtering in each of the two settings.

  3. Unclear hyperparameter selection despite substantial sensitivity. Performance depends strongly on the jitter window and filtering thresholds. For example, changing beta with alpha fixed moves Llama's pooled mathematical accuracy from 70.4% to 80.4% (Table 12). Although sensitivity analyses are provided, the paper does not explain how the main configurations were selected or whether an independent validation set was used. This leaves uncertainty about whether the reported gains transfer beyond the selected configurations.

Questions for authors
  1. Clarification: token confidence. How exactly is pip_i computed? Is it the probability of the generated token before or after temperature scaling and top-p truncation?

  2. Confidence gating. How often does the hard verbalized-confidence gate discard all correct candidate paths? What evidence supports excluding low-confidence paths rather than retaining them?

  3. Fallback selection. How robust are the small high-jitter gains over no filtering in Table 7? Please provide paired uncertainty estimates for these comparisons.

  4. Hyperparameter selection. How were the window sizes and filtering thresholds selected? Please specify the selection criterion and whether selection used an independent validation set or any of the reported test questions and labels.

Reviewer_pipgOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper mainly investigates how to better use multiple reasoning paths generated by large language models to select more reliable answers. Specifically, the proposed selection strategy combines verbalized confidence (VC) with token confidence: it selects paths with lower fluctuations in token confidence in the high-VC regime and paths with higher fluctuations in the low-VC regime, while. With the parameter setting in the paper, the method outperforms generally outperforms most of the evaluated baselines., including simple majority voting, across the evaluated datasets and models.

Critical strengths

The experiments cover multiple models and benchmarks, include a broad set of baselines, and demonstrate overall improvements in average accuracy. The ablation studies are also extensive. A distinctive aspect of the method is its use of verbalized confidence to determine whether to favor low- or high-jitter reasoning paths. The initial experiments effectively illustrate that stable token confidence does not necessarily imply a correct reasoning process.

Critical weaknesses

VeTo’s performance depends on several hyperparameters, particularly the window size used to compute jitter. The paper uses different window sizes across model families and modalities, but does not clearly explain how they were selected, leaving their transferability to new settings uncertain. VeTo requires access to token probabilities and additional model self-evaluations, introducing extra inference overhead and limiting its applicability to interfaces that expose the required token probabilities.

Questions for authors
  1. In the low-VC regime, Figure 2 shows no statistically significant difference in jitter between correct and incorrect reasoning paths. Although Appendix D.1 reports benefits from high-jitter filtering, the gains in final voting accuracy are small. Could the authors provide confidence intervals and further evidence supporting this fallback strategy?
  2. How were the model- and modality-specific window sizes selected? Can a fixed window size achieve comparable performance across models without additional tuning?
Reviewer_FqQoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose VeTo, which uses verbalized confidence and token jitter together to select reasoning paths. If there are high-confidence paths, the method keeps more stable paths. If all paths have low confidence, it keeps paths with higher jitter. Then majority voting is used for final answer. Experiments are conducted on eight benchmarks and three model families.

Critical strengths
  1. The motivation is clear and the paper is easy to understand.
  2. The finding that stable path can be wrong and unstable path can be correct is interesting.
  3. The experiments include different tasks, models, model sizes, and inference budgets.
  4. The ablation shows both verbalized confidence and jitter are useful.
Critical weaknesses
  1. The method is similar to existing works such as CISC, Soft-SC, and DeepConf. Verbalized confidence and token confidence filtering are already studied. The main new part is to use different jitter filtering under high- and low-confidence cases, but the overall method is still quite heuristic.
  2. The high-jitter filtering under low-confidence case is mainly tested on only two settings. The improvement is also small, from 57.42% to 58.06% on MATH500 and from 34.33% to 34.83% on MathVista.
  3. When using Qwen to evaluate Llama paths, the result is changed. This means the method depends much on the quality of confidence evaluator.
  4. Each reasoning path needs confidence evaluation. It is not clear whether the method is still better when comparing under the same total compute budget.
Questions for authors
  1. What is the main difference between VeTo and CISC, DeepConf, and other confidence-based path selection methods?
  2. Can the authors provide more experiments to show high-jitter paths are useful in low-confidence cases?
  3. Can the authors compare all methods under the same total computation budget, including confidence evaluation cost?
  4. How are the window size and filtering thresholds selected? Are they tuned on validation set?
  5. The verbalized confidence prompt checks logic and calculation errors. Is it actually more like a verifier instead of confidence estimation?

Comments on the paper

1 comment

Dear Reviewers FqQo and dDuE,

Thank you for your time and effort during the rebuttal phase. As the discussion period is set to close in 24 hours, we would appreciate it if you could let us know whether our rebuttal has addressed your concerns.

If there are any remaining concerns or additional feedback you would like us to consider, please do not hesitate to let us know.

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.