acceptodds
Under review as a conference paper at ICLR 2027

Beyond Final Confidence: Leveraging Confidence Dynamics for Test-Time Reasoning

Abstract

Test-time scaling improves language model reasoning by sampling and aggregating multiple reasoning trajectories. Recent methods further leverage model confidence to prioritize reliable trajectories. However, final confidence alone can be misleading, as failed trajectories may remain highly confident while repeatedly following an incorrect reasoning direction with little effective self-correction. We refer to this failure mode as high-confidence reasoning stagnation. To distinguish trajectories within the high-confidence regime, we measure the change in token confidence from the penultimate to the final layer and aggregate it over the tail of each trajectory. Among high-confidence samples, stronger inter-layer confidence dynamics are associated with higher accuracy, shorter reasoning trajectories, and less observed stagnation, providing a complementary signal beyond final confidence. Based on this observation, we introduce a training-free aggregation method that jointly filters trajectories using tail confidence and inter-layer confidence dynamics, and then weights the retained trajectories by their confidence dynamics. Across Qwen3-1.7B, Qwen3-8B, and Gemma4-12B, our method consistently improves overall accuracy over standard majority voting and confidence-based aggregation on the evaluated reasoning benchmarks. Further analyses show that these gains remain robust under increasing sampling budgets, while confidence dynamics are most informative when measured across layers close to the final readout.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time scaling, inter-layer change, confidence dynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TiMuOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how model-internal confidence signals can improve multi-trajectory aggregation for test-time reasoning. To handle high-confidence but incorrect or stagnating trajectories, it introduces the inter-layer confidence difference between the final and penultimate layers as a complementary signal to static confidence. The method independently ranks trajectories by tail-averaged final-layer confidence and tail-averaged inter-layer confidence dynamics, retains the top-ρ\rho under each criterion, takes their intersection, and then performs weighted majority voting using the inter-layer confidence dynamics as weights.

Critical strengths
  1. The method is simple and requires no additional training or external verifier. It uses the model’s own inter-layer confidence differences to support trajectory filtering and weighted voting.
Critical weaknesses
  1. The paper lacks comparisons with vanilla performance and with state-of-the-art training-free baselines such as Certaindex [1], DEER [2], and ASAG [3]. It also does not report generation-length or efficiency metrics, such as average generated tokens or end-to-end inference cost.
  2. The method requires multiple sampled trajectories per problem: with N=1, joint filtering and weighted aggregation degenerate, so it cannot be applied in single-sample settings. In contrast, Certaindex [1], DEER [2], and ASAG [3] do not require multi-sample comparison. This is a significant limitation of the proposed method. In addition, Comparing N-sample and single-sample settings is also unfair.
  3. The paper ignores computational overhead: the cost of sampling N trajectories is not accounted for, and the extra vocabulary distribution computed from the penultimate layer is not discussed.
  4. The evaluation primarily focuses on mathematical and scientific reasoning, but do not include code generation. The method’s applicability to these tasks therefore remains unverified.
  5. The retention ratio ρ\rho is set to 60% for Qwen3 and 50% for Gemma4. Why are these values chosen? The need to tune this parameter limits the effectiveness and generality of the method.
  6. ΔC\Delta C can also be negative, which would assign a negative voting weight to that trajectory. The paper does not discuss or address this issue.

[1] Efficiently Scaling LLM Reasoning Programs with Certaindex. NeurIPS, 2025.

[2] Dynamic early exit in reasoning models. ICLR, 2026.

[3] Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models. ICML, 2026.

Questions for authors

Could the authors provide an ablation using the same trajectory pool that filters trajectories solely by inter-layer confidence differences while retaining the same dynamics-weighted voting, to assess whether adding final-layer confidence filtering provides additional benefits?

4 comments
AuthorsOpenReview ↗

Response to Reviewer TiMu [part 1]

1. Baseline Comparability and Computational Efficiency (W1–W3)

1.1 Methodological Scope and Baseline Comparability (W1, W2)

Our method focuses on post-hoc aggregation under a fixed sampling budget, rather than single-sample reasoning or adaptive generation.

Multi-trajectory aggregation is an established test-time scaling paradigm that improves reasoning reliability by combining complementary reasoning paths. Like self-consistency and majority voting, our method requires N>1N>1 trajectories, which is inherent to the aggregation setting. Importantly, we do not compare NN-sample aggregation against single-sample inference to claim improvements. Instead, all methods operate on exactly the same NN sampled trajectories, enabling fair comparisons with standard majority voting, confidence-weighted voting, and confidence-based filtering.

The suggested methods, Certaindex, DEER, and ASAG, primarily address complementary problems, including adaptive sampling, early stopping, and generation-time intervention to reduce redundant reasoning. Our method instead improves the aggregation of completed trajectories without modifying generation. These approaches could potentially be combined with ours to construct and aggregate more efficient trajectory pools. We have clarified this distinction in the revised manuscript.

1.2 Computational Overhead (W1, W3)

Our method incurs only modest additional computation beyond standard NN-sample generation.

All aggregation methods share the same trajectory generation cost. Our additional overhead comes primarily from applying the LM head to the penultimate-layer hidden states to obtain an extra vocabulary distribution. Final-layer logits are already available during decoding, while the additional confidence calculations are comparatively inexpensive. We estimate the per-token computational cost as follows:

ModelStandard DecodingWith Confidence DynamicsRelative Cost
Qwen3-1.7B12.62 GFLOPs13.24 GFLOPs1.05×
Qwen3-8B38.73 GFLOPs39.97 GFLOPs1.03×
Nemotron-Nano-4B10.52 GFLOPs11.35 GFLOPs1.08×

These theoretical FLOPs estimates indicate only 3%–8% additional computation per generated token. Computing confidence dynamics only over tail tokens can further reduce this overhead when implemented without full-sequence intermediate-layer projections. We emphasize that our contribution is improving aggregation accuracy under the same multi-sample generation budget, rather than reducing the number of generated tokens. We have clarified this distinction and the associated computational overhead in the revised manuscript.

2. Effectiveness on Coding Tasks (W4)

Our additional evaluation demonstrates that confidence-dynamics-weighted voting also improves code generation performance.

We thank the reviewer for this suggestion. We extend our evaluation to LiveCodeBench v6 using Qwen3-1.7B and Qwen3-8B under sampling budgets N∈{20,25,30}N \in \{20,25,30\} (Appendix F).

ModelSampling PoolStandardCWDWCᵀWDᵀW
Qwen3-1.7BN = 3030.5333.5934.3531.3033.59
N = 2530.6933.7434.1231.6033.51
N = 2030.8433.1333.8932.7533.36
Qwen3-8BN = 3048.8552.6754.2052.6751.91
N = 2549.0852.5253.5152.4452.37
N = 2049.0153.2854.3551.9152.60

Voting accuracy (%) on LiveCodeBench v6. CW and DW denote global confidence- and dynamics-weighted voting; CᵀW and DᵀW use their tail-based counterparts.

Dynamics-weighted voting consistently outperforms standard majority voting and confidence-weighted voting across all six settings, improving over standard voting by 3.05–3.82 percentage points on Qwen3-1.7B and 4.43–5.35 points on Qwen3-8B. These results provide additional evidence that confidence dynamics is useful beyond mathematical and scientific reasoning, extending to code generation.

AuthorsOpenReview ↗

Response to Reviewer TiMu [part 2]

3. Retention Ratio and Generality (W5)

Our revised experiments remove dynamics-specific filtering thresholds while maintaining consistent improvements across different trajectory-pool settings.

As detailed in our General Response, we have redesigned the main experiments to use confidence dynamics directly as a voting weight, without requiring an additional dynamics-based filtering threshold. We compare standard, confidence-weighted, and dynamics-weighted voting on identical trajectory pools under three shared settings: the full pool (100%) and the top-80% and top-50% confidence-filtered subsets. The revised results show that dynamics-weighted voting consistently outperforms its confidence-weighted counterparts across these settings, demonstrating that its effectiveness does not depend on a particular retention threshold. These experiments also clarify the complementary roles of confidence filtering and dynamics weighting.

4. Handling Negative Confidence Dynamics (W6)

We apply a non-negative shift to confidence-dynamics scores before using them as voting weights.

We thank the reviewer for identifying this missing implementation detail. For each sampling pool S(x)\mathcal{S}(x), we compute:

ΔC~(τi)=ΔC(τi)−min⁡τj∈S(x)ΔC(τj).\widetilde{\Delta C}(\tau_i)=\Delta C(\tau_i)-\min_{\tau_j\in\mathcal{S}(x)}\Delta C(\tau_j).

This ensures non-negative voting weights while preserving the relative ordering and pairwise differences of the original scores. We have clarified this implementation in Section 3.2 of the revised manuscript.

5. Ablation of Confidence and Dynamics Filtering (Q1)

Our ablations show that confidence is more effective for filtering trajectories, while confidence dynamics provides a stronger voting weight.

Our revised main experiments (see General Response) already demonstrate that dynamics-weighted voting generally outperforms confidence-weighted voting on identical trajectory pools, including confidence-filtered subsets. To directly address the question, we additionally compare confidence-based and dynamics-based filtering while keeping tail dynamics-weighted voting fixed, using the same top-50% retention ratio.

ModelCF-DᵀWCᵀF-DᵀWDF-DᵀWDᵀF-DᵀW
Qwen3-1.7B49.8449.2448.9649.06
Qwen3-8B68.7169.0968.8768.95
Nemotron-Nano-4B55.2556.1654.6455.45

Overall accuracy (%) under top-50% retention. CF/DF denote global confidence/dynamics filtering, while CᵀF/DᵀF denote tail-based filtering. All methods use tail dynamics-weighted voting (DᵀW).

Confidence-based filtering achieves the highest accuracy on all three models: 49.84%, 69.09%, and 56.16%, compared with 49.06%, 68.95%, and 55.45% using tail dynamics filtering. Together with our weighting ablations, these results support our revised design: confidence-based filtering selects more reliable candidates, while confidence dynamics provides a complementary signal for weighting their contributions during aggregation.

Thank you for the detailed rebuttal and additional experiments. Some of my concerns have been addressed, but I still have the following concerns:

  1. The authors only report an additional 3%–8% in theoretical FLOPs, without providing evaluations of actual inference latency, memory consumption, or end-to-end computational cost.

  2. The proposed method requires generating multiple reasoning trajectories, but the authors do not provide accuracy–compute curves or demonstrate whether comparable accuracy can be achieved with fewer sampled trajectories. Therefore, it remains unclear whether the modest accuracy improvements justify the additional computational cost.

We thank the reviewer for this follow-up.

Response to Concern 1: Actual Inference Latency and Memory Overhead

To complement our theoretical FLOPs analysis, we conducted an additional end-to-end inference benchmark using Qwen3-1.7B on a single NVIDIA H100 GPU. We evaluated 10 AIME problems with 10 trajectories per problem, using identical random seeds, generated sequences, and sequence lengths for both settings.

MetricStandard (C)Ours (C + ΔC)Overhead
Generation Time878 s935 s+6.5%
Throughput1,949 tok/s1,830 tok/s−6.1%
Peak GPU Memory62,464 MiB62,584 MiB+0.19%

The results show that computing inter-layer confidence dynamics introduces only 6.5% additional inference time and 0.19% peak memory overhead, consistent with our theoretical FLOPs estimates (3%–8%). These measurements confirm that confidence dynamics can be extracted with modest additional computational cost, without requiring extra trajectory generation.

Response to Concern 2: Accuracy–Compute Tradeoff

Our Section 4.5 and additional experiments demonstrate that confidence dynamics improves sample efficiency, achieving higher accuracy with substantially fewer trajectories.

We extend our evaluation in Section 4.5 to sampling budgets N∈{3,5,10,15,20,25,30}N \in \{3,5,10,15,20,25,30\} across three models and five reasoning benchmarks. For each problem, we draw 20 random subsets of NN distinct trajectories without replacement from a fixed pool of 50, reporting the mean accuracy. All methods use identical subsets for fair comparison.

As shown in the new accuracy–sampling-budget curves (Appendix E, Figure 7), dynamics-weighted voting consistently outperforms standard majority voting across all evaluated budgets, with an increasingly pronounced advantage over confidence-weighted voting at larger budgets. Notably, tail dynamics-weighted voting with only N=10N=10 surpasses both standard and confidence-weighted voting with N=30N=30 across all three models, even without filtering. These results demonstrate that confidence dynamics enables more effective use of the sampling budget, and offsets its modest additional computational overhead (3%–8% theoretical FLOPs per token).

Reviewer_cRMmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduced a new method for improving test-time generation on certain benchmarks. The method is an improvement over the DeepConf approach, wherein they model the difference in confidence between the last and penultimate layer, ΔC\Delta C, alongside the standard confidence measure introduced in the DeepConf paper. They show that using the additional ΔC\Delta C for filtering and majority weighting improves performance and how their criterion is complementary to just using CC. The paper reads nicely and clearly defines the methodology, but has certain limitations that are explained in the weaknesses and questions sections.

Critical strengths
  1. Improves on prior work.
  2. Very clearly explains the method used to improve performance on benchmarks in a voting setup.
Critical weaknesses
  1. With respect to the filtering and retention ratio it is stated that "ρ=50\rho = 50% provides a robust general-purpose setting across models", but the method is only tested on 3 models, where the retention ratio results for one of them is reported in the appendix, and where the experimental setup and results are slightly different. That this threshold is both robust and general-purpose is not a claim that can be made based on the evidence provided.

  2. No discussion of the mechanism. Please correct me if I am mistaken, but the results show that the tail for correct trajectories have a bigger difference in confidence, ΔC\Delta C, between layers LL and L−1L - 1 then incorrect trajectory, what is the mechanism behind this (Section 3.3)? Furthermore, when you study the approach using layers L−1L-1, L−2L-2, and L−3L-3 the results indicate contrasting behaviors (Section 4.6). Qwen3-1.7B has higher gains when using L−2L-2 than L−1L-1 whereas Qwen3-8b has higher gains when using L−1L-1 as opposed to L−2L-2, and the study is not performed for the third model.

  3. Severely lacking related work section. Related work is important to contextualize the work and should be contained in the main text.

Questions for authors
  1. The paper seems to be somewhat observational. While the method seems somewhat novel and improves on prior work in terms of accuracy, there are no comments or hypotheses as to why it works. Do you have any hypotheses for why it works. A study of why it should and does improve the performance of a language model would improve the relevance of the paper.

  2. Why have you not included Gemma in the experiments through the entire paper, and why keep some results (with different experimental details) in the appendix (e.g.g section 4.4 and section 4.6)?

  3. How have you controlled for the length of a trace in your analysis? It is noted that trace length is often correlated with correctness, but as far as I can tell it isn't discussed more than that.

3 comments
AuthorsOpenReview ↗

Response to Reviewer cRMm [part 1]

1. Retention Ratio and Generalizability (W1)

Our revised experiments remove dynamics-specific filtering thresholds and demonstrate consistent improvements across models and sampling-pool settings.

To reduce reliance on heuristic retention ratios and better isolate the contribution of confidence dynamics, we have redesigned our main experiments (see General Response). Specifically, we now use confidence dynamics directly as a voting weight, allowing fair comparisons with standard majority voting and confidence-weighted voting on the same trajectory pools, either without filtering or after confidence-based filtering. This eliminates the need for a separate dynamics-specific retention threshold.

We also expand our evaluation to three models (Qwen3-1.7B, Qwen3-8B, and Nemotron-3-Nano-4B), each tested on five reasoning benchmarks under a unified experimental protocol. The revised results show that dynamics-weighted voting consistently outperforms its confidence-weighted counterparts across the full pool and the top-80% and top-50% confidence-filtered subsets. These findings indicate that the benefits of confidence dynamics do not depend on a particular retention ratio and provide stronger evidence of its effectiveness across the evaluated models and tasks.

2. Mechanism Analysis and Layer Selection. (W2 & Q1)

2.1 Mechanism: Late-Stage Predictive Consolidation vs. Premature Commitment

We hypothesize that confidence dynamics capture the extent of late-stage predictive refinement, distinguishing continued consolidation from potentially premature commitment. Even with similarly high final confidence, trajectories can exhibit different layer-wise behaviors. Low ΔC\Delta C indicates that predictions are already concentrated in the penultimate layer with limited further refinement, whereas high ΔC\Delta C reflects substantial final-layer updating through top-1 revision or alternative-token suppression.

To investigate this, we divide the top 20% highest-confidence decoding steps of Qwen3-8B on HMMT into five equal-sized groups by ΔC\Delta C. The lowest and highest groups show distinct patterns:

MetricLow ΔC\Delta CHigh ΔC\Delta C
Mean final confidence25.0225.37
Final-layer Top-1 probability0.99900.9996
Top-1 switch rate0.53%22.72%
Alternative-token suppression7.1418.02
Penultimate-layer alternative probability mass0.00550.1720

Notably, 22.72% of high-ΔC\Delta C steps switch the top-1 prediction, and even when the top-1 prediction preserved (77.3%), their alternative-token probability mass drops from 0.0778 to 0.000295, demonstrating substantial refinement after the penultimate layer.

We further find that the correct–incorrect dynamics gap originates almost entirely from ranks 2–20, rather than top-1. Correct trajectories exhibit stronger alternative-token suppression (gap: +0.589, 95% CI: [+0.116, +1.093]). Moreover, after stratifying trajectories by final confidence, stronger alternative-token suppression remains associated with higher accuracy (63.5% vs. 53.7%).

We hypothesize that this difference reflects reliable predictive consolidation versus potentially premature commitment. Such premature commitment may contribute to long-CoT reasoning stagnation, where reflection or verification repeatedly reinforces an existing reasoning direction without substantive revision. This is consistent with our findings in Section 3.3 that weaker dynamics correlate with lower correctness, longer trajectories, and greater stagnation.

2.2 Layer Selection: Informative Final-Layer Comparisons

Our analysis shows that confidence dynamics involving the final layer are consistently informative across models, without requiring a universally optimal reference layer. Our original experiments show that different reference layers can yield different aggregation gains across models. Rather than optimizing the reference depth for each model, we adopt L−(L−1)L-(L-1) as a simple and uniform default.

To further investigate layer selection, we extend our analysis to Qwen3-1.7B, Qwen3-8B, and Nemotron-3-Nano-4B, measuring the correct–incorrect confidence-dynamics gap across different layer pairs. As shown in the new figure (Figure 7 in the revision), earlier adjacent-layer comparisons generally exhibit small gaps, whereas comparisons between the final layer and L−1L-1, L−2L-2, or L−3L-3 yield larger positive gaps across all three models, particularly over the final 2,048 tokens. Notably, for Nemotron-3-Nano-4B, using L−2L-2 or L−3L-3 as the reference produces a slightly larger tail discrimination gap than L−1L-1, suggesting potential benefits from model-specific layer selection. These results support our choice of L−(L−1)L-(L-1) as a simple, consistently informative configuration, while indicating that alternative final-layer comparisons may offer further improvements.

AuthorsOpenReview ↗

Response to Reviewer cRMm [part 2]

3. Related Work (W3)

We have expanded the related work discussion and incorporated it into the main manuscript.

We thank the reviewer for this suggestion. Due to space constraints, the original submission placed part of the related work discussion in the appendix. In the revised manuscript, we have moved and expanded it into Section 5, including comparisons with intermediate-layer prediction analysis, early-exit methods, and cross-layer contrastive decoding. We also clarify our distinction from these approaches: our work focuses on using inter-layer confidence dynamics to assess reasoning-trajectory reliability and improve post-hoc aggregation, rather than interpreting intermediate representations, reducing generation costs, or modifying token generation.

4. Consistency of Model Evaluation (Q2)

We have unified the evaluation protocol across three models to ensure more consistent and directly comparable results.

In the original submission, the computational cost of Gemma4-12B limited some experiments to smaller sampling budgets. The additional inference required for layer-depth analysis was also not completed for Gemma at the submission time, resulting in incomplete or differently configured evaluations listed in Appendix.

To address this, we have revised our experiments to use Qwen3-1.7B, Qwen3-8B, and Nemotron-3-Nano-4B under a unified protocol. All three models are now evaluated on the same five reasoning benchmarks, with an identical sampling budget and repeated subsampling procedure (see General Response). This revision provides a more consistent evaluation across models and avoids the differences in experimental settings present in the original submission.

5. Controlling for Reasoning Trajectory Length (Q3)

Confidence dynamics remains an effective voting signal after length-based filtering, suggesting that its benefits are not solely attributable to shorter reasoning trajectories.

We thank the reviewer for this suggestion. Since trajectory length may correlate with correctness, we conduct additional experiments that retain the shortest 80% or 50% of trajectories in each sampling pool. We then compare different voting strategies on exactly the same length-filtered subsets, isolating the contribution of their weighting signals.

ModelStandardShortest 80% CWShortest 80% DᵀWShortest 50% CWShortest 50% DᵀW
Qwen3-1.7B44.8446.4048.3247.7649.15
Qwen3-8B65.0566.2268.4568.1169.14
Nemotron-3-Nano-4B51.4652.0753.7151.7353.16

Overall accuracy (%) averaged across five benchmarks. CW denotes global confidence-weighted voting, and DᵀW denotes tail confidence-dynamics-weighted voting. All methods within each length-filtered setting use the same retained trajectories.

As shown above, dynamics-weighted voting consistently outperforms confidence-weighted voting across all three models. The improvements are +1.92, +2.23, and +1.64 percentage points under the shortest-80% setting, and +1.39, +1.03, and +1.43 points under the shortest-50% setting, respectively.

These results provide additional evidence that confidence dynamics captures useful information beyond trajectory length, rather than simply benefiting from length-based selection. We have added this analysis to Section 4.6 of the revised manuscript.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

Dear Reviewer cRMm,

Thank you again for your constructive feedback and helpful suggestions.

We would like to kindly follow up on our rebuttal and ask whether our responses have addressed your concerns. We would greatly appreciate any further feedback or questions you may have.

Thank you for your time and consideration.

Reviewer_NwAQOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes a layer confidence difference indicator method to better predict reasoning quality. It computes the confidence gain from the second-to-last layer to the last layer of the LLM, using the same output module. The work supports the hypothesis that higher gain is related to more reliable results through experiments on HMMT, GPQA-D, Olympiad, and Countdown.

Critical strengths

The idea is concise and can be used as a plug-in. It does not need a complex decoding policy and is applicable to a wide range of base inference-time scaling methods, although the paper focuses on the majority voting setting.

The experiments are solid, especially the analysis experiments across hyperparameters, considering a range of thresholds and ratios.

The writing is clear and easy to follow.

Critical weaknesses

The gain is relatively marginal, about 1–2 points across tasks. It is thus questionable whether the method is generalizable and has potential.

Given that the method is simple and concise, I am concerned about its novelty, given related work on cross-layer confidence analysis, such as LogitLens4LLMs, Tuned Lens, and other works such as DoLa, CALM, and LayerSkip.

Questions for authors

See the weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer NwAQ

We thank the reviewer for these constructive comments. We address the concerns about generalizability and novelty through expanded experiments and a clearer distinction from prior work.

1. Generalizability and Effectiveness (W1)

Our expanded experiments demonstrate consistent improvements across multiple models, reasoning benchmarks, and coding tasks, providing stronger evidence of the generalizability and practical potential of confidence dynamics.

(1) Expanded reasoning evaluation. As detailed in our General Response, we revise our evaluation to cover three models (Qwen3-1.7B, Qwen3-8B, and Nemotron-3-Nano-4B) and five reasoning benchmarks. For each problem, we generate a pool of N=50N=50 trajectories and evaluate 20 random subsamples of N=30N=30 distinct trajectories drawn without replacement. All methods share the same subsets to ensure fair comparisons.

Even without trajectory filtering, tail confidence-dynamics-weighted voting outperforms its tail confidence-weighted counterpart by +1.97, +2.12, and +2.06 percentage points on the three models, respectively. These advantages are also maintained under confidence-based filtering, showing that confidence dynamics provides a complementary voting signal across different models and candidate-pool settings without requiring dynamics-specific filtering thresholds.

(2) Additional coding evaluation. We further evaluate our method on LiveCodeBench v6 using Qwen3-1.7B and Qwen3-8B, with sampling budgets N∈{20,25,30}N \in \{20,25,30\}.

ModelSampling PoolStandardCWDWCᵀWDᵀW
Qwen3-1.7BN = 3030.5333.5934.3531.3033.59
N = 2530.6933.7434.1231.6033.51
N = 2030.8433.1333.8932.7533.36
Qwen3-8BN = 3048.8552.6754.2052.6751.91
N = 2549.0852.5253.5152.4452.37
N = 2049.0153.2854.3551.9152.60

Voting accuracy (%) on LiveCodeBench v6. CW and DW denote confidence- and confidence-dynamics-weighted voting, respectively; the superscript T indicates tail-based scoring.

Dynamics-weighted voting consistently outperforms both standard majority voting and confidence-weighted voting across all six settings, improving over standard voting by 3.05–3.82 points on Qwen3-1.7B and 4.43–5.35 points on Qwen3-8B.

Together, these results demonstrate that confidence dynamics provides a useful aggregation signal beyond mathematical and scientific reasoning, extending to code generation.

2. Novelty and Distinction from Prior Work (W2)

Our contribution is not intermediate-layer readout itself, but identifying inter-layer confidence dynamics as a complementary signal of reasoning-trajectory reliability and leveraging it for training-free, post-hoc aggregation.

While the suggested studies also utilize intermediate-layer information, they differ in their objectives and applications:

  • LogitLens4LLMs and Tuned Lens focus on interpreting intermediate Transformer representations through vocabulary-space projections. We build on this general readout technique but investigate whether inter-layer confidence changes can distinguish reliable from unreliable reasoning trajectories, particularly when final confidence alone is insufficient.

  • CALM and LayerSkip leverage intermediate-layer predictions for early-exit inference and computational efficiency. In contrast, our method does not modify generation or perform early exit; it evaluates completed trajectories to improve answer aggregation.

  • DoLa uses token-level contrastive decoding between intermediate and final-layer predictions to improve factuality during generation. Although also evaluated on reasoning tasks, it does not explicitly assess trajectory-level behaviors in modern long-form reasoning, which involves extensive exploration, verification, and reflection. In contrast, we aggregate inter-layer confidence dynamics over complete trajectories or their tail portions to characterize late-stage predictive refinement and identify potentially unreliable high-confidence reasoning, without modifying token generation.

Importantly, we investigate a distinct failure mode: reasoning trajectories can remain highly confident despite producing incorrect answers and exhibiting reasoning stagnation. Our analysis shows that inter-layer confidence dynamics provides additional discrimination within this high-confidence regime and improves subsequent aggregation. Thus, our central contribution is establishing and exploiting the relationship between late-stage confidence dynamics and reasoning-trajectory reliability, rather than introducing another layer-probing, early-exit, or token-level contrastive decoding method.

AuthorsOpenReview ↗

Gentle Follow-up on Our Rebuttal

Dear Reviewer NwAQ,

Thank you again for your constructive feedback and helpful suggestions.

We would like to kindly follow up on our rebuttal and ask whether our responses have addressed your concerns. We would greatly appreciate any further feedback or questions you may have.

Thank you for your time and consideration.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response: Revised Main Results and Evaluation Protocol

We thank the reviewers for their constructive feedback. To better demonstrate the effectiveness of confidence dynamics beyond static confidence and address concerns about generalizability and heuristic thresholds, we have redesigned our aggregation framework and substantially expanded our evaluation.

1. Simplified Aggregation Without Dynamics-Specific Filtering Thresholds

We now use confidence dynamics directly as a voting weight, allowing fair comparisons with standard majority voting and confidence-weighted voting without requiring dynamics-specific filtering thresholds.

Unlike our original joint-filtering approach, the revised method applies dynamics-weighted voting to any given trajectory pool, either without filtering or after confidence-based filtering. We evaluate three shared settings:

  • Direct Voting (100%): All sampled trajectories without filtering.
  • Top-80% Voting: The top 80% of trajectories ranked by confidence.
  • Top-50% Voting: The top 50% of trajectories ranked by confidence.

All voting methods within each setting use exactly the same trajectory pool, isolating the contribution of different weighting signals. These retention ratios are shared evaluation settings rather than model-specific hyperparameters.

2. Expanded Evaluation with Repeated Random Subsampling

We expand our experiments to three models and five reasoning benchmarks under a unified evaluation protocol.

Specifically, we evaluate Qwen3-1.7B, Qwen3-8B, and Nemotron-3-Nano-4B on GPQA-Diamond, HMMT, AIME, SuperGPQA, and Countdown6. For each problem, we generate a fixed pool of N=50N=50 trajectories, then randomly select 30 distinct trajectories without replacement, repeating this procedure 20 times. All methods share the same subsets in each repetition, and we report the mean accuracy across repetitions. This reduces sensitivity to random subset selection while ensuring fair comparisons under identical generation budgets.

3. Consistent Improvements Across Models and Sampling-Pool Settings

As summarized in the revised main table below (the detailed table on five datasets is provided as Table 1 in the revised submission), confidence-dynamics-weighted voting consistently improves aggregation accuracy across the evaluated models and pool settings, even without trajectory filtering. Importantly, even without filtering, tail confidence-dynamics-weighted voting outperforms its tail confidence-weighted counterpart by +1.97, +2.12, and +2.06 percentage points on Qwen3-1.7B, Qwen3-8B, and Nemotron-3-Nano-4B, respectively. The advantage is also maintained under confidence-based filtering, demonstrating that confidence dynamics provides complementary information beyond static confidence without requiring a dynamics-specific threshold.

Overall, these revised results provide stronger evidence for the effectiveness and generalizability of confidence dynamics as a training-free aggregation signal across different models, reasoning benchmarks, and sampling-pool settings. Detailed results are presented in the revised Section 4.2.

ModelStandardCWCᵀWDWDᵀWCF-CWCᵀF-CᵀWCF-DᵀWCᵀF-DᵀWCF-CWCᵀF-CᵀWCF-DᵀWCᵀF-DᵀW
Voting SettingDirectDirectDirectDirectDirectTop-80%Top-80%Top-80%Top-80%Top-50%Top-50%Top-50%Top-50%
Qwen3-1.7B44.8445.9945.9146.6647.8846.8847.1148.3448.5748.3748.1449.8449.24
Qwen3-8B65.0566.0465.9966.7368.1167.0266.6968.4768.2868.4068.2268.7169.09
Nemotron-Nano-4B51.4651.6951.8152.5953.8752.3552.3954.6454.6752.7454.2155.2556.16

We report overall accuracy (%) macro-averaged across five reasoning benchmarks under three sampling-pool settings: Direct Voting (100%), Top-80% Voting, and Top-50% Voting. CW and DW denote confidence- and confidence-dynamics-weighted voting, respectively; CF denotes confidence-based filtering, and the superscript T indicates tail-based scoring.

AuthorsOpenReview ↗

Summary of the Rebuttal and the revised PDF

We thank all three reviewers for their careful reading and for recognizing the simplicity and effectiveness of our confidence dynamics metrics in distinguishing correct from incorrect reasoning. In response to your feedback, we made six changes to the manuscript.

  1. We now revise our main comparison design that uses confidence dynamics directly as voting weights, enabling a fair comparison with majority voting and confidence-weighted voting without dynamics-specific filtering thresholds. Our expanded evaluation shows consistent gains across five datasets and three models under varying sample pool sizes and filtering methods. We aggregate results over repeated random subsamples to reduce the impact of sampling variability (Section 4.2).

  2. We added experiments evaluating our method on a coding task (Appendix G).

  3. We show that the metrics remain effective when we control for reasoning trajectory length (Section 4.6).

  4. We further introduce an online tree search algorithm that uses confidence dynamics and outperforms other baselines (Section 4.3).

  5. We clarify the computational overhead of confidence dynamics and compare how each aggregation method performs as the sampling budget varies (Section 4.5 and Appendix 7).

  6. We clarify the mechanism underlying confidence dynamics and relate our approach to prior work on layer-wise interpretation, early exit, and token reduction (Section 5).

We hope these revisions address the reviewers’ concerns. As the discussion period draws to a close, we would appreciate hearing whether any concerns remain so we can respond before it ends.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.