acceptodds
Under review as a conference paper at ICLR 2027

Believe Your Model: Distribution-Guided Confidence Calibration

Abstract

Large Reasoning Models have demonstrated remarkable performance with the advancement of test-time scaling techniques, which enhance prediction accuracy by generating multiple candidate responses and selecting the most reliable answer. While prior work has shown that internal model signals such as confidence scores can partly indicate response correctness and exhibit a distributional correlation with accuracy, the relative structure of each query's confidence distribution has not been fully utilized to guide answer selection. Motivated by this, we propose DistriVoting, which explicitly exploits this per-query distributional structure during voting. Specifically, our method (1) decomposes the mixed confidence distribution into higher- and lower-mean candidate components using a Gaussian Mixture Model, and (2) applies a reject filter that uses the lower-mean component as counter-evidence to mitigate overlap between the two components. Besides, to further alleviate the overlap from the perspective of the distribution itself, we propose SelfStepConf, which uses step-level confidence to dynamically adjust the inference process, increasing the separation between the two components to improve the reliability of confidence in voting. Experiments across 16 models and 5 benchmarks demonstrate that our method outperforms state-of-the-art approaches.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large reasoning model, test-time scaling, intrinsic confidence

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HqN1OpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes DistriVoting, a method for aggregating multiple LLM reasoning trajectories using model-internal confidence scores. It fits a two-component Gaussian mixture model to the scores for each query, uses the lower-confidence pool to identify an answer for potential rejection, and applies hierarchical confidence-weighted voting to the retained higher-confidence trajectories. The paper also introduces SelfStepConf, which triggers reflection during generation when step-level confidence drops. Experiments across 16 model configurations and five reasoning benchmarks report improvements over sampling and voting baselines.

Critical strengths
  1. The paper presents a clear and well-motivated approach to using model-internal confidence for answer selection without an external reward model. Adapting the filtering threshold to each query’s confidence distribution is also a practical idea.

  2. The experimental evaluation is broad, covering multiple model configurations, five benchmarks, and different sampling budgets. Component ablations help identify where the improvements come from, particularly showing that GMM filtering provides gains.

Critical weaknesses
  1. The BoN baseline is unclear. The standard BoN method will introduce an external reward model to score the final answer, while this work does not rely on such a reward model. Then how do we compare the BoN baseline with the paper's proposed methods? Or does the "BoN" refer to we simply select the trajectory with the highest confidence?

  2. Some useful ablations are missing. Please also compare against ordinary majority voting on the same filtered high-confidence trajectories. It would also help to test answer rejection without refitting the GMM, to show whether refitting is necessary.

  3. SSC needs stronger controls. Comparing confidence-based triggering with random or fixed-interval reflection at matched intervention counts would help explain its gains. Comparisons at equal token or time budgets are also needed, since equal trajectory counts do not imply equal computational cost.

  4. Some experiment results are inconsistent. In Table 17, SSC with Qwen3-14B-NonThinking matches Qwen3-32B-NonThinking on four benchmarks and scores higher on HMMT2025 (12.76 vs. 9.80), yet has a lower average (42.14 vs. 42.78). In Tables 5 and 18, DeepSeek-R1-8B’s baseline confidence on AIME2024 differs (17.52 vs. 17.96), while SSC’s value is 17.92 in both, reversing the direction of the change. The average baseline confidence also differs (14.74 vs. 14.92). Please correct these discrepancies and clarify whether they affect the conclusions.

Questions for authors

Please see the critical weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer HqN1 (1/2)

We thank the reviewer for the valuable comments. Below, we address the concerns about the BoN baseline, component ablations, SSC controls, and consistency of the reported results.

[W1] BoN definition and comparison

BoN-Conf selects the trajectory with highest internal confidence, r⋆=arg⁡max⁡rCrr^\star=\arg\max_r C_r (Eq.1). Candidate generation and answer extraction are shared with WSC and DistriVoting. MoB also uses internal confidence (Appendix C.1).

This distinguishes BoN's scorer from its selection rule. Self-Certainty[R2] uses internal probabilities, while Lightman et al.[R3] use reward models. Our comparison concerns selection under internal scores, not all external reward models.

Revision: Section 4.1 specifies internal scoring. Appendix G.17 clarifies shared candidates and answer extraction.

[W2] Majority voting after filtering and refitting

To test whether filtering helps with simple voting and whether refitting is necessary, we compare the variants in Table R6. It compares majority voting before and after GMM, then varies the aggregator on a fixed positive pool. Refit variants fix the initial pools and rejected answer. Without refit, deletion stays within the original positive pool. With refit, a new mixture is fitted to the remaining full pool after deletion.

Basic denotes unmodified generation. All rows report accuracy (%) on shared working sets at B=128B=128. All + Majority denotes SC.

Table R6. Filtering, aggregation and refitting ablations.

ConfigurationDeepSeek-R1-8BQwen3-32B
All + Majority73.0973.85
GMM + Majority76.5175.66
GMM + WSC76.6475.79
GMM + HierVoting76.8776.01
GMM + Reject + HierVoting, no refit76.8875.95
GMM + Reject + HierVoting, refit76.9575.99

As shown in Table R6, GMM adds 3.42/1.81 points with majority voting unchanged, so sophisticated weighting is not required for this filtering benefit. Refitting adds 0.07/0.04 points over deletion with original assignments retained. It updates the mixture but is unnecessary for the main gain.

Revision: Appendix G.17 adds unweighted voting and refitting ablations on matched candidate pools.

AuthorsOpenReview ↗

Response to Reviewer HqN1 (2/2)

[W3] Controls for trigger timing and matched computation

To test whether trigger timing matters beyond added reflection or tokens, we compare SSC with random and periodic controls in Table R7. Random and periodic controls use the same injection operation and paragraph boundaries. Rates are calibrated by model and benchmark on a pilot without test correctness. Matching targets total interventions per query at the same token budget, not a global mean per trajectory.

For this comparison, all methods use DistriVoting at 2,000,000 generated tokens/query. SSC accuracy is 77.62%/76.40% for DeepSeek/Qwen. Its intervention mean [P25,P75][P_{25},P_{75}] is 200 [81,272] / 154 [61,211]. P25P_{25} and P75P_{75} denote the 25th and 75th percentiles. Δ\Delta is SSC minus control in percentage points, with pointwise 95% confidence intervals (CIs) clustered by query.

The estimates in Table R7 come from 64 frozen pool replays that consume the cap, including reflection and unfinished tokens, with scores and answers taken only from visible prefixes. These are not independent full budget generations or elapsed time measurements. SSC versus Basic is shared with Table R5b. Random/Periodic test timing, unlike paper Tables 6/7's token and segmentation ablations.

Table R7. SSC versus matched reflection controls.

ControlDeepSeek: Δ\Delta [CI]Qwen32B: Δ\Delta [CI]Interventions/query: mean [P25,P75][P_{25},P_{75}], D / Q
Basic+0.78 [0.10, 1.52]+0.47 [−0.04, 0.97]0 [0,0] / 0 [0,0]
Random+0.61 [0.02, 1.20]+0.43 [−0.08, 0.94]198 [80,270] / 154 [60,210]
Periodic+0.50 [−0.08, 1.10]+0.33 [−0.18, 0.84]199 [82,268] / 153 [62,208]

Table R7's point estimates favor SSC over both controls. DeepSeek versus Random has a positive pointwise interval, but several others include zero. As matching is approximate, we also check counts, zero intervention rates and started/completed trajectories by model and benchmark.

Revision: Appendix G.17 combines token budget and trigger controls, with calibration and intervention statistics.

[W4] Table consistency and interpretation

We thank the reviewer for pointing out these inconsistencies. When consolidating the extensive experimental results, we did not synchronize a few updates across tables. The Qwen3-14B-NonThinking benchmark entries in Table 17 will be corrected while keeping its reported average of 42.14%. The Qwen3-32B-NonThinking row and Figure 8 remain unchanged, and Appendix Table 18 will be aligned with Table 5. These reporting corrections do not change our main conclusions.

Revision: Table 17's 14B entries and Table 18's baseline confidences are corrected. Table 5, Figure 8, and Table 17's reported averages remain unchanged.

[R2] Kang et al. "Scalable best-of-n selection for large language models via self-certainty." NeurIPS, 2026.

[R3] Lightman et al. "Let's verify step by step." ICLR, 2024.

I sincerely appreciate the authors' thorough rebuttal. While I agree that this is a solid work and was torn on whether to raise my score, I have decided to keep my rating unchanged. This is mostly a matter of personal taste, as I find the paper to be more engineering-oriented with insufficient deeper insights. Because of this, I have lowered my confidence accordingly. Let's see if confidence-aware voting also works here. Good luck!

AuthorsOpenReview ↗

Response to Reviewer HqN1 (Follow-up)

Thank you very much for your follow-up and for considering this a solid work. We respect that the final judgment is partly a matter of taste. Our presentation may have emphasized the pipeline more than the findings behind it, so we would like to briefly restate the insights that our analyses were designed to establish. The controls added during the discussion further support them.

Our central view is that confidence-based voting is limited less by the confidence signal itself than by how it is used. The paper develops this view through three findings, which we believe are not specific to our pipeline.

1. The useful signal is relative within each query, not absolute. Fixed thresholds and fixed retention ratios assume that the useful cutoff is similar across queries. Our analysis shows that it is not. The best fixed retention ratio for a single model ranges from 6% to 96% across benchmarks (Section 5.3, Figure 3), and every per-query splitter we tested, including K-Means and MeanShift, beats the fixed Top50 ratio in voting accuracy (Table 2). The structure these splitters capture is also not answer identity, since the two most frequent answers differ by only 0.022 in mean confidence (Section 5.4). The cutoff should be read from each query's own confidence distribution, which reflects relative trajectory quality rather than which answer a trajectory gives.

2. The gain comes from changing which trajectories vote, not from ranking them better. One might expect a filter to help only if it ranks correct trajectories well above incorrect ones. Our results show otherwise. Within the GMM positive pool, AUROC is only 0.5831 (Table 2), yet the correct fraction of the voting pool rises from 69.27% to 77.60% after GMM filtering and to 80.41% after rejection (Table 4). Voting depends on the total support each answer receives, so improving pool composition helps even when individual ranking stays weak. Consistently, interval-stratified voting clearly helps a fixed Top50 pool but has little effect after GMM filtering (Appendix F.4). The controls added in Appendix G.17 confirm this. With plain majority voting, GMM filtering adds 3.42/1.81 points for DeepSeek-R1-8B/Qwen3-32B, whereas replacing majority voting with HierVoting on the same pool adds only 0.36/0.35 points (Table 26). For confidence-aware voting, deciding which trajectories vote matters more than designing more elaborate weights.

3. Overlap is the remaining bottleneck, and it can be reduced during generation. Under Gaussian assumptions, Theorems 2.1 and 2.2 show that a larger gap between the correct and incorrect components strictly increases the midpoint tail ratio and a lower bound on multi-answer weighted-voting accuracy. Raising confidence alone is therefore not sufficient. A generation intervention benefits voting when it widens the gap between correct and incorrect trajectories. Section 5.2 tests this prediction for SSC, whose AUROC improves in 23 of 25 model and benchmark settings (Table 15). The added analyses are consistent with this account. The correct-class mean rises more than the incorrect-class mean in all five configurations (Table 27), and SSC's point estimates exceed random and periodic triggers at matched intervention counts and token caps (Table 28), although only DeepSeek-R1-8B versus random has an interval excluding zero. The analysis also marks the boundary of this effect. Pass@KK curves narrow as KK grows (Section 5.5), so SSC improves finite-budget sampling efficiency rather than the reasoning limit.

Together, these findings give a two-factor account of confidence-based voting: where to split each query's own distribution, and how separable that distribution is. DistriVoting addresses the first factor, SSC addresses the second, and the analyses test each factor and its boundary.

We sincerely appreciate your time and thoughtful engagement throughout the review process. Any further questions would be most welcome.

Reviewer_9t5eOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3(+1)
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes a confidence-based voting method for test-time scaling. It uses a two-component Gaussian Mixture Model to separate sampled reasoning trajectories, followed by a rejection filter and hierarchical voting. The authors also introduce SelfStepConf, which triggers self-reflection based on step-level confidence. Experiments across 16 models and five benchmarks show improvements over existing voting methods.

Critical strengths
  1. The method adapts to per-query confidence distributions without requiring additional reward models or training.

  2. The evaluation covers multiple model families, benchmarks, and sampling budgets.

Critical weaknesses
  1. Most improvements come from GMM filtering, add Reject Filter and Hierarchical Voting provides only 0.31 and 0.20 percentage points on the two main models (Table 1), their contribution beyond standard distribution-based filtering is relatively small.

  2. Eq.7 applies negative confidence weights, while the voting operator maximizes weighted support (Eqs.30–32) ,this can favor the least-supported answer rather than the most concentrated negative answer, moreover, the theoretical analysis assumes correctness-conditioned Gaussian distributions, which are not directly established by the unsupervised GMM.

Questions for authors
  1. Why does negating confidence weights in Eq 7 identify reliable counter-evidence?

  2. Please report results under matched token or computational budgets, particularly for SelfStepConf, and provide paired statistical tests for the improvements.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 9t5e (1/2)

We thank the reviewer for the valuable comments. Below, we address the concerns about component contributions, negative pool selection, theoretical scope, and matched computational budgets.

[W1] Contribution beyond GMM filtering

GMM Filter is a core contribution built on a Gaussian mixture assumption for each query's confidence distribution. Its effectiveness supports DistriVoting's central motivation: using distributional priors to improve confidence reliability for voting. Paper Table 2 (DeepSeek-R1-8B) provides evidence for this motivation. Voting accuracy within DistriVoting is 75.10% with Top50, versus 75.19%, 75.50%, and 76.95% with K-Means, MeanShift, and GMM.

Within this design, GMM Filter provides coarse distributional screening, while Reject Filter uses counter-evidence from the negative pool for targeted refinement. To examine this division of roles, Table R3 tracks cumulative filtering and retained-pool quality on DeepSeek-R1-8B (128 trajectories/query, 64 repeats). Its Acc/WAcc measures are those in paper Table 4. Stages I, II and III denote all candidates, after GMM, and after rejection and refitting. Filter Ratio is relative to the original candidate set.

Table R3. Average candidate quality and cumulative filtering.

MetricStage IStage IIStage III
Acc (%)69.2777.6080.41
WAcc (%)69.7477.6880.48
Filter Ratio (%)0.0023.4132.67

As Table R3 shows, the GMM stage produces the larger initial reduction and purity improvement, consistent with its coarse filtering role. From Stage II to III, cumulative filtering increases by 9.26 percentage points alongside further Acc and WAcc gains, consistent with targeted refinement.

Furthermore, DistriVoting supports different base voting methods for both rejection and final selection. HierVoting is one implementation, aggregating weighted votes within confidence bins and then combining the bin winners. Table R5a reports larger point gains from GMM filtering than from adding Reject Filter and HierVoting, whose combined gains have confidence intervals including zero. The central filtering step remains adaptive: GMM fits each query's unlabeled confidence distribution rather than retaining a fixed fraction.

Revision: Appendix G.17 adds stagewise filtering ratios, matched aggregation/refitting controls, and paired uncertainty summaries.

[W2 & Q1] Selection from the negative pool and theoretical scope

Eq.7 nominates a rejection candidate, not a calibrated error probability. For individual trajectories,

arg⁡max⁡r(−Cr)=arg⁡min⁡rCr.\arg\max_r(-C_r)=\arg\min_r C_r.

Under weighted answer aggregation, however, selection also depends on how many trajectories support each answer:

arg⁡min⁡a∑r:Ar=aCr\arg\min_a\sum_{r:A_r=a}C_r

HierVoting aggregates within and across intervals. Moreover, if equivalent answers are grouped and AposA_{\mathrm{pos}} is correct, the disagreement gate rejects only incorrect nominees. High removal precision can therefore reflect the gate, not the signed score alone.

To test whether answer aggregation adds value, Table R4 compares Eq.7 with MinC, which selects the lowest-confidence trajectory's answer from the negative pool. Only nomination changes, with initial pools and all other voting steps fixed. Each rule gates its own nominee, so triggers can differ. Using unmodified generation (Basic), B=128B=128, and 20,352 evaluations per model, trigger rates and final accuracy use all evaluations, while rates of removing correct answers use each rule's triggers. Fix/harm counts and Δ\Delta compare against GMM+HierVoting without rejection. Δ\Delta is in percentage points.

Table R4. Controls for negative answer selection.

ModelRuleTrigger %Correct removal %Fix / harmΔ\Delta
DeepSeek-R1-8BEq.731.202.0986 / 70+0.08
DeepSeek-R1-8BMinC31.952.4084 / 76+0.04
Qwen3-32BEq.728.702.8171 / 75−0.02
Qwen3-32BMinC29.452.9570 / 78−0.04

Table R4 shows small net accuracy changes under both nomination rules. The direct paired Eq.7 minus MinC differences are +0.04 [−0.11, 0.19] and +0.02 [−0.12, 0.16], with pointwise 95% CIs clustered by query. Both intervals include zero, so this comparison does not establish superiority over MinC. The nomination rule remains an empirical refinement, not the main source of gain.

The Gaussian theory similarly motivates separation without guaranteeing unsupervised label recovery. Theorem 2.2 assumes independent Gaussian scores conditional on correctness and fixed nonnegative weights, variances and incorrect mean. The pool diagnostics in Appendix G.17 provide the empirical link: higher mean pools contain 16.8/9.3 points more correct trajectories despite fitting without labels.

Revision: Sections 2.3 and 3.2 clarify theoretical scope and signed nomination. Appendix G.17 explains gate effects and the MinC comparison.

AuthorsOpenReview ↗

Response to Reviewer 9t5e (2/2)

[Q2] Equal token budgets and paired statistics

Matched trajectory budgets. To distinguish the contributions of filtering, later voting stages, and SSC, we compare their incremental gains in Table R5a. Its first contrast fixes WSC and Basic working sets to isolate filtering, without extra generation or a reward model. Basic denotes unmodified generation. Its selectors share working sets at B=128B=128, with SSC paired by query and repeat. Differences are in percentage points, with pointwise 95% confidence intervals (CIs) clustered by query. pHp_{\mathrm H} is the two-sided query-paired permutation p value, Holm-corrected for three contrasts per model.

Table R5a. Component gains at matched trajectory budgets.

ContrastDeepSeek-R1-8B: Δ\Delta [CI]pHp_{\mathrm H}Qwen3-32B: Δ\Delta [CI]pHp_{\mathrm H}
GMM+WSC − WSC-Top50+1.89 [0.35, 3.27]0.042+0.57 [−0.01, 1.15]0.162
DistriVoting − GMM+WSC+0.31 [−0.10, 0.72]0.140+0.20 [−0.13, 0.53]0.240
DistriVoting+SSC − DistriVoting+0.89 [0.14, 1.65]0.048+0.54 [−0.07, 1.13]0.162

In Table R5a, the second contrast adds Reject Filter and HierVoting to GMM+WSC under Basic, with both confidence intervals including zero. In paper Tables 10 and 11, the same addition under SSC yields 0.60/0.43 points for DeepSeek-R1-8B/Qwen3-32B. This is not SSC's own gain.

To test whether the gains persist at equal token budgets, we supplement the B=128B=128 results with the comparison in Table R5b. Generation is capped at 2,000,000 tokens per query, including reflection and unfinished trajectories. The 64 offline replays consume fixed random orderings of frozen pools. Only budget prefixes provide scores and answers, with no extra stop at 128 trajectories.

For this comparison, Basic selectors share a budget prefix, while SSC uses its own pool at the same cap. All replays consume that cap. These 64 matched replays are not independent full budget generations or online evaluations. Their intervals describe query variation conditional on available pools, not variation from independently regenerating them. Table R5b reports differences in percentage points, with pointwise 95% bootstrap intervals stratified by benchmark and clustered by query.

Table R5b. Gains under a matched token budget.

ContrastDeepSeek-R1-8BQwen3-32B
GMM+WSC − WSC-Top50+1.90 [0.28, 3.40]+0.54 [−0.06, 1.15]
DistriVoting − GMM+WSC+0.23 [−0.10, 0.55]+0.21 [−0.08, 0.48]
DistriVoting+SSC − DistriVoting+0.78 [0.10, 1.52]+0.47 [−0.04, 0.97]

Table R5b reports positive point gains at equal token budgets, with some paired intervals including zero. DistriVoting's corresponding absolute accuracies without/with SSC are 76.84%/77.62% on DeepSeek and 75.93%/76.40% on Qwen32B. These comparisons do not guarantee uniform significance or equal elapsed time.

Revision: Appendix G.17 reports matched token budget results, replay protocols, and conditional uncertainty.

Reviewer_9t5eOpenReview ↗

response to rebuttal

Thank you for the clarifications and additional experiments, which address my concerns, I will raise my score, good luck

AuthorsOpenReview ↗

Response to Reviewer 9t5e (Follow-up)

Thank you very much for your follow-up and for confirming that your concerns have been resolved. We are glad that the clarifications and additional experiments addressed them. We appreciate your time and support, and we wish you all the best!

Reviewer_SV5ZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses inference-time answer selection mechanisms, motivated by the Test-Time Scaling paradigm. The authors propose a new answer selection mechanism, DistriVoting, that examines the per-query confidence distributions across candidate generations. The authors fit a two-component GMM to the confidence of candidate generations to a prompt, removing the answer that looks the most clearly wrong (measured as the lowest confidence answer in the lower-mean component) via a rejection filter, and then after re-fitting the GMM take the answer as a vote over the highest mean component. On DeepSeek-R1-8B and Qwen3-32B across five benchmarks, DistriVoting leads to modest but persistent improvements over other test-time scaling answer selection methods. Various ablations support the authors’ formulation of DistriVoting using a two-component GMM and support DistriVoting’s effectiveness across various numbers of generations. Motivated by theoretical results suggesting that the greater the separation between GMM components’ means leads to greater accuracy of the higher-mean component’s answer, the authors additionally introduce StepStepConf, a forced reflection mechanism applied when a generation’s confidence level dips below a pre-specified threshold.

Critical strengths

This paper offers an interesting approach to Test-Time Scaling answer selection, and offers performance increases over several relevant baselines. The empirical scale of the paper is extensive, and I appreciate the number of benchmarks evaluated, as well as the number of ablations. These provide good evidence supporting the effectiveness of DistriVoting.

Critical weaknesses

This paper assumes a positive correlation between confidence and accuracy that does not necessarily exist. LLMs, especially those post-trained to reason, are often systematically overconfident, and the framework presented here cannot distinguish between a confidently correct and a confidently incorrect answer. Indeed, it seems this framework would be more vulnerable to errors caused by confidentially incorrect generations than simpler frameworks such as Best of N, which provide equal weight to less confident and more confident answers. Table 2 appears to support this hypothesis, with an AUROC near chance across all clustering methods (more knowledge of the balance between classes would help me evaluate accuracy in this table). This challenges the method’s effectiveness, as it would exacerbate a particularly harmful type of LLM error and potentially reduce robustness to out-of-distribution inputs.

Also, the link between StepStepConf and increasing the gap between the Gaussian component means is not justified. I understand StepStepConf as primarily a reflection mechanism, and while I can see how it could rescue trajectories in which a mistake is suddenly made, I do not understand how it could increase the confidence gap between correct and incorrect trajectories. In my read of the paper, the authors did not provide a clear justification for this. Additional empirical evidence (beyond section 5.2, which is one model and one benchmark) or theoretical results would help establish the effectiveness of StepStepConf.

Questions for authors

[I appreciate this work and am open to updating my score if weakness concerns are convincingly addressed during the rebuttal]

4 comments
AuthorsOpenReview ↗

Response to Reviewer SV5Z (1/2)

We thank the reviewer for the valuable comments. Below, we address the concerns about confidence reliability, confidently incorrect answers, and SSC's separation mechanism.

[W1] Confidence reliability and confidently incorrect answers

Eq.1 is not a calibrated correctness probability. Following DeepConf[R1] and Self-Certainty[R2], we test whether internal scores improve selection despite overconfidence, including the requested BoN comparison.

To test whether DistriVoting improves selection despite confidently incorrect candidates, we compare it with the baselines in Table R1. We draw 64 subsamples of B=128B=128 from frozen pools of unmodified generation (Basic), comparing selectors on identical candidates. The error subset contains an incorrect answer among the 26 highest confidence scores, but every selector still receives all 128 candidates. SC votes equally, whereas BoN selects by score. Each model has 20,352 evaluations. The DeepSeek/Qwen subsets contain 12,000/16,000 evaluations across 236/278 queries, including sets with no correct candidate. Here, Δ\Delta reports DistriVoting's accuracy gain over each comparator in percentage points, with query-clustered pointwise 95% confidence intervals (CIs).

Table R1. Voting gains over baseline selectors.

ModelComparatorAll queries: Δ\Delta [CI]Error subset: Δ\Delta [CI]
DeepSeek-R1-8BSC+3.86 [1.40, 6.29]+2.60 [0.34, 4.90]
DeepSeek-R1-8BBoN-Conf+3.52 [1.12, 5.88]+2.50 [0.40, 4.71]
DeepSeek-R1-8BWSC-Top50+2.20 [0.48, 3.96]+0.84 [−0.62, 2.30]
Qwen3-32BSC+2.14 [0.55, 3.83]+1.50 [−0.14, 3.21]
Qwen3-32BBoN-Conf+2.12 [0.50, 3.76]+1.50 [−0.10, 3.17]
Qwen3-32BWSC-Top50+0.77 [−0.12, 1.70]+0.12 [−0.60, 0.85]

As shown in Table R1, both overall BoN contrasts have positive intervals. For the BoN comparison on the error subset, DeepSeek also has a positive interval, while Qwen remains uncertain.

The selection gains in Table R1 need not imply strong ranking within the retained pool. Filtering can increase pool purity even when correct and incorrect scores overlap. Paper Table 2's GMM AUROC of 0.5831 concerns positive pool quality (Appendix G.2). In separate pool diagnostics, mean AUROC within queries is 0.7410/0.6820 before filtering and 0.5980/0.5600 in positive pools for DeepSeek/Qwen32B. AUROC excludes sets missing a class, whereas final accuracy includes all sets. By contrast, correct fractions in higher mean pools exceed those in lower mean pools by 16.8/9.3 points. Paper Table 4 also reports 69.27% → 77.60% → 80.41% correctness before filtering, after GMM and after rejection.

Besides, paper Table 4 provides the class-balance context: unfiltered class fractions are 60.36%, 64.54%, 86.83%, 79.92%, and 81.22%, averaging 69.27% when weighted by question counts. This imbalance informs accuracy interpretation but does not itself explain near-chance AUROC. Together, the ranking, pool quality and voting results support selection utility on the evaluated distributions, not universal error detection or OOD robustness.

Revision: Sections 2.1 and 4.1 clarify confidence and BoN scoring, with reliability diagnostics in Appendix G.17.

AuthorsOpenReview ↗

Response to Reviewer SV5Z (2/2)

[W2] Why SSC can increase confidence separation

Mechanism: targeted internal checking. SSC injects wait when step confidence is below its adaptive threshold and declining. If the model can check task constraints, reconsideration may corroborate valid reasoning or expose contradictions in faulty reasoning. This feedback changes the context for later predictions, potentially strengthening a supported answer or weakening an inconsistent one. Eq.1 measures confidence at the later step containing the final answer. Eq.16 preserves sorted probabilities, ruling out direct score inflation at injection. This is a mechanism hypothesis based on existing model ability, not an added verifier or guaranteed correction.

Prediction and evidence. Let Δμc\Delta\mu_c and Δμi\Delta\mu_i denote paired changes in correct and incorrect class means over the same valid queries. The mechanism predicts Δμc>Δμi\Delta\mu_c>\Delta\mu_i, yielding

Δg=Δμc−Δμi>0.\Delta g=\Delta\mu_c-\Delta\mu_i>0.

Fig.2's DeepSeek-R1-8B/HMMT2025 example shows mean changes of 16.74 → 18.00 for correct answers and 13.56 → 12.96 for incorrect answers.

To test this asymmetry across settings, Table R2 combines reported AUROC with new class-mean and standardized separation statistics. It retains paper Table 15's five model AUROC pairs, covering five benchmarks with improvements in 23 of 25 settings. New statistics use 128 trajectories per method/query on NN common valid queries out of 318. Mean changes are in confidence score units. To account for score variability, we use

dq=μq,c−μq,i(sq,c2+sq,i2)/2.d_q=\frac{\mu_{q,c}-\mu_{q,i}}{\sqrt{(s_{q,c}^2+s_{q,i}^2)/2}}.

For the new statistics in Table R2, s2s^2 denotes sample variance and Δd\Delta d averages dSSC−dBasicd_{\mathrm{SSC}}-d_{\mathrm{Basic}}, with query-clustered pointwise 95% CIs. We include only queries where both methods have at least two samples per class and a positive denominator. This paired subset differs from the reported AUROC evaluation and can exclude queries becoming all correct or all incorrect.

Table R2. Class confidence shifts and separation under SSC.

ModelReported AUROC: Basic / SSCΔμc\Delta\mu_cΔμi\Delta\mu_iΔd\Delta d [CI]NN
DeepSeek-R1-8B0.7701 / 0.8060+0.42+0.09+0.21 [0.08, 0.34]274
Qwen3-8B0.7023 / 0.7978+0.61+0.10+0.38 [0.20, 0.55]301
Qwen3-14B0.7007 / 0.7941+0.36−0.12+0.31 [0.12, 0.48]286
Qwen3-14B-NonThinking0.6763 / 0.7175+0.18−0.04+0.15 [0.04, 0.25]292
Qwen3-32B0.7129 / 0.8008+0.29−0.08+0.26 [0.09, 0.42]281

Table R2 shows mean shifts matching the predicted asymmetry and improved separation after accounting for score variability. Both means may increase while the gap widens. These results do not prove the proposed mechanism or establish a unique causal explanation. Class membership changes preclude interpreting these shifts as causal gains for fixed trajectories. Theorems 2.1 and 2.2 motivate separation's usefulness under their assumptions, not universal SSC improvement.

Revision: Appendix G.17 adds the SSC mechanism hypothesis, separation statistics, and validity criteria.

[R1] Fu et al. "Deep think with confidence." ICLR, 2026.

[R2] Kang et al. "Scalable best-of-n selection for large language models via self-certainty." NeurIPS, 2026.

Thank you to the authors for their thorough response. I find the additional results convincing, and I have updated my score to a 3. In the camera ready version the authors may wish to revise the manuscript further for clarity. For instance, many of the experimental descriptions are condensed to the point where they take several reads to understand, and details such as the fact that Table 2 and Table 4 describe the same set of trajectories are not immediately clear. Nevertheless, this is a solid work.

AuthorsOpenReview ↗

Response to Reviewer SV5Z (Follow-up)

Thank you very much for your follow-up, for confirming that your concerns have been resolved, and for your valuable comments on the presentation. We have clarified the experimental descriptions as follows:

Revision:

  • Section 5.1: separated the overview into method, analysis, and protocol paragraphs, and stated the default of 128 trajectories per query with 64 repetitions.
  • Section 5.2: stated that the three variants share the same candidate pool.
  • Section 5.3: defined the Table 2 metrics, including Acc, WAcc, AUROC, Voting Acc, and Predict Acc.
  • Section 6.1: stated that Table 4 reuses the Table 2 candidate pools, that Stage II is the GMM pool, and that Stage III reports retained-pool accuracy rather than final voting accuracy.

Your comments led to clear improvements, and the corresponding results are all included in the revised paper. We appreciate your time and support again!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.