- Rating
- 2
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes DistriVoting, a method for aggregating multiple LLM reasoning trajectories using model-internal confidence scores. It fits a two-component Gaussian mixture model to the scores for each query, uses the lower-confidence pool to identify an answer for potential rejection, and applies hierarchical confidence-weighted voting to the retained higher-confidence trajectories. The paper also introduces SelfStepConf, which triggers reflection during generation when step-level confidence drops. Experiments across 16 model configurations and five reasoning benchmarks report improvements over sampling and voting baselines.
Critical strengths
-
The paper presents a clear and well-motivated approach to using model-internal confidence for answer selection without an external reward model. Adapting the filtering threshold to each query’s confidence distribution is also a practical idea.
-
The experimental evaluation is broad, covering multiple model configurations, five benchmarks, and different sampling budgets. Component ablations help identify where the improvements come from, particularly showing that GMM filtering provides gains.
Critical weaknesses
-
The BoN baseline is unclear. The standard BoN method will introduce an external reward model to score the final answer, while this work does not rely on such a reward model. Then how do we compare the BoN baseline with the paper's proposed methods? Or does the "BoN" refer to we simply select the trajectory with the highest confidence?
-
Some useful ablations are missing. Please also compare against ordinary majority voting on the same filtered high-confidence trajectories. It would also help to test answer rejection without refitting the GMM, to show whether refitting is necessary.
-
SSC needs stronger controls. Comparing confidence-based triggering with random or fixed-interval reflection at matched intervention counts would help explain its gains. Comparisons at equal token or time budgets are also needed, since equal trajectory counts do not imply equal computational cost.
-
Some experiment results are inconsistent. In Table 17, SSC with Qwen3-14B-NonThinking matches Qwen3-32B-NonThinking on four benchmarks and scores higher on HMMT2025 (12.76 vs. 9.80), yet has a lower average (42.14 vs. 42.78). In Tables 5 and 18, DeepSeek-R1-8B’s baseline confidence on AIME2024 differs (17.52 vs. 17.96), while SSC’s value is 17.92 in both, reversing the direction of the change. The average baseline confidence also differs (14.74 vs. 14.92). Please correct these discrepancies and clarify whether they affect the conclusions.
Questions for authors
Please see the critical weakness.
4 comments
Response to Reviewer HqN1 (2/2)
[W3] Controls for trigger timing and matched computation
To test whether trigger timing matters beyond added reflection or tokens, we compare SSC with random and periodic controls in Table R7. Random and periodic controls use the same injection operation and paragraph boundaries. Rates are calibrated by model and benchmark on a pilot without test correctness. Matching targets total interventions per query at the same token budget, not a global mean per trajectory.
For this comparison, all methods use DistriVoting at 2,000,000 generated tokens/query. SSC accuracy is 77.62%/76.40% for DeepSeek/Qwen. Its intervention mean is 200 [81,272] / 154 [61,211]. and denote the 25th and 75th percentiles. is SSC minus control in percentage points, with pointwise 95% confidence intervals (CIs) clustered by query.
The estimates in Table R7 come from 64 frozen pool replays that consume the cap, including reflection and unfinished tokens, with scores and answers taken only from visible prefixes. These are not independent full budget generations or elapsed time measurements. SSC versus Basic is shared with Table R5b. Random/Periodic test timing, unlike paper Tables 6/7's token and segmentation ablations.
Table R7. SSC versus matched reflection controls.
| Control | DeepSeek: [CI] | Qwen32B: [CI] | Interventions/query: mean , D / Q |
|---|---|---|---|
| Basic | +0.78 [0.10, 1.52] | +0.47 [−0.04, 0.97] | 0 [0,0] / 0 [0,0] |
| Random | +0.61 [0.02, 1.20] | +0.43 [−0.08, 0.94] | 198 [80,270] / 154 [60,210] |
| Periodic | +0.50 [−0.08, 1.10] | +0.33 [−0.18, 0.84] | 199 [82,268] / 153 [62,208] |
Table R7's point estimates favor SSC over both controls. DeepSeek versus Random has a positive pointwise interval, but several others include zero. As matching is approximate, we also check counts, zero intervention rates and started/completed trajectories by model and benchmark.
Revision: Appendix G.17 combines token budget and trigger controls, with calibration and intervention statistics.
[W4] Table consistency and interpretation
We thank the reviewer for pointing out these inconsistencies. When consolidating the extensive experimental results, we did not synchronize a few updates across tables. The Qwen3-14B-NonThinking benchmark entries in Table 17 will be corrected while keeping its reported average of 42.14%. The Qwen3-32B-NonThinking row and Figure 8 remain unchanged, and Appendix Table 18 will be aligned with Table 5. These reporting corrections do not change our main conclusions.
Revision: Table 17's 14B entries and Table 18's baseline confidences are corrected. Table 5, Figure 8, and Table 17's reported averages remain unchanged.
[R2] Kang et al. "Scalable best-of-n selection for large language models via self-certainty." NeurIPS, 2026.
[R3] Lightman et al. "Let's verify step by step." ICLR, 2024.
I sincerely appreciate the authors' thorough rebuttal. While I agree that this is a solid work and was torn on whether to raise my score, I have decided to keep my rating unchanged. This is mostly a matter of personal taste, as I find the paper to be more engineering-oriented with insufficient deeper insights. Because of this, I have lowered my confidence accordingly. Let's see if confidence-aware voting also works here. Good luck!
Response to Reviewer HqN1 (Follow-up)
Thank you very much for your follow-up and for considering this a solid work. We respect that the final judgment is partly a matter of taste. Our presentation may have emphasized the pipeline more than the findings behind it, so we would like to briefly restate the insights that our analyses were designed to establish. The controls added during the discussion further support them.
Our central view is that confidence-based voting is limited less by the confidence signal itself than by how it is used. The paper develops this view through three findings, which we believe are not specific to our pipeline.
1. The useful signal is relative within each query, not absolute. Fixed thresholds and fixed retention ratios assume that the useful cutoff is similar across queries. Our analysis shows that it is not. The best fixed retention ratio for a single model ranges from 6% to 96% across benchmarks (Section 5.3, Figure 3), and every per-query splitter we tested, including K-Means and MeanShift, beats the fixed Top50 ratio in voting accuracy (Table 2). The structure these splitters capture is also not answer identity, since the two most frequent answers differ by only 0.022 in mean confidence (Section 5.4). The cutoff should be read from each query's own confidence distribution, which reflects relative trajectory quality rather than which answer a trajectory gives.
2. The gain comes from changing which trajectories vote, not from ranking them better. One might expect a filter to help only if it ranks correct trajectories well above incorrect ones. Our results show otherwise. Within the GMM positive pool, AUROC is only 0.5831 (Table 2), yet the correct fraction of the voting pool rises from 69.27% to 77.60% after GMM filtering and to 80.41% after rejection (Table 4). Voting depends on the total support each answer receives, so improving pool composition helps even when individual ranking stays weak. Consistently, interval-stratified voting clearly helps a fixed Top50 pool but has little effect after GMM filtering (Appendix F.4). The controls added in Appendix G.17 confirm this. With plain majority voting, GMM filtering adds 3.42/1.81 points for DeepSeek-R1-8B/Qwen3-32B, whereas replacing majority voting with HierVoting on the same pool adds only 0.36/0.35 points (Table 26). For confidence-aware voting, deciding which trajectories vote matters more than designing more elaborate weights.
3. Overlap is the remaining bottleneck, and it can be reduced during generation. Under Gaussian assumptions, Theorems 2.1 and 2.2 show that a larger gap between the correct and incorrect components strictly increases the midpoint tail ratio and a lower bound on multi-answer weighted-voting accuracy. Raising confidence alone is therefore not sufficient. A generation intervention benefits voting when it widens the gap between correct and incorrect trajectories. Section 5.2 tests this prediction for SSC, whose AUROC improves in 23 of 25 model and benchmark settings (Table 15). The added analyses are consistent with this account. The correct-class mean rises more than the incorrect-class mean in all five configurations (Table 27), and SSC's point estimates exceed random and periodic triggers at matched intervention counts and token caps (Table 28), although only DeepSeek-R1-8B versus random has an interval excluding zero. The analysis also marks the boundary of this effect. Pass@ curves narrow as grows (Section 5.5), so SSC improves finite-budget sampling efficiency rather than the reasoning limit.
Together, these findings give a two-factor account of confidence-based voting: where to split each query's own distribution, and how separable that distribution is. DistriVoting addresses the first factor, SSC addresses the second, and the analyses test each factor and its boundary.
We sincerely appreciate your time and thoughtful engagement throughout the review process. Any further questions would be most welcome.
Response to Reviewer HqN1 (1/2)
We thank the reviewer for the valuable comments. Below, we address the concerns about the BoN baseline, component ablations, SSC controls, and consistency of the reported results.
[W1] BoN definition and comparison
BoN-Conf selects the trajectory with highest internal confidence, r⋆=argmaxrCr (Eq.1). Candidate generation and answer extraction are shared with WSC and DistriVoting. MoB also uses internal confidence (Appendix C.1).
This distinguishes BoN's scorer from its selection rule. Self-Certainty[R2] uses internal probabilities, while Lightman et al.[R3] use reward models. Our comparison concerns selection under internal scores, not all external reward models.
Revision: Section 4.1 specifies internal scoring. Appendix G.17 clarifies shared candidates and answer extraction.[W2] Majority voting after filtering and refitting
To test whether filtering helps with simple voting and whether refitting is necessary, we compare the variants in Table R6. It compares majority voting before and after GMM, then varies the aggregator on a fixed positive pool. Refit variants fix the initial pools and rejected answer. Without refit, deletion stays within the original positive pool. With refit, a new mixture is fitted to the remaining full pool after deletion.
Basic denotes unmodified generation. All rows report accuracy (%) on shared working sets at B=128. All + Majority denotes SC.
Table R6. Filtering, aggregation and refitting ablations.
As shown in Table R6, GMM adds 3.42/1.81 points with majority voting unchanged, so sophisticated weighting is not required for this filtering benefit. Refitting adds 0.07/0.04 points over deletion with original assignments retained. It updates the mixture but is unnecessary for the main gain.
Revision: Appendix G.17 adds unweighted voting and refitting ablations on matched candidate pools.