- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper studies when score-based weighting improves plurality voting over sampled language-model responses. Its main theoretical result characterizes when a bounded, strictly positive score-only weighting rule can improve the asymptotic error probability bound, under the assumptions of i.i.d. sampling and a uniquely correct population mode. The criterion depends on whether the correct answer’s conditional score distribution lies outside the convex hull of those of the most frequent incorrect answers. The paper also characterizes weighting rules that universally preserve the correct population mode or prevent accuracy degradation under exact calibration, and provides guarantees for score-based tie-breaking and harmful winner reversals. Validation combines constructed distributions, numerical checks, and AIME experiments across three language models. At an eight-candidate budget, raw-score weighting improves accuracy by 2.78 percentage points for Math-7B and 2.91 percentage points for Phi-4, with most gains arising from resolving count ties.
Critical strengths
-
Comprehensive analysis. The paper examines score-weighted voting from several perspectives, and the theoretical results appear reasonable and provide useful insights into when weighting helps or hurts.
-
Attention to voting ties. The paper explicitly studies tie-breaking techniques in LLM test-time inference, which I think is new and interesting to the literature.
Critical weaknesses
-
Limited treatment of adaptive inference. The paper mainly studies voting under fixed sampling budgets or in the asymptotic limit. However, it remains unclear how the proposed weighting rules could support adaptive sampling, stopping, and aggregation. This limits their practical guidance for settings where the inference budget is chosen dynamically.
-
Practical value of tie-breaking needs further validation. The main empirical gains are demonstrated with eight candidates, where count ties may be relatively common. It would be useful to examine whether these gains persist at larger budgets, such as 40 candidates, where ties may become less frequent. Another natural baseline is to resolve a tie by sampling additional candidates until a unique count winner emerges. Comparing accuracy and sampling cost against this baseline would help establish the practical value of score-based tie-breaking.
-
The limitation of a correct population mode assumption. The main theoretical results assume that the correct answer is already the unique population mode. However, an important motivation for using scores is to address cases where this basic assumption of self-consistency fails. In particular, the paper could better explain when retaining one base vote per candidate through is preferable to directly using scores or selecting the highest-scoring candidate. A clearer comparison with Best-of-N, especially when the population mode is incorrect, would strengthen the motivation of introducing .
-
The presentation is difficult to follow. The paper contains repeated qualifications and defensive explanations that interrupt the main argument. Consolidating these caveats and presenting a clearer distinction between the central results and supplementary details would improve readability.
Questions for authors
Please see my critical weakness.
3 comments
I genuinely like the core idea of combining majority voting with score weighting. Thank you for the detailed response, which addresses several of my initial concerns.
However, a few important questions and issues remain:
-
Adaptive sampling vs. fixed-budget sampling: Regarding my first point, my main question is whether your framework can genuinely support adaptive sampling rather than relying on fixed-budget sampling (or merely breaking ties at the end of a fixed budget)—for example, how does it perform in an early-stopping setting compared with existing methods such as Adaptive Self-Consistency?
-
Scope of tie-breaking gains: Regarding my second point, my concern is not addressed simply by scaling from 8 to 40 samples. If you continue increasing the sampling budget (e.g., to 80) or switch to a stronger LLM, ties will naturally occur even less frequently. Perhaps the authors should explicitly scope their claim and highlight that their method is particularly helpful for less consistent LLMs (or harder regimes where output variance is high).
-
Theoretical characterization of and comparison between and : Regarding my third point, it is unclear where the threshold 2.2 in comes from. A promising improvement would be to add formal theoretical results (e.g., theorems or propositions) that rigorously characterize the required conditions and this exact threshold. Furthermore, the current experimental results do not seem to convincingly support that consistently outperforms .
-
Revision quality and presentation: Thank you for the significant effort you put into revising the manuscript. However, as Reviewer axPD noted, "such a substantially revised manuscript would be better evaluated through a new review cycle." Moreover, I am still not fully satisfied with the revised version. For example, (i) The newly added table on Page 2 does not work well and is likely to confuse readers rather than clarify the main message; (ii) Experimental Findings: Organizing the experimental section into 9 Findings dilutes the focus. Highlighting the 3 most important findings would be much more effective. (iii) Inconsistent Notation: Notation inconsistencies significantly detract from the paper's readability. For example, at least three different symbols are used for probability throughout the manuscript (, , and ). Please carefully check these instances and streamline the notation to improve overall readability.
I hope these comments can help the authors further improve their paper.
Thank you for your continued engagement and support for the core idea. Your follow-up identifies four points we should make clearer.
1. Adaptive stopping
Choosing when to stop and choosing how to combine answers are separate decisions. Our weighting rules can be applied when sampling stops based on the answers seen so far.
Proposition 7 remains valid at such stopping times under independent sampling and per-problem calibration. It bounds how often the half-floor rule overturns a correct count leader; it does not establish that the leader is correct.
Our experiment compares scoring a tie with sampling until one answer leads. We have not run a direct comparison with Adaptive Self-Consistency.
2. Where tie-breaking helps
Tie-breaking is useful when generated answers disagree enough to tie. This includes small budgets and difficult problems, including difficult problems for strong models. Answer disagreement is the relevant factor, rather than model size alone.
Full weighting also changes answers outside ties. Across 362 AIME problems, it beats count-first by 2.21 percentage points [0.83, 3.68] at 40 solutions. We distinguish this benefit from tie-breaking (Section 6).
3. Where the threshold 2.2 comes from
The threshold follows exactly from Equation (14). For weight c + score, each answer receives a base contribution from its frequency plus its score contribution. In that example, the expected totals per sample are:
- Correct answer: 0.35c + 0.19
- Leading wrong answer: 0.40c + 0.08
- Other wrong answer: 0.25c + 0.08
The correct answer beats both exactly when 0 ≤ c < 2.2, because:
0.35c + 0.19 > 0.40c + 0.08 exactly when c < 2.2.
Under independent sampling, these expected totals determine the unique long-run winner. Thus, 2.2 is an exact threshold for this example, rather than a fitted parameter or a universal recommendation.
The base controls how much agreement counts relative to scores. Its benefit depends on the problem. The separate half-floor rule has a precise preservation guarantee under calibration (Theorem 2), rather than a guarantee of always beating raw scores.
4. Presentation and review scope
We agree with your specific suggestions: move the page-2 table to the appendix, focus the experiments on three main findings, and use consistent probability notation.
The three main findings are gains from breaking ties, gains beyond ties, and comparisons with highest-score selection. This provides a simpler structure for the same central results.
We appreciate the limited review time and would be grateful for consideration of the submission’s theoretical contribution and the new experiments we ran during the rebuttal phase that answer your questions.
Thank you for highlighting adaptive inference and the case where the most common answer is wrong. We added the sampling comparison you suggested, clarified the stopping guarantee, and tested how scores correct a wrong vote leader.
1. Should we score a tie or generate more solutions?
We compared these two choices directly. In a separate adaptive-inference test on 120 AIME problems, we replayed the same saved Math-7B solutions with Qwen PRM:
Each policy uses only the solutions available so far. Starting with 40 solutions:
Scoring ties with 40 solutions also matched plain voting's observed accuracy with 128 solutions: 25.83%. That uses 88 fewer generated solutions, with fewer than one scored solution per problem on average. Our cost model estimates 24 GPU-seconds saved per problem compared with sampling until one answer leads (Appendix A.4; Table 16).
The stopping guarantee covers decisions made during sampling. The half-floor rule gives each solution weight
max(score, 0.5). Under independent sampling from the same model and scorer, with scores calibrated for that problem, the probability that this rule ever replaces a correct count leader ahead by at leastmvotes with a wrong answer is at mostexp(−2m). This holds across all sample sizes, including stopping based on the answers seen so far (Section 5.1; Proposition 7).2. What happens when ties become less common?
Scores also improve accuracy by choosing an answer with fewer votes. Across all 362 AIME problems unused in our submission, ties occur in 29% of groups at eight solutions and 6% at 128.
With Math-7B and Qwen PRM, weighting beats count-first at 16, 32, 40, and 64 solutions. At 40 solutions, the gain is 2.21 percentage points [0.83, 3.68]. Count-first uses scores only to break ties, so this comparison measures the benefit beyond tie-breaking.
With Phi-4 on 120 AIME problems, the gain over count-first appears at all six tested budgets, reaching 3.33 points [0.83, 6.67] at 128 solutions (Section 6).
3. Can scores correct a wrong vote leader? Why keep a base vote?
Yes. Equation (14) gives an example where a wrong answer appears 40% of the time and the correct answer appears 35%. Scores let voting recover the correct answer.
The weighting results use 200,000 simulated groups per budget; highest-score accuracy is calculated exactly. As more solutions are generated, weighted voting's accuracy approaches 100%, while highest-score selection approaches 80% (Appendix A.6; Table 21).
A base vote controls how much agreement matters. With weight
c + score, each solution contributescvotes plus its score. Settingc = 0uses scores directly;c = 1keeps one vote per solution and adds the score.In this example, the correct answer wins in the long run for 0 ≤ c < 2.2, including
c = 1. The half-floor rule also corrects this example. Under calibration, it additionally preserves every already-correct long-run winner (Theorem 2).This distinguishes two goals: correcting a wrong count leader and preserving a correct one. In practice, we keep one base vote and choose the score-based bonus using separate problems with known answers.
The model experiments also show the value of combining votes. With Math-Shepherd on all 362 AIME problems, score-weighted voting beats highest-score selection by 13.26 points [9.12, 17.13] at 128 solutions.
4. Presentation
We followed your suggestion to consolidate the qualifications. The main text now presents the central results with plain English language explanations, one scope table, and one limits paragraph. Section 6 summarizes the experiments, and the appendix contains the detailed procedures.