acceptodds
Under review as a conference paper at ICLR 2027

Compare to Correct: Pairwise-Guided Logit Adjustment for LLM Test-Time Scaling

Abstract

Sampling-based test-time scaling enhances LLM reasoning by aggregating multiple reasoning paths for the same query. Existing methods use answer frequencies or path confidence to compute candidate scores, which do not explicitly characterize comparisons between competing answers. Inspired by the Condorcet principle in social choice theory, we introduce pairwise confidence comparisons between paths supporting different answers to capture answer-level comparative advantage. We combine this comparative evidence with voting support through a KL-regularized objective. Theoretically, we show that this comparative-advantage is equivalent to the adjustment to the traditional voting logits. In this way, our proposed Pairwise-Guided Logit Adjustment (PGLA) framework allows comparative evidence to compensate for weaker voting support and correct voting-based ranking errors. Experiments on 10 challenging benchmarks including mathematical reasoning, knowledge reasoning, and code generation with 7 models ranging from 1.7B to 32B parameters speak to the effectiveness of this plug-and-play term. Additional evaluations with external reward scores and online sampling further demonstrate its applicability.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM reasoning

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_1Gb6OpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Pairwise-Guided Logit Adjustment (PGLA), a plug-and-play way to aggregate sampled reasoning paths. On top of standard voting, it builds a confidence based pairwise comparison matrix between competing answers (motivated by the Condorcet principle), folds that comparative advantage into a KL regularized objective, and shows the solution is just an additive logit offset on the base voting score. Across several model families and scales and a spread of math, knowledge, and code benchmarks, it reports a small but consistent average gain over voting baselines, plus calibration improvements and online sampling savings.

Critical strengths

The KL objective reducing to a clean closed form logit adjustment is elegant, and the proof plus the sorting and prefix sum acceleration make the method genuinely cheap and label free. Benchmark coverage is strong: multiple model families and scales, math, knowledge, and code, with means and standard deviations reported, which many aggregation papers skip. Nice honesty in the appendix: the ranking quality ablation directly ties gains to how well confidence separates correct from incorrect paths, and the two case framing gives a believable mechanism for when the method helps.

Critical weaknesses
  1. The gains are small relative to variance, so the central claim is not yet statistically established. The headline improvement over voting baselines sits within the range of the reported standard deviations on several settings, and the paper offers no paired significance testing across seeds even though its matched sampled pools make such a test straightforward. This is consequential because the entire contribution is an incremental accuracy gain from a plug in term: if that gain cannot be distinguished from sampling noise, the empirical case for the method collapses, regardless of how clean the theory is. The pattern that gains concentrate on smaller and math specialized backbones and fade on the strongest model deepens this concern, since it suggests the effect may be an artifact of weaker base models rather than a durable property.

  2. The evaluation never tests the regime where the method should fail, so the scope of the claim is unverified. Every benchmark is competition math, closed book STEM multiple choice, or code, all tasks where path confidence correlates with correctness. The method's own ranking quality analysis shows the gain tracks that confidence correctness correlation and can go negative when the correlation is weak. On open domain or multi-hop retrieval QA, the dominant failure mode is confident and wrong, exactly the regime where confidence based pairwise comparison should stop discriminating and the comparative term should degrade voting. By labeling GPQA alone as "knowledge reasoning" and omitting any retrieval benchmark, the paper claims broad applicability while testing only the favorable half of its own stated dependency. This is consequential because a plug in aggregation term that quietly hurts on a whole task family, without any guidance on when to turn it off, is risky to adopt as presented.

Questions for authors

1/Every benchmark here is competition math, closed book STEM multiple choice, or code, all tasks where path confidence tends to correlate with correctness. I was curious how PGLA behaves on open domain or multi-hop retrieval QA (HotpotQA, TriviaQA, Natural Questions), where the dominant failure mode is confident and wrong. Since the paper's own ranking quality analysis shows the gain tracks the confidence correctness correlation and can even go slightly negative when that correlation is weak, a single retrieval benchmark would test whether the comparative advantage term still helps or quietly degrades voting. It would also let the knowledge reasoning claim rest on an actual retrieval task rather than GPQA alone.

2/The headline average gain sits close to the reported standard deviations on several settings, with error bars that look like they overlap. It would strengthen the claim to add a paired significance test across seeds, since the matched sampled pools make a paired test natural. I was curious how much of the gain is just better confidence rather than the pairwise structure. A natural ablation: replace the pairwise term with a direct confidence weighted re-rank at the same compute, so readers can see what the Condorcet style comparison buys beyond the confidence signal itself.

3/The gains look concentrated on the smaller and math specialized backbones and shrink on the largest model. It would help to say plainly where PGLA stops helping as models get stronger, rather than reading the large model result as uniform support. 4/Correctness labeling is only lightly described (normalized answer match for math, execution for code). Since this is effectively the evaluation ground truth, how sensitive are the calibration numbers and the recoverable error rates to the matcher? A short robustness check with an alternative judge would reassure.

5/Confound worth addressing: is the comparative advantage partly tracking answer group size or path length rather than correctness? A control that regresses out block size would make the mechanism claim cleaner.

6/The online stopping results compare against voting based early stopping baselines, but the stability rule and warm up appear to differ slightly across methods. Could you confirm all methods see identical path prefixes so the savings are not a stopping artifact?

13 comments

Thank you for your detailed comments on the evaluation and the scope of the method.

Q1. The current benchmarks cover competition math, closed-book STEM multiple-choice questions, and code generation. On open-domain or multi-hop retrieval QA, models often answer incorrectly with high confidence. How does PGLA perform on benchmarks such as HotpotQA, TriviaQA, and Natural Questions? The ranking-quality analysis shows that gains can turn slightly negative when confidence is weakly related to correctness. A retrieval benchmark would help show whether the pairwise adjustment still helps in this setting.

A1. We evaluated PGLA on all 7,405 questions in the HotpotQA Fullwiki validation set and all 3,610 questions in the NQ-Open original development set, using a confidence score that incorporates retrieved evidence. PGLA improves EM by 2.28–3.35 percentage points and F1 by 1.64–2.28 points across SC, Weighted SC, and RPC. We use Qwen3-1.7B (non-thinking) and run 10 matched repetitions. In each repetition, all methods use the same ten retrieved passages and 32 sampled paths per question.

  • The table below reports answer exact match (EM) and token-level F1 on HotpotQA, two standard answer evaluation metrics used by its official evaluation protocol. EM measures whether a prediction matches the reference answer after normalization and F1 measures token overlap between the prediction and reference answer.
MethodEM (%) ↑F1 (%) ↑
SC15.5428.41
SC + PGLA18.0330.36
Weighted SC16.2428.98
Weighted SC + PGLA18.5230.70
RPC15.5328.39
RPC + PGLA18.0230.08
  • The table below reports EM and F1 on NQ-Open benchmark.
MethodEM (%) ↑F1 (%) ↑
SC4.4218.13
SC + PGLA6.7319.77
Weighted SC5.1118.64
Weighted SC + PGLA7.5920.36
RPC4.2917.91
RPC + PGLA7.6320.19

Q2a. Can the reported gains be distinguished from sampling variation in settings where the gains are comparable to the reported standard deviations? What does a paired significance test across seeds show when each baseline and its PGLA counterpart are evaluated on the same sampled paths?

A2a. We evaluate each baseline and its PGLA version on the same sampled paths in each of 10 repetitions in the original paper. All 17 positive-gain comparisons reported below have a 95% confidence interval above zero and a raw pp-value below 0.05. We use SC as the baseline, with DeepConf for the PGLA adjustment on Qwen3 reasoning models and PPL on math-specialized models. The tables below cover the positive mean gains in 8 of 9 Qwen3 settings and all nine math-specialized settings.

  • Paired SC comparisons on Qwen3 reasoning models using DeepConf for the PGLA adjustment.
ModelBenchmarkSC (%)SC + PGLA (%)Mean gain (pp)95% CI (pp)Raw pp-value
Qwen3-1.7BAIME2469.5075.00+5.50[+3.63, +7.37]9.273e-05
Qwen3-1.7BAIME2547.3351.33+4.00[+2.60, +5.40]0.0001
Qwen3-1.7BHMMT2527.0030.00+3.00[+0.91, +5.09]0.0100
Qwen3-4BAIME2480.0083.00+3.00[+2.25, +3.75]8.538e-06
Qwen3-4BHMMT2551.5054.00+2.50[+0.31, +4.69]0.0299
Qwen3-8BAIME2481.5086.67+5.17[+3.53, +6.80]5.344e-05
Qwen3-8BAIME2575.6779.67+4.00[+1.35, +6.65]0.0077
Qwen3-8BHMMT2555.9458.67+2.72[+1.50, +3.94]0.0007
  • Paired SC comparisons on math-specialized models using PPL for the PGLA adjustment.
ModelBenchmarkSC (%)SC + PGLA (%)Mean gain (pp)95% CI (pp)Raw pp-value
InternLM2-Math-Plus-1.8BMATH36.4839.77+3.28[+3.13, +3.44]4.611e-12
InternLM2-Math-Plus-1.8BMathOdyssey14.5718.28+3.71[+3.09, +4.32]2.504e-07
InternLM2-Math-Plus-1.8BAIME2.663.37+0.71[+0.55, +0.87]3.270e-06
InternLM2-Math-Plus-7BMATH50.5753.99+3.43[+3.31, +3.54]2.339e-13
InternLM2-Math-Plus-7BMathOdyssey28.3035.30+7.00[+6.51, +7.48]1.225e-10
InternLM2-Math-Plus-7BAIME9.409.95+0.55[+0.32, +0.78]0.0005
DeepSeek-Math-RL-7BMATH53.3353.52+0.19[+0.07, +0.31]0.0053
DeepSeek-Math-RL-7BMathOdyssey36.5737.87+1.29[+0.70, +1.89]0.0008
DeepSeek-Math-RL-7BAIME9.4210.02+0.60[+0.37, +0.84]0.0002

Details. The main results report mean accuracy and standard deviation across 10 sampling seeds. We apply a two-sided paired tt-test to the 10 accuracy differences under the null hypothesis of zero mean gain.

Q2b. I was curious how much of the gain is just better confidence rather than the pairwise structure. A natural ablation: replace the pairwise term with a direct confidence weighted re-rank at the same compute, so readers can see what the Condorcet style comparison buys beyond the confidence signal itself.

A2b. With the same sampled paths, confidence signals, and sampling budget, SC + PGLA outperforms both confidence weighting and direct confidence adjustment in all seven settings. It improves accuracy by 1.18–6.48 percentage points over Weighted SC and by 0.67–3.32 points over SC + Mean.

  • Confidence weighting (Weighted SC). We sum the path weights within each answer group, as in Eq. (4). We use inverse perplexity piPPLp_i^{\mathrm{PPL}} for PPL and the raw confidence scores for DeepConf and Self-Certainty.
  • Confidence adjustment (SC + Mean). We scale the path confidence scores to [−1,1][-1,1] within each pool and average them within each answer group. This gives the mean-confidence adjustment for answer uu.
\frac{1}{|\mathcal C_u|} \sum_{i\in\mathcal C_u} \left( 2\,\frac{p_i-\min_j p_j}{\max_j p_j-\min_j p_j}-1 \right).$$ Here $p_i=\log p_i^{\mathrm{PPL}}$ in the PPL runs, and $p_i$ is the raw confidence score in the other runs. If all path scores are equal, we set $s_u^{\mathrm{Mean}}=0$. SC + Mean adds $s_u^{\mathrm{Mean}}$ to $\log s_u^{\mathrm{SC}}$. SC + PGLA adds $s_u^{\mathrm{Pair}}$ to the same base score, as in `Eq. (10)`. The gain over SC + Mean therefore demonstrates the benefit of replacing confidence averaging with pairwise comparisons across groups. PGLA improves on SC + Mean by a further 0.67 to 3.32 percentage points. The largest difference is on MathOdyssey with InternLM2-Math-Plus-7B, where accuracy rises from 31.98% to 35.30%. - Accuracy (%, mean ± standard deviation) on MathOdyssey with PPL confidence. | Method | InternLM2-Math-Plus-1.8B | InternLM2-Math-Plus-7B | DeepSeek-Math-RL-7B | | --- | ---: | ---: | ---: | | SC | 14.57 ± 0.45 | 28.30 ± 0.61 | 36.57 ± 0.56 | | Weighted SC | 14.86 ± 0.47 | 28.82 ± 0.68 | 36.68 ± 0.61 | | SC + Mean | 15.89 ± 0.58 | 31.98 ± 0.50 | 36.99 ± 0.55 | | SC + PGLA | **18.28 ± 0.56** | **35.30 ± 0.55** | **37.87 ± 0.55** | - Accuracy (%, mean ± standard deviation) on AIME24 with DeepConf and Self-Certainty. | Method | Qwen3-1.7B (DeepConf) | Qwen3-1.7B (Self-Certainty) | Qwen3-4B (DeepConf) | Qwen3-4B (Self-Certainty) | | --- | ---: | ---: | ---: | ---: | | SC | 69.50 ± 1.98 | 69.50 ± 1.98 | 80.00 ± 0.00 | 80.00 ± 0.00 | | Weighted SC | 70.00 ± 1.49 | 70.33 ± 1.80 | 80.00 ± 0.00 | 80.00 ± 0.00 | | SC + Mean | 72.33 ± 2.13 | 73.33 ± 1.49 | 81.67 ± 1.67 | 81.00 ± 1.53 | | SC + PGLA | **75.00 ± 1.67** | **74.00 ± 1.33** | **83.00 ± 1.00** | **82.00 ± 1.63** |
Reviewer_xY3zOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the limitation of existing sampling-based test-time scaling methods, which largely overlook pairwise confidence comparisons between competing answers. The authors propose Pairwise-Guided Logit Adjustment (PGLA), which integrates pairwise comparative advantages with voting support through a KL-regularized objective and derives a simple closed-form logit adjustment. Experiments across seven LLMs and ten benchmarks covering mathematical reasoning, knowledge reasoning, and code generation demonstrate consistent improvements over representative aggregation baselines. The proposed method achieves approximately 2% average accuracy improvement while introducing negligible aggregation overhead and offering additional benefits in online sampling efficiency.

Critical strengths
  1. Well-motivated and principled methodology. The paper identifies an important limitation of existing voting-based aggregation methods. The proposed pairwise comparison mechanism is intuitive, theoretically well-formulated, and easily integrated into existing aggregation frameworks.

  2. Comprehensive experimental validation. The experiments cover diverse models, benchmarks, confidence estimators, and aggregation baselines. The additional analyses of confidence reliability, calibration, and online sampling further strengthen the empirical evidence.

Critical weaknesses
  1. Dependence on confidence reliability. PGLA assumes that pairwise confidence comparisons provide meaningful information about answer quality. Although the authors investigate this issue, the method lacks an adaptive mechanism to prevent performance degradation when confidence estimates are unreliable.
Questions for authors
  1. Could the authors discuss whether the strength of the pairwise adjustment could be adapted according to confidence reliability, particularly when confidence rankings are noisy?

  2. Does the online extension exhibit similar accuracy–efficiency tradeoffs on Qwen3 reasoning models or non-mathematical tasks?

3 comments

Thank you for your questions about confidence reliability and online sampling.

Q1. Could the authors discuss whether the strength of the pairwise adjustment could be adapted according to confidence reliability, particularly when confidence rankings are noisy?

A1: To control PGLA's reliance on pairwise confidence comparisons, we use a ridge regressor trained on 10 labeled questions to choose β\beta (in suVexp⁡(βsuPair)s_u^{\mathcal V}\exp(\beta s_u^{\mathrm{Pair}})). Larger values let reliable comparisons have more influence; smaller values reduce reliance on noisy comparisons. The regressor uses four features of the voting margin and pairwise comparisons to predict each candidate β\beta's gain over voting alone. It selects the candidate with the largest predicted gain and uses β=0\beta=0 if no positive gain is predicted. On the held-out questions, it selects β\beta using only the current pool's statistics. On the remaining held-out MathOdyssey questions, adaptive β\beta outperforms fixed β=1\beta=1 for both voting rules at every tested shuffling level from 0% to 90%.

Experimental setup. We evaluate InternLM2-Math-Plus-7B with K=128K=128, reporting mean accuracy over ten matched sampling seeds. To progressively weaken confidence rankings, we randomly reassign confidence scores among 0%–100% of the sampled paths while keeping the paths and answers fixed. Weighted SC weights and PGLA comparisons are recomputed from the same perturbed scores. The table also reports mean within-question path AUC to quantify the remaining ranking information.

According to the table below, we arrive at the following conclusions:

1) Weak rankings still help. At 80% shuffling, path AUC drops to 0.5250, yet both PGLA variants outperform their voting baselines. Adaptive PGLA improves accuracy by about 2.9 percentage points for both voting rules. 2) The gain disappears near chance. At 100% shuffling, path AUC is 0.5017. Both PGLA variants score below their voting baselines, and adaptive PGLA scores below fixed PGLA.

  • Accuracy (%) and mean within-question path AUC for InternLM2-Math-Plus-7B on MathOdyssey. Columns give the fraction of paths selected for confidence-score shuffling.
Method0%10%25%40%60%70%80%90%100%
AUC0.64070.62710.60570.58150.55040.53980.52500.51430.5017
SC27.9927.9927.9927.9927.9927.9927.9927.9927.99
SC + PGLA (fixed)35.1234.7533.8532.5131.1930.7429.7128.5527.52
SC + PGLA (adaptive)36.9436.7336.2535.4633.2732.6730.9028.9226.65
Weighted SC28.5228.5228.4428.2328.2628.1028.1528.0728.05
Weighted SC + PGLA (fixed)35.5735.0733.9632.7731.2730.8729.7928.5027.52
Weighted SC + PGLA (adaptive)36.7636.7336.3935.6233.4332.9031.0628.8726.65

Q2. Does the online extension show a similar accuracy–efficiency trade-off on Qwen3 reasoning models or on non-mathematical tasks?

A2. We extend online evaluation to Qwen3-4B on HMMT25 and to Qwen3-1.7B on HumanEval. Both PGLA variants improve accuracy and use fewer paths than Online SC in every setting, with accuracy gains of 0.91–3.67 percentage points and savings of 72.50%–76.10% relative to full-budget SC.

  • Online accuracy and path use for Qwen3-4B on HMMT25.
MethodAcc. (%)Gain vs. Online SC (pp)Mean pathsSaving
Full-budget SC51.50 ± 1.57—32.0—
Adaptive SC51.50 ± 1.57—15.751.00%
Online SC51.67 ± 3.42—9.470.50%
Online SC + PGLA (DeepConf)53.67 ± 3.79+2.007.975.40%
Online SC + PGLA (Self-Certainty)55.33 ± 3.48+3.677.676.10%
  • Online accuracy and path use for Qwen3-1.7B on HumanEval.
MethodAcc. (%)Gain vs. Online SC (pp)Mean pathsSaving
Full-budget SC83.84 ± 1.32—32.0—
Adaptive SC83.84 ± 1.32—28.012.40%
Online SC84.15 ± 1.26—17.345.90%
Online SC + PGLA (DeepConf)85.06 ± 1.78+0.918.872.50%
Online SC + PGLA (Self-Certainty)85.67 ± 1.52+1.528.872.60%

Details. The full budget is Kmax⁡=32K_{\max}=32. Online SC and Online SC + PGLA first check after two paths and then check every two paths. They use the same ordered path pool and stop when the same answer is the unique winner at three consecutive checks, making six paths the earliest stopping point. A tie resets the count, and each method returns its prediction at the budget limit. Adaptive SC uses its original stopping rule.

Thank you for your response, which fully addresses my concerns. I will keep my positive rating.

Reviewer_CNapOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to aggregate multiple sampled reasoning paths into one answer at test time. Motivated by the Condorcet principle, it compares the confidence scores of paths supporting different answers to build a pairwise matrix YuvY_{uv}, which is averaged with vote-share weights into a comparative advantage suPair∈[−1,1]s^{\text{Pair}}_u \in [-1, 1]. Combining this term with a base voting distribution through a KL-regularized objective gives the closed form suPGLA=suVexp⁡(suPair)s^{\text{PGLA}}_u = s^V_u \exp(s^{\text{Pair}}_u), which the paper interprets as a logit adjustment (Prop. 3.1). The adjustment is plugged into SC, confidence-weighted SC, Borda, DeepConf filtering, Adaptive SC, RPC, and PRM-based voting. Experiments with 7 models (1.7B–32B) on 10 math, GPQA, and code benchmarks report an average gain of about 2% over the voting baselines, with additional online-sampling and external-reward settings.

Critical strengths
  • The paper motivates the problem well with a concrete failure mode of voting. In Fig. 4(b), 69.87%–91.62% of recoverable SC errors are cases where the correct answer gets no more votes than a wrong one but wins the pairwise confidence comparisons, which makes the idea of adding pairwise evidence intuitive.
  • The method is simple and training-free, which makes it easy to adopt. It is a closed-form change on top of any voting rule, and it works with internal confidence scores as well as external PRM scores (Table 3, e.g., PRM-Weighted SC on InternLM2-Math-Plus-7B improves from 36.42 to 38.59 at the full budget).
  • The evaluation is broad and paired, and the gains are consistent in direction. Each baseline and its +Ours variant use the same problems and sampled subsets (App. F.4), and in Table 1 none of the 126 pairs gets worse (121 improve, 5 tie). I also like the sanity check in G.5, where the gain shrinks from +2.17% to −0.54% as the ranking quality of the scores drops to chance.
Critical weaknesses
  • I am not convinced by the Condorcet framing. Under the SC base, Prop. G.1 shows suPair=(2Rˉu−K−1)/(K−nu)s^{\text{Pair}}_u = (2\bar{R}_u - K - 1)/(K - n_u), a one-vs-rest mean confidence rank, so PGLA is closer to rank-based voting than to pairwise preference aggregation. In Table 1, SC+Ours (Self-Certainty) beats Self-Certainty Borda, which uses the same signals, by only 0.66 points on average.
  • It is hard to tell whether the gains on the competition benchmarks exceed noise. AIME24/25 and HMMT25 have 30 problems each, so the Qwen3-8B gains of 4.61%, 1.78%, and 2.63% correspond to 1.38, 0.53, and 0.79 problems. The ±\pm values may come from resampling subsets of a single pool (e.g., 80.00$\pm$0.00), the number of repetitions is not stated.
  • I would like to see how sensitive the results are to the strength of the adjustment. The KL weight in Eq. (8) is fixed at 1 and never varied (e.g., suVexp⁡(βsuPair)s^V_u \exp(\beta s^{\text{Pair}}_u)), which also limits how far an answer with weak voting support can be lifted.
Questions for authors

Please see the weaknesses.

5 comments

Thank you for your comments. We respond to each point below.

Q1. Condorcet framing and the comparison with Borda

I am unconvinced by the Condorcet framing. Under SC, Proposition G.1 expresses the adjustment using an answer group's average confidence rank against the other paths. This makes PGLA look closer to rank-based voting than to pairwise preference aggregation. In Table 1, SC + Ours (Self-Certainty) is only 0.66 percentage points ahead of Self-Certainty Borda, which uses the same confidence signal.

A1. Borda uses instance-wise scoring: it assigns each path a score based on its confidence rank and sums these scores within each answer group. PGLA applies a pair-wise adjustment using confidence comparisons between answers, so it can complement Borda by refining its answer-level scores. Applying PGLA directly to Borda improves accuracy in all 18 Qwen3 comparisons shown below and all six InternLM2 comparisons.

  • Accuracy (%, mean ± standard deviation), using Self-Certainty Borda with γ=1.2\gamma=1.2 and K=32K=32.
ModelMethodAIME24HMMT25GPQA-DHumanEvalMBPPLiveCodeBench
Qwen3-1.7BBorda73.33 ± 1.4929.33 ± 2.9143.69 ± 2.2786.59 ± 1.5485.19 ± 0.8632.95 ± 1.28
Qwen3-1.7BBorda + PGLA74.67 ± 1.6334.33 ± 3.9644.95 ± 2.0288.11 ± 1.8385.45 ± 0.7733.52 ± 1.43
Qwen3-4BBorda81.00 ± 1.5355.00 ± 1.6760.10 ± 0.6094.97 ± 0.9094.18 ± 0.4947.81 ± 1.18
Qwen3-4BBorda + PGLA82.33 ± 2.1355.33 ± 1.6360.94 ± 0.7595.73 ± 0.5894.26 ± 0.3148.00 ± 1.14
Qwen3-8BBorda83.67 ± 1.8059.33 ± 2.4963.97 ± 0.9595.33 ± 0.4995.59 ± 0.3352.00 ± 0.47
Qwen3-8BBorda + PGLA86.33 ± 1.8060.00 ± 2.1164.31 ± 0.8696.34 ± 0.9196.12 ± 0.5252.38 ± 1.08
  • Accuracy (%, mean ± standard deviation), using linear Borda with PPL ordering. K=64K=64 for MATH and K=128K=128 otherwise.
ModelBenchmarkBordaBorda + PGLAGain (pp)
InternLM2-Math-Plus-1.8BMATH38.83 ± 0.0840.58 ± 0.21+1.75
InternLM2-Math-Plus-1.8BMathOdyssey17.12 ± 0.6221.13 ± 0.57+4.01
InternLM2-Math-Plus-1.8BAIME3.23 ± 0.183.56 ± 0.20+0.33
InternLM2-Math-Plus-7BMATH53.07 ± 0.1653.60 ± 0.16+0.53
InternLM2-Math-Plus-7BMathOdyssey33.14 ± 0.5937.48 ± 0.58+4.34
InternLM2-Math-Plus-7BAIME9.97 ± 0.1210.20 ± 0.34+0.23

Details. Let RiR_i be the ascending confidence rank of path ii. Borda assigns answer uu the score

suBorda=∑i∈CuRiγ.s_u^{\mathrm{Borda}}=\sum_{i\in\mathcal C_u}R_i^\gamma.

We use suBordas_u^{\mathrm{Borda}} as the base support suVs_u^{\mathcal V} in Eqs. (7) and (10):

=\log s_u^{\mathcal V} +\frac{\sum_{v\ne u}s_v^{\mathcal V}Y_{uv}} {\sum_{v\ne u}s_v^{\mathcal V}}.$$ Each comparison uses the same sampled paths and confidence signal. `Proposition G.1` gives the rank representation of the special SC case. The Condorcet principle motivates comparisons between competing answers, which PGLA combines with the chosen voting support.

Q2. It is hard to tell whether the gains on the competition benchmarks exceed noise. AIME24/25 and HMMT25 have 30 problems each, so the Qwen3-8B gains of 4.61%, 1.78%, and 2.63% correspond to 1.38, 0.53, and 0.79 problems. The values may come from resampling subsets of a single pool (e.g., 80.00±0.0080.00\pm0.00), the number of repetitions is not stated.

A2. The original paper reports the mean and standard deviation across 10 repetitions with different sampling seeds. We use these same repetitions for the paired analysis. For each problem, we generate a pool of 64 paths once. Each repetition draws K=32K=32 paths from this pool, with the baseline and its PGLA version using the same subset. We report the mean and standard deviation of accuracy across these 10 repetitions.

To test whether the gains can be distinguished from variation across path draws, we apply a two-sided paired tt-test to the 10 accuracy differences under the null hypothesis of zero mean gain. We use SC as the baseline, with DeepConf for the PGLA adjustment on Qwen3 reasoning models and PPL on math-specialized models. The tables report the positive gains in 8 of 9 Qwen3 settings and all 9 math-specialized settings. In all 17 reported settings, the 95% confidence interval for the mean gain lies above zero and the raw pp-value is below 0.05.

  • Paired SC comparisons on Qwen3 reasoning models using DeepConf for the PGLA adjustment.
ModelBenchmarkSC (%)SC + PGLA (%)Mean gain (pp)95% CI (pp)Raw pp-value
Qwen3-1.7BAIME2469.5075.00+5.50[+3.63, +7.37]9.273e-05
Qwen3-1.7BAIME2547.3351.33+4.00[+2.60, +5.40]0.0001
Qwen3-1.7BHMMT2527.0030.00+3.00[+0.91, +5.09]0.0100
Qwen3-4BAIME2480.0083.00+3.00[+2.25, +3.75]8.538e-06
Qwen3-4BHMMT2551.5054.00+2.50[+0.31, +4.69]0.0299
Qwen3-8BAIME2481.5086.67+5.17[+3.53, +6.80]5.344e-05
Qwen3-8BAIME2575.6779.67+4.00[+1.35, +6.65]0.0077
Qwen3-8BHMMT2555.9458.67+2.72[+1.50, +3.94]0.0007
  • Paired SC comparisons on math-specialized models using PPL for the PGLA adjustment.
ModelBenchmarkSC (%)SC + PGLA (%)Mean gain (pp)95% CI (pp)Raw pp-value
InternLM2-Math-Plus-1.8BMATH36.4839.77+3.28[+3.13, +3.44]4.611e-12
InternLM2-Math-Plus-1.8BMathOdyssey14.5718.28+3.71[+3.09, +4.32]2.504e-07
InternLM2-Math-Plus-1.8BAIME2.663.37+0.71[+0.55, +0.87]3.270e-06
InternLM2-Math-Plus-7BMATH50.5753.99+3.43[+3.31, +3.54]2.339e-13
InternLM2-Math-Plus-7BMathOdyssey28.3035.30+7.00[+6.51, +7.48]1.225e-10
InternLM2-Math-Plus-7BAIME9.409.95+0.55[+0.32, +0.78]0.0005
DeepSeek-Math-RL-7BMATH53.3353.52+0.19[+0.07, +0.31]0.0053
DeepSeek-Math-RL-7BMathOdyssey36.5737.87+1.29[+0.70, +1.89]0.0008
DeepSeek-Math-RL-7BAIME9.4210.02+0.60[+0.37, +0.84]0.0002

On a benchmark with only 30 problems, one additional correct answer changes accuracy by 3.33 percentage points. Different path subsets can therefore produce the same total number of correct answers. For example, SC with Qwen3-4B on AIME24 answers 24 of 30 problems correctly in every repetition, giving 80.00±0.0080.00\pm0.00. The larger benchmarks in Table 2 provide finer accuracy resolution, and all standard deviations reported there are nonzero.

SeedSCSC + PGLA(DeepConf)
080.0083.33
180.0083.33
280.0083.33
380.0083.33
480.0080.00
580.0083.33
680.0083.33
780.0083.33
880.0083.33
980.0083.33
Mean ± SD80.00 ± 0.0083.00 ± 1.00

Q3. I would like to see how sensitive the results are to the strength of the adjustment. The KL weight in Eq. (8) is fixed at 1 and never varied (e.g., suVexp⁡(βsuPair)s_u^{\mathcal V}\exp(\beta s_u^{\mathrm{Pair}})), which also limits how far an answer with weak voting support can be lifted.

A3. With the proposed coefficient β\beta, the score is suVexp⁡(βsuPair)s_u^{\mathcal V}\exp(\beta s_u^{\mathrm{Pair}}). Setting β=0\beta=0 gives the original voting score, and setting β=1\beta=1 gives PGLA as reported in the paper. For β>0\beta>0, the score follows from the same objective as Eq. (8) with a KL weight of 1/β1/\beta.

\arg\max_{\bar s} \left[ \sum_{u\in\mathcal A}\bar s_u s_u^{\mathrm{Pair}} -\frac{1}{\beta}D_{\mathrm{KL}}(\bar s\Vert\bar s^{\mathcal V}) \right].$$ We perform a sensitivity analysis of $\beta$ in $[0.3,0.5,0.7,1.0,1.2,1.5,2]$ across 28 settings with SC and Weighted SC.. On MathOdyssey, $\beta=1.5$ or $2$ performed best in all six settings, improving on $\beta=1$ by 0.28–3.60 points. On historical AIME, the largest gain over $\beta=1$ was 0.31 points. - Models, benchmarks, and voting rules used in the sensitivity analysis of $\beta$. Setting|Model|Benchmark|Voting rule|Confidence|$K$ ---|---|---|---|---|---: S01|Qwen3-1.7B|AIME24|SC|DeepConf|32 S02|Qwen3-1.7B|AIME24|SC|Self-Certainty|32 S03|Qwen3-1.7B|AIME24|Weighted SC|DeepConf|32 S04|Qwen3-1.7B|AIME24|Weighted SC|Self-Certainty|32 S05|Qwen3-1.7B|AIME25|SC|DeepConf|32 S06|Qwen3-1.7B|AIME25|SC|Self-Certainty|32 S07|Qwen3-1.7B|AIME25|Weighted SC|DeepConf|32 S08|Qwen3-1.7B|AIME25|Weighted SC|Self-Certainty|32 S09|Qwen3-4B|AIME24|SC|DeepConf|32 S10|Qwen3-4B|AIME24|SC|Self-Certainty|32 S11|Qwen3-4B|AIME24|Weighted SC|DeepConf|32 S12|Qwen3-4B|AIME24|Weighted SC|Self-Certainty|32 S13|Qwen3-4B|AIME25|SC|DeepConf|32 S14|Qwen3-4B|AIME25|SC|Self-Certainty|32 S15|Qwen3-4B|AIME25|Weighted SC|DeepConf|32 S16|Qwen3-4B|AIME25|Weighted SC|Self-Certainty|32 S17|InternLM2-Math-Plus-1.8B|MathOdyssey|SC|PPL|128 S18|InternLM2-Math-Plus-1.8B|MathOdyssey|Weighted SC|PPL|128 S19|InternLM2-Math-Plus-1.8B|AIME|SC|PPL|128 S20|InternLM2-Math-Plus-1.8B|AIME|Weighted SC|PPL|128 S21|InternLM2-Math-Plus-7B|MathOdyssey|SC|PPL|128 S22|InternLM2-Math-Plus-7B|MathOdyssey|Weighted SC|PPL|128 S23|InternLM2-Math-Plus-7B|AIME|SC|PPL|128 S24|InternLM2-Math-Plus-7B|AIME|Weighted SC|PPL|128 S25|DeepSeek-Math-RL-7B|MathOdyssey|SC|PPL|128 S26|DeepSeek-Math-RL-7B|MathOdyssey|Weighted SC|PPL|128 S27|DeepSeek-Math-RL-7B|AIME|SC|PPL|128 S28|DeepSeek-Math-RL-7B|AIME|Weighted SC|PPL|128

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.