acceptodds
Under review as a conference paper at ICLR 2027

SELFOCUS: Mitigating Systematic Selection Bias in Inference-Time Scaling

Abstract

Inference-time reasoning methods often rank sampled chains of thought (CoTs) using within-trace token probabilities. Final-answer frequency, however, is a many-to-one trajectory marginal. Under an idealized outcome-only, KL-regularized trajectory model, we prove that self-consistency need not recover the highest-reward answer because its ranking also depends on the total reference-policy mass of all trajectories leading to answer . We identify a sufficient high-dimensional regime in which heterogeneous erroneous paths make a wrong answer more frequent than the uniquely rewarded answer, and demonstrate this distributional shift in a controlled multi-step experiment. We then introduce SELFOCUS, a training-free aggregator that uses cross-CoT semantic recurrence to estimate each reasoning path's deviation from the trajectory population and partially mitigate the influence of on inference-time answer selection. It clusters reasoning steps, accumulates question-local log-frequency offsets along each path, and selects the answer minimizing the ratio of mean cumulative offset to answer frequency. Across four model families and seven benchmarks, SELFOCUS improves paired majority vote in 15 of 17 model–dataset settings without token log probabilities, an answer-correctness verifier, or additional training. On larger models, measured compute also shows substantially lower GPU-hour usage than token-log-probability-based selectors.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Inference-Time Scaling, Chain-of-Thought Reasoning, Self-Consistency, Semantic Recurrence, Answer Selection, Training-Free Inference

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_j1LmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a limitation of majority voting in inference-time scaling: an incorrect answer may receive more votes than the correct answer because many different reasoning paths lead to it. The authors analyze this problem using a KL-regularized policy model and show that answer probability depends on both reward and the reference policy's probability mass for that answer. To address this issue, the paper proposes SELFOCUS, a training-free answer selection method. It clusters semantically similar intermediate reasoning steps across sampled CoTs, assigns lower scores to frequently occurring steps, and combines these scores with answer frequency to select the final answer. Experiments across four model families and seven benchmarks show improvements over majority voting

Critical strengths
  1. Consistent empirical improvements. SELFOCUS outperforms majority voting and Semantic Self-Consistency (SSC) in 15 of 17 settings, with particularly consistent gains on TheoremQA across three model families.

  2. A practical, training-free approach to answer selection. SELFOCUS uses intermediate-step semantic recurrence without requiring additional training or token log probabilities. This provides an alternative to probability-based selection methods. The paper also includes useful ablations and controlled experiments to examine the proposed method.

Critical weaknesses
  1. The advantage over existing selection methods is not sufficiently compelling

    Inference-time selection has been extensively studied, and SELFOCUS mainly extends semantic agreement from complete CoTs to intermediate reasoning steps. Although the results are encouraging, the improvement over SSC averages only 1.68 percentage points. Moreover, the full procedure involves step segmentation, semantic clustering, and NLI classification, but its computational cost is not compared with SSC. The small positional discounts in some settings (e.g., η = 0.1 in Table 11) also raise the question of whether a much simpler early-step selection method could achieve similar results.

  2. Insufficient evidence for the claimed path-volume correction

    The paper argues that incorrect answers can accumulate votes from diverse reasoning paths, while correct answers tend to have more consistent paths. However, correct answers can also be reached through different valid approaches, and systematic errors can produce highly consistent incorrect reasoning. Although Figure 3 shows a correlation between semantic recurrence and correctness, the paper does not directly establish that the proposed path-volume bias explains the majority-voting errors corrected by SELFOCUS. The controlled experiments in Appendix D.2 are useful, but they do not establish that path-volume correction explains the accuracy gains on the evaluated reasoning benchmarks.

Questions for authors
  1. How does SELFOCUS compare with a simpler selector using only the first few reasoning steps, in terms of both accuracy and computational cost? Can the authors also report its computational cost relative to SSC?

  2. For cases where SELFOCUS corrects majority voting, are the incorrect majority answers actually supported by more diverse reasoning paths? How does the method behave when correct answers are reached through different valid approaches?

4 comments
AuthorsOpenReview ↗

### 1. Comparison with Prefix-Only Selectors

We thank the reviewer for suggesting a comparison with simpler selectors that use only the first few reasoning steps. We evaluate two representative prefix-based methods:

  • PoLR generates short reasoning prefixes, clusters them according to prefix similarity, and continues only the trajectories belonging to the dominant prefix cluster.
  • Prefix-Confidence Scaling evaluates short prefixes using the model's token probabilities and continues only the prefix with the highest confidence.

We replay both methods using the same candidate trajectories as Vote and SELFOCUS. PoLR uses the first 256 tokens of each trajectory, whereas Prefix-Confidence uses the first 32 tokens. Compute is reported as GPU-hours per question at N=64N=64, consistent with the usage convention in our paper. Each entry below is formatted as accuracy / GPU-hours per question.

ModelDatasetVote Acc./GPU-h/qPoLR Acc./GPU-h/qPrefix-Conf. Acc./GPU-h/qSELFOCUS Acc./GPU-h/q
Gemma2-9BGPQA37.12 / 0.0054031.06 / 0.0039020.45 / 0.0011939.39 / 0.01890
Gemma2-9BMATH50054.70 / 0.0050652.90 / 0.0043149.10 / 0.0010855.50 / 0.01430
Gemma2-9BTheoremQA40.36 / 0.0049937.01 / 0.0043327.91 / 0.0011842.97 / 0.01440
Llama3.3-70BGPQA47.73 / 0.1075344.70 / 0.0630245.20 / 0.0083948.74 / 0.12750
Llama3.3-70BMATH50075.50 / 0.0702574.50 / 0.0545171.90 / 0.0082976.40 / 0.08610
Llama3.3-70BTheoremQA65.06 / 0.0889864.79 / 0.0678949.53 / 0.0094369.21 / 0.10510
Llama3.3-70BAIME2510.00 / 0.1134710.00 / 0.0739813.33 / 0.0086213.33 / 0.16250
Llama3.3-70BAIME2616.67 / 0.0965416.67 / 0.0633313.34 / 0.0080118.33 / 0.14080
Qwen3-32BGPQA59.09 / 0.0519157.07 / 0.0252358.58 / 0.0032762.37 / 0.08040
Qwen3-32BMATH50086.20 / 0.0458385.60 / 0.0337378.60 / 0.0036786.50 / 0.05900
Qwen3-32BTheoremQA77.51 / 0.0445975.64 / 0.0314854.62 / 0.0038381.73 / 0.05230
Qwen3-32BAIME2580.00 / 1.3685380.00 / 0.6938473.33 / 0.0260080.00 / 1.39330
Qwen3-32BAIME2685.00 / 1.0812385.00 / 0.6227978.33 / 0.0227187.50 / 1.09920
GPT-OSS-120BAIME2581.67 / 1.0596181.67 / 0.4725968.34 / 0.0233185.00 / 1.19750
GPT-OSS-120BAIME2693.33 / 0.9323183.33 / 0.3915760.00 / 0.0242495.00 / 1.05140
GPT-OSS-120BBRUMO2578.33 / 5.1439875.00 / 2.4536855.00 / 0.1183175.00 / 5.18940
GPT-OSS-120BHMMT2561.67 / 8.1228251.67 / 3.3709741.67 / 0.1380966.67 / 8.18870

The results confirm that the first few reasoning steps contain signals that can be used to filter trajectories and substantially reduce inference cost. PoLR uses approximately 41.5%41.5\%--86.9%86.9\% of the per-question compute required by full Vote, but its macro-average accuracy decreases from 61.76%61.76\% to 59.21%59.21\%. Prefix-Confidence is substantially cheaper, using approximately 1.7%1.7\%--23.6%23.6\% of the Vote compute, but its macro-average accuracy decreases to 50.54%50.54\%. In contrast, SELFOCUS achieves a macro-average accuracy of 63.74%63.74\%.

These results distinguish reducing the generation budget from improving final answer selection. Prefix signals can identify trajectories that are inexpensive to continue, but on difficult problems the first few reasoning steps do not contain sufficient information to distinguish final answers reliably. SELFOCUS instead uses the semantic recurrence and deviation structures accumulated throughout complete CoTs, retaining later reasoning information that prefix-only selectors cannot use.

Our results therefore do not suggest that prefixes contain no quality signal. Rather, under our evaluation settings, prefix information is primarily useful for reducing the inference budget. Using the step-level structure of complete CoTs requires more computation, but produces more consistent accuracy improvements across models and datasets.

AuthorsOpenReview ↗

2. Computational Cost Relative to SSC and Scaling Potential

SSC and SELFOCUS use exactly the same 64 generated CoTs, so their generation costs are identical. We compare only post-processing: SSC embeds complete CoTs and constructs similarity-based voting weights, whereas SELFOCUS clusters reasoning steps and computes trajectory offsets.

The first table reports post-processing GPU-hours per question at N=64N=64, following the convention used in our paper.

ModelDatasetSSC GPU-h/questionSELFOCUS GPU-h/question
Gemma2-9BGPQA0.001910.01350
MATH5000.001730.00920
TheoremQA0.001690.00940
Llama3.3-70BGPQA0.002880.02000
MATH5000.001960.01590
TheoremQA0.002010.01620
AIME250.002620.04900
AIME260.002600.04430
Qwen3-32BGPQA0.002820.02850
MATH5000.002250.01320
TheoremQA0.002170.00770
AIME250.023770.02470
AIME260.017160.01800
GPT-OSS-120BAIME250.110440.13790
AIME260.085420.11910
BRUMO250.004880.04540
HMMT250.007530.06590

Because accuracy gain is measured over a complete dataset, we multiply the per-question cost by the number of questions. Let AmA_m be the accuracy of method mm, let AVA_V be the accuracy of Vote, and let CmC_m be the total post-processing GPU-hours of method mm over the complete dataset. We define compute efficiency as

Em=Am−AVCm.E_m = \frac{A_m - A_V}{C_m}.

Let ESE_S denote the efficiency of SELFOCUS and ECE_C the efficiency of SSC. We compare the two methods using

ΔE=ES−EC.\Delta E = E_S - E_C.

A positive ΔE\Delta E indicates that SELFOCUS obtains more accuracy improvement per absolute GPU-hour.

ModelDatasetSSC total GPU-hSELFOCUS total GPU-hSSC gain (pp)SELFOCUS gain (pp)SELFOCUS - SSC (pp/GPU-h)
Gemma2-9BGPQA0.3792.673+0.51+2.270.849 - 1.347 = -0.498
MATH5000.8644.600+0.70+0.800.174 - 0.811 = -0.637
TheoremQA1.2647.022+0.80+2.610.372 - 0.633 = -0.261
Llama3.3-70BGPQA0.5703.960+0.51+1.010.255 - 0.895 = -0.639
MATH5000.9827.950+0.20+0.900.113 - 0.204 = -0.091
TheoremQA1.49912.101+1.20+4.150.343 - 0.801 = -0.458
AIME250.0791.4700.00+3.332.265 - 0.000 = +2.265
AIME260.0781.3290.00+1.661.249 - 0.000 = +1.249
Qwen3-32BGPQA0.5585.643+1.01+3.280.581 - 1.810 = -1.228
MATH5001.1236.600+0.20+0.300.046 - 0.178 = -0.133
TheoremQA1.6245.752+1.61+4.220.734 - 0.991 = -0.257
AIME250.7130.7410.000.000.000 - 0.000 = 0.000
AIME260.5150.5400.00+2.504.630 - 0.000 = +4.630
GPT-OSS-120BAIME253.3134.1370.00+3.330.805 - 0.000 = +0.805
AIME262.5633.5730.00+1.670.467 - 0.000 = +0.467
BRUMO250.1461.3620.00-3.33-2.445 - 0.000 = -2.445
HMMT250.2261.977-1.67+5.002.529 - (-7.391) = +9.920

SSC is cheaper in every setting and is therefore more cost-efficient on smaller models and conventional benchmarks, although SELFOCUS achieves larger raw accuracy gains in most cases.

This pattern changes on high-difficulty tasks. SSC matches Vote in seven of eight settings and declines on GPT-OSS HMMT25, whereas SELFOCUS improves accuracy in six. SELFOCUS therefore provides a higher return per GPU-hour in six settings, ties in one, and is lower only on BRUMO25.

Because SSC encodes complete CoTs, its cost increases with trajectory length. On Qwen AIME25/26, its cost is already close to SELFOCUS (0.023770.02377 vs. 0.024700.02470 and 0.017160.01716 vs. 0.018000.01800 GPU-hours per question), while providing no accuracy gain over Vote.

We also observe that SSC increasingly approaches ordinary Vote as model scale, task difficulty, and trajectory length increase. SSC compresses each complete CoT into one global embedding and weights votes using similarities between complete trajectories. Long trajectories leading to different answers can share substantial problem context, reasoning templates, and intermediate computations. This shared content can dilute local differences relevant to answer selection, making the resulting similarity weights less discriminative and causing weighted voting to approach ordinary Vote.

Avoiding a single whole-sequence representation is an explicit design motivation of SELFOCUS. It decomposes each CoT into reasoning steps and measures the recurrence and deviation of local semantic states across trajectories. Thus, even when complete CoTs share substantial content, SELFOCUS can preserve differences in their local reasoning structures. Its additional post-processing is intended to avoid excessive compression of long-horizon reasoning into one global embedding.

to be continued in next comment...

AuthorsOpenReview ↗

part of comment 2 and 3. Does Path Deviation Actually Drive Answer Correction?

We do not claim a universal scaling law or that SELFOCUS is always more cost-efficient than SSC. More precisely, SSC is a low-cost global semantic selector with limited accuracy gains, whereas SELFOCUS exchanges additional step-level processing for stronger answer discrimination. The current results suggest that this design may have greater potential as model scale, generated text volume, and task difficulty increase.

3. Does Path Deviation Actually Drive Answer Correction?

We interpret the reviewer's question as raising two concerns. First, because the final SELFOCUS score combines cumulative offset, positional weighting, and answer-frequency normalization, some Vote errors might be corrected by these additional operations rather than by a lower correct-answer offset. Second, correct answers supported by heterogeneous reasoning paths may themselves receive high offsets and be penalized.

Are Vote errors corrected because of lower offsets?

To isolate the offset signal, we set η=1\eta=1, removing positional weighting, and directly compare the correct answer with the most-voted incorrect answer across all instances in which SELFOCUS corrects Vote.

StatisticCorrect answerMost-voted incorrect answer
Mean cumulative offset0.98681.5145
Paired offset difference--+0.5276
95% cluster-bootstrap CI--[0.4343, 0.6259]
Proportion with higher incorrect-answer offset--91.3%

In 91.3%91.3\% of corrected instances, the most-voted incorrect answer has a higher cumulative offset. Moreover, answer-frequency normalization favors the incorrect majority: because SELFOCUS minimizes mean offset divided by answer frequency, the larger vote count lowers the incorrect answer's score. The correct answer must therefore have a sufficiently lower offset to overcome, rather than benefit from, the frequency term.

A small fraction of instances do not satisfy this ordering, so we do not claim that every correction is entirely offset-driven. Nevertheless, the paired means, confidence interval, and 91.3%91.3\% directional agreement show that lower step-level cumulative offset is the primary signal in most corrected cases.

What happens when correct reasoning paths are heterogeneous?

A correct answer reached through different reasoning approaches may have lower semantic recurrence and therefore a higher offset. We operationalize this heterogeneity using the mean undiscounted step offset of trajectories supporting the correct answer. Among instances that Vote originally answers correctly, we construct tertiles using fixed boundaries of 0.50020.5002 and 0.97440.9744 from the η=1\eta=1 analysis, and apply the same boundaries at η=0.5\eta=0.5.

HeterogeneityCorrect-answer mean step offsetRetention at η=1\eta=1Retention at η=0.5\eta=0.5
Low[0.0234, 0.5002]99.21%99.15%
Medium(0.5002, 0.9744]98.20%98.36%
High(0.9744, 3.5730]94.82%96.51%

Without positional weighting, the risk of overturning an originally correct Vote prediction increases with correct-answer path heterogeneity. Thus, heterogeneous correct reasoning can indeed be adversely affected by the offset score.

Early-step positional weighting partially mitigates this failure mode. In the high-heterogeneity group, correct-answer retention increases from 94.82%94.82\% to 96.51%96.51\%. In the fixed-coefficient diagnostic, positional weighting slightly reduces beneficial corrections but produces a larger reduction in harmful reversals. Its main empirical role is therefore to protect answers that Vote already selects correctly, particularly when they are supported by heterogeneous reasoning paths.

Under the final protocol with label-free estimated η\eta, SELFOCUS preserves 98.9%98.9\% of predictions that Vote originally answers correctly, with an overall correction rate of 3.08%3.08\% and a regression rate of only 0.68%0.68\%. Together, these results indicate that cumulative offset primarily identifies incorrect majority answers with high path deviation, while positional weighting reduces the damage to heterogeneous correct reasoning. Although the mechanism behind this protective effect requires further study, both the fixed-coefficient transitions and the heterogeneity-stratified results support this functional distinction.

If our interpretation correctly captures the reviewer's concerns, we will incorporate these analyses into the revised manuscript. Specifically, we will add the main findings and discussion to the main text and provide the complete experimental definitions, per-setting results, and computational-cost analysis in the appendix.

Thank you for your response. My concerns remain. I have decided to retain my score.

Reviewer_i8N1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a failure mode of self-consistency (i.e., majority voting): answer popularity depends on both correctness and unobservable reference policy trajectory mass. As a result, a wrong answer supported by many heterogeneous wrong reasoning paths may become the majority answer, reducing the accuracy of self consistency.

Specifically, this paper first develops a theoretical framework for this failure mode and verifies it through a controlled experiment. It then proposes SELFOCUS, a training-free aggregation method. SELFOCUS clusters intermediate reasoning steps across sampled reasoning chains, measures the semantic deviation of each reasoning path, and ranks candidate answers using both path deviation and answer popularity. Experiments show that SELFOCUS outperforms majority voting in 15 out of 17 model-dataset settings. It also achieves substantial GPU-hour savings over token-probability-based methods on larger models.

Critical strengths
  1. The paper identifies a new failure mechanism of self-consistency and studies it both theoretically and experimentally.

  2. Based on this motivation, the paper proposes SELFOCUS, a simple training-free method that shows consistent improvements over majority voting across different models and datasets.

Critical weaknesses
  1. There is still a gap between the theoretical analysis and the proposed method. The theory focuses on the reference-policy trajectory mass (Q_y), while SELFOCUS measures semantic recurrence from post-RL samples. Connecting the SELFOCUS score to (Q_y) relies on assumption A2. Although the paper acknowledges this limitation, A2 is not empirically validated on the evaluated language models. Therefore, it is unclear whether the improvement of SELFOCUS actually comes from correcting the proposed path-mass bias.
Questions for authors
  1. SELFOCUS seems closely related to step-level self-consistency: a good reasoning path should not only produce a frequent final answer, but should also contain intermediate reasoning steps that frequently appear in other trajectories. This seems different from correcting the reference-policy mass (Q_y). Could the authors further disentangle these two explanations? In particular, could the improvement of SELFOCUS mainly come from step-level semantic consistency rather than correcting (Q_y)?

  2. For some open-source RL-trained models, the corresponding SFT model is also available and could potentially serve as the reference policy. Could the authors sample from such an SFT model to directly estimate (Q_y), and test whether the semantic deviation measured by SELFOCUS is actually correlated with (Q_y)? I think this would provide useful empirical evidence for assumption A2 and strengthen the connection between the theory and the proposed method.

2 comments
AuthorsOpenReview ↗

Step-Level Consistency and Reference-Policy Mass

We thank the reviewer for asking how step-level semantic consistency relates to reference-policy mass in our analysis.

1. Does step-level semantic consistency drive the gains?

Step-level semantic recurrence is the signal SELFOCUS measures directly. In Table 9 (Appendix F), ASC-weighted voting uses this signal by weighting each vote by the semantic concentration of its trajectory. Across 17 model–dataset settings, ASC-weighted voting averages 0.64 percentage points below majority voting, with five improvements, four ties, and eight declines. On the same trajectories, the full SELFOCUS score at η=0.5\eta=0.5 improves over ASC-weighted voting by 2.19 percentage points on average, with 15 improvements and two ties. This comparison establishes the added value of the full scoring rule over this particular use of step-level recurrence.

The paired correction analysis shows how the offset contributes to reranking. Among cases where SELFOCUS corrects a majority-vote error, the correct answer has a lower undiscounted mean offset than the most-voted incorrect answer in 91.3% of cases. The mean paired difference, computed as incorrect minus correct, is 0.528 (95% cluster-bootstrap CI [0.434, 0.626]). In these cases, the lower offset helps the correct answer overcome the vote-count advantage of the incorrect majority. This analysis characterizes the observed corrections. Attributing the benchmark gains to reference-mass correction additionally requires evidence linking the semantic statistic to QyQ_y.

2. Reference-model comparison

Following your suggestion, we paired Mistral-Small-24B-Base-2501 with Mistral-Small-24B-SimpleRL-Zoo and Qwen2.5-14B-Instruct with its Search-R1 14B PPO counterpart. We used the published starting checkpoints as reference proxies and matched the two-stage decoding protocol within each pair. We estimated reference answer probabilities from independent samples, using 128 draws per question on GPQA Diamond and 128–384 on MATH500. SimpleRL-Zoo · Search-R1 14B PPO checkpoint

We compared pairs of incorrect answers to the same question, so the shared term hx(0)h_x(0) in Eq. (12) cancels under the binary evaluation reward. For each pair, we regressed the log ratios of RL vote counts and summed trajectory offsets on the log ratio of estimated reference probabilities, weighting each question equally. We scored held-out RL trajectories using semantic partitions fitted on separate RL trajectories.

Model pairDatasetAnalyzable / screened questionsWrong-answer pairsRL vote-count slopeSummed-offset slope
Mistral 24B Base → SimpleRL-ZooGPQA Diamond118 / 1981541.111 [0.843, 1.370]1.071 [0.813, 1.319]
Qwen2.5-14B-Instruct → Search-R1 14B PPOGPQA Diamond100 / 1981540.942 [0.844, 1.035]0.930 [0.836, 1.020]
Mistral 24B Base → SimpleRL-ZooMATH50038 / 500570.691 [0.189, 1.122]0.575 [0.092, 0.978]

Brackets indicate 95% question-bootstrap intervals.

RL vote ratios track the estimated reference probability ratios on the supported wrong-answer pairs, with slopes close to one for both GPQA model pairs. This is consistent with the same-reward prediction of Eq. (2) under the idealized Gibbs model. The summed offset equals the answer-conditioned mean offset multiplied by its RL vote count, so its slope reflects both terms by construction. A2 concerns calibration of the answer-conditioned semantic term itself, which remains a separate empirical question from the agreement of the summed statistic.

I appreciate the authors for their response on additional experiments and analysis. I will keep my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.