acceptodds
Under review as a conference paper at ICLR 2027

The Irony of Power Sampling: Sampling Better, Failing More

Abstract

Recent work shows that power sampling enables base models to achieve reasoning performance competitive with that of RL post-trained models without additional training (Karan & Du, 2026). The cost of sampling from the power distribution motivates more efficient approximation algorithms (Ji et al., 2026; Azizi et al., 2026), but how the inherent error from power sampling affects task performance remains underexplored. To address this gap, we conduct an in-depth study of how sampling fidelity relates to task performance through theoretical analysis and empirical evaluation. We begin with a Markov graph task that illustrates why power sampling helps when model likelihood aligns with task success. We then show theoretically that reducing the sampling error can systematically degrade task performance. Finally, we corroborate our theoretical findings through experiments with practical LLM power samplers across reasoning and instruction-following tasks. Our experiments show that higher likelihood does not consistently improve task performance and that immediate stopping occurs in practice as predicted by our analysis.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
language model, reasoning, power sampling, distribution

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_WV93OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies whether more accurate sampling from an LLM's power distribution improves task performance. It proves that increasing the number of SIR candidates strictly improves distributional fidelity, while under specified conditions it also increases failure probability. The paper further analyzes immediate stopping as a concrete failure mode. Synthetic examples and experiments on reasoning and instruction-following tasks illustrate that higher sequence likelihood does not consistently yield better answers, and that reducing immediate stopping can improve performance. The main insight is that sampling accuracy and the quality of the target distribution must be evaluated separately.

Critical strengths
  1. The paper provides an interesting and substantive theoretical insight. It proves that increasing the number of SIR candidates strictly improves sampling fidelity while, under specified conditions, increasing failure probability. This clearly demonstrates why better approximation to a target distribution need not improve task performance.

  2. The paper connects its theory to a concrete failure mechanism. The analysis of immediate stopping, supported by observations in real LLMs and experiments modifying the stopping rule, gives the theoretical findings practical relevance and suggests how this failure can be mitigated.

Critical weaknesses
  1. The effect of model capability is not sufficiently explored. It would be useful to examine whether stronger models show better alignment between trajectory likelihood and correctness, and whether this leads to larger gains from power sampling. This would help clarify when the method is likely to help or hurt.

  2. The empirical evidence for failure is relatively narrow. Immediate stopping mainly occurs in the two Qwen models tested. Analyzing accuracy after excluding empty responses would help determine whether improved sampling fidelity also increases other types of errors.

Questions for authors

Please see the critical weakness. Besides, can the analysis guide an adaptive power-sampling strategy that adaptively determine the sampling budget or sharpening exponent based on observable answers/failure signals?

3 comments

We thank the reviewer for their careful reading and constructive feedback.

W1: Model capability and power sampling gains

The effect of model capability is not sufficiently explored. It would be useful to examine whether stronger models show better alignment between trajectory likelihood and correctness, and whether this leads to larger gains from power sampling. This would help clarify when the method is likely to help or hurt.

Following the reviewer’s suggestion, we compared base accuracy, likelihood and Power-SMC gains. On both tasks, models that more often assign higher likelihood to correct responses than to incorrect responses tend to have higher base accuracy and larger Power-SMC gains.

Using 32 base samples per run at temperature 1, we measure how often a correct response has higher sequence log-likelihood than an incorrect response to the same question. Sequence log-likelihood is the sum of token log-probabilities. The table gives Spearman correlations of this percentage with base accuracy and Power-SMC gain.

GPQA: 198 questions, 16 particles. MATH500: 100 questions, 128 particles, excluding Mistral. Power-SMC gain is its accuracy at α=4\alpha=4 minus token sampling accuracy at temperature 14\frac{1}{4}.

TaskCorrelation with base accuracyCorrelation with Power-SMC gain
GPQA0.530.53
MATH5000.620.77

When the comparison includes only the Qwen, Llama and Phi models, the correlation with base accuracy is negative on GPQA and weak on MATH500. Base accuracy alone therefore does not consistently predict how well likelihood distinguishes correct from incorrect responses.


W2: Immediate stopping in other model families

The empirical evidence for failure is relatively narrow. Immediate stopping mainly occurs in the two Qwen models tested.

Following the reviewer’s suggestion, we conducted experiments on additional base models. On DeepSeek-Math-7B-Base, SmolLM3-3B-Base and Mistral-7B-v0.3, Power-SMC stops immediately more often than token sampling on both GPQA and MATH500.

Please refer to our response in General Response G2.


W2: Errors after excluding empty responses

Analyzing accuracy after excluding empty responses would help determine whether improved sampling fidelity also increases other types of errors.

Following the reviewer’s suggestion, we compared accuracy after excluding empty responses. On Qwen2.5-Math-7B GPQA, non-empty accuracy changes little for Power-SMC but falls for Scalable Power Sampling as the budget increases from 8 to 128. For both samplers, incorrect non-empty responses account for a smaller percentage of all outputs at the larger budget.

Please refer to our response in General Response G4.


Q1: Adaptive budgets and exponents

... can the analysis guide an adaptive power-sampling strategy that adaptively determine the sampling budget or sharpening exponent based on observable answers/failure signals?

Thank you for suggesting this interesting direction. Our analysis motivates a preliminary approach to adaptively selecting the sharpening exponent α\alpha using observable failure signals.

For an intuitive description, increasing α\alpha favors higher likelihood responses among candidates drawn from the base model. We can reduce α\alpha when SIR assigns empty candidates more selection probability than their fraction among the candidates.

Following this intuition, we conducted experiments with α∈4,2,1\alpha\in\\{4,2,1\\}. We checked these values in descending order and selected the first for which the SIR probability of an empty response did not exceed the fraction of empty candidates. We used the same 32 base samples per run at temperature 1 and calculated accuracy from the SIR selection probabilities. To compare this strategy with directly avoiding empty responses, we also removed empty candidates before SIR selection while keeping α=4\alpha=4.

Qwen2.5-Math-7B

GPQA Diamond: 198 questions. MATH500: 100 questions.

SamplingGPQA accuracy (%)MATH500 accuracy (%)
Base sampling at temperature 1116.9244.72
SIR, fixed α=4\alpha=411.5969.36
SIR, adaptive α\alpha17.5372.07
SIR, fixed α=4\alpha=4, empty candidates removed18.4773.91

As suggested by the reviewer, we implemented adaptive α\alpha. On Qwen2.5-Math-7B, adaptive α\alpha improves accuracy over fixed α=4\alpha=4, while our method of removing empty candidates gives higher accuracy than adaptive α\alpha on both tasks. Developing more general adaptive strategies that adjust the sharpening exponent or sampling budget based on observable signals, and evaluating their effectiveness across models and tasks, is an interesting direction for future work.

Thank you for your response. I think this is a solid work and I'll keep my positive score.

We are glad that our response addressed your concerns. Thank you for your positive assessment and for maintaining your score!

Reviewer_JwGcOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether a more accurate approximation of the power distribution necessarily leads to better task performance. Using SIR, it shows that increasing the number of candidates improves sampling fidelity but, under specified conditions linking failure to importance weight, can also increase failure probability. The paper identifies immediate stopping as one concrete instance of this behavior. Experiments with SIR, MH-MCMC, likelihood best-of-N, and Power-SMC across four language models and three tasks show that higher-likelihood responses do not consistently perform better. Overall, the paper demonstrates that sampling fidelity and task performance need not improve together.

Critical strengths

The paper asks an important question that has received relatively little attention. Existing work mainly focuses on approximating the power distribution more efficiently and accurately, whereas this paper examines whether a more faithful approximation is actually beneficial for task performance. The theoretical analysis shows that greater SIR fidelity can, under clearly stated conditions, increase failure probability. The decomposition between the effect of the target power distribution and approximation error is particularly useful and could inform future work on power-sampling algorithms.

Critical weaknesses

The main empirical finding is less consistent than the title suggests. In Table 3, increasing the number of SIR candidates improves accuracy for LLaMA and Phi, and no immediate stopping is observed for these models. In Table 4, likelihood best-of-N helps or hurts depending on the model and task, while several power-sampling settings do not perform worse than token-temperature sampling. The experiments support the claim that sampling fidelity and task performance can diverge, but not a general pattern in which better sampling leads to worse performance.

Immediate stopping is the only concrete LLM failure mechanism analyzed in depth. The experiments also show that likelihood-based selection can choose non-empty incorrect responses, but the paper does not examine why this occurs. It is therefore unclear how much of the observed performance loss can be attributed to immediate stopping rather than other relationships between likelihood and correctness.

As minor presentation issues, some important equations should be numbered, and several figure and table captions could be more self-contained.

Questions for authors
  1. In Table 4, Qwen2.5-Math-7B obtains 24.58% GPQA accuracy with Power-SMC, compared with 24.41% under token-temperature sampling, despite a 7.58% immediate-stopping rate. Can the authors provide further evidence about whether this small performance difference comes from the target power distribution itself or from approximation error?

  2. Corollary 3 is proved for complete-response SIR with token-temperature proposals. Does an analogous maximal-weight or monotonicity argument apply to Power-SMC with sequential weighting and intermediate resampling, or is the immediate-stopping behavior of Power-SMC currently supported only empirically?

  3. For LLaMA and Phi in Table 3, accuracy increases with the number of SIR candidates, and no immediate stopping is observed. Is correctness more positively associated with the SIR importance weight for these models, and can the authors provide evidence explaining their different behavior?

  4. Since the two stopping interventions move the output distribution away from the original power distribution, can the authors report or estimate the resulting TV distance to π_α, or otherwise quantify the associated loss of sampling fidelity?

4 comments

We thank the reviewer for their careful reading and constructive feedback.

W1: When accuracy decreases

The main empirical finding is less consistent than the title suggests. ... The experiments support the claim that sampling fidelity and task performance can diverge, but not a general pattern in which better sampling leads to worse performance.

Our experiments support the claim that a better approximation to a fixed power target can lower task accuracy. In Table 2, adding SIR candidates lowers both TV and accuracy on the same response tree.

On GPQA, increasing the budget for Power-SMC and Scalable Power Sampling produces more empty responses and lowers accuracy on Qwen2.5-Math-7B. Keeping the base stopping probability with the same number of particles recovers the Power-SMC accuracy loss. The gains for Llama and Phi in Table 3 and the mixed results in Table 4 show that the accuracy change depends on the model and decoding rule.

After the rebuttal discussion, we will revise the title and Takeaway 2 to reflect the discussion and the conditions under which accuracy decreases.

Please refer to our responses in General Response G1, G4, and G5.


W2: Accuracy conditional on non-empty responses

It is therefore unclear how much of the observed performance loss can be attributed to immediate stopping rather than other relationships between likelihood and correctness.

Immediate stopping accounts for the likelihood best-of-N accuracy drop on Qwen2.5-Math-7B and about half of the drop on Qwen2.5-7B. We combine the empty response rate of best-of-N with base sampling accuracy among non-empty responses.

GPQA accuracy (%).

SettingQwen2.5-Math-7BQwen2.5-7B
Base sampling17.3428.11
Only empty response rate changed (calculated)11.5822.20
Likelihood best-of-N11.6216.33
Best-of-N after removing empty candidates18.5220.03

In the budget comparisons on Qwen2.5-Math-7B, more empty responses account for most of the Power-SMC accuracy drop. Lower accuracy among non-empty responses accounts for most of the Scalable Power Sampling drop.

Please refer to our response in General Response G4.


W2: Why likelihood selection chooses non-empty incorrect responses

The experiments also show that likelihood-based selection can choose non-empty incorrect responses, but the paper does not examine why this occurs.

Sequence log-likelihood sums token log-probabilities, so a shorter incorrect response can score higher than a longer correct response even when its mean token log-probability is lower.

We analyze the GPQA runs where best-of-N selected a non-empty wrong answer and a correct candidate was available. For each run, we compare lengths with the correct candidate that had the highest likelihood. For comparison, we draw one non-empty incorrect response at random from the same 32 candidates.

Median number of tokens. The percentages give how often the wrong answer was shorter than the correct candidate.

ModelRuns analyzedSelected incorrect response (tokens)Correct candidate (tokens)Selected response shorter (%)Random incorrect response shorter (%)
Qwen2.5-Math-7B3018351281.140.4
Qwen2.5-7B3603030187.231.0
Llama-3.2-3B-Instruct37232741473.119.8
Phi-3.5-mini-instruct36535547080.035.5

For Llama and Phi, the selected wrong answer also usually has a higher mean token log-probability than the correct candidate.

Q1: Target effects versus approximation error

In Table 4, Qwen2.5-Math-7B obtains 24.58% GPQA accuracy with Power-SMC, compared with 24.41% under token-temperature sampling, despite a 7.58% immediate-stopping rate. Can the authors provide further evidence about whether this small performance difference comes from the target power distribution itself or from approximation error?

The small accuracy difference comes from two opposing effects: the power target lowers accuracy relative to token sampling, while approximation error from Power-SMC with 16 particles offsets that loss. We apply the decomposition of Section 5.1 with FF the set of incorrect responses and EE the event that the first token is EOS. The empty response is always incorrect, so π4(F)≥π4(E)\pi_4(F)\ge\pi_4(E), and a lower bound on π4(E)\pi_4(E) bounds both terms without sampling from π4\pi_4.

We compute a lower bound on πα(E)=p(E)α/Zα\pi_\alpha(E)=p(E)^\alpha/Z_\alpha, with Zα=∑yp(y)αZ_\alpha=\sum_y p(y)^\alpha, from the next token probabilities of the base model, without sampling.

Responses extending a prefix uu contribute at most p(u)αp(u)^\alpha to ZαZ_\alpha. Expanding the prefix replaces this bound with the sum over its continuations, which cannot increase for α≥1\alpha\ge1. We expand the prefix with the largest bound for 512 passes per question, rounding upper bounds upward and lower bounds downward. The table averages the resulting lower bounds on πα(E)\pi_\alpha(E). Since empty responses are incorrect, target accuracy is at most one minus this bound.

Qwen2.5-Math-7B on GPQA at α=4\alpha=4.

DistributionImmediate stopping (%)Accuracy (%)
Token sampling, temperature 14\frac{1}{4}1.3524.41
Power target π4\pi_4≥86.86\ge 86.86≤13.14\le 13.14
Power-SMC (16 particles)7.5824.58

The effect of the power distribution is independent of the sampler. From 8 to 128 particles, the approximation error in failure probability moves closer to zero and accuracy falls. Keeping the base stopping probability at 128 particles recovers the accuracy loss. We will add the bound and its computation to the appendix.

Please refer to our responses in General Response G1 and G5.


Q2: Immediate stopping in Power-SMC

Corollary 3 is proved for complete-response SIR with token-temperature proposals. Does an analogous maximal-weight or monotonicity argument apply to Power-SMC with sequential weighting and intermediate resampling, or is the immediate-stopping behavior of Power-SMC currently supported only empirically?

A similar weight argument compares Power-SMC with token sampling at a fixed number of particles. It does not prove monotonicity as particles are added. We use the full vocabulary, allow EOS at the first token and fix α>1\alpha>1. Empty particles keep their weights after the first token, while continuing particles receive factors below one. Resampling preserves the normalized weight of empty particles in expectation, so the probability of an empty response is at least that of token sampling. We will formalize this argument in the revision.

The increase in immediate stopping from 8 to 128 particles is supported by our GPQA experiments.

Please refer to our response in General Response G1.


Q3: Weight and correctness for Llama and Phi

For LLaMA and Phi in Table 3, accuracy increases with the number of SIR candidates, and no immediate stopping is observed. Is correctness more positively associated with the SIR importance weight for these models, and can the authors provide evidence explaining their different behavior?

Correct responses tend to receive larger SIR importance weights in Llama and Phi, consistent with their accuracy gains as candidates are added. We compare correct and incorrect candidates from the same question. For Qwen2.5-Math-7B, the incorrect candidate more often has the larger weight.

Llama and Phi produced no empty candidates in these runs. For Qwen2.5-Math-7B, empty candidates have the largest importance weight, so adding SIR candidates increases the probability of selecting an empty response, as Corollary 3 predicts.

Please refer to our response in General Response G3.

Q4: TV distance after changing the stopping rule

Since the two stopping interventions move the output distribution away from the original power distribution, can the authors report or estimate the resulting TV distance to πα\pi_\alpha, or otherwise quantify the associated loss of sampling fidelity?

Following the reviewer’s suggestion, we estimated TV between the Power-SMC output distribution under each stopping rule and the original power target. Normalizing the power target is infeasible for an LLM. We therefore used the randomly initialized GPT-2 models from G1, with one layer, three vocabulary tokens and a maximum of four generated tokens, so all 31 possible responses can be enumerated. We estimated the Power-SMC output distribution from 10,000 samples per model.

Power-SMC at α=4\alpha=4. Estimated TV to π4\pi_4, averaged over random initializations.

Stopping rule4 particles16 particles64 particles
Keep the base stopping probability0.6670.6670.667
Block EOS at the first token0.9660.9660.966

The power target favors empty responses in these models, so reducing immediate stopping keeps TV large. Proposition 5 considers distributions that change only the probability of an empty response EE, leaving the conditional distribution of non-empty responses unchanged. Setting that probability to the base model’s stopping probability p(E)p(E) gives TV ∣πα(E)−p(E)∣|\pi_\alpha(E)-p(E)| to πα\pi_\alpha; setting it to zero gives TV πα(E)\pi_\alpha(E).

On Qwen2.5-Math-7B GPQA, both stopping rules improve accuracy with the same number of particles.

Please refer to our responses in General Response G1 and G5.


Minor points: Equation numbers and captions

As minor presentation issues, some important equations should be numbered, and several figure and table captions could be more self-contained.

We will number the equations in the main text and state the experimental settings and metric definitions in the figure and table captions.

Thank you for the detailed response. I believe the additional analyses and clarifications have adequately addressed my concerns. I will keep my positive score.

Reviewer_1w7mOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper builds on Karan and Du (2026), who investigate the hypothesis that RL post-training primarily sharpens the output distribution of pretrained language models , rather than introducing fundamentally new reasoning capabilities (which is a popular hypothesis in the community, see [1]). Their work suggests that sampling from the associated power distribution does the same sharpening and thus can recover reasoning performance competitive with RL-post-trained models without additional training. In this original paper the authors show evidence for that hypothesis investigating the task performance across various base-models.

Building on this motivation, this paper investigates whether more accurate approximations to the aforementioned sampling method necessarily improve downstream task performance.

Theoretical Results / Thought experiments

  • The authors first introduce a toy-example example demonstrating that power sampling can outperform conventional token-level temperature sampling when sequence likelihood aligns with task success.
  • The main theoretical contribution establishes that increasing the number of candidates in sampling importance resampling (SIR) monotonically improves approximation fidelity, measured by total variation distance and several other divergences.
  • However, under certain conditions on the relationship between importance weights and task failures, each additional candidate simultaneously increases failure probability.

Empirical Evaluation

  • Their experiments use SIR, Metropolis–Hastings MCMC, and sequential Monte Carlo (Power-SMC) across four language models and three benchmarks (GPQA Diamond, MATH500, and UltraFeedback).
  • The results demonstrate that … I am not sure TBH I will discuss my impression of the empirical evaluation in a later section .

[1] Huang, Audrey, Adam Block, Dylan Foster, Dhruv Rohatgi, Cyril Zhang, Max Simchowitz, Jordan Ash, and Akshay Krishnamurthy. "Self-improvement in language models: The sharpening mechanism." In International Conference on Learning Representations, vol. 2025, pp. 76687-76739. 2025.

Critical strengths

Section 5

The paper’s strongest contribution is its theoretical analysis of sampling importance resampling. Building on Cremer et al (2017), the authors establish strict monotonic improvements in approximation fidelity with increasing candidate count and derive sufficient conditions under which this improvement is accompanied by monotonically increasing failure probability. The result is formulated for general finite distributions and may therefore be relevant beyond language-model sampling.

Writing

I find the paper well written and easy to follow.

Critical weaknesses

Can you explain the significance of the derived result in Section 5?

While the finite-sample monotonicity result for SIR appears potentially interesting, I find the broader conceptual contribution less compelling. The paper frames the observation that better sampling fidelity can lead to worse task performance as somewhat surprising. However, sampling fidelity and downstream task utility are fundamentally different objectives, and there is no general reason to expect them to improve monotonically together.

The distinction between accurate probabilistic inference and downstream task performance has been extensively studied. In particular, Stahlberg and Byrne (2019) already demonstrated that more accurate sequence-level inference in autoregressive language models can produce worse outputs, including empty sequences. Similarly, relationships between sequence likelihood, reward, and distribution sharpening have been investigated in more recent work, including Tomihari and Sato (2026).

Furthermore, SIR, MCMC, and autoregressive sequence sampling have a substantial existing theoretical literature. Although the paper cites some relevant work, it is not sufficiently clear which aspects of the finite-sample SIR results go beyond known properties of self-normalized importance sampling and related ordering results. I would not assume that the monotonicity theorem has previously been established, but its precise novelty relative to classical Monte Carlo theory requires a more convincing discussion.

Overall, the paper’s most distinctive contribution appears to be a relatively specific theoretical characterization of SIR, whereas the broader framing emphasizes a phenomenon that is already well understood.

Can you lay out the vision for the experimental evaluation?

The main theoretical result concerns the simultaneous monotonic improvement in sampling fidelity and deterioration in task performance. However, the practical experiments do not directly measure approximation fidelity for MH-MCMC or Power-SMC. Consequently, they demonstrate that different sampling procedures can produce different task outcomes, but do not establish that more faithful practical approximations to the power distribution systematically reduce task performance.

The empirical evidence is also relatively limited and heterogeneous. Results vary across models and benchmarks, the experiments use only four relatively small models, and uncertainty estimates are missing. In particular, accuracy does not consistently decrease with additional SIR candidates, although the immediate-stopping result is supported for one model.

Additionally, the introduction motivates power sampling through results suggesting that sharpening a pretrained base model’s output distribution can recover performance comparable to RL post-training. Yet the experimental evaluation mixes pretrained base models and instruction-tuned models without systematically analyzing how post-training affects the relationship between sequence likelihood and task success. This is particularly relevant because the strongest immediate-stopping failures occur in the base models, while neither instruction-tuned model exhibits this behavior. Although these observations do not establish a causal relationship, they raise the possibility that the reported failures depend substantially on the underlying model distribution and training procedure.

A controlled comparison of matched base and post-trained checkpoints, together with direct measurements of sampling fidelity, would substantially strengthen the empirical contribution and clarify its relevance to the paper’s motivating hypothesis.

Questions for authors
  1. In reference to my first claim of weakness, can the authors clarify the importance of their theoretical results? The following references may be useful;

[1] Chatterjee, Sourav, and Persi Diaconis. "The sample size required in importance sampling." The Annals of Applied Probability 28, no. 2 (2018): 1099-1135. [2] Sanz-Alonso, Daniel. "Importance sampling and necessary sample size: An information theory approach." SIAM/ASA Journal on Uncertainty Quantification 6, no. 2 (2018): 867-879.

  1. Do the authors contribute new understanding to the phenomenon that more accurate inference can harm task performance bejond the results from Stahlberg and Byrne (2019) & Tomihari and Sato (2026) ?

  2. Can the authors provide additional experimental results? The paper’s central theoretical result concerns improving approximation fidelity to a fixed target distribution. However, in the practical MH-MCMC and Power-SMC experiments, fidelity to the power distribution is not directly evaluated. The authors could do this by for example running MCMC chains of varying length (assuming that longer chains lead to better approximations) or IS.

Without this evidence, how should the empirical comparisons be interpreted as validating the main theoretical finding, rather than illustrating differences in task performance between sampling algorithms?

  1. Present Emirical evaluations The paper is motivated by the possibility that power sampling on pretrained base models can recover some benefits of RL post-training. However, the experiments combine base and instruction-tuned models, and the strongest immediate-stopping failures occur only in the base models.

Do the authors have evidence indicating whether this difference is related to post-training, rather than other model-specific characteristics? More generally, how do the authors interpret the relevance of their findings to the original motivation of using power sampling as an alternative to RL post-training, when using a mix of base and instruct models in their experiments?

TBH at the moment, I would prefer there to be no empirical evaluation, due to the lack of insight. I also wish the empirical results would be presented in a way that is more easily digestable.

3 comments

We thank the reviewer for their careful reading and constructive feedback.

W1/Q1: Comparison with importance sampling theory

... Although the paper cites some relevant work, it is not sufficiently clear which aspects of the finite-sample SIR results go beyond known properties of self-normalized importance sampling and related ordering results. ...

In reference to my first claim of weakness, can the authors clarify the importance of their theoretical results?

The most important conclusion of Section 5 is that increasing the sampling budget can make empty responses more likely even as the sampler better approximates the same target distribution. With SIR, we vary only the number of candidates while keeping the proposal and the power target unchanged. Under the conditions of Corollary 3, each additional SIR candidate strictly reduces TV to the same power target while making an empty response more likely. Thus, approximation error at a smaller budget can reduce the probability of an empty response.

Chatterjee and Diaconis (2018) and Sanz-Alonso (2018) study the sample size required for accurate importance sampling. Sample size bounds for accurate estimation do not determine whether the distribution of the selected SIR response moves closer to the target from every NN to N+1N+1.

Related work on importance-weighted variational inference (Burda et al., 2016; Cremer et al., 2017; Domke and Sheldon, 2018) establishes monotonic improvement of the IWAE bound and relates this bound to the KL divergence of the resampled distribution. However, a monotonically decreasing upper bound does not itself imply a monotonic decrease in the underlying divergence. Our analysis directly establishes this property for the marginal distribution of a single SIR output.

We summarize the theoretical contributions of our work and prior studies below. Here, qq is the proposal, π\pi is the target, and μN\mu_N is the distribution of the response selected from NN SIR candidates.

WorkQuantityGuarantee
Chatterjee and Diaconis (2018, Theorems 1.1–1.2)Importance sampling estimate of Eπ[f]\mathbb E_\pi[f]Sample size on the scale of exp⁡(DKL(π∥q))\exp(D_{\mathrm{KL}}(\pi\Vert q))
Sanz-Alonso (2018, Theorems 1–2)Importance sampling estimateNecessary sample size in terms of divergences between target and proposal
Agapiou et al. (2017, Theorem 2.1)Bias of the self-normalized estimateO(1/N)O(1/N) bound, which also bounds TV for the SIR output
Burda et al. (2016), Cremer et al. (2017), Domke and Sheldon (2018)IWAE gap and SIR output KLThe gap is non-increasing in NN and upper bounds DKL(μN∥π)D_{\mathrm{KL}}(\mu_N\Vert\pi)
This work (Theorem 1 and Appendix A.1)Distribution of the single SIR outputTV, both KL directions, χ2\chi^2 and squared Hellinger decrease strictly from NN to N+1N+1 under our assumptions when q≠πq\ne\pi.

The formula for the SIR output density ratio follows Skare et al. (2003). Appendix A.1 proves strict monotonicity for a finite response space with a proposal of full support. We will add the comparison with prior work to Section 2.

W1/Q2: Comparison with earlier inference results

... The distinction between accurate probabilistic inference and downstream task performance has been extensively studied. ...

Do the authors contribute new understanding to the phenomenon that more accurate inference can harm task performance bejond the results from Stahlberg and Byrne (2019) & Tomihari and Sato (2026)?

Our analysis explains when increasing the sampling budget makes a known failure more frequent, even as sampling becomes more accurate. When the conditional probability of failure is non-decreasing in the importance weight and strictly higher at some weights, each additional SIR candidate increases failure and reduces TV to the same target (Section 5.2). For immediate stopping, an empty response has the largest importance weight under the proposal in Corollary 3. Approximation error at a smaller budget therefore makes empty responses less frequent than they are under the power target.

Stahlberg and Byrne (2019) compare exact MAP decoding with beam search. Tomihari and Sato (2026) compare power sampling with token temperature sampling and relate expected reward to reward–likelihood covariance. Our result explains the direction of the change in failure probability at each finite SIR budget while the target remains fixed.

WorkWhat variesResult
Stahlberg and Byrne (2019)Exact MAP search versus beam searchExact MAP can return empty translations
Tomihari and Sato (2026)Power target versus token temperature samplingNext-token odds (Proposition 1). The EOS case compares the two endpoints of our SIR analysis.
This workSIR budget with the target and proposal fixedEach additional candidate lowers TV and raises immediate stopping (Theorem 1, Corollary 3)

Power-SMC and Scalable Power Sampling also produce more empty responses at larger budgets on Qwen2.5-Math-7B GPQA. Changing the Power-SMC stopping rule with the same number of particles recovers the lost accuracy. We will revise the introduction and Takeaway 2 to state when additional candidates increase the probability of failure.

Please refer to our responses in General Response G1 and G5.


W2: Experimental design

TBH at the moment, I would prefer there to be no empirical evaluation, due to the lack of insight. I also wish the empirical results would be presented in a way that is more easily digestable.

Our experiments examine whether additional sampling compute improves accuracy, and whether empty responses explain the loss when accuracy falls. In Table 3, adding SIR candidates raises immediate stopping on Qwen2.5-Math-7B while accuracy changes little; accuracy rises for Llama and Phi. Better approximation to the target therefore need not have the same effect on accuracy across models.

To measure approximation directly, we use small Transformers with three vocabulary tokens and at most four generated tokens. Their 31 possible responses let us compute the normalized target and compare TV across budgets. We also change the stopping rule while fixing the model and the number of particles to measure whether reducing empty responses recovers accuracy.

We will present the TV comparisons separately and place accuracy beside immediate stopping in the LLM tables.

Please refer to our response in General Response G1.


Q3: Additional budget experiments and fidelity measurements

... The authors could do this by for example running MCMC chains of varying length (assuming that longer chains lead to better approximations) or IS. ... Without this evidence, how should the empirical comparisons be interpreted as validating the main theoretical finding, rather than illustrating differences in task performance between sampling algorithms?

Following the reviewer’s suggestion, we varied the MH-MCMC chain length and the number of particles in Power-SMC. The proposal and power target stay fixed for each sampler.

Measuring TV requires the normalized power target. We use small Transformers whose 31 possible responses can be enumerated. For MH-MCMC we compute TV from its output law, and for Power-SMC we estimate TV from sampled responses. Increasing the budget lowers TV and raises immediate stopping for both samplers in this setting.

On Qwen2.5-Math-7B for GPQA, longer MH-MCMC chains and more Power-SMC particles both raise immediate stopping at α=4\alpha=4. At α=1\alpha=1, Power-SMC has the base model as both proposal and target, so the output distribution does not change with the number of particles. At α=4\alpha=4, MH-MCMC with zero steps and Power-SMC with one particle both return a draw from token sampling at temperature 14\frac{1}{4}.

Please refer to our response in General Response G1.

W2/Q4: Relevance to RL post-training

A controlled comparison of matched base and post-trained checkpoints, together with direct measurements of sampling fidelity, would substantially strengthen the empirical contribution and clarify its relevance to the paper’s motivating hypothesis.

The paper is motivated by the possibility that power sampling on pretrained base models can recover some benefits of RL post-training. ... More generally, how do the authors interpret the relevance of their findings to the original motivation of using power sampling as an alternative to RL post-training, when using a mix of base and instruct models in their experiments?

Note that our main contribution is to understand power sampling. We believe that establishing whether it can replace RL post-training is beyond the scope of this work.

Our results show that Power-SMC improves accuracy over direct sampling from the same Qwen2.5-Math-7B base checkpoint on GPQA and MATH500 (Table 4).

To investigate the relationship between post-training and immediate stopping, we additionally measured the probability of generating EOS as the first token on the same GPQA questions for two pairs of base and post-trained checkpoints. We use the corresponding chat templates for post-trained models.

GPQA. Mean probability of EOS as the first token.

MetricQwen2.5-Math-7BQwen2.5-Math-7B-InstructQwen3-4B-BaseQwen3-4B
EOS as the first token (%)1.648.7×10−58.7\times10^{-5}1.283.3×10−53.3\times10^{-5}

Following the reviewer’s suggestion, we plan to compare direct sampling and Power-SMC on matched checkpoints before and after RL post-training, using the same prompt format. We will add accuracy and immediate stopping results to the revision.


W2: Broader evaluation and uncertainty

The empirical evidence is also relatively limited and heterogeneous. Results vary across models and benchmarks, the experiments use only four relatively small models, and uncertainty estimates are missing. In particular, accuracy does not consistently decrease with additional SIR candidates, although the immediate-stopping result is supported for one model.

To address the reviewer’s concern, we ran experiments on additional models and all 500 MATH500 questions.

The confidence interval for Llama’s SIR accuracy change is above zero; the intervals for Phi and Qwen2.5-Math-7B include zero. Section 5.2 predicts the direction from how failure probability varies with importance weight.

GPQA Diamond. SIR accuracy changes from N=1N=1 to N=8N=8, in percentage points. The 95% confidence intervals use resampled questions, keeping each question’s runs together and using the same questions at both budgets.

ModelAccuracy change95% confidence interval
Llama-3.2-3B-Instruct+8.494.49 to 12.53
Phi-3.5-mini-instruct+2.53−1.07 to 6.26
Qwen2.5-Math-7B−0.91−4.37 to 2.49

Please refer to our responses in General Response G2 and G3.

Reviewer_HAXHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the relationship between sampling fidelity and task performance in LLM power sampling. Through theoretical analysis of sampling importance resampling (SIR), the authors show that increasing the number of candidates can monotonically improve approximation fidelity while degrading task performance under certain conditions. They further identify immediate stopping as a concrete failure mode. Experiments with SIR, MH-MCMC, and Power-SMC across reasoning and instruction-following tasks demonstrate that higher likelihood does not consistently lead to better performance, and that modifying stopping rules can mitigate this issue.

Critical strengths
  1. Interesting and meaningful problem. The paper challenges the assumption that better approximations to the power distribution necessarily improve downstream task performance, highlighting an important distinction between sampling fidelity and task utility.

  2. Solid theoretical analysis. The SIR analysis establishes monotonic improvements in approximation fidelity and characterizes conditions under which failure probability increases. The response-tree example and theoretical results on immediate stopping provide clear insights into the phenomenon.

  3. Empirical investigation. The evaluation covers multiple sampling algorithms, models, and tasks. The immediate-stopping experiments and corresponding mitigation strategies provide useful empirical evidence and practical insights.

Critical weaknesses
  1. Limited novelty relative to prior work. The mismatch between sequence likelihood and task quality, including degenerate outputs such as empty responses, has been studied previously. Recent works, including Tomihari & Sato (2026) and Yang et al. (2026), also investigate related limitations of power sampling. Although the SIR monotonicity analysis provides a useful theoretical contribution, the broader conclusions and identified failure modes overlap substantially with existing observations.

  2. Gap between theoretical results and empirical validation. The theoretical degradation result relies on specific conditions relating failure probability to importance weights, but these conditions are not verified for practical LLMs. Moreover, approximation fidelity is not directly measured for MH-MCMC or Power-SMC, making it difficult to establish whether observed performance degradation actually results from improved fidelity. The empirical accuracy trends are also mixed, rather than consistently following the theoretical prediction.

  3. Limited experimental scope and generalizability. The experiments focus on four relatively small models and a limited set of benchmarks, including only 100 MATH500 questions. Immediate stopping is mainly observed in the Qwen models, while the other models do not exhibit this failure mode. The proposed mitigation is evaluated in a relatively narrow setting, and the lack of uncertainty estimates further limits the strength of the empirical conclusions.

Questions for authors

Please see my pros and cons.

4 comments

We thank the reviewer for their careful reading and constructive feedback.

W1: Novelty relative to prior work

Limited novelty relative to prior work. The mismatch between sequence likelihood and task quality, including degenerate outputs such as empty responses, has been studied previously. Recent works, including Tomihari & Sato (2026) and Yang et al. (2026), also investigate related limitations of power sampling. Although the SIR monotonicity analysis provides a useful theoretical contribution, the broader conclusions and identified failure modes overlap substantially with existing observations.

Our analysis explains when reducing sampling error makes task failure more likely, and proves how the failure probability changes at each SIR budget. Under the conditions of Theorem 1 and Corollary 3 in our work, every additional SIR candidate reduces TV to the same power target and increases the probability of an empty response. The EOS comparison implied by Tomihari and Sato (2026, Proposition 1) is the endpoint case of our SIR analysis, comparing token temperature sampling at N=1N=1 with the power target as N→∞N\to\infty. Yang et al. (2026) primarily investigate how distributional sharpening can degrade multi-sample aggregation, particularly self-consistency. While they also consider sampling budget, they do not establish how increasing the number of SIR candidates simultaneously affects sampling fidelity and the failure probability of a single selected response under a fixed power target.

WorkComparisonEmpty responsesSIR budget guarantee
Stahlberg and Byrne (2019)Exact MAP vs. beam searchMAP output for about half of WMT15 English–German sentences—
Tomihari and Sato (2026)Power target vs. token temperature (Proposition 1)——
Yang et al. (2026)Power target with self-consistency voting——
This paperSampling toward a fixed πα\pi_\alphaLargest SIR importance weight (Corollary 3)SIR TV falls and immediate stopping rises at each N→N+1N\to N+1

Our analysis identifies the power target's preference for empty responses. In G5, keeping the base stopping probability improves Power-SMC accuracy with the same number of particles. We will add this discussion to Section 2.


W2: Verifying the failure condition

The theoretical degradation result relies on specific conditions relating failure probability to importance weights, but these conditions are not verified for practical LLMs.

For immediate stopping, the condition in Section 5.2 holds for the token sampling proposal used in our LLM experiments with SIR. The conditional probability of immediate stopping is one at the largest importance weight and zero at every smaller weight. For overall error, however, the condition depends on the model and need not hold.

With full vocabulary token sampling at temperature 1α\frac{1}{\alpha} and α>1\alpha>1, each token contributes a factor ∑vp(v∣u)α<1\sum_v p(v\mid u)^\alpha<1 to the importance weight, where uu is its prefix. An empty response ends after the first factor. Every non-empty response has at least one more factor (Appendix A.2). Our response to Q2 from Reviewer JwGc give the argument for Power-SMC.

For overall error, we ranked candidates by importance weight within each GPQA question and compared error rates in the lowest and highest quartiles.

Error rate (%) in the lowest and highest importance weight quartiles.

ModelLowest quartileHighest quartile
Qwen2.5-Math-7B77.0276.85
Llama-3.2-3B-Instruct80.7272.90
Phi-3.5-mini-instruct74.5868.94

For Llama and Phi, the highest weight quartile has a lower error rate than the lowest, consistent with their accuracy gains in Table 3. For Qwen2.5-Math-7B, the two quartiles have similar error rates, and overall accuracy changes little. These comparisons illustrate that the relationship between importance weight and overall error differs across models, but they do not establish the monotonicity condition in Section 5.2.


W2: Accuracy at different sampling budgets

The empirical accuracy trends are also mixed, rather than consistently following the theoretical prediction.

Section 5.2 predicts more immediate stopping under SIR as candidates are added. Overall accuracy depends on how errors vary with the importance weight. In Table 3, SIR accuracy rises for Llama and Phi and changes little for Qwen2.5-Math-7B.

Please refer to our response in General Response G3.

W2: Distance to the power target

Moreover, approximation fidelity is not directly measured for MH-MCMC or Power-SMC, making it difficult to establish whether observed performance degradation actually results from improved fidelity.

Increasing the budget lowers TV to the power target and raises immediate stopping for both MH-MCMC and Power-SMC in our fidelity experiments. We use small Transformers because we can enumerate all 31 responses and compute the normalized power target. Summing over all possible responses is intractable for the GPQA models.

On Qwen2.5-Math-7B GPQA, Power-SMC accuracy among non-empty responses changes little as the number of particles increases. Keeping the base stopping probability with the same number of particles recovers the lost accuracy.

Please refer to our responses in General Response G1, G4, G5.


W3: Immediate stopping beyond Qwen

Immediate stopping is mainly observed in the Qwen models, while the other models do not exhibit this failure mode.

On DeepSeek-Math-7B-Base, SmolLM3-3B-Base and Mistral-7B-v0.3, Power-SMC stops immediately more often than token sampling on both GPQA and MATH500. The instruct models returned no empty responses on GPQA, where their base probabilities of EOS as the first token are negligible.

With KK particles, Power-SMC can stop immediately only if at least one particle draws EOS as the first token, so its stopping probability is at most KK times that of token sampling.

Please refer to our response in General Response G2.


W3: Evaluation scope and uncertainty

The experiments focus on four relatively small models and a limited set of benchmarks, including only 100 MATH500 questions. ... The proposed mitigation is evaluated in a relatively narrow setting, and the lack of uncertainty estimates further limits the strength of the empirical conclusions.

We ran experiments on additional models and all 500 MATH500 questions. Keeping the base stopping probability improved Power-SMC accuracy for Mistral and SmolLM3 on GPQA Diamond and for SmolLM3 on MATH500.

We computed the 95% confidence intervals by resampling questions, keeping each question’s runs together and using the same questions for both settings.

GPQA Diamond. Accuracy gains from keeping the base stopping probability, relative to unmodified Power-SMC, in percentage points.

ModelAccuracy gain95% confidence interval
Qwen2.5-Math-7B+6.404.04 to 9.09
Mistral-7B-v0.3+4.212.36 to 6.40
SmolLM3-3B-Base+12.299.43 to 15.32

Please refer to our responses in General Response G2 and G5.

Thanks for the detailed response. I have no further questions and would like to raise the score.

We are happy that our response addressed your concerns and thank you for raising the score! We promise to incorporate all the discussions in our rebuttal in the revision.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response to Reviewers

We thank the reviewers for their constructive feedback and thoughtful suggestions. Below, we address several questions shared across reviews by presenting additional experiments on sampling fidelity, task performance, and immediate stopping. We discuss the theoretical contributions and their relationship to prior work in our reviewer-specific responses.

G1: Sampling budget, TV, fidelity, and task performance

Increasing the budget lowers TV to the same power target and raises immediate stopping in small Transformers, where we can compute the power target. On Qwen2.5-Math-7B GPQA, increasing the budget for Power-SMC and Scalable Power Sampling raises immediate stopping and lowers accuracy at α=4\alpha=4.

Power sampling targets πα(y)∝p(y)α\pi_\alpha(y)\propto p(y)^\alpha for a base model pp. Every sampler below starts from token sampling at temperature 1/α1/\alpha and corrects toward πα\pi_\alpha. The budget is the number of candidates for SIR and Scalable Power Sampling (Ji et al., 2026), the number of particles for Power-SMC and the number of chain steps for MH-MCMC. EE denotes immediate stopping, EOS as the first token, which returns an empty response that is always graded incorrect. TV is 12∑y∣μ(y)−πα(y)∣\frac12\sum_y|\mu(y)-\pi_\alpha(y)| between a sampler's output distribution μ\mu and πα\pi_\alpha.

Small Transformers. TV needs the normalized target, which is infeasible to compute for an LLM. We use randomly initialized GPT-2 models with one layer, a vocabulary of three tokens {a, b, EOS}, and a maximum of four generated tokens. We can therefore enumerate all 31 possible responses and compute the power target. For MH-MCMC, we compute the output distribution after each number of steps from its transition kernel. We estimate Power-SMC TV from 2,000 outputs per model. For comparison, we also estimate TV from 2,000 direct samples from π4\pi_4.

Small Transformers at α=4\alpha=4. Results are averaged over random initializations.

SamplerTV to π4\pi_4Immediate stopping (%)
MH-MCMC, 0 steps (token sampling)0.78418.2
MH-MCMC, 4 steps0.55940.7
MH-MCMC, 16 steps0.21075.6
MH-MCMC, 64 steps0.00696.0
Power-SMC, 1 particle (token sampling)0.78917.7
Power-SMC, 4 particles0.44851.9
Power-SMC, 16 particles0.06490.3
2,000 direct samples from π4\pi_40.00496.5
π4\pi_4096.6

In this controlled setting, increasing the budget reduces TV while increasing immediate stopping. The exact power target itself assigns 96.6% probability to immediate stopping. These results illustrate how a closer approximation can amplify a failure mode already present in the target distribution.

For SIR, Theorem 1 proves that TV falls with every added candidate, and Table 2 shows the decrease on an exact response tree. The small-Transformer results provide complementary evidence for MH-MCMC and Power-SMC at the evaluated budgets, without implying an analogous monotonicity guarantee for these algorithms.

GPQA Diamond. We next evaluate how increasing the sampling budget affects immediate stopping and accuracy on Qwen2.5-Math-7B across all 198 questions. SIR candidates are those of Table 3.

Qwen2.5-Math-7B on GPQA Diamond.

Sampler and budgetImmediate stopping (%)Accuracy (%)
MH-MCMC, α=4\alpha=4, steps 0 → 321.35 → 6.9024.41 → 21.89
SIR, α=4\alpha=4, candidates 1 → 8 (Table 3)2.53 → 6.2323.06 → 22.16
Power-SMC, α=4\alpha=4, particles 8 → 1286.40 → 21.0426.43 → 21.04
Scalable Power Sampling, α=4\alpha=4, candidates 8 → 1285.05 → 23.2321.72 → 5.56
Power-SMC, α=1\alpha=1, particles 8 → 1281.35 → 1.1817.00 → 18.52

Scalable Power Sampling at α=4\alpha=4.

Candidates8163264128
Immediate stopping (%)5.058.5913.6417.6823.23
Accuracy (%)21.7220.2015.6610.105.56

At α=1\alpha=1, the proposal and target both equal the base distribution, so increasing the number of particles does not change the Power-SMC output distribution. The small empirical differences in this setting are consistent with sampling variability.

Together, these experiments distinguish two aspects of our analysis: the small-Transformer experiments directly measure fidelity, while the practical LLM experiments show how task performance and immediate stopping change with the approximation budget.

AuthorsOpenReview ↗

General Response to Reviewers

G2: Immediate stopping across models

Following the reviewers’ suggestions, we conducted experiments on additional models and all 500 MATH500 questions. Power-SMC returns more empty responses than token sampling for DeepSeek-Math-7B-Base, SmolLM3-3B-Base and Mistral-7B-v0.3 on both GPQA and MATH500.

GPQA Diamond. Power-SMC uses 16 particles at α=4\alpha=4 on the 198 questions, as in Table 4. Token sampling at temperature 14\frac{1}{4} is the Power-SMC proposal and equals Power-SMC with one particle. EE is the event that the first token is EOS. The table gives its probability p(E)p(E) before temperature scaling, averaged over questions.

GPQA Diamond. Arrows go from token sampling to Power-SMC.

Modelp(E)p(E) (%)Immediate stopping (%)Accuracy (%)
Qwen2.5-Math-7B1.641.35 → 7.5824.41 → 24.58
Qwen2.5-7B1.310.84 → 1.6828.11 → 29.29
Qwen3-4B-Base1.280.34 → 2.1932.83 → 29.80
DeepSeek-Math-7B-Base5.460.17 → 3.208.08 → 8.59
SmolLM3-3B-Base26.6772.05 → 95.294.04 → 0.51
Mistral-7B-v0.313.7014.81 → 81.998.25 → 1.35
Llama-3.2-3B-Instructbelow 10−510^{-5}0 → 020.54 → 23.23
Phi-3.5-mini-instructbelow 10−510^{-5}0 → 028.79 → 27.95
Phi-4-mini-instructbelow 10−510^{-5}0 → 032.83 → 29.46

The upper bound on p(E)p(E) for the instruct models holds on every GPQA question. Neither sampler returned empty responses from these models on either task.

MATH500, all 500 questions. We additionally evaluated token-temperature sampling and Power-SMC on the full MATH500 benchmark, extending our original evaluation on a subset of 100 questions. Power-SMC uses 128 particles at α=4\alpha=4.

MATH500, all 500 questions. Arrows go from token sampling to Power-SMC.

ModelImmediate stopping (%)Accuracy (%)
Qwen2.5-Math-7B0.00 → 0.4068.60 → 79.00
Qwen2.5-7B0.00 → 0.0063.80 → 72.30
Qwen3-4B-Base0.00 → 0.0073.60 → 76.90
DeepSeek-Math-7B-Base0.00 → 4.204.20 → 1.00
SmolLM3-3B-Base79.00 → 94.4012.70 → 4.30
Mistral-7B-v0.37.40 → 61.201.00 → 0.50
Llama-3.2-3B-Instruct0 → 043.70 → 45.70
Phi-3.5-mini-instruct0 → 045.50 → 50.50
Phi-4-mini-instruct0 → 067.20 → 66.70

On MATH500, Power-SMC improves accuracy for several models, including Qwen2.5-Math-7B and Qwen3-4B-Base. However, DeepSeek-Math, SmolLM3, and Mistral exhibit increased immediate stopping and lower accuracy. These results show that power sampling can be beneficial or harmful depending on the model and task, and that immediate stopping is a particularly prominent failure mode for some base models.

G3: Importance weights and SIR accuracy

With SIR, adding candidates increases the failure rate when the chance of failure is non-decreasing in the importance weight and is strictly higher at some weights. Llama-3.2-3B-Instruct gains accuracy as candidates are added, and its correct candidates tend to have larger weights.

We use 24 candidates for each of the 198 questions, as in Table 3. The weight comparison uses only questions with both correct and incorrect candidates; the remaining questions have all 24 candidates correct or all incorrect. SIR accuracy uses all 198 questions. The table gives the percentage of pairs from the same question in which the correct candidate has the higher importance weight ρ(y)=p(y)4/q(y)\rho(y)=p(y)^4/q(y), where p(y)p(y) is the base model’s probability of response yy and qq is token sampling at temperature 14\frac{1}{4}.

GPQA Diamond. SIR accuracy from N=1N=1 to N=8N=8.

ModelQuestions with correct and incorrect candidatesCorrect candidate has higher weight (%)SIR accuracy (%)
Llama-3.2-3B-Instruct16155.719.36 → 27.85
Phi-3.5-mini-instruct15252.528.62 → 31.15
Qwen2.5-Math-7B16148.323.06 → 22.16
Qwen2.5-Math-7B, empty candidates removed16148.9

G4: Errors after excluding empty responses

On Qwen2.5-Math-7B, both samplers return more empty responses at the larger budget. Accuracy among non-empty responses changes little for Power-SMC but falls for Scalable Power Sampling.

Power-SMC uses 594 responses per budget and Scalable Power Sampling uses 198.

Qwen2.5-Math-7B on GPQA Diamond at α=4\alpha=4.

MetricPower-SMC (budget 8)Power-SMC (budget 128)
Accuracy (%)26.4321.04
Accuracy among non-empty responses (%)28.2426.65
Empty responses (% of all outputs)6.4021.04
Incorrect non-empty responses (% of all outputs)67.1757.91
MetricScalable Power Sampling (budget 8)Scalable Power Sampling (budget 128)
Accuracy (%)21.725.56
Accuracy among non-empty responses (%)22.877.24
Empty responses (% of all outputs)5.0523.23
Incorrect non-empty responses (% of all outputs)73.2371.21
AuthorsOpenReview ↗

General Response to Reviewers

G5: Stopping rules

Keeping the base model’s stopping probability improves Power-SMC accuracy at α=4\alpha=4 on GPQA Diamond for Qwen2.5-Math-7B, Mistral and SmolLM3, and on MATH500 for SmolLM3.

EE is EOS as the first token and p(E)p(E) is its probability under the base model. Proposition 5 defines two modified targets: keeping the base stopping probability assigns probability p(E)p(E) to an empty response, and blocking EOS at the first token assigns probability zero. Both targets preserve the power target’s conditional distribution over non-empty responses. We approximate these targets with Power-SMC; both rules allow EOS at later positions.

Qwen2.5-Math-7B on GPQA Diamond at α=4\alpha=4, 128 particles. Accuracy changes are relative to unmodified Power-SMC.

Stopping ruleImmediate stopping (%)Accuracy (%)Accuracy change
Base model p(E)p(E)1.64
Unmodified21.0421.04
Keep base stopping probability1.0127.44+6.40
Block first-token EOS0.0028.11+7.07

GPQA Diamond, 16 particles. Arrows go from standard Power-SMC to Power-SMC that keeps the base stopping probability.

Modelp(E)p(E) (%)Immediate stopping (%)Accuracy (%)Accuracy change
Mistral-7B-v0.313.7081.99 → 11.111.35 → 5.56+4.21
SmolLM3-3B-Base26.6795.29 → 25.760.51 → 12.79+12.29

On GPQA Diamond, keeping the base stopping probability also raises SmolLM3-3B-Base accuracy over token sampling at temperature 14\frac{1}{4}.

SmolLM3-3B-Base on all 500 MATH500 questions at α=4\alpha=4 with 128 particles. The base probability of EOS as the first token, p(E)p(E), averages 57.03%.

SamplerImmediate stopping (%)Accuracy (%)
Token sampling at temperature 14\frac{1}{4}79.0012.70
Power-SMC94.404.30
Power-SMC, keeping the base stopping probability56.1024.30

Keeping the base stopping probability brings immediate stopping close to p(E)p(E) and raises accuracy above both Power-SMC and token sampling.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.