acceptodds
Under review as a conference paper at ICLR 2027

More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It

Abstract

Power Sampling sharpens a language model's distribution over complete generation trajectories, offering a verifier-free way to improve reasoning at inference time. It also has the potential to serve as a general-purpose front end for a broad range of downstream sampling methods. However, we uncover a striking paradox: Power Sampling can drive more probability mass toward correct trajectories while degrading the downstream inference it is intended to enhance. Using self-consistency as a representative case, we observe accuracy drops of up to 10.1 percentage points across models and reasoning benchmarks. We trace this paradox to two mismatches. Dose mismatch arises because a fixed exponent induces drastically different amounts of distributional change across problems. Coverage mismatch arises because global sharpening concentrates mass on a narrow set of dominant paths: high pass@k, often interpreted as evidence of preserved diversity, can therefore coexist with the loss of broad reasoning-path support required for downstream aggregation, search, and selection. Guided by this diagnosis, we replace uniform trajectory exponentiation with a deformation-controlled, support-preserving Power target that calibrates sharpening across problems while limiting the suppression of moderate-probability paths. In a same-budget instantiation with weighted self-consistency, the repaired sampler reverses the losses caused by global Power and, when combined with self-evaluation, outperforms standard multi-sample inference across reasoning benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, reasoning, inference-time scaling, power sampling, self-consistency, sampling diversity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tXh9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether sharpening a model's distribution with Power Sampling helps self-consistency. Using Power weights on a shared pool of 8 base samples, the authors find that SC accuracy drops in 28 of 33 model/benchmark settings, even though on HARP the average correct mass goes up. They blame this on two things: sharpening can favor one strong wrong path over several weaker paths that agree on the right answer, and a fixed alpha changes some problems much more than others. Their fix, Relative-Rank SoftSat, weights samples by their likelihood rank in the pool and caps the gain at high ranks, optionally together with a same-model self-evaluation label. They test it on three open models and 11 benchmarks.

Critical strengths

I found the main observation useful. Many people assume that sharpening toward likely trajectories should help multi-sample methods, and the 33 settings show fairly clearly that it can hurt SC instead. The HARP analysis, where correct mass rises but the margin shrinks because the top trajectory takes most of the weight, makes the reason easy to see. The fix is also simple and cheap, since it only reweights samples that were already generated for SC, and the rank argument in Lemma 1 is clean.

Critical weaknesses

My main concern is that the title, the contributions and the conclusion talk about Power Sampling in general, but almost all experiments only reweight 8 base samples with p^(alpha-1). With alpha = 4 and long outputs, these weights put nearly all the mass on the most likely sample. On the HARP slice ESS is about one sample, and every alpha from 1.5 up gives almost the same scores as alpha = 4. So the "Power" baseline is close to taking the answer of the highest-likelihood sample, and the drop may come mostly from degenerate weights in a small pool. It may say little about the Power target itself. The only test with real Power-SMC is 9 single-seed settings. Those numbers do look lower than the Uniform results in Tables 1 and 2, which partly supports the paper, but they come from different runs and there is no Uniform baseline on the same seed.

Also, SoftSat on its own does not really improve over plain uniform SC. It wins and loses about equally against Uniform with the same average accuracy, so its main benefit is recovering from a baseline that the paper itself sets up. With a 75% uniform floor (eta = 0.25) it stays close to uniform voting by design, and on the HARP pool linear rank does a bit better, so I am not sure the saturation part matters much. The real gains come from self-evaluation, which needs an extra pass over every trajectory, and SE-only already beats Uniform in most settings. There is no comparison with weighted SC methods like CISC or RPC, even though they are cited. And the gaps between SoftSat, SelfEval and Uniform are mostly within the standard deviations, with no paired tests reported.

A smaller point on presentation: the results come from many different pools, seeds and slices, which made the paper hard to follow, and the writing is very dense.

Questions for authors
  1. Table 7 suggests that real Power-SMC with uniform SC also trails Uniform, but it is single-seed and from different runs. Can you run Power-SMC and Uniform SC on the same seeds and on more benchmarks? Without this, I read the results as being about importance weighting on 8 samples.
  2. How does SoftSat+SelfEval compare with confidence-weighted SC such as CISC or RPC? And how does it compare with simply drawing more samples using the extra compute that self-evaluation needs?
  3. Only beta is varied in the sensitivity study. How does SoftSat behave with a smaller uniform floor (larger eta)? I would like to know if it does anything beyond staying close to uniform.
  4. Did you try length-normalized log-likelihood, for both the Power weights and the ranks? Summed log-probs depend a lot on length, which may explain part of the collapse.
Reviewer_xNMdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether pushing sampling weight toward high-likelihood reasoning trajectories, as in power sampling, improves answer consensus. It separates three things: the probability of a correct trajectory, the vote margin for an answer, and the final consensus decision. The main finding is that likelihood weighting often lowers consensus accuracy even as average correct mass rises, because gains land on already-correct problems while decisions regress elsewhere. It shows this by reweighting the same fixed pool of samples under uniform, likelihood, and rank-based schemes, across three models and eleven benchmarks. It proposes a conservative rank-based weighting as a repair, plus an optional same-model correctness judge.

Critical strengths

By reweighting the same fixed pool of samples, the paper shows that likelihood weighting often hurts consensus accuracy even while average correct mass goes up. The effect holds across three models and eleven benchmarks, and the mechanism is made concrete: gains concentrate on already-correct problems while decisions regress elsewhere.

Critical weaknesses
  • The proposed fix does not beat the trivial baseline. The rank-based weighting mostly just repairs the damage that likelihood weighting does, landing essentially at plain uniform voting, and it keeps a large uniform component by design. The real gains come from a separate self-evaluation judge, which costs a full reread per candidate and beats the judge-only variant only about half the time. So the actionable takeaway reduces to "do not use likelihood weighting, plain voting is fine," which undercuts the paper's "how to fix it" framing.
  • It is unclear how far the failure transfers to actual power sampling. The strong failures are on small reweighted pools whose importance weights collapse, not on samples drawn from a real power sampler. The asymptotic argument does not cover the finite budgets used, the direct-sampler comparisons skip the largest failures, and because scores are taken before top-k renormalization the implemented weights target a different distribution than the ideal one. So the result may be partly about finite-pool reweighting rather than power sampling itself.
Questions for authors
  1. On one or two high-loss settings, does an actual power sampler show the same failure, or is the effect specific to reweighting a small fixed pool?
  2. Under the top-k proposal used, what target do the implemented weights actually converge to, and does a matched-proposal check change the main result?
  3. At matched effective sample size or uniform-mixture strength, does the saturating rank weighting beat linear rank and a simple floor-regularized version of likelihood weighting?
Reviewer_3fomOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper examines how trajectory-level likelihood sharpening affects answer-level consensus. It shows that base-pool power weighting can increase average correct-trajectory mass while reducing consensus accuracy, and analyzes this behavior through dose and coverage mismatch. Relative-Rank SoftSat replaces likelihood gaps with rank-based, saturated weights to reduce concentration, with an optional self-evaluation signal.

Critical strengths

S1: Distinguishing correct-trajectory mass from the margin over competing answers is useful. The shared candidate pools make the comparison between aggregation rules controlled, and the observed losses show that likelihood sharpening is an unreliable default for the evaluated consensus pipeline.

S2: The proposed rank-based rule directly addresses variation in reweighting strength across prompts. Its finite-pool calibration property is clearly stated, and the broad comparisons show that it recovers much of the accuracy lost to power weighting.

Critical weaknesses

W1: The main experiments establish a limitation of reweighting small base-sampled pools more directly than a limitation of sampling from the power target. Section 6.1 uses eight shared trajectories per problem. In this regime, concentrated importance weights can produce poor estimates even when the target distribution itself is useful. The asymptotic connection to target sampling does not resolve this distinction at the evaluated budget. This limits the broader interpretation of the negative result.

W2: The evidence for the specific SoftSat design is weaker than the evidence against aggressive weighting. Section 6.2 reports mixed results against uniform aggregation, and Equation 16 retains a substantial uniform component. The recovery could therefore come largely from shrinking toward uniform weights. A comparison at matched concentration is needed to identify the additional benefit of rank calibration and saturation. Together with W1, this leaves the central diagnosis and proposed repair insufficiently separated from finite-sample weight concentration.

Questions for authors

Q1: Can the authors separate target-distribution effects from finite-pool estimation effects in the settings with the largest losses?

Q2: For SoftSat, how much of the recovery remains when compared with simple power-weight shrinkage using the same uniform component or effective sample size?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.