- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies whether more accurate sampling from an LLM's power distribution improves task performance. It proves that increasing the number of SIR candidates strictly improves distributional fidelity, while under specified conditions it also increases failure probability. The paper further analyzes immediate stopping as a concrete failure mode. Synthetic examples and experiments on reasoning and instruction-following tasks illustrate that higher sequence likelihood does not consistently yield better answers, and that reducing immediate stopping can improve performance. The main insight is that sampling accuracy and the quality of the target distribution must be evaluated separately.
Critical strengths
-
The paper provides an interesting and substantive theoretical insight. It proves that increasing the number of SIR candidates strictly improves sampling fidelity while, under specified conditions, increasing failure probability. This clearly demonstrates why better approximation to a target distribution need not improve task performance.
-
The paper connects its theory to a concrete failure mechanism. The analysis of immediate stopping, supported by observations in real LLMs and experiments modifying the stopping rule, gives the theoretical findings practical relevance and suggests how this failure can be mitigated.
Critical weaknesses
-
The effect of model capability is not sufficiently explored. It would be useful to examine whether stronger models show better alignment between trajectory likelihood and correctness, and whether this leads to larger gains from power sampling. This would help clarify when the method is likely to help or hurt.
-
The empirical evidence for failure is relatively narrow. Immediate stopping mainly occurs in the two Qwen models tested. Analyzing accuracy after excluding empty responses would help determine whether improved sampling fidelity also increases other types of errors.
Questions for authors
Please see the critical weakness. Besides, can the analysis guide an adaptive power-sampling strategy that adaptively determine the sampling budget or sharpening exponent based on observable answers/failure signals?
3 comments
Thank you for your response. I think this is a solid work and I'll keep my positive score.
We are glad that our response addressed your concerns. Thank you for your positive assessment and for maintaining your score!
We thank the reviewer for their careful reading and constructive feedback.
W1: Model capability and power sampling gains
Following the reviewer’s suggestion, we compared base accuracy, likelihood and Power-SMC gains. On both tasks, models that more often assign higher likelihood to correct responses than to incorrect responses tend to have higher base accuracy and larger Power-SMC gains.
Using 32 base samples per run at temperature 1, we measure how often a correct response has higher sequence log-likelihood than an incorrect response to the same question. Sequence log-likelihood is the sum of token log-probabilities. The table gives Spearman correlations of this percentage with base accuracy and Power-SMC gain.
GPQA: 198 questions, 16 particles. MATH500: 100 questions, 128 particles, excluding Mistral. Power-SMC gain is its accuracy at α=4 minus token sampling accuracy at temperature 41.
When the comparison includes only the Qwen, Llama and Phi models, the correlation with base accuracy is negative on GPQA and weak on MATH500. Base accuracy alone therefore does not consistently predict how well likelihood distinguishes correct from incorrect responses.
W2: Immediate stopping in other model families
Following the reviewer’s suggestion, we conducted experiments on additional base models. On DeepSeek-Math-7B-Base, SmolLM3-3B-Base and Mistral-7B-v0.3, Power-SMC stops immediately more often than token sampling on both GPQA and MATH500.
Please refer to our response in General Response G2.
W2: Errors after excluding empty responses
Following the reviewer’s suggestion, we compared accuracy after excluding empty responses. On Qwen2.5-Math-7B GPQA, non-empty accuracy changes little for Power-SMC but falls for Scalable Power Sampling as the budget increases from 8 to 128. For both samplers, incorrect non-empty responses account for a smaller percentage of all outputs at the larger budget.
Please refer to our response in General Response G4.
Q1: Adaptive budgets and exponents
Thank you for suggesting this interesting direction. Our analysis motivates a preliminary approach to adaptively selecting the sharpening exponent α using observable failure signals.
For an intuitive description, increasing α favors higher likelihood responses among candidates drawn from the base model. We can reduce α when SIR assigns empty candidates more selection probability than their fraction among the candidates.
Following this intuition, we conducted experiments with α∈4,2,1. We checked these values in descending order and selected the first for which the SIR probability of an empty response did not exceed the fraction of empty candidates. We used the same 32 base samples per run at temperature 1 and calculated accuracy from the SIR selection probabilities. To compare this strategy with directly avoiding empty responses, we also removed empty candidates before SIR selection while keeping α=4.
Qwen2.5-Math-7B
GPQA Diamond: 198 questions. MATH500: 100 questions.
As suggested by the reviewer, we implemented adaptive α. On Qwen2.5-Math-7B, adaptive α improves accuracy over fixed α=4, while our method of removing empty candidates gives higher accuracy than adaptive α on both tasks. Developing more general adaptive strategies that adjust the sharpening exponent or sampling budget based on observable signals, and evaluating their effectiveness across models and tasks, is an interesting direction for future work.