- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper evaluates a simple inference-time procedure, where they generate N candidate responses and select the one with the lowest average negative log-probability under the model. This minimum surprisal rate criterion is equivalent to selecting the candidate with minimum perplexity. Experiments use Qwen2.5-Math-7B, Qwen2.5-7B, and Phi-3.5-mini-instruct on MATH500 and HumanEval, comparing against low-temperature sampling, Power Sampling MCMC, and GRPO models trained on MATH. The empirical observation is potentially useful, but novelty, theoretical applicability, and efficiency claims are insufficiently established.
Critical strengths
S1: The method is simple, parallelizable, and requires neither training nor an external verifier.
S2: Results across three models and two benchmarks establish a useful baseline against more complex sampling methods.
S3: Prefix-based scoring suggests a potentially useful direction for adaptive stopping.
Critical weaknesses
W1: Minimum surprisal is equivalent to minimum perplexity, already evaluated as a Best-of-N selector in prior self-certainty work. Closely related baselines are missing.
W2: The lower bound permits probability queries, while the algorithm also requires samples. Its hard instance also has probability exactly beta, violating the stated strict >beta condition.
W3: Accuracy does not consistently exceed MCMC, and coding comparisons use GRPO trained on math.
W4: Temperature-dependent scoring and generation details are unclear, and prefix length is selected using evaluation results.
Questions for authors
Q1: How does the method compare with existing perplexity selection, self-certainty, and self-consistency using identical candidates?
Q2: Can the theorem be corrected under a common sampling-and-probability access model relevant to autoregressive inference?
Q3: Can you report matched-token and matched-time comparisons with uncertainty, clarify scoring probabilities, and validate prefix selection on separate data?
2 comments
Thank you for the detailed rebuttal
Through their rebuttal, the authors have addressed most of my initial concerns, so I increased my score.
We are truly glad to see that you found our method simple, parallelizable, and requiring neither training nor an external verifier while results across three models and two benchmarks establish a useful baseline against more complex sampling methods.
The goal of our paper is to show that a very simple baseline, min-surprisal sampling, matches or outperforms much more complex MCMC methods [1] at a matched sampling budget, while also being much lower latency (MCMC require N sequential samples while min-surprisal can do N in parallel). We also provide a theoretical justification, that the number of samples required by min-surprisal is close to optimal in the worst-case for any such sampling method.
Min-surprisal sampling can be applied to literally any question asked to an LLM, all that is necessary is to keep track of cumulative log-probs. In contrast, methods like self-consistency are only applicable to questions where there is a unique correct solution e.g. math problems with a numerical answer. Thus, these methods cannot even be applied to coding problems, where multiple syntactically different coding solutions can all implement the same logical function.
[1] Karan, A. and Du, Y. Reasoning with sampling: Your base model is smarter than you think. ICLR 2026. Oral Presentation.
There is no need to correct the theorem, because if you have access to computing probabilities then you immediately have access to sampling! In fact, this is precisely how autoregressive LLMs work: to sample the next token we use the model to compute the probability of each possible next token, and then we sample from that discrete distribution.
This is a typo. The condition in the definition should be ≥β rather than >β.
The results are already matched-token budgets: we use N=10 samples for min-surprisal and compare this to N=10 MCMC iterations for Power Sampling. The scoring probabilities are from the original LLM distribution.
For the prefix selection validation question, we took a random split of the MATH500 dataset into a train set and a test set of 250 problems each. The peak threshold for the train set was a t=1250 token prefix, with an accuracy of 78.8%. Using this same threshold t=1250 on the held-out test set gave accuracy 78.0%, which was also the peak accuracy for that set. Thus, the prefix threshold selection method generalizes to held-out problems.