acceptodds
Under review as a conference paper at ICLR 2027

LLM Benchmark Means Can Be Blind to the Tail

Abstract

LLM benchmark failure rates are often read as properties of model weights, but they are also properties of the decoder used to sample outputs. We show that this matters most in the tail. On independently selected SecurityEval prompts where -detected weaknesses are rare under an untruncated baseline, lowering the temperature to reduces prompt-weighted medium-severity rates by roughly one third to one half across three models, with rate ratios between and . On the full benchmark, the corresponding ratio for Qwen2.5-1.5B is , with a 95% paired-prompt bootstrap interval of . All code-weakness rates are conditional on parseable output, and parseability is reported separately. This contrast motivates treating the decoder as part of the estimand. We study when samples collected under one decoder can estimate a rate under another. For configurations that share the same model logits, a teacher-forced pass over stored completions gives exact likelihood ratios for target decoders, without additional generation. The transfer is valid only when the audit support contains the target's event-relevant support. When it does, we use a per-prefix Rényi-2 quantity computed from the stored logits as an empirical predictor of transfer cost, and show that prefix weighting can improve efficiency for prefix-measurable events. The experiments also show a limit of reuse: measured effective sample size and other sample-based diagnostics can miss poorly sampled mass, and do not by themselves support a family-wide certificate. We recommend reporting the decoding configuration with every failure rate, and checking support, statistical cost, and parseability before transferring conclusions across decoders.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
LLM evaluation, rare-event estimation, importance sampling, Rényi divergence, secure code generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2LU3OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I'm not entirely sure what's being communicated in this paper. The authors declare an intention to explore the effects of various decoding configurations on the means and tails of various benchmark distributions. There are theoretical results and experimental results presented as well.

Critical strengths

none

Critical weaknesses
  • I can't follow this paper. It appears to be fully or heavily AI-generated without much critical intervention on the part of the authors. I would suggest doing a thorough manual review of the manuscript as well as the ideas behind it to ensure the submission is communicating conceptually what you are aiming to get across
  • As far as I can tell, one key experiment involves conditioning on a set of prompts that did not cause errors in one configuration and comparing to another? This seems like it may induce selection bias, so the authors should be careful about that. However, take this with a grain of salt as I did not really follow what was going on at any point
Questions for authors

n/a

Reviewer_ankFOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that LLM failure rates depend on the decoder, and that this matters mostly on prompts where failures are rare. On low-rate SecurityEval prompts, lowering the temperature from 1.0 to 0.7 roughly halves the bandit-detected weakness rate, while the full-benchmark average barely moves. The paper then studies reusing samples across decoders via importance weights computed from shared logits. Identification requires that the audit decoder can produce everything the target can, and a Rényi-2 quantity predicts the cost. The transfer is validated on a synthetic log-probability event, and the paper reports that standard diagnostics can miss unsampled mass.

Critical strengths

The work is carefully executed: independent prompt selection, prompt-clustered inference with Holm correction, and openly reported negative results. The implementation checklist in Appendix C is practically useful. The strengths are in execution rather than ideas.

Critical weaknesses
  1. No new perspective. The main conclusions follow almost directly from definitions:
  • Greedy decoding can’t represent sampling.
  • Lowering temperature makes usually-safe prompts safer and usually-unsafe ones worse, so the mean cancels. The paper’s own Table 5 shows the aggregate slightly rising at T=0.7.
  • Cross-decoder transfer is standard importance sampling. I didn’t find a result that would surprise someone familiar with decoding or importance sampling.
  1. The method doesn’t reach its motivating problem.
  • The transfer is never applied to the safety event. By the paper’s own Table 11, it would be unusable at the safety setting (L=160).
  • Validation in the rare regime rests on cells with only a handful of direct events.
  • Only 2 of 8 primary safety tests survive Holm correction.
Questions for authors
  1. Is there a case where the decoder changes a conclusion people actually draw, such as flipping model rankings?
  2. On high-rate prompts, does T=0.7 raise the rate, so that the flat mean is simply cancellation?
  3. Can the transfer recover Table 1 on the actual safety comparison?
Reviewer_RthBOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that an LLM benchmark failure rate depends on the decoder, not just the model weights. On SecurityEval, changing the sampling temperature has a much larger effect on prompts with rare Bandit-detected code weaknesses than on the benchmark average. The authors then use stored completions and teacher-forced logits to estimate event rates under other decoding settings. They explain when this reweighting is possible, propose a R'enyi-2-based check for transfer cost, and show where common diagnostics fail. I found the support analysis interesting. My reservation is that the transfer experiments and the security finding are not tested together.

Critical strengths

I think the basic warning is useful. An average over a benchmark can stay almost unchanged while some prompts become more likely to produce a failure. The authors selected their low-rate prompts independently of the samples used to measure the temperature effect, which makes that comparison more credible.

The support example in Section 6.2 is also strong. Reweighting from top-k=50k=50 to an untruncated decoder underestimates one event rate by about seven times, even though the measured effective sample size is 0.989$N$. That is a clear demonstration that the usual weight diagnostics can look reassuring when important mass is missing. I also appreciate that the paper shows the failed transfer cells and does not claim that its attempted family-wide certificate works.

Critical weaknesses

My main concern is that the cross-decoder method is never evaluated on the safety event used to motivate it. The SecurityEval experiment counts Python weaknesses flagged by Bandit. Sections 6.2-6.4 instead transfer a reference-log-probability atypicality event, and the prefix experiment uses a string indicator defined on the first 16 tokens. These are reasonable tests of the statistical machinery, but neither shows that a stored safety audit can estimate the Bandit weakness rate under another decoder. The authors say direct comparisons for that event would be expensive. I understand the reason, although that is also the difficult setting where the proposed method needs to be useful. Even a smaller experiment using the actual weakness event would make the contribution more convincing.

I am less certain about how far the rare-event finding generalizes. The security result comes from 121 Python stubs and a single static analyser. Only two of the four temperature-effect magnitude tests survive Holm correction. For Qwen2.5-7B at medium severity, the prompt-weighted rate ratio is 0.665, but its 95% interval includes one ([0.35, 1.15]). There is stronger evidence for the direction than for a stable effect size. Also, around 29-34% of that model's completions are unparseable in the comparison, so the reported weakness rate is conditional on a changing subset of outputs. The authors report parseability separately, which is good, but I would still like to see the joint rate of producing parseable insecure code. The full-benchmark and low-rate results come from different sampling campaigns and their contrast was not directly tested. Finally, the R'enyi-based admission rule is chosen retrospectively on 32 testable cells. It looks useful there, yet the later family experiment has failures that the diagnostics do not catch. I think the paper is careful about these limits, but they leave the practical reliability of a transferred safety rate unresolved.

Questions for authors
  1. Could the authors report at least one cross-decoder estimate for the actual Bandit-defined weakness event, including a direct target estimate and parseability? This would connect the safety experiment to the transfer method.
  2. How much do the low-rate results change with a different prompt-selection threshold? I would also like to know whether the divergence-based admission rule works similarly on new decoder settings or an independent audit, rather than only the retrospective comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.