acceptodds
Withdrawn from ICLR 2027

How Much Correctness Signal Is in the Top- Log-Probability Channel? An Attribution Study of Single-Pass LLM Uncertainty

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reliable LLM deployment needs uncertainty estimates that are both faithful and cheap, yet the strongest methods resample the model, while single-pass methods trade accuracy for cost. We ask how much response-correctness signal a single generation's token-wise predictive distributions actually carry once three everyday factors are accounted for: response length, supervision, and model access. To answer, we build PULSE, a supervised sequence probe that reads only the per-step top- log-probabilities of a single generation, without access to hidden states or weights, and compare it with reference probes that read progressively more of a response, across five LLMs and three mathematical reasoning benchmarks. Most of the black-box signal comes for free: response length alone is a strong predictor, and a zero-shot entropy score comes within about two AUROC points of PULSE. Supervision adds nothing over these free scalars on its own; reading the full top- channel adds about a point, and modeling sequence order a smaller increment on top. White-box hidden states, by contrast, open a much larger gap. Within the black-box setting, PULSE is the strongest option we test, never significantly worse than the best baseline and matching a five-pass perturbation-based estimator at one-fifth of its cost. The resulting attribution shows where black-box uncertainty signal comes from, and how little of it lies beyond free, zero-shot scores.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Uncertainty quantification, arge language models, Black-box uncertainty estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jdd4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work does uncertainty estimation for LLMs. PULSE uses a small, trained transformer model to learn a mapping from token probabilities to response correctness. Basically, I think the authors try to model uncertainty estimation as a regression problem that can be learned directly from token probabilities.

However, in this paper, the motivation and problem definition are unclear. The authors also acknowledged that their method design won’t help. For example, “adds nothing over these free scalars on its own” and “models sequence order in a smaller increment” in abstract. The experimental setting is also not fair. For example, most of the baselines in the paper are training-free, whereas PULSE requires training.

Critical strengths

The authors conduct extensive ablation studies

Critical weaknesses
  1. Lack of correct position against the previous works. line 13-14: “yet the strongest methods resample the model, while single-pass methods trade accuracy for cost.” Not true. Many sampling-based methods won’t affect the performance. For example, TokUR doesn’t affect generation quality, which is the main baseline used in this paper. Lines 128-130: “We answer this empirically by learning the readout instead, and find that the answer is: not much." The authors said the gap this paper wants to fill is “not much”.
  2. Overclaim: Line 25: “PULSE is the strongest option we test, never significantly worse than the best baseline”: not true. See table 1 Llama-3.1-8B DeepScaler and gemma-4-E2B MATH500, PULESE is not the best one.
  3. The motivation is confusing. For example, the authors said that “compressing the whole generation into one handcrafted statistic” would “discard the temporal structure”. But it seems the authors' method still uses a scalar as the uncertainty. The problem this paper wants to study is also not so clear to me. In Line 047, it says, “asks a narrower, more testable question,” but followed by many questions: “how much response-correctness signal …. ”, “where ….”, “how much is ….”, “How much…..”, “how much …”. I’m trying to understand, but I really cannot grasp the gap this paper aims to fill.
  4. An important component of this paper's conclusions concerns “response length”. The authors aim to show the importance of response length for uncertainty using experiments. But this conclusion has been widely studied by previous work [1~3].
  5. The experimental results in this paper make me feel that the author doesn’t think the proposed method can really work. For example, in section 5.3, the authors acknowledged that the length itself is sufficient, and that supervised learning and sequence modeling in PULSE contribute little. They also said “PULSE’s remaining edge is the top-k evidence itself, not architectural sophistication on top of it," in Line 349-353.

[1] SAUP: Uncertainty Propagation on LLM Agent (ACL 2025)

[2] Sequential Probabilistic Uncertainty Estimation for Parallel Multi-Agent Reasoning Systems

[3] From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents

Questions for authors
  1. Please see the weaknesses.
  2. The presentation needs substantial improvement.
  3. I’m confused about the design that combines the top-k probability of each token to do simple regression. Could the author provide any supportive experimental evidence or theoretical proof?
Reviewer_3KjaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Authors present PULSE, a one-pass uncertainty estimator that only takes in top-k probabilities across a sequence of LLM next-token predictions and produces a score reflecting the probability that the LLM is wrong. PULSE involves normalizing trajectory probabilities and then a prediction head that outputs a pair of Dirichlet parameters that encode the relative probaiblities of the output being correct/incorrect. Authors show that PULSE outperforms other standard uncertainty measurements within a triplet of datasets but that unsupervised zero-shot methods like TECP come close. The authors also show that the trained uncertainty models are somewhat transferable across datasets/models.

Critical strengths
  • Training objective and model is easy to follow and reproduce.
  • Model performs well in the settings provided and at reasonable inference cost.
  • Analysis is interesting, especially around the TECP low-cost alternative model.
Critical weaknesses
  • The paper explicitly avoids comparison to multi-pass methods like semantic entropy. However, it is generally accepted that single-pass methods only based on softmax probabilities are a fairly weak baseline for uncertainty estimation. Even if the authors show that they can beat out other weak baselines within this restrictive setting, I question whether the end result is really important enough given that these methods are somewhat outdated. Even if the multi-pass methods win out, it would be interesting to have that comparison. This type of analysis is even more critical if naive baselines that don't even need labels like TECP plus response length is enough to almost beat the proposed method.
  • There is not much discussion around why the model architecture and loss function proposed were designed as such. In many papers that are testing auxiliary modules these types of design choices can be considered ansatz approaches but given that the main point here seems to be the specific architecture and output design of PULSE I think a lot more discussion is warranted around why instinctively these choices were made.
  • There is an implicit conflation between accuracy/correctness versus uncertainty. This is often the case with uncertainty papers, but it would be good to address this point explicitly. Incorrectness does not necessarily mean the model is uncertain, as models can be confidently incorrect.
Questions for authors

(1) How does the proposed method compare to multi-pass methods like semantic entropy?

(2) What was the insight behind the model design and loss function design of PULSE? Why is this better than other approaches and what can we learn from this design?

(3) How is correctness related to uncertainty and what are we really measuring here?

Nit: also generally the paper could be grammatically improved. Run-on sentences occur very frequently and spacing between paragraphs is uneven.

Reviewer_6hXhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors ask how much response correctness signal is contained in the per step of top-k log-probs of a single LLM generation, and also where that signal comes from. They introduce PULSE, a small causal Transformer with an evidential head over perstep top 10 log-probabilities, trained on labeled responses. It is compared on the same responses and labels, with zero-shot single-pass scores (summed entropy, mean entropy, self-certainty, EDIS, verbalized confidence), with 2-pass P(True), with 5-pass TokUR and with supervised controls, and on a separate split, with a hidden state probe. The study covers five open models of different sizes (1B-8B) and three math benchmarks. Additionally, they find that the response length alone reaches 73.9AUROC, the summed entropy score TECP(78.2) comes within around 2 points of PULSE(80.1), and hidden states add substantially more.

Critical strengths
  • Every main-table estimator is scored on the same responses and labels, baseline signs are cross-fitted and there are paired bootstraps are used. Negative results are included.

  • It is a useful practical message for mathematical correctness. They include summed entropy, which is a strong free baseline, and a supervised top-k readout adds around 2 points.

Critical weaknesses
  • The headline result shrinks under the paper's own controls, and the attribution depends on the order of the steps. TECP captures 94% of PULSE's above-chance AUROC overall, but only 82% within length quintiles [Table A16] and 71% within questions [Table A24]. A hidden state probe that sees only the prompt scores 77.2, about the same as TECP [Table A26], so it appears much of the signal is question difficulty. The increments depend on the order in which factors are added in table A17, adding width first gives width +4.4 and order +0.8; adding order first gives order +3.1 and width +2.1. This interaction (-2.3) is larger than the whole PULSE to TECP gap (1.9).

  • The "blackbox" framing and the title are bit broader than the evidence.(Though the title changed?) The "free" scores like (TECP, PredEnt, SelfCert, EDIS) are computed from the full vocabulary distribution in Section 5.1, which top-k only access does not provide. Versions computed from the same top-10 log-probs that PULSE sees aren’t reported. Also, the cost comparison is weak as a zero-shot TECP already beats the five-pass TokUR so one-fifth of the cost of TokUR is important context.

Questions for authors
  • Can you report within-question AUROC for every probe in the step by step comparison (length, TECP, the supervised probes, PULSE and the hidden-state probe)?

  • What do TECP and PredEnt score when computed only from the top-10 log-probabilities that PULSE sees?

  • Section 4.1 says the sampled token is in the top-10 on only 45-69% of Llama3.2-1B steps at temperature 0.8, which seems low. Could the tokens and log-probabilities be misaligned?

  • How does PULSE differ from LOS-Net [1], which also trains a Transformer on top-K output probabilities?

[1] Bar-Shalom et al. Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions. AAAI 2026. arXiv:2503.14043.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.