- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper performs an empirical analysis of various LLM confidence elicitation methods and studies their failure rates when used together on the task of fact verification.
The authors give a detailed breakdown of what % of errors are caught by distributional, self-evaluative, and behavioural approaches to confidence elicitation. They also give a sense of how approaches perform when combined together, and what % of cases are covered by various combinations of approaches.
The authors show that there are often confident consensus decisions among methods, where all elicitation methods collectively fail to flag an erroneous prediction.
Critical strengths
-
The topics of claim verification and uncertainty estimation are highly relevant.
-
The paper exposes benchmark errors, and sharing these errors with the community could help increase the quality of shared research resources.
Critical weaknesses
- The paper is hard to read, and the statistics are poorly presented.
Many sentences carry several statistics with little contextualization or explanation of what these statistics imply. This issue extends to the abstract as well. Several terms like "a nine-verifier panel from five vendors" are used without proper definition of the key terms.
- The paper reports many results, but these do not particularly help inform future decisions on how to think about confidence elicitation or claim verification more broadly:
The signal-level findings largely confirm prior work: the verdict-token margin is a strong free baseline, verbalised confidence is degenerate, and P(True) offers a modest gain at the cost of an extra call. The core is hard to act on. The recommendations in Section 7 are either standard practice or not well supported by the paper's own data. For example, the claim that an agreeing second verifier "adds no evidence of correctness" conflicts with Section F, where error rates are substantially lower on claims Gemini agrees with.
- The paper omits study of arguably the most useful confidence elicitation method:
The paper does not evaluate trained probes on hidden states, which prior work identifies as among the most effective failure predictors and which can recover precisely the self-consistent errors that make up the core. The authors acknowledge this gap but attribute it to scope, even though their panel includes open-weight models. Without them, the claim that the core is set by "the information all signals share" is established only for output-level signals. Likewise, the decomposition is computed only for GPT-4o; the other eight verifiers enter only through error co-occurrence, so it is unknown whether the size and composition of the core hold for smaller open models or for fine-tuned verifiers such as MiniCheck.
Questions for authors
- Concretely what should a practitioner or verifier designer do differently after reading this paper?
4 comments
To Reviewer NQJ5
Dear Reviewer NQJ5,
We understand this is a busy period and we are sorry to add to it. We would be very grateful if you could take a moment to look at our response and the revised paper. In short, we corrected the Section 7 sentence you showed to conflict with Appendix F, and we ran the experiments you asked for: hidden-state probes on the open-weight panel verifiers, and the decomposition on five further verifiers and a fine-tuned one (Tables 1b and 1c). If any concern remains, please let us know and we will respond right away. If your concerns have been resolved, we would be very glad to hear that as well. Thank you again for your time and for the careful review.
Too much AI-generated content for my liking
Frankly, there is too much Claude-style writing in both the response to my review and the text of the paper for me to actively consider raising my score. It's not that AI writing is by itself bad, but rather it makes the paper one that simply does not communicate an effective argument.
For example, even in the above comment: ", Algorithm 1 is in §3.3 with a worked example, and the abstract goes from 25 numbers to 9 and §1 from ~45 to 12." is classic Claude-Code style output, and it shows a lack of genuine engagement with the spirit of what my review was.
I am pretty sure: "§7 vs. §F: the reviewer is right, and the sentence is corrected. Among accepted claims the error rate is 3.9% when Gemini agrees and 23.5% when it disagrees (2.7% vs 16.9% in the top margin bin; new Table 6b), so agreement is evidence. The old sentence stated a different fact (Gemini agrees with 70.6% of GPT-4o's confident misses) as if it were this one." is copy pasted from a claude console.
I would prefer not to spend more time on this review as a result.
The core issue is, once again, not that AI is in principle bad at writing papers, but rather it has some error modes which this paper falls into as a consequence.
The above sentence: "Among accepted claims the error rate is 3.9% when Gemini agrees and 23.5% when it disagrees (2.7% vs 16.9% in the top margin bin; new Table 6b), so agreement is evidence. " is simply not as coherent to human readers as it is to LLMs.
I prefer to reject papers that are not written with human readers in mind.
Response to Reviewer NQJ5
Thank you for clarifying your concerns. We understand your point about the readability of our paper and acknowledge that some of our responses were overly condensed. However, we would like to distinguish the style of our writing from the scientific effort behind it. We took your original review seriously and devoted considerable time to addressing your concerns, including conducting additional analyses and revising the manuscript. We respect your assessment of the writing quality, but we hope that our research and the substance of our responses will also be considered independently of assumptions about how the text was produced.
Response to Reviewer NQJ5
We thank the reviewer for reading §7 against Appendix F and for naming the experiment the paper lacked. The reviewer is right on both counts.
W1 (hard to read; statistics poorly presented; undefined terms). Agreed. Every term is defined before first use (glossary in §3.3; the panel is introduced at the start of §4.2), Algorithm 1 is in §3.3 with a worked example, and the abstract goes from 25 numbers to 9 and §1 from ~45 to 12.
W2 (confirms prior work; hard to act on; §7 vs. Section F).
W3 (no hidden-state probe; floor shown only for output-level signals; only GPT-4o; fine-tuned verifiers). Done on exactly those models (Tables 1b, 1c). The reviewer's expectation holds for errors in general and not for the consensus core.
Q (what should a practitioner or verifier designer do differently?). Table 17 gives one row per decision. A deployer: (1) rank by the free margin instead of paying for a second call; (2) budget 13.9% for a 5% residual-risk target and choose the guarantee's loss first, since the loss definition accounts for 25.1pp of the cost; (3) monitor with a ~17× cheaper verifier (AUROC 0.778 vs 0.784); (4) review disagreements, not agreements; (5) supply evidence, which turns silent errors into abstentions. A designer: (6) avoid agreement-weighted pseudo-labels, which inherit a 7.75× consensus error; (7) add vocabulary-preserving meaning violations to training data (+4.3pp confident acceptance); (8) fit any white-box probe on the target benchmark.
Strength 2 (sharing the benchmark errors). The 210 adjudicated labels are released keyed by benchmark IDs, with the per-dataset distribution in Table 18.
If any of these leaves a weakness open, we would be grateful to know which, so that we can address it before the discussion closes.