- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors study confidence estimation for LLMs, and specifically for tasks where model outputs are continuously graded.
Instead of treating correctness as binary (labels are either or ) it is continuous (), and the authors thus formulate confidence estimation as a probabilistic regression problem.
They evaluate different regression methods on 7 datasets across 4 different tasks, with 2 LLMs per task.
Critical strengths
-
The studied problem is interesting, I have never seen anyone formulating confidence estimation as probabilistic regression (predicting ) for continuous correctness labels before. This is relevant both for someone interested in LLM confidence estimation/evaluation, and someone interested in probabilistic regression.
-
The paper is well written and easy to follow overall, I think the proposed approach makes sense, and the experimental evaluation covers multiple different datasets/tasks and two different LLMs per task.
Critical weaknesses
- The paper evaluates different models/methods within the proposed approach of probabilistic regression for continuous correctness labels, but the practical utility of this approach itself hasn't really been demonstrated. It remains unclear what this actually adds in practice compared to the common approach of treating correctness as binary.
Questions for authors
Questions/suggestions:
-
Can you directly compare your proposed continuous approach with the standard binary approach in some way, to demonstrate what this adds in practice? Binarize the correctness labels and implement some common method, directly compare with this as a baseline?
-
The paper evaluates different methods (Beta Reg, Quantile Reg, etc) within the proposed continuous approach for confidence estimation, but how would you actually apply this in practice? For example, to determine whether a given LLM output should be trusted or if the model should abstain? Or, to compare different LLMs in terms of reliability/trustworthiness? You have results for both LLM #1 and LLM #2 in the paper, can you use your approach to evaluate which of these is the best?
-
Also, if using your approach e.g. to evaluate and compare different LLMs, does the choice of regression method matter? Could e.g. Beta Reg give that LLM #2 is better than #1, while Quantile Reg gives that #1 is better than #2? If so, how do you decide which regression method to use?
-
Section 6, "validating that Beta regression methods achieve superior performance with fewer samples": Has this really been evaluated properly though? Could do a controlled evaluation, by creating subsets of the data (e.g. 10%, 25%, 50%, 100%) and comparing how the performance scales with more samples?
-
Section 5.2 - Selective Prediction: Why specifically this Selective-MAE@50% metric (why not e.g. 25% or 10% instead)? Could it not make more sense to evaluate using sparsification plots and/or the AUSE metric, instead of arbitrarily picking one threshold?
-
Section 5.2 - Selective Prediction: You write that "Table 3 shows Hurdle-Beta achieves lower selective error on XSum and HotpotQA" and "on multimodal BigCodeBench, Random Forest is comparable". Perhaps not quite fairly worded? 0.051 vs 0.053 on XSum --> "Beta achieves lower", but 0.322 vs 0.333 on BigCodeBench --> "Random Forest is comparable"? If Beta is better on XSum, then Random Forest is better on BigCodeBench?
Minor things:
- The caption for Table 2 should probably be above the table instead.
- Section 2, "Kadavath et al. (2022)elicit verbalized": formatting.
- Section 4, "is empirically validated in Zhang et al. (2020)": use \citep instead of \citet here?
Clarification of rating:
I found this paper interesting and quite enjoyed reading it. I think the studied problem and proposed approach are interesting and overall make sense. I want to be able to recommend accept for this paper. But, right now I'm quite borderline on it, and I think some clarifications will be required during the rebuttal/discussion.
4 comments
Thank you for the response.
I have read the other reviews and all author responses.
My view of the paper remains similar to my original review. I think my main concern ("the practical utility of this approach itself hasn't really been demonstrated. It remains unclear what this actually adds in practice compared to the common approach of treating correctness as binary") mostly remains.
"Or, to compare different LLMs in terms of reliability/trustworthiness? You have results for both LLM #1 and LLM #2 in the paper, can you use your approach to evaluate which of these is the best?", I don't really think this question from my review has been adressed.
"The reviewer is correct that a retention curve is more informative, and we will add Selective-MAE at other retention levels (e.g., 25% and 75%) or a sparsification curve as suggested", I do think some new result for this should be added already in the rebuttal.
I will keep my "3: Weak acceptance" score for now, but I remain quite borderline on the paper.
We thank the reviewer for the follow-up. We were unable to run new experiments during this period as we focused heavily on reworking some theoretical statements, but we appreciate the suggestions (such as around Selective-MAE) and will include them as mentioned previously. Here we clarify two points.
Practical utility compared with binary correctness. In our approach, a single fitted model replaces a set of binary classifiers, each tied to a threshold chosen in advance. This model can answer whether a response clears any quality threshold, including one chosen after deployment, and also gives interval estimates for flagging uncertain cases. Because it is one distribution, its answers for different thresholds are consistent with each other, which separately fitted classifiers do not guarantee. We see this as a practical benefit in itself, in addition to the richer representation of a distribution for continuous correctness.
Comparing LLMs. It may help to clarify the paper's scope. Our primary goal is per-query confidence estimation for a given LLM, not ranking LLMs. The estimators are fit separately for each LLM and each fit is evaluated for calibration. The point is to make the approach agnostic to the choice of LLM, so that the method can be deployed for any LLM. As the reviewer inferred, since each LLM gets its own predicted distribution over correctness, the framework also lends itself to comparing LLMs on the same queries.
We are grateful for the reviewer's feedback and hope they will continue to support the paper in the final discussion.
Thank you again.
I will keep my "3: Weak acceptance" score but remain quite borderline on the paper.
Rebuttal
We thank the reviewer for valuable feedback and the careful reading, including the formatting points, and for finding the problem interesting. We respond to specific questions below and describe edits in our revised document to incorporate the feedback.
Q1 (comparison with the binary approach): This comparison is already in the paper. Multi-threshold binary regression is the standard binary approach: logistic regression on correctness labels binarized at a threshold, applied here at five thresholds (0.1, 0.3, 0.5, 0.7, 0.9) with the same features and splits (Section 3.3). CRPS is the Brier score of the threshold event averaged over all thresholds, so Table 1 compares how well each method predicts P(y≥τ∣x). Beta and Hurdle-Beta have lower CRPS than multi-threshold on both LLMs for all three datasets (within one standard deviation on BigCodeBench and for HotpotQA with LLM 2), with one model for all thresholds instead of one per threshold. Calibrated data-free confidence scores are further standard baselines (Appendix Tables 4 to 10). We will also include a per-threshold AUROC and Brier comparison in an appendix.
Q2 / Q3 combined (use in practice, choice of method): To decide whether to trust a response, one computes its predicted distribution and abstains or escalates when P(y≥τ∣x) falls below a cutoff or the predicted interval is wide. Both τ and the cutoff can be chosen after deployment, per user or task, without retraining, which a binary classifier cannot offer. Selective-MAE (Section 5.2) shows that predicted uncertainty identifies where predictions are reliable.
The choice of regression method is determined mainly by the task's score distribution, not by the LLM (Fig. 2): Hurdle-Beta when mass sits near 1, plain Beta for smooth unimodal scores, and piecewise methods for multimodal or discrete scores, which can be checked on training scores before fitting. Consistently, within each dataset, the same methods are best on ECE, IS and CRPS for both LLMs in Table 1, up to differences within one standard deviation. Since methods differ mainly in their uncertainty estimates and much less in their predicted means on smooth data (Section 5.1), we expect comparisons of LLMs by predicted mean correctness to agree across methods there. Because the regressors are supervised, one can also fit the same regression for each LLM on the same queries and compare the resulting distributions, which could inform the choice of LLM when that is a consideration. We have expanded the discussion of practical use in Section 6.
Q4 (Section 6 claim): We agree that the evidence for this statement is limited, so we have removed it from the Conclusion. The controlled evaluation that the reviewer suggests is Fig. 4(A) and Appendix D.1: training subsets of 100 to 4000 generations on HotpotQA, now referenced in Section 5.2. All methods converge by about 1000 generations (about 200 queries), and Hurdle-Beta is worse than the others at the smallest sizes. The theoretical statement is restricted to plain Beta versus an equal-width histogram (Theorem 1).
Q5 (retention level): The 50% level is a median-retention summary, not a tuned choice, and the metric is defined for any retention rate. The reviewer is correct that a retention curve is more informative, and we will add Selective-MAE at other retention levels (e.g., 25% and 75%) or a sparsification curve as suggested.
Q6 (Table 3 wording): The reviewer is correct. Table 3 reports LLM 1, where the gaps are 0.051 vs. 0.053 on XSum and 0.333 vs. 0.322 on BigCodeBench, both within one standard deviation, while HotpotQA shows a larger gap (0.166 vs. 0.181). With LLM 2, Random Forest is comparable or better (Appendix Tables 4, 8, 10), so the evidence does not support a consistent advantage for Hurdle-Beta in selective prediction, and we have reworded the text accordingly.
We are glad the reviewer found the problem and approach interesting. If these clarifications resolve the open points, we would be grateful for the reviewer supporting the paper in the discussion. We thank them again for helpful comments and suggestions.