acceptodds
Under review as a conference paper at ICLR 2027

When LLM Confidence Fails: External Calibration for Tabular Predictions

Abstract

Large language models (LLMs) are increasingly applied to structured clinical predictions, making reliable uncertainty quantification essential for safe deployment. Evaluating across three model families (both open and closed weights), three prompting conditions and nine binary clinical prediction tasks with over 16,000 combined patient records, we demonstrate that verbalized confidence from LLMs is fundamentally uninformative regarding instance-level correctness on tabular data. By formally disentangling two commonly conflated evaluation metrics, we show that an LLM carrying zero correctness information can deceptively attain a classifier AUROC of 0.806, while its true self-knowledge inverts to below random chance on complex etiologies like acute kidney injury. To address this, we introduce a scalable, two-call external calibrator that predicts per-instance correctness using task-level descriptors, rank-aggregation statistics, and the divergence between the LLM's self-reported feature attribution and a data-derived reference. Because our calibrator requires no model internals, it is directly applicable to proprietary APIs. While cross-task transfer without target adaptation reveals that external calibration fails on certain targets due to concept shift, adapting the calibrator with a small few-shot target set resolves this degradation. Crucially, source-task pretraining reduces the required target labeling budget by roughly an order of magnitude, achieving correctness AUROC gains (up to +0.363) and providing a robust, lightweight guardrail for LLM deployment.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Large Language Models (LLMs), Calibration, Clinical Machine Learning, Tabular Data, Rank Aggregation, Feature Attribution Divergence, Transfer learning, Uncertainty Quantification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_aiNYOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper examines whether verbalized confidence predicts LLM correctness on clinical tabular tasks. It distinguishes outcome AUROC from correctness AUROC and proposes an external XGBoost calibrator using model outputs, attribution divergence from mutual information references, and task descriptors. Experiments across three model families, nine tasks, and three prompting conditions demonstrate substantial gains from target task adaptation alongside inconsistent transfer without adaptation.

Critical strengths

1.The paper illustrates a practically important distinction between clinical-outcome discrimination and the ability to identify incorrect predictions.

2.The evaluation covers multiple model families, clinical tasks, and prompting conditions, and includes informative analyses of negative transfer and source-versus-target-only training.

3.The proposed formulation is compatible with proprietary APIs, and the attribution-only results suggest potentially useful signals beyond verbalized confidence on difficult tasks.

Critical weaknesses

1.The supplied code constructs MI references from the full labeled target cohort before forming evaluation splits. Appendix G describes few-shot adaptation but implements within-task cross-validation. Support sets are also balanced using known prediction correctness. These choices undermine the claimed held-out evaluation and labeling efficiency.

2.Correctness AUROC measures ranking, not probability calibration, and no calibration or selective-risk evaluation substantiates the deployment claims. Table 3 contradicts the claim that confidence is fundamentally uninformative. The related-work discussion also mischaracterizes prior studies, including Xiong et al. (2024), that already evaluate correctness prediction.

3.The main adaptation protocol lacks matched output-only and logit-only baselines; within-task ablations do not establish attribution gains at the claimed label budget. Moreover, fixed JSON examples of 0.85 confidence and 0.75 self-assessment match Qwen’s dominant outputs, leaving prompt anchoring as an uncontrolled explanation for confidence collapse.

Questions for authors

1.Can you rerun the full pipeline with MI references estimated exclusively from the permitted support labels, a disjoint test set, and support acquisition that does not condition on unknown correctness? Please also reconcile Appendix G with the released implementation.

2.Under identical support sets and test patients, how does the full method compare with predictors using only the predicted label, confidence, and self-assessment, or only logits for Qwen? Please report paired uncertainty estimates, probability-calibration metrics, and risk–coverage performance.

3.Do the confidence-collapse findings persist after removing or varying the numerical examples in the output schemas? How would these controls affect the claim that verbalized confidence is fundamentally uninformative?

2 comments
AuthorsOpenReview ↗

Corrected protocol (results hold), new baselines and calibration metrics added

We thank the reviewer for the detailed review and for examining the code. The issues identified were real, and we have corrected them.

W1 / Q1. MI references, Appendix G, and support selection.

The reviewer is correct on all three points. We reran the full pipeline so that, at each k, (i) MI reference is estimated only from the k labeled support patients, (ii) supports are drawn at random rather than stratified by correctness, and (iii) evaluation uses only the remaining, disjoint patients. Results are essentially unchanged, at k=200k = 200 (Gemini, zero-shot), improvements over verbalized confidence are +0.351 on AKI (previously +0.363), +0.304 on PE (+0.295) and +0.098 on IHM (+0.082). Under the same protocol, GPT-5.5 shows the same pattern (AKI +0.328, PE +0.138, IHM +0.077). MI rankings from random kk subsets are mostly stable and give unchanged within-task calibrator AUROC. We agree Appendix G mixed a stability analysis with an adaptation experiment. At k=0k = 0 (LOTO), we state that the reference uses labeled target data, the revision separates them. The idea is to use disagreement of the model with a good reference which might give some insight into the model's confidence and correctness, this reference can be from a domain-expert or from the data (MI or weighted MI). The calibrator is aligned to the quality of the gold-standard reference.

W2. Calibration and selective-risk metrics.

We reported AUROC because the intended use is ranking predictions for review, where only ordering of scores matter. We agree these metrics are necessary and report both. At k=200k = 200, the calibrator reduces mean ECE from 0.126 (raw verbalized confidence) to 0.046 and mean Brier from 0.201 to 0.160, with the largest gains where confidence is least reliable (for PE: ECE 0.214 to 0.057; revisit: 0.272 to 0.076). On both tasks, deferring by raw confidence is worse than deferring at random (overall error 36.7% and 37.6%). On high-accuracy ED tasks where verbalized confidence is well calibrated (ICU transfer, ECE 0.024), the calibrator does not improve further. For selective prediction, mean AURC falls from 0.187 (raw confidence) to 0.142. At 80% coverage, the error rate drops from 38.5% to 25.9% on PE and from 39.2% to 25.6% on AKI. For GPT-5.5, mean ECE falls from 0.095 to 0.040 and AURC from 0.186 to 0.148; on AKI, error at 80% coverage drops from 38.1% to 24.8%. Paired bootstrap CIs will be in the revision.

W2. Contradiction with Table 3 and related work.

We agree that our claims were too strong about all models. Table 3 supports it for Qwen, (3 distinct confidence values, self-knowledge AUROC 0.50 on most tasks). Gemini and GPT-5.5 are moderately informative (mean 0.659, 0.699) but unreliable under a single threshold. We now describe confidence as a weak and inconsistent correctness signal. Our work is complementary to Xiong et al. (2024), they report text Q&A outcomes and model’s correctness, we do it for tabular datasets and compare how the AskA_{sk} can diverge from AclfA_{clf}, benchmarking these numbers and analyzing the gaps, we do not claim that self-knowledge metric as a contribution. Further, they use multiple inferences for a single testcase, whereas our method only needs two calls at most which is much more scalable, we do this by taking advantage of the tabular structure of the data.

W3 / Q2. Matched output-only and logit-only baselines.

We now report these under the adaptation protocol itself, with identical supports and evaluations. At k=200k = 200, the full calibrator for Gemini exceeds the output-only model (ŷ, c, p_true) on all 9 tasks (mean AUROC 0.758 vs. 0.668). For Qwen, we add logit-only and output-plus-logit baselines. At k=200k = 200, the full calibrator exceeds the output-plus-logit model on 7 of 9 tasks (mean AUROC gain +0.101; PE +0.285, AKI +0.169) and the logit-only model on 8 of 9 (+0.081). At k<=10k <= 10, logits alone transfer better; attribution features become beneficial from k = 50. We agree our baseline of choosing raw confidence served the purpose of comparing the model's raw confidence which did not test the method. Our new results show gains over outputs-only models, which establish the need for these external features along with model-output features. It is worth noting that, combination of both types of features is what that performs well not each alone.

W3 / Q3. Prompt anchoring.

The example values in our schema (0.85, 0.75) match Qwen's dominant outputs. Consistent with prior findings, verbalized confidence often depends on prompt template instead of task-difficulty or answer correctness which is again a failure mode. The same schema was used for Gemini and GPT-5.5, which produce a wider range of values (Table 4,5,6), suggesting the collapse reflects Qwen’s susceptibility to anchoring rather than the schema alone. We agree this means Qwen’s verbalized confidence reflects the prompt more than its own confidence.

AuthorsOpenReview ↗

Follow-up: corrected few-shot protocol in revised PDF

Dear Reviewer,

Thank you again for your careful review and for examining our code. We have posted our responses and uploaded a revised PDF along with the revised code, with changes marked in blue and summarized with section pointers in our general comment. Following the issues you identified, we corrected the few-shot protocol: support sets are now drawn at random, the mutual information reference is estimated from the support set only, and evaluation excludes the support patients. All few-shot results were rerun under this protocol with 5 support draws and confidence intervals (Sec. 5.4, App. I.2), alongside new baselines and calibration metrics. We would be grateful to know whether these changes address your concerns, and we are happy to clarify anything before the discussion period ends.

Reviewer_NWqeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work examines the performance of LLMs in medical applications, identifies a discrepancy between confidence levels and actual outcomes, and proposes and validates a concise solution to address this issue.

Critical strengths

The phenomena observed in this work are interesting and insightful; addressing this issue broadens the application of LLMs in real-world scenarios.

Critical weaknesses
  1. The proposed solution appears a little naive; it seems more like an engineering heuristic than a scientific method.
  2. I suspect that the discrepancy between the reported confidence levels and the actual performance might stem from a lack of domain-specific pre-training. Would using a model pre-trained on the relevant domain yield results comparable to those of the proposed method? Alternatively, could it be that the definition of "confidence" employed is ill-suited for this specific domain—and would refining this definition offer a more sound approach?
Questions for authors

Could the authors conduct comparative experiments regarding the methods mentioned in the "weakness" section to demonstrate the necessity of the approach proposed in the manuscript, or provide a compelling analysis explaining why such comparative experiments are not feasible?

2 comments
AuthorsOpenReview ↗

New baselines test the hypothesis, result replicates across two models

We thank the reviewer for finding the phenomena interesting and for suggesting concrete comparisons, which we have added.

W1. The solution appears to be an engineering heuristic.

We made the method’s hypothesis explicit and tested it. The hypothesis is that disagreement between features the model states as driving its prediction and a data-derived reference ranking carries a signal regarding the model's confidence in its prediction errors beyond just model outputs. In our evaluations, the full calibrator exceeds the output-only baseline (ŷ, c, p_true) on all 9 tasks at k = 200 (mean AUROC 0.758 vs. 0.668) with the largest gains where verbalized confidence fails (PE +0.237, AKI +0.188). The result replicates on GPT-5.5 (0.779 vs. 0.702, all 9 tasks). We also empirically show that AUROC can diverge into classifier score and self-knowledge score, having constant confidence can give us higher classification score (which is traditionally used) but that doesn’t mean that its stated confidence is reliable (for Qwen, Aclf=0.806A_{clf} = 0.806 vs. Ask=0.50A_{sk} = 0.50).

W2 / Q. Domain-specific pretraining, alternative definitions of confidence, and necessity of the approach

We deliberately evaluated general-purpose LLMs, keeping API-based models in mind. They are the models most commonly deployed in practice, including in clinical workflows, and most domains have no specialized model. Our external calibrator is a correctness estimator which does not depend on domain information and treats LLM as a black-box. Therefore, our methods transfer directly with little to no modification needed and since our method works on general-purpose models, it therefore applies more broadly than a domain-specific model which is a narrower setting. We agree it is interesting to see whether domain pretraining changes how confidence behaves, and we view it as a natural extension to analyze domain-specific models (e.g., MedGemma) and leave it to future work.

No new definitions of confidence were introduced in this work but rather we use three definitions of confidence established in the previous works, verbalized confidence, P(true) - a self-assessment call of correctness (Kadavath et al., 2022), and for local models (Qwen), token probabilities. Our findings show that these three might be weak signals for the model's correctness. Any refinement of these definitions, such as learned combinations, remains a function of the model’s own outputs. Our output-only baseline (trained XGBoost) learns such a combination resulting in an AUROC 0.668 at k=200k=200. Our calibrator reaches 0.758 (GPT-5.5: 0.779) and reduces ECE from 0.126 to 0.046 (GPT-5.5: 0.095 to 0.040), because attribution divergence adds information outside the model's outputs. Moreover, the standard definition in clinical risk prediction, a probability of the outcome, corresponds to our signed score, for Qwen it attains a classifier AUROC of 0.806 while carrying no correctness information (self-knowledge AUROC 0.50). Finally, sampling-based confidence (Xiong et al., 2024) requires repeated generations per recored, which conflicts with the two-call, black-box setting we target.

Table R2. Gemini (GPT-5.5 in parentheses), zero-shot, k=200k = 200, supports excluded.

Verbalized confidenceOutput-onlyFull calibrator
Mean AUROC0.662 (0.700)0.668 (0.702)0.758 (0.779)
Mean Brier0.201 (0.193)0.184 (0.189)0.160 (0.155)
Mean ECE0.126 (0.095)0.082 (0.111)0.046 (0.040)
PE AUROC0.560 (0.689)0.627 (0.790)0.854 (0.827)
AKI AUROC0.448 (0.463)0.611 (0.541)0.798 (0.791)

Across the nine tasks, the full calibrator improves mean AUROC by +0.096 over verbalized confidence and +0.090 over the output-only model, and reduces mean ECE by 63%. Gains are most where verbalized confidence fails (PE +0.294, AKI +0.350). On high-accuracy ED tasks (critical outcome, ICU transfer, hospitalization) the calibrator matches raw confidence while the output-only model falls below it.

AuthorsOpenReview ↗

Follow-up: hypothesis and feature-group ablation in revised PDF

Dear Reviewer,

Thank you again for your review. We have posted our responses and uploaded a revised PDF, with changes marked in blue and summarized with section pointers in our general comment. In the revision we state the hypothesis behind the method explicitly (Sec. 1, Sec. 3.3), a prediction is less likely to be correct when the features the model reports driving its prediction disagree with a gold-standard reference (MI reference in this work). We now test this directly. Under the same protocol and label budget, a calibrator on disagreement features alone and one on the model’s own outputs alone reach similar mean AUROC (0.638 and 0.653 for Gemini), while combining them reaches 0.752 (App. J.1, Table 26). The attribution signal is strongest where verbalized confidence is inverted. On AKI, attribution features alone reach 0.777 versus 0.602 for the model’s outputs. For Qwen2.5-7B, whose confidence carries no correctness information, attribution features raise AUROC from 0.662 with token log-probabilities alone to 0.735 (Table 27). We would be grateful to know whether this addresses your concerns, and we are happy to clarify anything before the discussion period ends.

Reviewer_pFhROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors tackle the issue of using zero-shot LLMs in clinical settings where the confidence is not known. They propose training a small auxiliary model to calibrate and better describe the outputs of the LLM so clinicans can make informed decisions. They test this on a few different medical datasets, and study and analyse different combinations and ablations.

Critical strengths

A clear strength is the ability to use this in black box or isolated settings, as some experiments include open weight models that can run in settings where data cannot leave isolated environments. The XGBoost approach can be trained locally on modest clinical hardware (as long as the LLMs can operate, but don’t require much additional compute).

Critical weaknesses

I have three key issues here.

The first is that verbalised confidence by the models themselves is described through the paper as “fundamentally uninformative”, yet the experiments themselves show that self-knowledge often not random at all and contains strong signal (Table 3). And LLM confidence is by far the top feature in the SHAP importance feature ranking. I believe this issue is partly due to using pooled confidences instead of using the yes/no classes as conditionals, which seems to carry strong signal.

This seems like a fundamental conflict between motivation/claims and actual results and I find it hard to reconcile them in the paper as it is presented.

Secondly, the claimed benefits of the attribution-divergence features are not isolated sufficiently. They are compared against the raw verbalised self-confidence values which makes for a very weak baseline. A much fairer alternative would be XGBoost using just the yes/no, self-confidence and P(true) to compare against. The within task ablations in table 19 show some aspects of this but it would be better tested in a complete setting.

Finally, the motivation for using LLMs over specialised prediction models, as stated in the paper, is that they require no training on specific tasks, yet the presented solution is to train an XGBoost classifier. That seems to defeat the purpose of using LLMs for their zero-shot capabilities. I can see this potentially being useful in data limited cases where one cannot train a task specific model, but this solution would only work if it is known that the data-requirement boundary for the proposed XGBoost classifier is sufficiently lower than just training a task specific model on the data. That’s not clear here.

Questions for authors
  1. What is the AURUC confidence when split into YES / NO predictions, for each model and class?
  2. How does an XGBoost classifier perform using only the yes/no prediction, self-confidence and P(true) perform?
  3. How were the cohorts sampled? At natural rates or upsampled? If not using natural rates, how does the system perform at natural rates?
2 comments
AuthorsOpenReview ↗

Per-class AUROC and output-only baseline, attribution features add signal beyond model outputs

We thank the reviewer for the careful reading and several thoughtful suggestions which led to new analyses.

W1 / Q1. Is verbalized confidence really uninformative?, and per- class AUROC

We agree that our wording was stronger than our results supported across all models. For Qwen, verbalized confidence is uninformative as it elicits only 3 distinct values and its self-knowledge AUROC is 0.50 on most tasks, consistent with prior findings of verbalized confidence (Xiong et al., 2024). For API models, confidence carries moderate signals but is unreliable. Following the reviewer's suggestion, we split self-knowledge AUROC by predicted class (all models in appendix of revised paper)

Table R1. A_sk within predicted class, Gemini, zero-shot.

TaskPooledWithin y^=1\hat{y}=1Within y^=0\hat{y}=0Within-class (pair-weighted)
PE0.5500.5830.7790.771
AKI0.4470.4560.4580.458
IHM0.5880.6690.5680.635
Revisit0.6710.6460.5120.604
Hosp.0.7110.6970.6920.696
Critical0.7480.6980.7730.737
ED IHM0.7420.6580.8590.816
ICU transfer0.7270.6970.7490.727
ACS/MI0.7450.7340.8050.748
Mean0.6590.6490.6880.688

The reviewer’s intuition was correct, within a predicted class, confidence ranks correct above incorrect predictions on 8/9 tasks. However, this is unreliable as a single score, if a model whose "no" answers are 50% accurate with confidence 0.88–0.92, and whose "yes" answers are 90% accurate with confidence 0.68–0.72, can rank errors perfectly within each class yet scores 0.46 when pooled, because ‘no’ is ranked above. On PE, predicted negatives receive higher median confidence (0.88 vs. 0.78) yet are less accurate (52% vs. 92%), so pooled AUROC falls to 0.550 despite 0.771 within class and on AKI it is inverted even within class. On SHAP, confidence is important along with predicted class and attribution features. On its own, it is a weak signal (mean AUROC 0.662 vs. 0.758; 0.700 vs. 0.779 for GPT-5.5 for the full calibrator at k = 200). We now describe it as a weak and inconsistent correctness signal, from uninformative (Qwen) to moderately informative but unreliable (Gemini, GPT-5.5).

W2 / Q2. XGBoost using only the prediction, confidence and P(true).

We agree that this is the right baseline. Comparison against raw confidence is the deployment baseline but that does not isolate what our features add. We now report it under identical random support sets and evaluation. At k = 200 (Gemini, zero-shot), the full calibrator exceeds the output-only model on all 9 tasks by more than 0.03 on 7 (mean AUROC 0.758 vs. 0.668), with the largest gains on PE (+0.237), AKI (+0.188), IHM (+0.138), hospitalization (+0.089) and revisit (+0.087). It also improves Brier on at least 8 and ECE on 7 of 9 tasks (Table R2, in response to reviewer 2). For GPT-5.5 the result replicates (0.779 vs. 0.702), with large gains on AKI (+0.250), IHM (+0.210) and hospitalization (+0.124) and near-ties on ED tasks, where confidence is already informative. The output-only model barely beats raw confidence (0.702 vs. 0.700). This shows that adding attribution features improves calibrator performances where raw confidence is the worst.

W3. Training a calibrator seems to defeat the purpose of zero-shot LLMs.

Our work is to build a calibrator, which is an external correctness estimator determining if an LLM will give a correct response for some input. Our goal is not to argue for LLMs over task-specific models, but to provide a guardrail for LLMs that are already deployed. This is the similar motivation to that of standard post-hoc calibration methods like Platt and Isotonic scaling that train on model confidences with a labeled data but since the verbalized confidences are unreliable we take some information from the dataset to get insights using feature ranking disagreements. The calibrator works for multiple tasks and does not take raw task features unlike a task-specific model. Additionally, large datasets are needed to train a model on a particular task but the calibrator takes only 100-200 rows to adapt to the new task.

Q3. Cohort sampling.

Cohorts were sampled with N = 2,000 patients per task (IHM: 658 due to cohort limits), stratified by positive rates of 36–61%. Since the LLM was not trained on these datasets and the calibrator doesn’t learn on the actual datasets with an aim is to predict LLM correctness and not task-outcome, the sampling mechanism may not affect its performance. We noted our sampling rates in Table 1. All comparisons of task-specific models, LLM predictions and calibrators are from the same datasets, and studies (Brown et al., 2025) have shown that gradient-boosted tree models for tasks are better than LLMs (different purpose than presented our work).

AuthorsOpenReview ↗

Follow-up: within-class analysis and revised PDF

Dear Reviewer,

Thank you again for your detailed review. We have posted our responses and uploaded a revised PDF, with changes marked in blue and summarized with section pointers in our general comment. Following your suggestion, we added a within-class self-knowledge analysis (Appendix F.4): within each predicted class, confidence ranks correct above incorrect predictions on 8 of 9 tasks for both API models but this is not reliable as a single score. We also added new baselines, calibration and selective-risk metrics, and confidence intervals over support draws (Sec. 5.4). We would be grateful to know whether these changes address your concerns, and we are happy to clarify anything before the discussion period ends.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of revisions: corrected protocol (results hold), new baselines, calibration metrics

We thank the reviewers for careful and constructive reviews. Several concerns were correct, and we have rerun the core experiments to address them. We summarize the main changes here and answer each reviewer individually below.

Our work is to study whether LLMs’ verbalized confidence identifies its own errors in tabular predictions. We show that the standard classifier AUROC can be high while confidence carries little information about correctness. We benchmark this claim and empirically show how classifier AUROC and self-knowledge AUROC can diverge, along with their gaps for these types of tasks. We propose an external calibrator, which is a correctness estimator, that predicts LLM correctness from the model’s outputs and from the disagreement between the features that the LLM says drove its predictions and a data-derived reference ranking (which could be replaced with any gold standard, e.g., expert-specified reference). A calibrator built on such a method is scalable and can be used for API models as well. The reviews led us to correct the adaptation protocol, add new output-only and logit-only baselines, report calibration and selective-risk metrics. We summarize the changes below.

Corrected adaptation protocol: Reviewer 3 correctly identified that our code estimated the mutual-information (MI) reference on the full target cohort and selected support sets stratified by using prediction correctness. We reran the full pipeline with (i) MI reference estimated only from the kk labeled support set, (ii) supports drawn at random, and (iii) evaluation on disjoint sets. Results are essentially unchanged: at k=200k = 200 (Gemini, zero-shot), the calibrator improves over verbalized confidence by +0.351 AUROC on AKI (previously +0.363) and +0.304 on PE (previously +0.295). We also state explicitly that in the k=0k = 0 (LOTO) settings, the calibrator is trained for multiple tasks and is assessed for its transfer on the new task, in our experiments the reference is computed from the full set but one can replace it with a known gold standard.

Baselines: Under identical supports and evaluation instances, we compare the full calibrator against an output-only XGBoost trained on the prediction, verbalized confidence and P(true). The full calibrator reaches mean AUROC 0.758 vs. 0.668 for Gemini and 0.779 vs. 0.702 for GPT-5.5. It clearly exceeds the output-only model where verbalized confidence is weak, by +0.09 to +0.25 on AKI, IHM and hospitalization for both models, and on PE for Gemini (+0.237). Where confidence is already informative (GPT-5.5 on the ED tasks), the two are within 0.02. The output-only model barely improves on raw confidence (GPT-5.5: 0.702 vs. 0.700).

Calibration and selective-risk metrics: At k=200k = 200, the calibrator reduces mean ECE from 0.126 to 0.046 (Gemini) and from 0.095 to 0.040 (GPT-5.5), and mean Brier score from 0.201 to 0.160 and from 0.193 to 0.155. For selective prediction, mean AURC falls from 0.187 to 0.142 (Gemini) and from 0.186 to 0.148 (GPT-5.5). On PE and AKI, deferring the 20% least-trusted predictions by raw confidence leaves a higher error rate than deferring at random (38.5% vs. 36.7% on PE); the calibrator reduces it to 25.9%.

Revised claims: We have replaced the claim that verbalized confidence is “fundamentally uninformative.” Following Reviewer 1’s suggestion, AUROC within each predicted class shows that confidence carries signal for Gemini and GPT-5.5 but is anchored by predicted class, which makes it unreliable as a single score. We now describe it as a weak and inconsistent correctness signal, from uninformative (Qwen) to moderately informative but unreliable (Gemini, GPT-5.5). Further, we agree that failure-prediction AUROC is a standard metric (Kadavath et al., 2022; Xiong et al., 2024) and we did not claim the metric as a contribution. Our contribution is showing, for clinical tabular prediction, how outcome AUROC can mask uninformative confidence, and an external calibrator that shows disagreement signals add correctness information beyond the model’s own outputs.

We will upload a revised PDF with these changes and post a comment here when it is available.

AuthorsOpenReview ↗

Revised PDF uploaded: summary of changes

We thank the reviewers again for their thoughtful comments. We have uploaded a revised PDF, the new/corrected content marked in blue.

Here is the summary of the changes:

  1. We corrected the few-shot protocol. The support set is now drawn at random and the mutual information is re-estimated from this support set SS and evaluation uses disjoint set of the target set. At k=200, we report means over 5 support draws with 95% bootstrap intervals, smaller k use one draw. LOTO uses a reference from labeled target data and is not label-free. All revised numbers use 5 draws, some values in our initial responses came from a single draw and are superseded by those below. Conclusions are unchanged. (Sec. 3.4, 6; Fig. 2; App. G, Table 13)

  2. We also added the new baseline of calibrator training only on output-only, additionally we added logits where there are available under the same protocol and label budget. This model barely improves on verbalized confidence (mean AUROC 0.653 vs. 0.659 for Gemini; 0.702 vs. 0.699 for GPT-5.5). The full calibrator reaches 0.752, 0.780 and 0.735 and exceeds this baseline on 8, 9 and 8 of 9 tasks (Gemini, GPT-5.5, Qwen2.5-7B). For Qwen2.5-7B, log-probabilities alone reach 0.662 vs. 0.500 for its confidence; adding attribution features gives 0.735. Neither feature group suffices alone (Gemini: outputs 0.653, attribution 0.638, combined 0.752) (Sec. 5.4, Table 3; App. J.1, Tables 26–27)

  3. Following the suggestions from the reviews, we added calibration and selective prediction metrics. For Gemini (zero-shot), ECE falls from 0.126 to 0.046 and AURC from 0.187 to 0.142. On PE and AKI, deferring by verbalized confidence is worse than random deferral (38.5% vs. 36.7%, 39.2% vs. 37.6% error). The full calibrator reduces it to 25.9% and 25.6%. The calibrator has the lowest Brier, ECE and AURC for every model and prompting condition. (Sec. 5.4, Table 4; App. I.3, Table 25)

  4. Following Reviewer 1, we report within-class self-knowledge: within a predicted class, confidence ranks correct above incorrect predictions on 8 of 9 tasks for both API models, but the two classes receive different confidence levels and pooling loses this signal. The more accurate class differs across tasks, which explains why the output-only baseline transfers poorly. (Sec. 5.2; App. F.4, Tables 11–12)

  5. The full calibrator significantly improves over verbalized confidence on 5 of 9 tasks for Gemini (zero-shot), 8 for GPT-5.5 and 9 for Qwen2.5-7B, and is within 0.01 where confidence is already informative. "Improves on all nine tasks" is replaced by these counts. Per-model results are now tables instead of figures; chain-of-thought adaptation uses the original protocol, as labeled and will be updated in the final revision (Sec. 5.4; App. I.2, Tables 19–24)

  6. As a consequence of using a corrected protocol, we also updated our results for source-task pretraining vs. training on the same k target labels only, pretraining adds +0.127, +0.207 and +0.104 mean AUROC at k=10 and +0.033, +0.035 and +0.040 at k=200 (zero-shot); with 50 labels it matches target-only training with 200 for all three models. (Sec. 5.4; App. J.2, Tables 28–30)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.