acceptodds
Under review as a conference paper at ICLR 2027

When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning

Abstract

When a model knows when it does not know, many possibilities emerge. The first question is how to enable a model to recognize that it does not know. A promising approach is to use confidence, computed from the model’s internal signals, to re- flect its ignorance. Prior work in specific domains has shown that calibration can provide reliable confidence estimates. In this work, we propose a simple, effective, and universal training-free method that applies to both vision and language mod- els, performing model calibration, cascading, and data cleaning to better exploit a model’s ability to recognize when it does not know. We first highlight two key empirical observations: higher confidence corresponds to higher accuracy within a single model, and models calibrated on the validation set remain calibrated on a held-out test set. These findings empirically establish the reliability and compara- bility of calibrated confidence. Building on this, we introduce two applications: 1. Model cascading with calibrated advantage routing and 2. Data cleaning based on model ensemble. Using the routing signal derived from the comparability of cali- brated confidences, we cascade large and small models to improve efficiency with almost no compromise in accuracy, and we further cascade two models of com- parable scale to achieve performance beyond either model alone (e.g., combining two models with around 82% accuracy yields over 90% accuracy). Leveraging multiple experts and their calibrated confidences, we design a simple yet effec- tive data-cleaning method that balances precision and detection rate to identify mislabeled samples in ImageNet and Massive Multitask Language Understand- ing (MMLU) datasets. Our results demonstrate that enabling models to recognize when they do not know is a practical step toward more efficient, reliable, and trustworthy AI.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Model Calibration, Model Cascades, Data Cleaning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uLkJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper calibrates pretrained vision and language models. Its router uses the models' average confidence difference within bins to decide when to call the second model. Its cleaning rule flags labels rejected by several models whose predictions fall in their highest confidence bins. Experiments evaluate routing and label detection on vision and language benchmarks.

Critical strengths
  1. Calibrated confidence is comparable across models, so the difference between two of them serves as a routing signal for cascading and cleaning with almost no extra training.
  2. The cascading result goes beyond common practice. The usual setting has a large model cover a small one, and here two models of comparable scale are cascaded and beat either one alone. Cleaning reports precision and detection rate together on ImageNet and MMLU.
Critical weaknesses
  1. The paper sets out to show that calibration improves routing. Algorithm 1, however, decides from the average confidence difference between the two models inside a bin, and the validation set that builds the bins carries labels, so the accuracy difference inside the same bin can be computed without any calibration and could provide an alternative routing rule. The two rules are never compared, so the experiments cannot say how much of the gain comes from calibration.
  2. The cleaning experiments do not isolate what calibration contributes. Algorithm 2's high-confidence disagreement rule selects samples using each model's top K confidence bins. If calibration preserves cross-sample confidence rankings and predictions, the equal-count bins and flagged set remain unchanged under consistent tie handling. However, this invariance is not guaranteed for multiclass temperature scaling. Figure 4 varies the detection rate without a matched before-and-after calibration comparison, so the independent contribution of calibration remains unclear.
Questions for authors
  1. Inside the bins Algorithm 1 uses, how does the accuracy difference between the two models compare with the confidence difference near the threshold? Would routing change where the two disagree?
  2. With models, bin count and K fixed, how many flagged samples change after calibration? Is precision at the same detection rate different?
  3. Does the routing experiment report the fraction of inputs sent to the large model before and after calibration, and if that fraction barely moves, where does the accuracy gain come from?
Reviewer_PkXWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether calibrated confidence can serve as a routing signal for model cascading and a filtering signal for data cleaning. The authors use post-hoc calibration to bin samples by the first model's confidence, retaining the first model on bins where the second model's calibrated confidence advantage is smallest, and routing to the second model otherwise. For cleaning, samples where multiple models confidently and unanimously disagree with the label are flagged. The results show that cascading a small and large model can balance efficiency and accuracy, and that cascading two large models can slightly outperform either alone.

Critical strengths
  1. The method is simple and broadly applicable. It needs only cached logits and a small labeled split, and works the same way for vision and language tasks.
  2. The experimental coverage is broad. The paper evaluates five language benchmarks (MMLU, ARC, GSM8K, MBPP, BigCodeBench) and multiple ImageNet variants (ImageNet-C, ImageNet-R, ImageNet-Sketch, VQA) in Fig 3 and Appendices A and G. Ablations cover the calibrator, bin count and scoring method in Tables 4, 5, 15.
  3. The efficiency result for large–large cascades is useful. Table 11 shows that the cascade matches a plain ensemble at about 53% of two-model compute, using the proxy in Eq. 6.
Critical weaknesses
  1. The Fig 2b result needs verification. Two models with accuracies around 82% reach a peak of ≈0.907 when cascaded. If routing is decided per bin of the first model's confidence, accuracy is bounded by ∑iwimax⁡(vˉ1,i,vˉ2,i)≤P(at least one correct)\sum_i w_i\max(\bar v_{1,i},\bar v_{2,i})\le P(\text{at least one correct}), so a ~9 pp gain requires nearly complementary errors within bins, which is unusual for ImageNet models trained on the same data. The same pair gains only ~0.7 pp on ImageNet-Sketch (Table 9), the x-axis is labeled "model(eva02) ratio" although the pair is DeiT-B/TinyViT, and lines 390–394 even repeat a duplicate sentence. Please report the exact checkpoints, each model's accuracy, the at-least-one-correct rate, and the per-bin upper bound.
  2. The simple baselines are missing. There is no comparison against a calibrated-confidence threshold on the small model, which is the natural baseline when the large model dominates. The calibration split already has labels, yet the method ranks bins by aica^c_i (the approximation) rather than the empirical per-bin accuracy advantage aiva^v_i. The RouteLLM-MF baseline falls below random on BigCodeBench (APGR 0.4320 vs. 0.5; Table 7). Please add threshold deferral and aiva^v_i-ranked routing to Fig 3. Both need only cached outputs.
  3. Operating points are chosen on the test curve. The "best performance" points (stars in Figs 2/3) look like test-split maxima. Appendix J (l.1296–1298) says operating points are chosen on calibration data, but the paper does not report test accuracy when KK is chosen on the calibration split. The SOTA claim compares a test-selected point on half of the val set against single models, yet a plain ensemble also reaches 90.05% (Table 11). Please select KK on the calibration split, report test accuracy, and revise the SOTA statements.
  4. The post-cleaning evaluation is circular. Removing samples that the models get wrong necessarily raises accuracy. On the matched cleaned subset, the single models score 97.43/97.47 and the cascade 97.80, a gap of only ~0.3 pp, not "substantially surpassing". App I.1 (l.1238–1240) states that cleaned-subset accuracy "cannot be compared directly ... as a state-of-the-art result". Please report matched single-model numbers in the main text and remove the saturation claim.
  5. The cleaning evaluation has several issues. Annotators see the model predictions, so annotation is not blind. The guideline defaults to "GT wrong" when uncertain. Multi-Object is treated as "GT correct" in B.1 but listed as an error type in App C. Table 3 labels the metric "Unique Accuracy (%)" but reports fractions (0.665/0.537). Confident Learning is evaluated as a single model while the proposed method uses a pair. Table 3's counts match the unthresholded candidate pool in App I.1, so calibration is not what beats CL. Please evaluate against ReaL labels [1] and the verified errors of [2]; give CL the same model pair; report CIs.
  6. The theoretical contribution is overstated. Eq. 3 is a definitional identity and Eq. 5 a triangle inequality, both in-sample. They say nothing about transfer from the calibration split to the test split, as the authors note on p.19 (l.1018–1022). Please recast them as diagnostics, or provide a finite-sample bound.
Questions for authors
  1. Which checkpoints are used in Fig 2b, and what are their individual accuracies, the at-least-one-correct rate, and the per-bin upper bound? Why does the x-axis say "eva02" when the pair is DeiT-B/TinyViT?
  2. How do threshold deferral and aiva^v_i-ranked routing (both computed on the calibration split) compare with the proposed method in APGR on every dataset in Fig 3?
  3. On which split are the starred "best performance" points in Figs 2/3 chosen? If KK is selected on the calibration split, what is the test accuracy?
  4. In Table 3, what is the exact configuration of Confident Learning (single model or the same pair), why is the metric named "Unique Accuracy (%)" with fractional values, and what are the precision/recall against ReaL labels and [2]?

[1] Are we done with ImageNet?, arXiv 2006.07159, 2020.

[2] Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks, arXiv 2103.14749, 2021.

Reviewer_d9RPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies confidence calibration across vision and language models for model cascading and data cleaning. It introduces a metric and theoretical bound to assess cross-model confidence comparability. The authors then use calibrated confidence for model routing and label-error detection. Experiments on vision and language benchmarks evaluate the effectiveness of the proposed methods.

Critical strengths
  1. The proposed method is simple and requires minimal additional training, making it practical and computationally efficient.
  2. The method is evaluated on vision and language tasks, demonstrating its potential applicability across different domains.
Critical weaknesses
  1. The paper lacks a Related Work section, and the novelty between the proposed approach and existing methods are not sufficiently discussed. Equations are not numbered, making them difficult to reference and discuss, and the reference formatting does not fully comply with the template requirements.
  2. The advantages of the proposed routing strategy over existing methods are not entirely clear from the experimental comparisons.
Questions for authors
  1. The lack of a systematic discussion of related work makes it difficult to understand the novelty of the proposed method.
  2. Although the authors compare routing performance with and without calibration in Section 3.2, the benefits of the proposed cross-model Confidence Advantage could be further demonstrated through comparisons with more recent or established routing methods, including simpler confidence-based strategies.
  3. In Section 3.4, would it be possible to further clarify the individual contributions of model ensembling and confidence calibration to the data-cleaning performance? An additional ablation study might help better understand their respective effects.

Comments on the paper

3 comments

Thanks for the new experiments. They do answer some of my questions, but the answers make me more cautious about the main claim. On routing, calibration seems to matter mainly when labeled data is limited. On cleaning, raw and calibrated confidence give very similar results. So the role of calibration is smaller than the paper presents, and I think fixing this needs more than adding tables. I will keep my score for now.

Thank you for the additional experiments, which address most of my concerns. However, confidence-threshold routing performs similarly to the proposed method on most two-model tasks, so the proposed routing rule's significance remains limited. I will maintain my original score.

Thanks for the response and the additional experiments. The new baselines and cleaning ablations address several of my concerns. However, the gains are still not consistent or substantial enough to fully resolve my concerns about the contribution. Therefore, I will keep my current rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.