acceptodds
Under review as a conference paper at ICLR 2027

How Far Can Reliability Drift Without Changing the Confidence Distribution?

Abstract

A classifier's confidence distribution can stay fixed while its conditional accuracy changes. We ask how far the reliability relation can move under covariate shifts that preserve the distribution of the confidence score, with a budget on the reweighting inside each confidence level; the worst-case movement as a function of the budget is a fragility profile. On an interval of budgets computable from the source, the profile is exactly the square root of the budget times the within-level variance of the correctness propensity, the grouping-loss term of calibration–refinement decompositions. Beyond that interval the profile depends on the tails of the propensity law, and the whole upward profile identifies the centred within-level law; calibration residual and grouping variance therefore do not determine fragility in general, although they do when labels and predictions are deterministic. Because the propensity is unobserved, we restrict reweightings to a learned finite readout within confidence bins, bound what the restriction misses by the grouping variance left inside readout cells, estimate the restricted profile with role-separated labels, and give a separate split-sample lower confidence bound. On ImageNet the bound is positive in both splits for four of six primary classifiers and nine of twelve additional ones as released, and for three of eighteen after temperature scaling. Held-out drift under optimised reweightings, fitted without evaluation labels, tracks the estimated profile; an exploratory label-permutation diagnostic gives near-zero agreement for this statistic but largely reproduces the correlation seen for unsigned random reweightings. On natural shifts, the change in within-bin composition, which the source profile bounds, is smaller than the within-cell change, which it does not control.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Trustworthy, reliability, calibration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RUioOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper measures the gap of reliability that can arise under covariate shift on a level-confidence setting. The paper defines "fragility profile", which identifies the propensity distribution. Under low levels of confidence-preserving covariate shifts, variance itself is sufficient, while in high-level shifts it is insufficient. This is derived theoretically (Sec.3) and evaluated experimentally (Sec 5. What the full~). To make the framework applicable in practice, they propose a confidence-binned and readout approaches.(Sec.4) Experiments on ImageNet classifiers evaluate the estimated frgility profiles, compare them with variance-based methods, and examine relevance and limitations.

Critical strengths
  • The introduction of the fragility provides a fresh perspective. This characterization on analyzin reliability is an interesting approach.
  • The paper provides extensive experimental evaluations to support its various theoretical and methodological claims.
Critical weaknesses
  • This paper has MAJOR writing issues. The explanation of the main idea is not only insufficient, but leaves the core components and implications highly ambiguous, and jargons used without proper definition. Upon reviewing, I had to go over the written statement repeatedly to at least guess what the authors were trying to argue. The sentences are overly abbreviated, and almost no intuitive explanation or motivation for the possible design choices. To list a few, (Line number),

    • (47) What is budget? Why is it this treated as a “budget”? Why can the \rho value be seen as a budget? Why is worst-case reliability drift a function of a budget that is \chi-square supremum difference?
    • (53) Why is stop-loss transform applied?
    • (48-50) What is P? What is \rho_+ ? why is this interval used, and why is it not defined?
    • (124) What is Q?
    • (235) What is readout? what is finite readout? what is nested readout? what is learned readout?
    • What is R_1 and R_2? What is \sigma? Is it \sigma-algebra or standard deviation?
    • What is cell? Why is cell needed needed? What is readout cell? What is \pi?
    • (244) What is binned "oracle”?
    • (248) : “The proof is conditional Jensen and nested feasible sets.” Is this a logical English sentence?
    • (264) What is "outer”? Is there "inner”?
    • (264) What value does the method "plug-in”? These questions arise while reading through the paper, but no answers can be found, I only had to guess through.
  • Despite the extensive experimental evaluation, the presentation lacks a clear focus on the paper's main contributions. Numerous experimental results are presented with similar emphasis, making it difficult to distinguish the key findings from supplementary analyses. Most of the experiments including tables and figures are placed in appendix, which hinders the reader to grasp the essential concepts. Only an abbreviated version is contained in the results section, most of the tables and figures are placed outside the main paper.

Questions for authors
  • Could the authors clarify which theoretical and empircal findings they consider the central contributions of the paper? Also, could authors clarify how the main experiments support the contributions?
Reviewer_TH7rOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper quantifies how classifier reliability can change under covariate shifts that preserve the confidence-score distribution, making them invisible to confidence-only monitoring. Under a Pearson (\chi^2) divergence budget within each confidence level, the authors characterize the largest possible reliability change—the fragility profile. For small budgets, this follows an exact square-root law determined by the variance of correctness propensities among inputs sharing a score. At larger budgets, the full propensity distribution matters.

For practical estimation, the authors bin confidence scores and partition inputs into learned groups, restricting shifts to group reweightings that preserve each confidence bin’s mass. They propose a plug-in estimator of the resulting restricted fragility profile and a separate lower confidence bound on population restricted fragility; the plug-in estimate itself provides no confidence-bound guarantee.

Experiments evaluate ImageNet classifiers under constructed group reweightings and natural distribution shifts, including ImageNet-C, ImageNetV2, and ImageNet-Sketch.

Critical strengths
  • The paper highlights an important limitation of confidence-based monitoring: reliability can deteriorate under distribution shift even when the confidence distribution remains unchanged.
  • Using an auxiliary correctness predictor as a “readout” offers an interesting way to group inputs and estimate restricted fragility, and the resulting estimator seems to show some correlation to observed fragility under natural distribution shifts.
  • The theoretical analysis explains the source of fragility and quantifies how the potential reliability change depends on the strength of the distribution shift.
Critical weaknesses
  • The presentation is difficult to follow. Dense terminology and notation, sometimes used before being clearly defined, obscure the significance of the results. More intuitive explanations of the main theoretical and experimental takeaways would help readers understand the paper’s contributions.
  • The practical utility of the theory is not convincingly established. The original exact-score fragility profile depends on unobserved correctness propensities, while the practical formulation introduces two effects that can act in opposite directions: coarse input groups hide within-group fragility, whereas coarse confidence bins permit redistribution across scores that the exact-score formulation excludes. This complicates the interpretation of the estimated quantity. Moreover, although estimated FAUC correlates with the composition component of natural drift on ImageNet-C, the experiments do not establish that it predicts overall reliability changes under natural distribution shifts, and results across benchmarks are mixed.
  • The plug-in estimation procedure is insufficiently clear in the main text, particularly how the confidence bins and input groups are learned and how the different data splits are used. An explicit estimator definition and a concise algorithm describing the full procedure would improve clarity and reproducibility.
Questions for authors

What do you see as the key contribution of this theoretical analysis? What takeaways are most relevant to users of ML classifiers, or ML researchers interested in designing more robust classifiers?

Reviewer_yGi3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper considers a particular assumption of score-preserving covariate shifts (covariate shift + Q_S = P_S, i.e., EP[r(X)∣S=s]=1\mathbb{E}_P[r(X) \mid S=s] = 1). Under this assumption, a fixed classifier’s reliability μ(s)=E[C∣S=s]\mu(s) = \mathbb{E}[C \mid S=s] can degrade subject to a conditional χ2\chi^2 budget. By exploiting the gap between the score algebra σ(S) and input algebra σ(X), the authors derive a closed-form bound (Theorem 1) and a split-sample lower confidence bound (LCB).

Critical strengths

The paper is theoretically dense and seems solid.

Critical weaknesses

My main concern is the major assumption of score-preserving covariate shift seems artificially fabricated. Natural shifts toward harder inputs usually push the marginal score distribution S(X) downward (model becoming less confident), leading to QS≠PSQ_S \neq P_S. I feel this assumption may not be practically useful in real world. E.g., in ImageNetV2 and ImageNet-Sketch results, profile-bounded composition term δcomp\delta_{\text{comp}} accounts for only 11.1% and 9.6% of total drift, and most of the degregation comes from within-cell accuracy collapse δcond\delta_{\text{cond}}, showing source fragility profile offers limited practical warning value.

Also regarding using S(X) distribution shift as a probe to detect shifts: usually there is a evaluation set on the target domain, and the calibration curve can be used together with S(X) distribution providing a more complete view of the joint distribution.

Questions for authors

The weakness is my main concern. Can the authors provide real world examples of this attack, and how a combined profile of S(X) distribution and calibration curve fails to defend?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.