acceptodds
Under review as a conference paper at ICLR 2027

Image-Dependent Calibration Maps for Compound Segmentation Losses

Abstract

In many domains, a segmentation network has to provide more than a binary decision: a calibrated probability map. For example, in radiotherapy planning, treatment margins and dose can be defined based on the probability that a specific area (i.e., voxel in the image) belongs to a tumor. However, models trained with compound losses, such as cross-entropy plus soft Dice, encode different posterior probabilities into the same output. This shift is caused by the optimum forced by the compound loss. By balancing the opposing loss gradients at a stationary point, we derive a calibration map in closed form, parameterized by two properties of the image, the foreground fraction and the attained Dice . We describe this map for the example of the cross-entropy plus soft Dice compound loss, and show that no beta, Platt, or temperature map can represent , and no map fitted once per model can recover the true posterior of multiple images. In addition to , we propose a foreground-conditioned version of isotonic regression that has sufficient capacity to model and improves on standard isotonic regression. Without any held-out labels or retraining, correcting with alone consistently reduces NLL and ECE scores compared to raw outputs. Furthermore, applying a beta or isotonic map fitted on held-out data in conjunction with is more effective than the respective fitted map alone. Both and the foreground-conditioned isotonic regression improve the placement of uncertainty margins across three tumor segmentation tasks, measured by the sample-wise IoU of the 0.25–0.75 probability margin.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
calibration, segmentation, tumor modeling, post-hoc calibration, class imbalance

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_9C7GOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces calibration maps for image segmentation losses

Critical strengths

Th topic of calibration maps is of great interest, going beyond simple classification in image segmentation tasks.

Critical weaknesses

The paper does not cover the main related work. This includes recently published papers on calibration for medical segmentation, such as: Axel-Jan Rousseau, Thijs Becker, Simon Appeltans, Matthew Blaschko, and Dirk Valkenborg. Post hoc calibration of medical segmentation models. Discover Applied Sciences, 7(3):180, 2025. Wenhao Liang, Wei Zhang, Lin Yue, Miao Xu, Olaf Maennel, and Weitong Chen. We care each pixel: Calibrating medical segmentation models. In Proceedings of the 28th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2025. Mijoo Kim and Junseok Kwon. Uncertainty calibration with energy based instance-wise scaling in the wild dataset. In Eur. Conf. Comput. Vis. (ECCV), 2024. Dongdong Wang, Boqing Gong, and Liqiang Wang. On calibrating semantic segmentation models: Analyses and an algorithm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

The paper is written in a way that is difficult to read and understand. I do not know if this is due to used generative AI. For example, consider the first definitions. It is not clear the decomposition of the binary loss ℓ(q) into its two branches. Is it the conventional one of the form y ℓ_1(q)+(1−y)ℓ_0(q)? Or yℓ_0(q)+(1−y)ℓ_1(q)? Or another form? And then (1) presents directly the gradients, which are not given from the main definition. It is also not clear what means that the gradient step on a voxel with label 1 raises q in proportion to g_1, …

The propositions are also not clearly presented or described. For example, there is no proof for Proposition 1, even in Appendix K. To understand it, one needs to check Reid and Williamson’s paper on composite binary losses and connect it to the so-called inverse link.

At the end, one has to read each section several times in order to understand some supposedly simple notions. For example, the concept of pull: pull up, pull down, region pulls, total pull…

============================

Rebuttal.

We thank the authors for their feedback and the modifications carried out to address the raised issues. For this reason, I have increased my score from Weak rejection to Weak acceptance.

Questions for authors

none

1 comment

We thank Reviewer 9C7G for the review of our work. Of the four suggested related papers, one was already cited, and we now cite the other three. We revised the parts you found difficult to follow: the loss decomposition is now stated explicitly, a glossary of notation and terms opens the supplement, and the proof of Proposition 1 is a labeled paragraph in Sup. K. We would be glad to hear whether these changes improved the readability.

  • A:W1-R:9C7G

Thank you for the suggested additions to our related work. We address each suggested paper below:

  • We already cite Rousseau et al. (2025) in our related work (Sec. 2, second paragraph), where we summarize their finding that isotonic regression and beta calibration are effective, that temperature scaling is the weakest parametric method, and that a fitted map improves most images but not all.
  • Wang et al. (2023) propose a post hoc calibration method for semantic segmentation, which we now cite in Sec. 2. Their selective scaling is fitted on labeled held-out data, whereas our map is label-free and derived from the training loss.
  • The title "We care each pixel: Calibrating medical segmentation models" does not appear in the MICCAI 2025 proceedings. We found the arXiv preprint and a related version at CIKM 2025 (Liang et al., 2025), which calibrates during training through the loss, complementary to our post hoc map. We now cite the CIKM version in Sec. 2.
  • Kim and Kwon (2024) propose an instance-wise scaling for classification under distribution shift. We now cite it next to Ding et al. as a per-instance scaling, but it treats neither segmentation nor the miscalibration that a compound training loss induces.

 

  • A:W2-R:9C7G

Yes, it is the conventional decomposition ℓ(q,y)=yℓ1(q)+(1−y)ℓ0(q)\ell(q, y) = y\ell_1(q) + (1-y)\ell_0(q), which we now state explicitly in Sec. 3.1. Eq. (1) defines the two pulls as the derivatives of these branches, signed by the direction in which a gradient step moves qq: g1=−ℓ1′g_1 = -\ell_1' and g0=ℓ0′g_0 = \ell_0'. A gradient step of size α\alpha changes qq by −αℓ1′(q)=αg1(q)-\alpha\ell_1'(q) = \alpha g_1(q) on a voxel labeled 11 and by −αℓ0′(q)=−αg0(q)-\alpha\ell_0'(q) = -\alpha g_0(q) on a voxel labeled 00. This is what "raises qq in proportion to g1g_1" refers to.

We added an overview of the notation and technical terms at the start of the supplement, with a pointer at the start of Sec. 3. It describes each term and references where it is introduced. As the page limit leaves no room for a longer introduction in the main text, this overview serves as a reference wherever a term or symbol is not immediately clear. Moreover, we unified the notation and terminology throughout the paper.

  • A:W3-R:9C7G

Proposition 1 follows from setting the derivative of the risk at a voxel to zero at its minimum, and its proof does not require information from Reid and Williamson (2010). By Eq. 1, R′(q)=ηℓ1′(q)+(1−η)ℓ0′(q)=−ηg1(q)+(1−η)g0(q)R'(q) = \eta\ell_1'(q) + (1-\eta)\ell_0'(q) = -\eta g_1(q) + (1-\eta)g_0(q). At an interior minimum q∗q^\ast, R′(q∗)=0R'(q^\ast) = 0, which is Eq. 2. Rearranged, Eq. 2 reads η(g1(q∗)+g0(q∗))=g0(q∗)\eta(g_1(q^\ast) + g_0(q^\ast)) = g_0(q^\ast), and dividing by the positive sum of the two pulls gives Eq. 3. The submitted Sup. K presented this proof in a single unlabeled sentence at the start of the section. We now give it as a labeled paragraph and point to it from Proposition 1.

  • A:W4-R:9C7G

We reduced the vocabulary to three terms: the pull up g1g_1 and the pull down g0g_0 (Eq. 1), and the region pulls aa and bb that a region term adds to them (Eqs. 4 and 5). The term "total pull" is no longer used, and all three terms are described in the glossary.

Reviewer_V7dWOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates loss-induced miscalibration in medical image segmentation models trained using compound objectives, particularly CE combined with soft Dice loss. The authors analyze that, unlike CE alone, the compound objective does not necessarily correspond to the true posterior probability. They derive a closed-form calibration map to correct this gap. Since these quantities are unavailable at inference time, they propose estimating them directly from the predicted probability maps through a fixed-point formulation. Additionally, they introduce foreground-conditioned variants of conventional calibration methods, including IR and beta calibration. Experiments using nnU-Net across five medical segmentation tasks demonstrate improvements in ECE, NLL and uncertainty-margin localization.

Critical strengths

-- The paper provides a useful perspective on why compound segmentation losses can distort probability estimates even when the optimization objective is satisfied.

-- The proposed calibration map estimation is applied in post-hoc without requiring an additional labeled calibration dataset. This is practical especially in medical segmentation applications where annotated calibration data may be limited.

-- In addition to ECE and NLL, the paper investigates uncertainty-margin localization. This is useful because improved calibration metrics do not necessarily translate into better downstream segmentation decisions.

Critical weaknesses

-- The theoretical formulation is intuitive, but the central posterior-recovery map in Proposition 3 relies on approximating the conditional expected Dice gradients using image-level statistics (Eq. 5). Since the Dice gradient is nonlinear in the random mask-level quantities, this replacement is not generally exact. Although Section 3.4 discusses the underlying assumptions, the paper does not sufficiently characterize the approximation error or establish when the resulting map closely recovers the true posterior. A quantitative analysis, particularly for small foreground regions or variable mask volumes, would strengthen the theoretical justification.

-- A central component of the proposed approach is the estimation of f and D through the coupled fixed-point equations in Eq. (8). However, the paper provides limited analysis of the convergence, and stability of these estimates especially with the noisy parameters induced during model training. Since both quantities are inferred from the model’s own potentially miscalibrated predictions, estimation errors may directly affect the resulting calibration map.

-- This is also highlighted from the experiments itself. Although the proposed calibration map depends jointly on image-specific foreground fraction and Dice, the results in Table 1 show that estimating D does not consistently improve calibration. For example, on Spleen full-resolution, replacing fixed D* with estimated D_eff increases ECE from 0.035 to 0.047 and NLL from 0.172 to 0.194. Similar degradation is observed for Pancreas full-res. These results suggest that image-dependent foreground estimation may account for a substantial portion of the improvement, while the benefit of estimating D remains configuration-dependent. Further analysis of Dice-estimation errors would help explain these differences.

-- In this view, it is also key to analyze the generalization of models to different architectures or is it very specific to nnUnet v2 only. All experiments rely on nnU-Net v2 trained using the same CE + soft-Dice formulation. Since the proposed calibration map is derived from the training objective, its robutness to different CE–Dice weightings, Dice implementations and segmentation architectures remain unclear. Experiments varying the relative loss weights and including additional segmentation backbones would provide stronger evidence for the generality of the proposed approach.

-- The uncertainty-margin experiments in Section 6.3 provide an interesting application of the proposed calibration maps. However, the reference posterior used to construct the target uncertainty margins is derived from segmentation annotations rather than directly observed conditional probabilities. Consequently, improvements in margin IoU demonstrate better agreement with the constructed reference but do not necessarily establish recovery of the true posterior uncertainty. An analysis of sensitivity to the reference construction, or evaluation using multiple independent annotations, would strengthen the interpretation of these results.

-- Additionally, the analytical calibration map in Eq. (6) depends on both the foreground fraction f and Dice statistic D, while the proposed image-conditioned beta and isotonic calibrators in Section 5 use only f. Although conditioning on foreground fraction is a reasonable practical choice, the paper does not establish whether f alone captures the relevant image-dependent variation. An ablation comparing conditioning on f, estimated D, and their combination would clarify whether the Dice statistic provides complementary calibration information.

Questions for authors

In view of the Weakness mentioned above, the following concerns should be addressed:

-- The authors should provide a quantitative analysis of the approximation error between the exact conditional expected Dice gradients and their image-level approximations? How does this approximation affect posterior recovery when foreground regions are small or mask volumes vary substantially?

-- Regarding the fixed-point estimation in Eq. (8), are there any theoretical or empirical guarantees concerning the convergence of the estimated (f) and (D)? How sensitive is the resulting calibration map to estimation errors, initialization, and noisy model predictions?

-- Table 1 shows that estimating D_eff improves calibration in some configurations but degrades it in others, such as Spleen and Pancreas full-resolution. Can the authors explain these differences and investigate whether they arise from inaccuracies in estimating (D)?

-- How does the proposed approach perform with different CE–Dice loss weightings, alternative Dice formulations, or segmentation architectures beyond nnU-Net v2? Is the derived calibration map applicable to these settings without modification?

-- Regarding uncertainty-margin evaluation: How sensitive are the results in Section 6.3 to o construct the reference posterior from segmentation annotations? Could the authors provide additional validation using multiple independent annotations?

-- Since Eq. (6) depends on both (f) and (D), what motivates conditioning the beta and isotonic calibrators only on (f)? Would conditioning on estimated (D), or jointly on (f) and (D), provide additional improvements?

4 comments

We thank Reviewer V7dW for the very thorough review, which raised several important questions. We address all of them below, from the theory of the fixed-point estimation to the uncertainty margins derived from independent annotators. We hope our answers resolve your questions, and we are happy to follow up on anything that remains open.

  • A:W1/Q1-R:V7dW

TLDR: We test the approximation on real annotations and quantify it on synthetic data. On KiTS, a model trained on the mask of a randomly drawn annotator for every sample (median volume deviation between annotators 3.2%) has its ECE and cross-entropy at least halved by the map, like the models trained on the majority vote and on soft labels; for the latter, Eq. 5 holds exactly (Sup. N). With a known voxel-wise posterior, we compare our mean-sum approximation, the Dice gradient against the soft target, with the exact expected Dice gradient, averaged over masks whose contour is dilated or contracted. The synthetic experiments show that the map still removes 80-85% of the raw error when the volume's standard deviation is 50% of its mean (Sup. E).

We use the three different annotations provided in the KiTS dataset for 333 of its 560 tumors to quantify cross-annotator volume variation in a real dataset. The KiTS annotators' tumor volumes vary by a median of 3.2% (90th percentile 7.4%; Sup. N). We use these masks to train a network by randomly sampling the ground-truth contour for samples that contain multiple annotations, a scenario where the mean-sum approximation is not exact. In the evaluation, we see that ECE and cross-entropy are still at least halved for this model with our calibration map, similar to the relative improvements observed for a soft-target-trained model, where the approximation is exact (see Sup. N, Table 13).

Second, we designed a synthetic experiment to quantify our approximation error. For a known posterior, we sampled masks consistent with it by dilating and contracting the sphere's contour, with volume standard deviations of up to 77%. On these masks, we trained a network and, separately, optimized each voxel's logit directly, without a network, which reaches the stationary point (Sup. E). We quantify the approximation error as the relative difference between the pulls of Eq. 5 and the exact expected pulls of Prop. 2, averaged over 4,096 sampled masks, and as the mean absolute difference of the calibrated output to the known posterior. When the whole contour shifts together, the pulls differ by 12% at a standard deviation in volume of 48%. The posterior falls from 0.95 to 0.05 across a band around the sphere's surface of about 3.3σ3.3\sigma, where σ\sigma, the standard deviation of the contour's position, is 1 or 1.6 voxels. We denote the relative standard deviation of the mask volume by ρ\rho. At the stationary point, the map's error (mean absolute difference to the known posterior) is 0.001-0.002 with independent voxel labels (ρ≤0.02\rho \le 0.02), against 0.04 for the raw output on the largest object (20% of the volume, σ=1\sigma = 1) and 0.18 on the smallest (1% of the volume, σ=1.6\sigma = 1.6). On the smallest object, where the same contour uncertainty is the largest share of the volume, the raw error grows to 0.133 at ρ=0.48\rho = 0.48 (σ=1\sigma = 1) and 0.199 at ρ=0.77\rho = 0.77 (σ=1.6\sigma = 1.6), and the map's error to 0.020 and 0.045, so the map still removes 85% and 77% of the error, against 99% for the map built from the exact pulls. On trained networks, the map removes 80% of the raw error at ρ=0.47\rho = 0.47 (0.121 to 0.025).

Eq. 18 (Sup. E) gives the difference between our approximation and the linear approximation of the exact pull. We now provide a reference to Eq. 18 in the main paper, describing that the error of our approximation grows with the spread of the volume.

  • A:W2/Q2-R:V7dW

TLDR: Eq. 8 has exactly one solution with f,D>0f, D > 0 (Prop. 8), and a bracketed solve is guaranteed to converge to it. Empirically, both of our solvers return the same pair wherever both converge, and this pair is close to the one measured from the labels.

We checked the convergence of the fixed-point estimation empirically and found that all but 6 of the 833 patches of Table 1 converged (using SciPy's fsolve, MINPACK's hybrid method started at (0.1,0.75)(0.1, 0.75) in the box f∈(0,0.99)f \in (0, 0.99), D∈(10−3,1)D \in (10^{-3}, 1)). On the 6 failed patches, the solution lay outside the box we set, or the solver stopped close to the root without meeting its convergence criterion. Theoretically, Eq. 8 has exactly one solution with f,D>0f, D > 0, and its ff lies in [qˉ/2,(qˉ+1)/2][\bar q/2, (\bar q+1)/2] (Prop. 8 in Sup. B, proof in Sup. K). We now state it in Sec. 4, together with the 6 patches on which the root-finder did not converge. During the revision period, we also implemented a bracketed solve: solving the second equation of Eq. 8 for its positive DD in closed form leaves one residual in ff, which is negative at qˉ/2\bar q/2 and positive at (qˉ+1)/2(\bar q+1)/2, so Brent's method on this bracket always converges, and by Prop. 8 to the unique solution. It returns the same pairs as before on all other patches and also finds the roots of the 6 patches. We will add this solver, as well as the initial one we used, to our code repository. Regarding estimation errors, on held-out labels the label-free ff is within 6% and DD within 0.06 of the pair measured from the labels in the median over patches, on nine of ten models (Sec. 4, Sup. B.2, Table 3). We also measured how sensitive the calibrated output is (new paragraph in Sup. B.2). First, changing ff by 10% or DD by 0.05 changes the NLL by at most 0.017. Second, solving the pair from outputs with logit noise of standard deviation 0.5 changes it by at most 0.001. Third, solving it from under- or overconfident outputs (logits scaled by 0.8 or 1.25) raises it by at most 0.049.

  • A:W3/Q3-R:V7dW

TLDR: On spleen and pancreas 3d_fullres, the DD measured from the labels calibrates no better than our estimate, so the gap to the constant D⋆D^\star does not come from estimation errors. On these two models, DD barely varies between patches.

We agree with your analysis that "image-dependent foreground estimation may account for a substantial portion of the improvement, while the benefit of estimating DD remains configuration-dependent." The more prominent role of ff makes sense, as empirically the between-patch variance for ff is much larger than the variance for DD (Sup. O). Consequently, models have stronger incentives to adapt to ff to minimize the risk, and the model's internal estimate of ff is probably easier to learn than estimating DD. Regarding inaccuracies in estimating DD, we tested how strongly the estimates deviate from the true (f,D)(f, D) derived from the labels (Sup. B.2, Table 3). The results did not indicate that estimation errors are causing the deviating behavior of Spleen and Pancreas at full resolution. On spleen 3d_fullres, the estimated DD is within 0.003 of the label-measured DD (median over patches), and using the label-measured DD gives the same ECE/NLL as the estimate (0.046/0.193 against 0.047/0.194), while the constant D⋆D^\star reaches 0.035/0.172 (Sup. B.2). Pancreas 3d_fullres behaves the same way (0.074/0.438 with the labels' DD, 0.072/0.435 estimated, 0.070/0.434 with D⋆D^\star). On these two models, DD hardly varies between patches: the middle 80% of the patches span a factor of at most 1.2 in DD. Our assumption is that these models do not learn to shift their outputs with DD, but this remains speculative.

  • A:W4/Q4-R:V7dW

TLDR: The maps derived for 4 further loss configurations, and the unchanged CE+Dice map on 3 further architectures, improve calibration as in the nnU-Net v2 CE+Dice experiments.

We followed your requests on this and ran additional experiments with different backbones (Primus-M and PrimusV3-S; Wald et al., TMLR 2026; MedNeXt-B; Roy et al., MICCAI 2023) and also with different loss configurations on KiTS (3d_lowres, fold 0, scored as Table 1):

  • CE + 0.5 soft Dice
  • CE + 2 soft Dice
  • Focal loss (γ=2\gamma = 2) + soft Jaccard (IoU)
  • CE + soft Dice with squared denominator (the V-Net form)

Our approach generalizes to these settings, again improving both calibration metrics compared to the raw output (significantly on six of seven) and label-derived uncertainty margins. The derived map reduces ECE by 44-77% across all seven models and NLL by 51-62% across six of them. On the model trained with the squared-denominator Dice, whose raw output is already the best-calibrated, NLL falls by 17%. Its derived map is also the closest to the identity, i.e., the closest to a loss that needs no post hoc correction. For the different architectures, the calibration map is unchanged from Eq. 6. Changing the Dice weighting replaces ff by f/wf/w in Eq. 6. For the other two loss configurations, which change the pointwise loss (focal loss) or the region loss (soft Jaccard, soft Dice with squared denominator), we insert their pulls into Eq. 3 and solve the label-free (f,D)(f, D) from Eq. 8 with the resulting map in place of Ψf,D\Psi_{f,D} (the squared-denominator Dice map is additionally parameterized by the patch's mean output and mean squared output, both computed directly from the prediction). Besides the new squared-denominator Dice, the paper already covers nnU-Net's two Dice implementations, the per-sample Dice (3d_lowres) and the batch Dice (3d_fullres), for which the map uses the pair (feff,Deff)(f_{\mathrm{eff}}, D_{\mathrm{eff}}) of Eq. 9. We summarize the results in the new supplement section "Other compound losses and backbones" (Sup. M). So the map carries over unchanged to other backbones, while a different loss needs its own map, which the same derivation provides.

  • A:W5/Q5-R:V7dW

TLDR: Against three independent annotations (new Sup. N), the image-dependent maps raise the margin IoU from 0.04-0.05 to 0.17-0.18, close to an oracle bound of 0.20-0.21.

KiTS23 provides three independent annotations for 333 of its 560 tumors, and its label is their majority vote. The voxels marked by one or two of the three annotators define a margin directly from independent annotations, without constructing a posterior from a single label. We trained three KiTS models (on the KiTS label, on a randomly drawn annotator's mask, and on the annotator fraction c/3c/3) and compared each map's margin with this annotator-derived margin (new Sup. N, Table 13). Ψ\Psi raises the IoU from 0.04-0.05 (raw output) to 0.17-0.18; it is the best map on two models and tied with isotonic$_f$ on the third, and significantly above the better one-map-per-model fit on two of the three. The IoU stays low for every map because the output itself only partly separates the voxels on which annotators disagree: an oracle fitted on each patch with its own annotations reaches only 0.20-0.21.

  • A:W6/Q6-R:V7dW

TLDR: We added the requested ablation (new Sup. O). In ECE and NLL, conditioning on DD alone performs close to one map per model, and conditioning on (f,D)(f, D) close to ff alone. In the uncertainty margins, adding DD raises the IoU on some lesion models without changing the models on which Ψ\Psi is best.

As you note, conditioning on the foreground fraction is a reasonable practical choice, and it motivated our design: across the patches of a model, ff varies far more than DD. The middle 80% of patches span a factor of 2.7 to 76 in ff, but at most 1.7 in DD (5.3 on KiTS lowres). Moreover, conditioning jointly on (f,D)(f, D) needs labeled reference patches that are similar in both ff and DD, and these are scarce when held-out labels are limited.

Following your suggestion, we conditioned beta and isotonic calibration on the estimated DD alone and on (f,D)(f, D) jointly, with the same kernel as for ff. We scored them as in Table 1 and Table 2 (Sup. O, Tables 14 and 15):

  • ECE and NLL: conditioning on DD alone performs about as well as one map per model (NLL lower on 1 of 10 models, higher on none, for both families). Conditioning on (f,D)(f, D) performs about as well as ff alone (NLL lower on 2 and higher on 1 of 10 models, for both families).
  • Uncertainty margins: adding DD to ff raises the margin IoU of isotonic regression on KiTS lowres from 0.34 to 0.39 and on liver fullres from 0.40 to 0.44, and changes it by at most 0.01 on the other four lesion models. Ψ\Psi remains the best method on the same three of the six lesion models as in Table 2.

We therefore keep ff as the conditioner in the main paper, a reasonable choice when few labeled reference patches with similar ff and DD are available, and report the DD and (f,D)(f, D) variants in the supplement (Sup. O), with a pointer in Sec. 5.

Reviewer_qPq2OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an image-dependent, label-free calibration method for medical image segmentation models trained with cross-entropy and soft Dice loss. It derives a theoretical mapping from model outputs to posterior probabilities and validates its effectiveness through synthetic and real-world experiments. While results show improved calibration, the theoretical assumptions and limited experimental validation warrant further investigation.

Critical strengths
  1. The paper derives an explicit, image-dependent mapping to correct probability miscalibration induced by cross-entropy and soft Dice loss, providing a principled calibration approach.
  2. The method enables label-free calibration at inference time, reducing the need for additional annotated data and making it potentially useful in medical segmentation.
  3. Experiments on multiple medical segmentation tasks and synthetic data demonstrate improvements in calibration metrics and support the potential effectiveness of the proposed method.
Critical weaknesses
  1. The derivation assumes approximate label statistics and near-optimal model outputs. While the paper discusses these assumptions and tests them synthetically, their validity under real-world label uncertainty, spatial correlations, and imperfect optimization remains unclear.
  2. Experiments cover five tasks but primarily use one fold, with limited cross-fold or external validation. Evaluation also focuses on selected prediction regions, so broader generalization and clinical applicability remain uncertain.
  3. The derivation is specific to CE + soft Dice, and it is unclear whether the method extends to other loss functions, training setups, or data distributions.
Questions for authors

This is an interesting and well-motivated paper that addresses an important problem in medical image segmentation. The theoretical analysis and label-free calibration strategy are promising. I have a few questions that could help clarify the method's assumptions and practical applicability:

  1. How sensitive is the proposed calibration method to violations of the theoretical assumptions, particularly label uncertainty and imperfect model optimization? Could this sensitivity be further quantified on real-world data?
  2. Could the authors provide results across multiple folds or an independent external dataset to better establish the robustness and generalizability of the method?
  3. How does the method perform under different loss weights, Dice implementations, or training configurations? To what extent can the proposed framework be extended beyond cross-entropy and soft Dice loss?
4 comments

We thank Reviewer qPq2 for the time and effort and for the thoughtful questions about our work, in particular the suggestions to analyze the generalization and robustness of our approach in more detail and to extend it to other loss and training configurations. We answer each point below, and we are glad to address any further questions.

  • A:W1/Q1-R:qPq2

TLDR: On real-world data, the map improves calibration both under annotator disagreement (KiTS, three annotations per tumor) and at every checkpoint of a training run, from epoch 10 on.

We used the multiple annotations in the KiTS dataset to test how the derived calibration maps perform under real-world label uncertainty and violations of the theoretical assumptions. The dataset contains three independent annotations for 333 of its 560 tumors (in 489 cases), which we first used to create soft targets, defined as the fraction of annotators who mark a voxel, c/3c/3, and second, to randomly sample one of the annotations as the ground-truth mask during training. In the second setting, the mask volumes fluctuate, so that the approximation of Eq. 5, which is exact only when the mask's volume does not vary for a given image, no longer holds exactly. The median relative standard deviation of the three volumes is 3.2%, and it is 7.4% at the 90th percentile.

Our map still improves calibration in this setup. It at least halves ECE and cross-entropy for all three models, trained on the KiTS label, with a random annotator, and on the soft target (ECE from 0.068, 0.089 and 0.099 to 0.020, 0.040 and 0.047; new Sup. N, Table 13). Moreover, this setting allows us to compare the model-output-derived margins with the margin derived from the annotators (the voxels marked by one or two of the three annotators). Ψ\Psi raises the IoU with this margin from 0.04-0.05 (raw output) to 0.17-0.18. The IoU stays low for every map because the output itself only partly separates the voxels on which the annotators disagree: an oracle isotonic regression (used as an upper bound), fitted on each patch with its own annotations, reaches only 0.20-0.21.

Spatially correlated label noise is the setting of Sup. E, where at a fixed posterior the label noise ranges from independent voxels to a dilation or contraction of the whole contour (see our answer to W1/Q1 of Reviewer V7dW).

Regarding imperfect model optimization, we apply the label-free map to archived checkpoints of a single KiTS training run (3d_lowres, fold 4, epochs 10 to 2000; new Sup. I.1, Table 6). The map lowers ECE and NLL significantly at every checkpoint, already at epoch 10, where nnU-Net's validation pseudo-Dice is only 0.29 (ECE from 0.519 to 0.178, NLL from 1.620 to 0.315; final model of this run: ECE from 0.117 to 0.057, NLL from 0.832 to 0.418). In this run, the improvement of the map therefore did not require converged training.

  • A:W2/Q2-R:qPq2

TLDR: Five-fold cross-validation (30 models: 3 datasets ×\times 2 configurations ×\times 5 folds), an evaluation region that also contains the missed foreground of the scored patches, and an external cohort (CRLM, 197 patients, 170 and 152 of them scored) reproduce the trends of Table 1.

We extended our experiments to provide a more detailed view on the generalization of our approach. We add the results of a five-fold cross-validation for the colon, liver, and kidney datasets on both lowres and fullres configurations (Sup. L.1, Table 8). Additionally, we rescore the calibration metrics on an evaluation region grown from the union of the predicted and the ground-truth foreground (dilated by 3 voxels), so that the foreground the model missed in the scored patches also enters the scores (Sup. L.2, Table 9). For these results, we added the new supplement section "Generalization" (Sup. L) and refer to it in the main paper, in Sec. 6.1 (evaluation region) and Sec. 6.2 (folds and external cohort).

Finally, we evaluate the two liver models (lowres and fullres) on the Colorectal-Liver-Metastases (CRLM) cohort from the Cancer Imaging Archive (TCIA; 197 patients, of which 170 are scored for the lowres and 152 for the fullres model; Simpson et al. [1]). We include a Table 1-style evaluation of this external dataset in the new supplement section (Sup. L.3, Table 10), since Table 1 is already crowded. The trends are consistent with the evaluation on MSD. Additionally, we evaluate the uncertainty margins for this dataset, with the fitted maps transferred from MSD without fitting them on any CRLM label, and present the results as two new rows in Table 2 of the main paper. This transfer reproduces the deployment setting: the models and the fitted maps come from MSD, no label of the new cohort is used, and Ψ\Psi needs no labels at all. There, Ψ\Psi lowers ECE by 41% and 46% and NLL by 43% and 51% (lowres and fullres; Table 10).

  • A:W3/Q3-R:qPq2

TLDR: The maps derived for 4 further loss configurations, and the unchanged CE+Dice map on 3 further architectures, improve calibration as in the nnU-Net v2 CE+Dice experiments.

We chose the CE+Dice compound loss (with equal weighting) as the common thread of the paper, as it is the most prominent combination. However, the theoretical framework applies to diverse combinations of pointwise and region losses (Sup. M). To also examine whether the derived calibration map works for other compound loss formulations, we ran 4 additional experiments on the KiTS data:

  • CE + 0.5 soft Dice
  • CE + 2 soft Dice
  • Focal loss (γ=2\gamma = 2) + soft Jaccard (IoU)
  • CE + soft Dice with squared denominator (the V-Net form)

For CE + w⋅w\cdotDice, the derived calibration map keeps its form, with ff replaced by f/wf/w. For the other two configurations, we insert the pulls of the respective pointwise and region loss into Eq. 3. The map of the squared-denominator Dice also depends on the patch's mean and mean-squared output. We find that for these additional configurations, the derived calibration map also improves the calibration of the outputs. Besides the new squared-denominator Dice, the paper already covers nnU-Net's two Dice implementations, the per-sample Dice (3d_lowres) and the batch Dice (3d_fullres), for which the map uses the pair (feff,Deff)(f_{\mathrm{eff}}, D_{\mathrm{eff}}) of Eq. 9. We summarize the results in an additional supplement section, "Other compound losses and backbones" (Sup. M), and refer to them before deriving the CE+Dice-specific calibration map in Sec. 3.3.

Regarding other training configurations, we applied the unchanged CE + Dice map to three further backbones trained with CE + Dice on KiTS, the transformers Primus-M and PrimusV3-S [2], and the convolutional MedNeXt-B [3]. It lowers ECE/NLL from 0.129/0.902 to 0.059/0.412 (Primus-M), from 0.112/0.890 to 0.063/0.437 (PrimusV3-S) and from 0.121/0.928 to 0.055/0.356 (MedNeXt-B). Regarding other data distributions, we refer to the external validation on the CRLM cohort in our answer to W2/Q2.

References

[1] Simpson, Amber L., et al. "Preoperative CT and survival data for patients undergoing resection of colorectal liver metastases." Scientific data 11.1 (2024): 172

[2] Wald, Tassilo, et al. "Primus: Enforcing attention usage for 3D medical image segmentation." Transactions on Machine Learning Research (2026).

[3] Roy, Saikat, et al. "MedNeXt: Transformer-driven scaling of ConvNets for medical image segmentation." International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2023).

I thank the authors for their thoughtful responses and additional experiments, which have satisfactorily addressed my concerns. I therefore maintain my positive recommendation.

Dear Reviewer qPq2,

We thank you for your positive assessment of our work and your constructive feedback, which helped us to further improve the paper.

Reviewer_1PWcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper studies loss-induced miscalibration in segmentation models trained with CE+Dice. It derives an image-dependent calibration map parameterized by foreground fraction and attained Dice, proposes label-free parameter estimation, and develops image-conditioned fitted calibrators. Experiments on five medical segmentation tasks show improvements in calibration and label-derived uncertainty-margin localization.

Critical strengths
  1. The analysis provides an interpretable explanation of why compound losses distort probabilities and why a shared calibration map may be insufficient.
  2. The label-free correction is practically useful when labeled calibration data are unavailable.
  3. Controlled synthetic experiments and supplementary analyses of batch-pooled Dice and multiclass segmentation strengthen the contribution.
Critical weaknesses
  1. The derivation relies on conditional-risk stationarity and approximations to region-level gradients. Although these are investigated, their validity for real trained networks remains difficult to assess.
  2. The self-consistent estimator has a unique solution, but this does not guarantee accurate parameter recovery. More analysis linking estimation errors to calibration performance would be helpful.
Questions for authors

Can the authors identify practical indicators of when the label-free correction becomes unreliable?

2 comments

We thank Reviewer 1PWc for the updated review.

  • A:W1/W2-R:1PWc

Both raised weaknesses were also raised by Reviewers qPq2 and V7dW, and we refer you to our answers to these weaknesses/questions in the specific answer paragraphs for Reviewers qPq2 and V7dW:

The stationarity assumption and the mean-sum approximation are addressed in A:W1/Q1-R:V7dW (Sup. E, Sup. N) and A:W1/Q1-R:qPq2 (Sup. I.1). The link between estimation errors and calibration results is discussed in A:W2/Q2-R:V7dW and A:W3/Q3-R:V7dW, including a new sensitivity analysis in Sup. B.2. Changing ff by 10% or DD by 0.05 changes NLL by at most 0.017.

  • A:Q1-R:1PWc

The reliability of the label-free correction relies mainly on the model architecture and the training distribution:

  • Models that do not have the architectural capacity to read global statistics, like the foreground fraction, cannot shift their outputs in response to the foreground fraction. If the model cannot shift its outputs in a loss-optimal manner (Eq. 3 / Eq. 6 specifically for CE+Dice), then removing the shift becomes ineffective or even harmful. We performed experiments for networks that lack this capacity in Sup. D.2. We do not expect this to be a major limitation for SOTA models, as this capacity is expected to be present (e.g., through the bottleneck).

  • Secondly, we believe that the distribution of the shift-defining parameters ff and DD in the training data is essential for the model to learn the ff- and DD-dependent loss-optimal shift. If the training data contains only objects of roughly the same size and the achieved Dice values remain similar during training, then the model is not forced to learn the ff- and DD-dependent calibration map. Consequently, if the model is then applied to, e.g., different-sized objects than in the training data, we would not expect the correction to work, since the model did not learn the size-dependent output shift. For the spleen and pancreas 3d_fullres models, DD varies little across patches. Here, the constant D⋆D^\star during deployment even beats the label-derived DD. In practice, this means that knowing the spread of ff and DD in the training patches, and whether the values during inference fall outside the range encountered during training, is an important piece of information for identifying instances or applications where the map might not work as expected.

Thank you for your response. After considering it, I have decided to maintain my original score.

Comments on the paper

1 comment

Dear Reviewers, Dear AC

We thank all Reviewers for their feedback, the discussions, and their positive evaluation of our work. The reviews raised interesting points that sparked new experiments and insights, which we have now included in the paper. Here, we provide a brief recap of many of the important discussion points and experiments.

The new experiments included generalization experiments (Reviewers qPq2, V7dW) for our approach across different architectures (Sup. M) and multiple folds (Sup. L), as well as deriving maps for other pointwise and regionwise losses and testing them on networks trained with these losses (Sup. M). Additionally, we used the two models trained on MSD on an external dataset (CRLM, Reviewer qPq2), where the map almost halves ECE and NLL without CRLM labels and yields better uncertainty margins than beta and isotonic regression fitted on MSD (Table 2). These generalization experiments solidified the experimental evidence. Moreover, the review resulted in a more detailed look at the estimation of ff and DD (Reviewers V7dW, 1PWc), stating the uniqueness of the estimated solution in the main text (Prop. 8, Sec. 4), the robustness to variations in ff and DD, and the analysis of errors in the estimated ff and DD relative to the label-measured values (Sup. B.2). Another important point was the effect of the mean-sum approximation (Reviewers V7dW, 1PWc), which we tested on synthetic data (Sup. E) and on real-world data with multiple annotations (Sup. N). The real-world annotator experiment (Reviewers qPq2, V7dW) showed that the map at least halves ECE and NLL when training with random masks from different annotators, and that it raises the IoU of the uncertainty margin from 0.04-0.05 for the raw model outputs to 0.17-0.18 (against an oracle upper bound of 0.20-0.21). Finally, interesting additional related work was added to the paper (Reviewer 9C7G), the presentation of the paper was improved by adding a glossary at the start of the supplement and unifying the notation (Reviewer 9C7G), and the relevant question of practical indicators for when the map might not work as expected during deployment was discussed (Reviewer 1PWc).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.