- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper introduces calibration maps for image segmentation losses
Critical strengths
Th topic of calibration maps is of great interest, going beyond simple classification in image segmentation tasks.
Critical weaknesses
The paper does not cover the main related work. This includes recently published papers on calibration for medical segmentation, such as: Axel-Jan Rousseau, Thijs Becker, Simon Appeltans, Matthew Blaschko, and Dirk Valkenborg. Post hoc calibration of medical segmentation models. Discover Applied Sciences, 7(3):180, 2025. Wenhao Liang, Wei Zhang, Lin Yue, Miao Xu, Olaf Maennel, and Weitong Chen. We care each pixel: Calibrating medical segmentation models. In Proceedings of the 28th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2025. Mijoo Kim and Junseok Kwon. Uncertainty calibration with energy based instance-wise scaling in the wild dataset. In Eur. Conf. Comput. Vis. (ECCV), 2024. Dongdong Wang, Boqing Gong, and Liqiang Wang. On calibrating semantic segmentation models: Analyses and an algorithm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
The paper is written in a way that is difficult to read and understand. I do not know if this is due to used generative AI. For example, consider the first definitions. It is not clear the decomposition of the binary loss ℓ(q) into its two branches. Is it the conventional one of the form y ℓ_1(q)+(1−y)ℓ_0(q)? Or yℓ_0(q)+(1−y)ℓ_1(q)? Or another form? And then (1) presents directly the gradients, which are not given from the main definition. It is also not clear what means that the gradient step on a voxel with label 1 raises q in proportion to g_1, …
The propositions are also not clearly presented or described. For example, there is no proof for Proposition 1, even in Appendix K. To understand it, one needs to check Reid and Williamson’s paper on composite binary losses and connect it to the so-called inverse link.
At the end, one has to read each section several times in order to understand some supposedly simple notions. For example, the concept of pull: pull up, pull down, region pulls, total pull…
============================
Rebuttal.
We thank the authors for their feedback and the modifications carried out to address the raised issues. For this reason, I have increased my score from Weak rejection to Weak acceptance.
Questions for authors
none
We thank Reviewer 9C7G for the review of our work. Of the four suggested related papers, one was already cited, and we now cite the other three. We revised the parts you found difficult to follow: the loss decomposition is now stated explicitly, a glossary of notation and terms opens the supplement, and the proof of Proposition 1 is a labeled paragraph in Sup. K. We would be glad to hear whether these changes improved the readability.
Thank you for the suggested additions to our related work. We address each suggested paper below:
Yes, it is the conventional decomposition ℓ(q,y)=yℓ1(q)+(1−y)ℓ0(q), which we now state explicitly in Sec. 3.1. Eq. (1) defines the two pulls as the derivatives of these branches, signed by the direction in which a gradient step moves q: g1=−ℓ1′ and g0=ℓ0′. A gradient step of size α changes q by −αℓ1′(q)=αg1(q) on a voxel labeled 1 and by −αℓ0′(q)=−αg0(q) on a voxel labeled 0. This is what "raises q in proportion to g1" refers to.
We added an overview of the notation and technical terms at the start of the supplement, with a pointer at the start of Sec. 3. It describes each term and references where it is introduced. As the page limit leaves no room for a longer introduction in the main text, this overview serves as a reference wherever a term or symbol is not immediately clear. Moreover, we unified the notation and terminology throughout the paper.
Proposition 1 follows from setting the derivative of the risk at a voxel to zero at its minimum, and its proof does not require information from Reid and Williamson (2010). By Eq. 1, R′(q)=ηℓ1′(q)+(1−η)ℓ0′(q)=−ηg1(q)+(1−η)g0(q). At an interior minimum q∗, R′(q∗)=0, which is Eq. 2. Rearranged, Eq. 2 reads η(g1(q∗)+g0(q∗))=g0(q∗), and dividing by the positive sum of the two pulls gives Eq. 3. The submitted Sup. K presented this proof in a single unlabeled sentence at the start of the section. We now give it as a labeled paragraph and point to it from Proposition 1.
We reduced the vocabulary to three terms: the pull up g1 and the pull down g0 (Eq. 1), and the region pulls a and b that a region term adds to them (Eqs. 4 and 5). The term "total pull" is no longer used, and all three terms are described in the glossary.