- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes a method for cross-site generalization in medical image segmentation. It proposes to jointly do teacher adaptation to the task, where the teacher is a vision foundation model, and knowledge distillation to a smaller student model. The student model is exposed to augmented pseudo-site data allowing better cross-site generalization. The claim of the paper is to follow a novel bilevel optimization approach coupling the optimization of the teacher and students for better performance.
Critical strengths
Domain generalization is an important but much addressed problem in medical imaging. I have not identified a clear strength due to the main problems below.
Critical weaknesses
Firstly, the main weakness of the paper is its methodological soundness. The positioning of the paper is around the bilevel optimization and that is the main claim of methodological novelty. The bilevel optimization process assumes an outer level loss optimized with respect to with a dependence on the optimal value of the parameters of another model w.r.t. an inner-level loss . is optimized w.r.t. , giving . The gradient of the outer-level introduces a dependence to , and bilevel optimization addresses this problem.
The main issue is that used in the present paper, as defined in Eq. 9, does not depend on . As far as I can tell, this is not a typo or a notational shortcut. Therefore I could not understand the point of the whole bilevel optimization, since the two problems are decoupled. The paper even includes the following sentence: "This source-supervised term is independent of the student parameters θ; it does not, by itself, define the differentiable TA–SD coupling required by the response formulation in Sec. 2.3.". I could not assert from the rest of the paper which part of the formulation defines the coupling required by the response formulation.
The paper makes no anonymous github available at this stage, and therefore it is not possible to directly assert this point by checking the codebase.
Secondly, the novelty is lacking: the paper uses foundation models, adaptation, knowledge distillation, in ways that are fairly standard.
Third, given the large body of literature on domain generalization, the experimental section should include relevant baselines. For a segmentation paper at ICLR, I would also expect a straightforward nnU-Net baseline.
Questions for authors
Depending on the rating of other reviewers, clarifications with respect to the points mentioned above would be welcome.