- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Here, the authors study the local adaptation of PanEcho and especially disentangle the adaptation improvement in terms of output calibration, residual prediction, and uncertainty-based abstention.
Critical strengths
- Studying what actually is happening when adapting these medical foundation models is important, especially when the finding is what the authors find: that in certain settings a simple recalibration would be enough to get the same outcome.
Critical weaknesses
- W1. For me, the main weakness is the contribution of the paper. The evaluation covers one model, one center, and seven related measurement targets, and it is unclear what general insight readers should take from it. The calibration finding is interesting, but whether it generalizes across domains or even across models for the same domain remains open. The other findings, that retaining pretrained predictions helps with fewer labels and that abstention reduces errors among retained cases, are expected. For me, the paper offers a detailed local evaluation of PanEcho, but the contribution beyond that is too limited for ICLR.
- W2. Another aspect is that recovering part of the error gap with an affine transformation does not explain what caused the gap or what the replacement head learned. The transformation could correct an offset, rescale predictions, or shrink them toward the local target distribution. So, for me, the interpretation as "attribution" is too strong for what the results actually show.
Questions for authors
See weaknesses.