- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates hidden forgetting in continual self-supervised medical pretraining. The authors show that frozen-probe performance declines while fine-tuning performance remains stable. They also examine dual-memory mechanisms and evaluation sensitivity.
Critical strengths
Addresses a meaningful evaluation problem in continual learning.
The main finding is supported by three seeds and pathology-level analysis.
Critical weaknesses
-
Limited generalizability. The experiments are limited to one medical image encoder. It would be interesting to see whether similar findings hold for other models or self-supervised learning approaches.
-
The anchored methods retain more information from the original task, but their performance on the new task is less clear. Without this comparison, it is difficult to assess the stability–plasticity trade-off.
-
Limited experimental evidence. Some important results, particularly those involving CaSSLe, PFR, and PadChest, are based on a single seed. The PadChest experiments also involve different numbers of training updates, which makes the comparison harder to interpret.
-
The paper mainly relies on linear probes to measure forgetting. I wonder whether the same conclusions would hold with more expressive probes, since lower linear-probe performance does not necessarily mean the information has been lost.
Questions for authors
-
Do the main findings hold for another medical foundation encoder or self-supervised objective?
-
Could the authors provide target-task performance for CaSSLe, PFR, and EWC to better assess the stability–plasticity trade-off?
-
Does the severe PadChest forgetting persist under matched optimization steps?
-
Would a nonlinear frozen probe lead to similar conclusions about representation preservation?