- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces RetinaLPA-2S, a few-shot continual adaptation method for retinal disease screening, along with Retinapack, a retinal imaging benchmark for longitudinal analysis. The core idea is to identify and protect a subspace (low-dimensional) associated with pathology changes while allowing the model to adapt to new conditions. The paper also explores deployment on resource-constrained hardware.
Critical strengths
-
The paper works on a very important problem, and its healthcare relevance cannot be overstated, especially in rural and resource-constrained settings where access to ophthalmologists is limited. As the paper points out, development of reliable AI systems that can adapt (or continually learn) and can operate offline could have meaningful impact on early disease screening and clinical care. Here early disease screening could be for the eye disaease and this could also be applicable for other systematic diseases (i.e., oculomics).
-
The paper introduces RetinalPack which is a valuable contribution that could support future AI research in retinal imaging. This benchmark includes more than 13K participants, longitudinal disease-change and stable pairs, repeated acquisitions across cameras (also operators), and separate cohorts for adaptation and evaluation. The longitudinal aspect is particularly interesting as I'm not aware of any such benchmark. This will allow the studies of disease progression beyond conventional cross-sectional classification/detection.
Critical weaknesses
-
The paper is strongly motivated by an important clinical application, but I'm not fully convinced about the strength of its methodological contribution for ICLR. The proposed approach combines generalized eigenvalue-based subspace identification, low-rank adaptation, and existing continual-leanring techniques, and while overall framework is interesting, it is not entirely clear what fundamentally new AI methodology is being contributed here.
-
The central methodological contribution of subspace identification techniques is used to identify a low-dimensional subspace associated with longitudinal pathology changes. But its unclear if the identified subspace genuinely captures pathology-related changes. The authors seem to construct it using clinically labeled change and stable pairs, but I'm not sure if the experiments directly demonstrate that these directions correspond to meaningful pathological changes rather than other correlated variations.
-
Related to point 2, another concern is whether the identified subspace genuinely captures semantic changes in data and if the proposed method can be generalized to other datasets and settings.
-
The question of generalization is also relevant for the presented continual learning and parameter efficient approach. While the paper made comparison with existing works, these comparisons are primarily conducted on the authors' own retinal benchmark.
-
Finally, the paper emphasize offline deployment but the evaluation is largely based on controlled hardware experiments rather than actual clinical deployment. I can understand that the acutual clinical deployment is perhaps out of scope of this study but since deployment is presented as contribution, I think this warrant the understanding of how well the benefits would hold under different hardware configurations and real-world operating conditions.
Questions for authors
-
Please answer the points raised in the weakness section.
-
I am also curious about the clinical acceptance criteria. I see that the paper establish specific criteria before an adapted model can be released. But there is also report about how well the model correctly detects C3 cases but still fails under predefined confidence-bound requirement. So my question is how often can the proposed continual learnign/adaptation system actually deploy its updated models if we see failure in terms of clinical acceptance criteria?