- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes CARES, a coreset selection method for continual learning that does not require model optimization. The experimental results demonstrate the efficiency and effectiveness of the proposed method.
Critical strengths
- The proposed coreset selection method avoids model optimization and bilevel optimization, making it computationally efficient.
- The writing is generally clear, and the equations and algorithm are well presented.
- The experiments are comprehensive and cover different evaluation protocols.
Critical weaknesses
- As acknowledged in line 186, the coreset selection method used in this work was originally proposed in [1]. Therefore, the selection method itself is not novel. While it is well established that selecting representative samples for replay can improve continual learning performance, this work does not sufficiently explain why the method from [1] is particularly suitable for continual learning.
- Beyond the empirical results, this work provides limited explanation of why CARES outperforms model-optimization-based coreset selection methods. Model-optimization-based methods typically have explicit objectives that guide sample selection. This work should further explain the relationship between the coverage objective and coreset effectiveness in continual learning, and why this objective can lead to better performance.
- The proposed method relies on an external feature encoder, specifically a frozen ImageNet-pretrained Inception-V3 model. Its effectiveness may therefore depend on the quality of the pretrained representations. It is unclear whether CARES would remain effective on domains that differ substantially from the pretraining distribution.
- The efficiency analysis does not sufficiently account for the computational cost associated with the pretrained encoder.
[1]. SubZeroCore: A Submodular Approach with Zero Training for Coreset Selection
Questions for authors
In line 176, the authors raise the question of how to evaluate the value of a sample when its contribution to the trained model is realized in the future rather than at the time of selection. How does the proposed method address this issue? In particular, how does the coverage-based selection objective account for the future utility of selected samples in continual learning?