- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Dual-State Predictive Momentum for online continual learning. DSPM maintains a teacher model from exponential averaging while utilizing this teacher model to distill the student learner during training for online continual learning.
Critical strengths
- The method implementation is simple and is relatively effective.
- The paper is well written and easy to undrestand
Critical weaknesses
- Lack of novelty. The main idea is to use Holt's linear trend exponential smoothing on model parameters, the idea is not new and not as novel as claimed.
- The experimental result showed limited and inconsistent improvement over other benchmarks.
- Through ablation studies, we see the main improvement comes from the teacher-student distillation, further weakening the effect of the trend EMA developed by the author.
Questions for authors
- This paper Parabolic Continual Learning [1] suggested another type of interpolation between data. Curious on the author's take on the connection between their methods (or EMA of parameter weights) and this paper.
- The author needs to address clearly on how is their method novel, and how is this novelty translated towards the empirical gains, otherwise the author's claim is severely weakened.
Yang, H., Hasan, A. & Tarokh, V.. (2025). Parabolic Continual Learning. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:2620-2628 Available from https://proceedings.mlr.press/v258/yang25b.html.