- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a replay-free framework for general continual learning, which is named MIST. MIST maintains historical second-order statistics in a frozen feature space to associate current inputs with past distributions, and uses these associations to condition orthogonal transformations of a shared latent prompt. It further combines historical EMA classifier predictions with an online plastic classifier using statistical novelty and confidence-based fusion. Results show strong anytime and final accuracy and retention.
Critical strengths
-
The paper comprehensively evaluates MIST on three benchmarks with six pretrained backbones.
-
MIST consistently achieves competitive or superior accuracy and retention, especially under self-supervised pretraining.
-
MIST achieves strong continual-learning performance with only a small number of trainable parameters.
Critical weaknesses
-
The current cumulative ablation does not establish whether isometric transport itself is the main contributor.
-
The gains compared to FlyPrompt are very large under some self-supervised settings, but the paper does not clearly state whether optimizer, learning rate, augmentation, local optimization steps, and tuning budgets are strictly matched across methods. In particular, MIST explicitly performs two optimization steps per incoming mini-batch, and their actual optimization budgets are different?
-
Continuous Statistical Measurement (CSM) is always enabled, so its individual contribution is not evaluated, and the cumulative design does not reveal interactions among CIT, GC, and APF. The authors explicitly acknowledge that the current ablation does not isolate CSM.
-
The necessity of the isometric constraint is not sufficiently demonstrated. The paper lacks a controlled comparison against simpler conditional prompt transformations using the same statistical associations and classifier-side components. Therefore, it remains unclear whether preserving latent prompt geometry is essential for the observed gains.
-
The theoretical invariant does not directly guarantee preservation of the actual prompts or representations used by the backbone.
-
The task-free setting still relies on manually scheduled internal intervals. This partially weakens the motivation of fully boundary-free GCL.
-
Hyperparameter sensitivity is incomplete, including E_max, d_m, etc.
-
The performance advantage is not universal. On ImageNet-R with the supervised Sup-21K backbone, FlyPrompt outperforms MIST in accuracy, and this trend persists across the evaluated blurry-ratio settings. This suggests that the advantage of MIST depends on the pretraining and dataset regime.
Questions for authors
See above.
Official Comment by Reviewer ASbv
Thank you for the author's response, but I still have some concerns. In Table R2.1, prototype supervision proves effective across multiple evaluation metrics. For example, its AUC value improves by 7.89 compared to the baseline model and by 7.36 to the w/both. However, MIST improves by only 5.36 compared to the baseline model with enabling prototype supervision, so it is puzzling that MIST’s performance boost is weaker than that of simple prototype supervision. I am unsure whether this trade-off between performance gains and computational cost is reasonable. The independent contributions or interactions of the various components seem to remain unclear.