- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes to incorporate test-time inference into class-incremental learning. The authors introduce a test-stream setting and propose a prompt composition method to leverage relationships among test samples. The method demonstrates improvements under the constructed test streams. However, I am not fully convinced that the proposed setting reflects the real-world scenarios used to motivate this work. The design of the anchor also requires further clarification and justification. In addition, the experimental evaluation is relatively limited, making it difficult to fully assess anti-forgetting ability of the proposed method.
Critical strengths
The paper considers continual learning from the perspective of test-time inference rather than focusing only on the training process. I think exploiting correlations among test samples is an interesting direction and could be useful in some real-world applications, although the current experiments do not fully validate this motivation.
Critical weaknesses
-
The paper motivates stream-aware inference using examples from surveillance and autonomous driving, where consecutive observations naturally share similar scenes or environmental context. However, in the experiments, the correlation is introduced by simply arranging test samples according to their incremental tasks. Since these tasks are defined by class partitions, samples from the same task do not necessarily share meaningful visual similarities. It is also unclear why test samples would arrive in this task-specific order in practice, especially for image classification datasets such as ImageNet-R and ImageNet-A. I am therefore not convinced that this experimental setting reflects the type of correlation used to motivate the method.
-
The anchor is learned jointly with the residual on Task 0 and then frozen for all subsequent tasks. However, I am not sure why the first task is sufficient for learning an anchor that is supposed to capture shared information across all tasks. Would the learned anchor be sensitive to which classes are assigned to Task 0? I did not find an analysis of this issue. Another concern is that both the anchor and the Task 0 residual are trained using the same cross-entropy loss, without any explicit constraint to encourage them to learn different information. It is therefore unclear what ensures that the anchor captures shared knowledge while the residual captures task-specific knowledge.
-
The main experiments only compare final accuracy and average incremental accuracy with prior methods, without reporting multi-seed results or forgetting rates. Although the authors provide these results in the appendix, they are not compared with existing methods. In continual learning, both the stability across random seeds and the ability to mitigate catastrophic forgetting are important for evaluating a method. Prior works such as DualPrompt and CODA-Prompt have reported results across multiple seeds. Without these comparisons, it is difficult to assess whether the proposed method consistently outperforms existing approaches and whether it is actually more effective at preventing forgetting.
Questions for authors
See weaknesses.