- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper is addressing modern rehearsal-free class incremental learning. Specifically, the proposed method is built upon CL-LoRA and EASE. In this paper, the fundamental assumption is that the contextual evidence MAY be available across nearby observations and Contexture Adapter Routing (CAR) is introduced. A frozen context-conditioned inference strategy that exploits a small unlabeled coherent test context without replay, task labels, retraining, or test-time parameter updates. CAR observes that task-specialized modules obtain weak but useful task-discrminative information in their predictive uncertainty. Although this signal is noisy for individual samples, aggregating it across a coherent context yields substantially more reliable module preferences. CAR converts this contextual preference into a soft task prior while retaining image-specific all-class prediction, thereby improving routing without replacing the underlying continual learner.
Critical strengths
-
This paper has attention to an adapter routing method instead of solving the catastrophic forgetting problem in CIL. Furthermore, this paper formulates frozen context-conditioned inference as a distinct CIL inference settings that lies between independent frozen inference and parameter updating TTA.
-
The paper proposed a training-free contextual routing mechanism applicable to modular continual learners via frozen predictive responses like CL-LoRA and EASE.
-
They tried to show theoretically that, when observations share a local distribution and the appropriate module has a positive expected entropy margin, the probability of routing error decreases exponentially with context size. However, there is still some uncertainty.
-
They claimed the proposed method is superior over 10% to the previous method in the traditional CIL evaluation protocols. However, it is not a fair comparison because the proposed method does not follow the traditional protocols (e.g., multiple seeds)
Critical weaknesses
-
One of the major concerns for this paper is that it is not a fair comparison with the previous CIL methods. In previous CIL methods followed the standard CIL evaluation protocol, where each test sample is classified independently. However, the proposed method primarily evaluates its performance using a context of eight test images (e.g., M=8) belonging to the same class. The paper claims that the labels information of context images are not used but we can think the same bird images (in Fig. 2) are flocked when building the context set. In TTA evaluation protocol, the same context test images are not deployed in the same batch. Furthermore, in Table 1, The results of previous methods are directly taken from their original publications without re-evaluating them under the same contextual inference protocol. Consequently, the reported performance comparisons do not establish a fair or convincing superiority over existing CIL methods.
-
The proposed method assumes that semantically coherent test contexts are available (like video stream), but does not provide a sufficiently effective mechanism for constructing such contexts in image classification tasks. I just wonder why the paper does not address the video recognition problem, not the CIFAR-based image classification. To solve this concern, the paper investigated label-free context discovery methods, including feature kNN, DBSCAN, and pseudo-class grouping, but it is too simple and naive. Moreover, the substantial performance degradation observed with randomly constructed contexts further demonstrates the method's sensitivity to context quality. In this respect, the effectiveness of the proposed method under realistic test streams containing heterogeneous classes or tasks remains insufficiently validated.
-
On CIFAR-100, the reproduced CL-LoRA baseline shows 87.94% final accuracy, while the proposed method with M=1 achieves 97.95%. It indicates no improvement under single-sample inference by the proposed routing strategy. The paper already noted the single image is noisy but the unlabeled contexts also can be noisy.
-
The proposed method assumes that the adapter producing the lowest task-local prediction entropy is the most appropriate adapter for a given context. However, low prediction entropy does not necessarily indicate correct task identification, as an incorrect adapter may produce overconfident predictions.
-
The reproduced method and the proposed method comparisons are primarily based on a single random seed, limiting the assessment of robustness across different class orders, adapter initializations, and context sampling strategies.
Questions for authors
- Code should be provided for checking the fair comparison concerns.
- Detailed analyses of inference latency, FLOPs, memory consumption, and throughput should be provided.
4 comments
The main concerns on this method is the fairness on the evaluation between the proposed one and related works. However, the rebuttal does not sufficient address this concern. Because this significant concern is not solved, it is natural to conclude that the rebuttals for other weakness are not proper now. In this respect, I will keep my original rate.
We thank the reviewer for the follow-up. We understand the concern regarding evaluation fairness and would like to clarify one point.
Our intention is not to claim that CAR outperforms existing CIL methods under the same standard sample-wise inference protocol. Rather, CAR studies whether additional contextual information, when available, can improve the predictions of an already trained continual learner. For this reason, our main comparison uses exactly the same frozen model and test images, with and without CAR.
We agree that the original same-class results represent a controlled setting and should not be interpreted as a direct comparison under the standard CIL protocol. This is also why we added the ImageNet-VID experiment, where contexts are formed naturally from consecutive video frames without using labels for grouping.
We appreciate the reviewer's concern and will make this distinction and the scope of our claims clearer in the paper.
Comment from the Reviewer WkR5
Thank you for the clear explanaition. I now better understand the intended scope of CAR. However, my main concern remains that CAR relies on additional contextual information that is not available under the standard sample-wise CIL evaluation protocol. The additional ImageNet-VID experiment is useful, but it does not fundamentally resolve my main concern regarding the evaluation fairness now. So, I keep my rating.
We thank the reviewer for the careful evaluation and constructive comments. We appreciate the recognition of CAR as a different inference-time approach to continual learning. We would like to clarify the intended comparison and present additional experiments addressing the concerns.
1. Comparison with existing CIL methods
We agree that conventional CIL evaluates each image independently, whereas CAR uses information from multiple observations. However, this difference is intentional and is the main motivation of our work.
CAR is not intended to replace existing CIL training methods. Instead, it improves inference using contextual information from an already trained model. In our experiments, we compare exactly the same frozen CL-LoRA or EASE model, with and without CAR. Therefore, the improvement comes from the additional contextual information rather than changes in training.
We agree that the original same-class groups were constructed using ground-truth labels. These experiments were intended to demonstrate CAR's potential under coherent contexts, not to claim a direct comparison under identical standard CIL inference conditions. We will make this distinction clearer in the paper.
2. Video evaluation and context construction
We agree that evaluating naturally occurring sequences is important. We therefore conducted an additional experiment on ImageNet-VID, where contexts are formed using consecutive video frames rather than class labels.
Native CL-LoRA achieves 86.93% accuracy, while CAR achieves 92.89%, 95.18%, and 96.56% for M=4,8,16, respectively. At M=16, the improvement is +9.63 percentage points over 436 frames, with a paired video-bootstrap 95% CI of [+4.17, +16.51] pp.
We also tested synthetic CIFAR-100 streams where images from different classes arrive consecutively within the same task. Using fixed windows without labels or task boundaries, CAR improves accuracy from 87.94% to 95.63% at M=4 when the average task duration is 32 observations.
These results demonstrate that CAR can benefit from contextual information without requiring same-class grouping.
3. Single-image routing and incorrect adapter selection
We agree that low entropy does not always indicate the correct adapter. This is one of the motivations for CAR: entropy from individual images can be unreliable, while averaging across related observations can make adapter selection more reliable.
We would also like to clarify that CAR at M=1 achieves approximately 87.95% on CIFAR-100, not 97.95%. The larger improvements come from aggregating multiple related observations.
Our additional experiments confirm that CAR can harm images from minority tasks when observations from different tasks appear in the same context. We also found that the original λ=1 effectively behaves like hard task selection.
Reducing λ to 0.05 improves accuracy from 88.47% to 95.11% at M=8 on synthetic CIFAR-100 streams. These results suggest that giving less weight to the selected task can reduce errors in mixed contexts. We present this as a sensitivity analysis rather than a new validated parameter setting.
4. Multiple training seeds
We agree that testing multiple training seeds is important. We have now reproduced the complete TUNA framework using three independent training seeds with a fixed class order.
Native TUNA achieves 91.60 ± 0.06% final accuracy on CIFAR-100. Adding contextual adapter selection improves accuracy to 94.54 ± 0.23% at M=16 under same-class grouping. On synthetic task-persistent streams, accuracy reaches 92.80–94.25%, depending on task duration.
These results show that contextual aggregation can also benefit another trained continual learning method across multiple training runs. The original CL-LoRA experiments remain single-seed.
5. Code and computational cost
We agree that the implementation and computational costs should be explained more clearly. We will clarify the experimental configurations, class orders, and implementation details.
We also measured inference time. With 10 adapters and batch size 16, native CL-LoRA takes 200.30 ms, compared with 200.77 ms for CAR at M=8. CAR does not require additional backbone forward passes. Peak GPU memory was also unchanged at the measured precision, although contexts require buffering observations.
We thank the reviewer again. We hope these new experiments clarify that CAR is intended to complement existing continual learners and that its benefits extend beyond the original same-class evaluation.