- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper focuses on the training-free few-shot class-incremental learning at the Bi-level Modality Calibration (BiMC) interface. This study finds that the within-novel ranking and the probability mass assigned to the novel groups are impacted by the log-sum-exp (LSE) of the fused logit drifts. The authors conduct the experiments on CIFAR100, CUB200, and miniImageNet, and the results show that the fusion lowers the performance, while MP-VTF stays close to BiMC with slightly higher novel accuracy.
Critical strengths
-
The method is training-free with two invariances.
-
The comparison between Naive fusion and MP-VTF is fair.
-
The authors explicitly discuss the limits of their approach.
Critical weaknesses
- The novelty is very limited. The contribution is incremental compared to existing studies.
- The improvement is negligible compared to the baseline. For example, Avg by +0.023 / +0.002 / +0.016 and Final by +0.103 / +0.013 / +0.020 (CIFAR100 / CUB200 / miniImageNet), while the standard deviations are 0.1 - 0.4. These differences are more like noise, and a comprehensive test with three seeds is impossible.
- This paper states it can hold the novel-group mass fixed, but does not show that fixed mass is desirable. Appendix B shows that Visual 1NN helps and hurts nearly the same (28 vs. 27 on CUB200, seed 2), while mass preservation itself does not guarantee semantically correct reranking. Thus, the method appears to mainly mitigate Naive fusion’s accuracy loss, leaving its substantive contribution unclear beyond correcting a side effect of the baseline.
- The hyperparameter selection and sensitivity are not properly evaluated. For example, the visual weight w is applied the same to all datasets, and there is no sensitivity analysis of it, as well as experimental evaluation across different values.
- The experiments should also evaluate the performance across other backbones, training-free FSCIL methods, and class-split protocols.
Questions for authors
-
How does MP-VTF compare with Naive fusion methods that use a tuned weight, a learned or fixed base/novel bias, or a global temperature, under the same settings?
-
What are the results under different w values?
-
What are the key novelty and technical contributions of this work rather than the incremental efforts?