- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates why tail classes with similar training frequencies exhibit different degrees of forgetting during adaptation of pretrained models. It introduces Feature Support Erosion and uses the Average Gradient Outer Product to characterize the overlap between pretrained discriminative subspaces and gradients from head classes. The proposed SADP method applies stronger directional preservation to classes with weaker feature support and achieves consistent improvements across four benchmarks for recognition under class imbalance.
Critical strengths
- The paper presents a useful perspective on tail class forgetting by examining feature support beyond class frequency.
- The analysis of feature support, feature retention, and recognition margins provides a coherent motivation for the proposed method.
- SADP selectively preserves vulnerable feature directions, with subspaces and support weights computed only once before training.
- Experiments across benchmarks show gains on tail classes while maintaining competitive performance on frequent classes.
Critical weaknesses
- The proposed explanation relies mainly on correlations, since directional coverage does not distinguish beneficial from harmful updates.
- The ablations lack directional preservation with uniform weights, making the contribution of support weighting difficult to isolate.
- The reliability of AGOP subspaces is not empirically verified, particularly for tail classes with fewer samples than the retained subspace dimension.
- The paper should clarify how the exact feature difference is obtained without an additional backbone evaluation when adapters are inserted throughout the Transformer.
Questions for authors
Please refer to weaknesses.