- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Selective Backpropagation (SBP), a parameter-budgeting framework for exemplar-free few-shot class-incremental learning. The method divides network parameters into three groups: previously assigned parameters that are frozen, currently assigned parameters that can be updated, and free parameters reserved for future tasks. The free parameters are randomly reinitialized at each training epoch to prevent the model from relying on them. The authors also combine SBP with prototype-based weight imprinting and a partial-distance classifier to support incremental learning. Experiments are conducted on CIFAR-100, MiniImageNet, and CUB-200, along with cross-domain FSCIL settings and a long-horizon ImageNet-1K benchmark.
Critical strengths
- The proposed parameter allocation strategy is simple and does not require additional optimization to learn importance masks. This makes the method relatively easy to implement and potentially useful for resource-constrained continual learning.
- The paper goes beyond standard FSCIL benchmarks by considering cross-domain and long-horizon learning scenarios. These experiments are useful for understanding how different methods perform under distribution shifts and over longer learning sequences.
- The authors provide ablation studies on parameter isolation, free-parameter reinitialization, and prototype imprinting. The results help show how these components contribute to the overall performance.
Critical weaknesses
- SBP randomly selects parameters for each incremental session, but the authors do not clearly explain why random selection is a good choice. Different parameters and layers may play different roles in learning new classes. It would be helpful to compare random selection with other strategies, such as importance-based or gradient-based parameter selection, to better justify this design.
- SBP uses a predefined parameter budget, with a larger portion assigned to the base session and a fixed portion to each incremental session. However, different tasks may require different amounts of capacity, especially under distribution shifts. It is unclear whether a fixed budget works well across different learning scenarios. The authors should further investigate how the allocation ratio affects performance and discuss what happens when the available parameters are exhausted.
- The relationship between parameter-level masks and feature-level masks in the partial-distance classifier is not very clear. The authors should explain how the feature dimensions are selected based on the assigned parameters, particularly when using different backbone architectures.
- The paper emphasizes that SBP achieves a good balance between stability and plasticity. However, most experiments only report overall classification accuracy across incremental sessions. This makes it difficult to tell whether the improvements come from better retention of old classes or better learning of new classes. It would be helpful to report base-class accuracy, novel-class accuracy, and forgetting measures separately across sessions.
- The cross-domain experiments are interesting, but the analysis is somewhat limited. A more detailed analysis of representation changes and performance on old and new classes would help explain why SBP performs better in these settings.
Questions for authors
- What is the main advantage of random parameter assignment over importance-based or gradient-based selection? Have the authors considered comparing these strategies under the same parameter budget?
- How sensitive is SBP to the predefined parameter allocation ratios? Would an adaptive allocation strategy work better when incremental tasks have different levels of complexity or distribution shifts?
- How are the feature-level masks in the partial-distance classifier determined from the parameter-level masks? Does this mapping work consistently across different backbone architectures?
- Can the authors provide separate results for old-class retention and new-class learning? This would help clarify whether the improvements mainly come from reducing forgetting or improving adaptation.
We thank the reviewer for the constructive comments and questions. We denote the reviewer's weaknesses and questions as W1, W2, … and Q1, Q2, ….
W1,Q1: SBP deliberately avoids importance- or gradient-based selection, as used by prior masking-based methods such as SoftNet (Kang et al., 2024). SBP pre-allocates future capacity without letting current training implicitly shape it: free parameters remain unused and are reset at the beginning of each epoch, so later sessions start from unbiased capacity and earlier tasks do not depend on parameters needed later (L78-83). Empirically, random allocation outperforms the two importance-based baselines in our tables: SoftNet (learned importance scores) and WaRP (gradient-based importance in a rotated weight space). SBP improves final accuracy over SoftNet / WaRP by 3.14 / 3.16 points on CIFAR-100 (Table 1), 9.12 / 7.21 on MiniImageNet (Table 2), and 7.49 / 2.45 on MiniImageNet→CUB-200 (Table 3), while training 4.79× / 1.43× faster on CIFAR-100. Over 80 ImageNet-1K sessions, WaRP falls to 12.90% final accuracy versus 39.08% for SBP (Table 4).
W2,Q2: Our sensitivity study (Table 11) varies the base budget from 0.5 to 0.99. Performance is stable across the practically relevant range of 0.75 to 0.99, with average accuracy varying by only 2.38%. Thus, SBP does not require precise dataset-specific tuning. The same fixed schedule is used in the cross-domain experiments (Tables 3 and 9) without retuning.
Regarding exhaustion: like prior parameter-isolation methods (e.g., PackNet, SoftNet), SBP uses a fixed backbone, so parameter capacity will eventually saturate in never-ending scenarios. Once capacity runs out, the model behaves like frozen-backbone methods such as FACT and NC-FSCIL: all prior parameters stay frozen, and novel classes are still added through prototype imprinting. We state this limitation in the revised paper (Sec. 5). To test SBP over a much longer horizon, we constructed an 80-session ImageNet-1K benchmark (Table 4), where SBP retains 39.08% final accuracy, compared with 18.40% (OrCo), 14.50% (ADBS), and 12.90% (WaRP).
W3, Q3: SBP allocates capacity at the granularity of output channels: each output channel of a convolutional or linear layer, together with its incoming weights, is assigned to at most one session and frozen thereafter. In ResNet-18, the 512 output channels of the last convolution index the 512 feature dimensions after the residual sum and global average pooling (Appendix A.1). The feature-level mask for session j is the set of these channels assigned at session j, and the active mask is their union over all sessions up to the current one. The ViT follows the same procedure, with the feature-level mask given by the allocated output units of the layer that produces the final features. We add this in our revised paper (L141-143).
W4, Q4: We agree that overall accuracy alone does not separate these effects. Below are base-class and novel-class (all incremental classes seen so far) accuracy on CIFAR-100 (single run):
SBP loses only 2.9 points on base classes across all 8 incremental sessions, versus 5.3–22.7 for the baselines, while its novel-class accuracy is comparable to the other backbone-updating methods (SoftNet, WaRP). OrCo learns novel classes better but forgets substantially more. On this in-distribution benchmark, SBP's gains therefore come mainly from reduced forgetting.
W5: We thank the reviewer for this suggestion and agree that overall accuracy alone does not explain the cross-domain results. Below we report base-class (MiniImageNet) and novel-class (CUB-200, all seen so far) accuracy on MiniImageNet→CUB-200 (single run) for SBP and OrCo, a frozen-backbone method:
To measure representation change, we compute the mean cosine similarity between base-class backbone features after the base session and after the final session: 0.983 for SBP, and 1.00 for OrCo by construction. SBP's backbone continues to adapt while preserving most of the base representation, and learns substantially more CUB classes than OrCo, whose fixed features learn almost none (0.8%). The cost of this adaptation is a larger drop in base accuracy (80.1% to 66.7%, vs. 69.9% to 66.2% for OrCo), and learning 200 CUB classes from five shots remains challenging for both. We will include this analysis for all methods in the final version and revise Sec. 3.2 accordingly.