acceptodds
Withdrawn from ICLR 2027

Are We Forgetting or Just Confused?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Class incremental learning enables models to continuously learn new categories sequentially. Nevertheless, mainstream incremental learning approaches follow a disjoint stage-wise training pipeline and inevitably suffer from severe inter-stage class confusion across different learning phases. To address this issue, this paper presents a Cross-stage Confusion Suppression (CCS) framework built upon pre-trained vision models. The framework first develops a Confusion-Aware Connected Component Generation module, which leverages the Union-Find algorithm to dynamically cluster categories with high inter-stage confusion. On this basis, a Connected Component-Guided Differential Adapter mechanism is proposed to alleviate feature confusion induced by isolated task-specific feature spaces. In particular, we learn a shared adapter to capture common feature patterns within each confused class group, and further subtract the shared feature response from task-specific adapters to obtain more discriminative incremental representations. Moreover, to cope with the inherent unavailability of historical data in exemplar-free incremental learning, we develop a Noise-Driven Pseudo-Sample Generation strategy to compensate for the missing prior category information. Finally, we adopt a two-stage inference procedure to adaptively activate differential adapters during testing, which effectively bridges the gap between training and inference. Extensive experiments on four standard benchmarks verify that the proposed CCS method significantly reduces inter-stage category confusion and achieves superior performance in both classification accuracy and forgetting mitigation. Code will be made publicly available.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Class Incremental Learning, Exemplar-free Incremental Learning, Adapter Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6qJVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Cross-stage Confusion Suppression (CCS) for exemplar-free class-incremental learning. It groups potentially confusing classes using prototype similarity and connected components, trains shared adapters using generated pseudo-samples, and applies weight subtraction and two-stage inference to improve cross-stage discrimination. Experiments on four benchmarks report improvements over the selected baselines.

Critical strengths

CCS provides a practical approach to improving discrimination between classes learned at different stages. Table 2 reports consistent improvements over the authors' baseline across all four benchmarks, suggesting that targeted refinement can be useful.

Critical weaknesses

1. The novelty is not sufficiently distinguished from prior work, and the omission of a key baseline is a significant concern. The distinction between catastrophic forgetting and cross-task confusion has already been explicitly investigated in [1], [2], and [3], weakening the novelty of the paper's motivation. While CCS introduces a specific mechanism for addressing this issue, its contribution relative to these existing studies could be more clearly established.

More importantly, the authors discuss the limitations of RSIAT [4] in Sec. 2 but omit it from Table 1. RSIAT reports 86.92% average accuracy and 82.75% final accuracy on ImageNet-R B0 Inc20, compared with 86.49% and 81.68% for CCS. Although these cross-paper results do not constitute a controlled comparison, they raise legitimate questions about the claimed superiority of CCS and make the omission particularly consequential.

2. The core mechanism and efficiency claims require stronger justification. The method combines established techniques, including graph-based grouping, weight-space subtraction, and gradient-based pseudo-sample generation. In particular, the assumption that subtracting shared adapter weights suppresses common semantic information lacks a theoretical guarantee, as acknowledged in Appendix B.4. Although Table 18 compares weight-level and feature-level subtraction, the existing ablations do not sufficiently isolate the benefit of differential adapter subtraction from simpler component-specific prediction refinement strategies.

Furthermore, CCS retains separate task-specific adapters and introduces additional shared adapters, resulting in growing parameter storage as tasks accumulate. Eq. (14) also requires task-specific feature evaluations during inference. While Tables 4 and 9 provide partial efficiency measurements, they do not establish cumulative parameter storage or end-to-end training and inference costs, particularly when pseudo-sample generation and repeated task-specific evaluations are considered. Consequently, the claimed efficiency advantage remains insufficiently substantiated.

References

[1] On the Importance of Cross-Task Features for Class-Incremental Learning.

[2] Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning.

[3] Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings.

[4] Representation-Steered Incremental Adapter-Tuning for Class-Incremental Learning with Pre-Trained Models.

Questions for authors
  1. How are the historical rectified prototypes in Eq. (9) computed and updated without access to historical training samples? Are they estimated using the generated pseudo-samples, and are they recomputed as the shared adapters are optimized?
  2. What explains the differences between Table 1 and the three-seed results in Table 19? For example, VTAB final accuracy is 94.95% in Table 1 but 92.67 ± 0.97% in Table 19. Additionally, although Appendix A.3 reports dataset-specific similarity thresholds, what validation protocol was used to select them, and were test-set results involved in this selection?
  3. Can the authors provide a controlled comparison with RSIAT [4] using the same pretrained backbone checkpoint, class orders, random seeds, and evaluation protocol? This would help clarify whether CCS provides a consistent advantage over this closely related method.
  4. Can the authors isolate the benefit of differential adapter subtraction against a simpler component-specific reranker with comparable parameters and computational cost? Please also report cumulative parameter storage and end-to-end training and inference costs, including pseudo-sample generation, as the number of incremental tasks increases.
Reviewer_kyoKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary
  1. The authors address an important problem in class-incremental learning. Cross-stage class confusion is a meaningful issue beyond conventional catastrophic forgetting. The motivation is clear and the proposed framework is interesting.

  2. The method achieves strong results on four benchmarks. CCS obtains the best final accuracy across all five reported experimental settings. The improvements over recent methods are promising.

  3. The authors provide extensive ablation studies and additional analysis. They investigate confusion groups, pseudo-sample generation, hyperparameters, and computational efficiency. The results provide useful evidence for the effectiveness of the overall framework.

Critical strengths
  1. The authors study class-incremental learning, where cross-stage class confusion is a meaningful issue beyond conventional catastrophic forgetting. The motivation is clear and the proposed framework is interesting.

  2. The method achieves strong results on four benchmarks. CCS obtains the best final accuracy across all five reported experimental settings. The improvements over recent methods are promising.

  3. The authors provide extensive ablation studies and additional analysis. They investigate confusion groups, pseudo-sample generation, hyperparameters, and computational efficiency. The results provide useful evidence for the effectiveness of the overall framework.

Critical weaknesses
  1. The authors directly subtract shared adapter weights from task-specific adapter weights to obtain discriminative representations. However, weight subtraction does not correspond to feature subtraction in nonlinear adapters, as the authors also acknowledge. It remains unclear why this operation should remove shared semantic information rather than useful discriminative features.

  2. The pseudo-sample generation only minimizes the feature distance to historical class prototypes. This does not guarantee that the generated samples preserve the actual class semantics or intra-class diversity. The reported prototype similarity mainly reflects the optimization objective and provides limited evidence of semantic fidelity.

  3. The authors identify confused classes by comparing prototypes from independently trained task-specific adapters. However, these adapters may learn different feature transformations even with the same frozen backbone. High prototype similarity does not necessarily indicate actual classification confusion, so the grouping criterion lacks sufficient justification.

  4. The authors use Union-Find to group confused classes based on transitive similarity relationships. Two classes can belong to the same component even when they have little direct similarity or actual confusion. This may introduce unrelated classes into shared adapter training and weaken the quality of the learned common representations.

  5. The authors rely on the initial top-k predictions to trigger the second-stage refinement. If the correct class falls outside this candidate set, the refinement stage cannot reliably recover it. This limitation makes the method heavily depend on the first-stage predictions, especially under severe distribution shifts.

Questions for authors

Questions in weaknesses.

Reviewer_9SHNOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Cross-stage Confusion Suppression (CCS) for exemplar-free class-incremental learning with a frozen pre-trained backbone. CCS groups potentially confusable classes using a prototype similarity graph, learns component-specific shared adapters, and constructs differential adapters through weight subtraction. Historical-class pseudo-samples are generated by optimizing current-task images toward stored historical prototypes. A two-stage inference procedure selectively refines predictions within confusion groups. Experiments cover four benchmarks, with the highest final accuracy reported across all five evaluated settings.

Critical strengths
  • The method directly addresses cross-stage class confusion through targeted grouping and refinement, with a clear connection between the identified problem and the proposed design.
  • Table 1 reports competitive results across four benchmarks. The component-level error analyses provide additional evidence that CCS reduces confusion among selected classes.
  • The ablations examine random grouping, standard adapters, alternative pseudo-sample generation methods, and feature-level versus weight-level subtraction.
Critical weaknesses
  1. The reported superiority is not established across random seeds. For example, CIFAR100 final accuracy decreases from 91.55% in Table 1 to 90.04 ± 1.30% in Table 19. Corresponding results for strong baselines under matched seeds and class orders are needed.
  2. Table 20 compares whole-dataset PTM accuracy with CCS’s average incremental accuracy. These metrics are not directly comparable, so the reported improvements require reevaluation.
  3. The exemplar-free update procedure is underspecified. Equation 9 defines rectified prototypes using historical data distributions, but their practical estimation and refresh schedule are unclear, particularly when components expand or merge.
  4. The efficiency analysis remains incomplete. Table 4 reports per-batch training time, and Table 9 reports additional refinement costs, but neither establishes the complete cost of pseudo-sample generation, component training, and inference through all task-specific adapters.
  5. The title suggests a diagnosis of forgetting versus confusion. The experiments demonstrate reduced confusion but do not quantify how much performance degradation results from cross-stage competition versus deterioration of previously learned knowledge.
Questions for authors
  1. How were the dataset-specific similarity thresholds selected?
  2. How important is the seed-selection strategy in NPG? Does selecting samples nearest to historical prototypes outperform randomly selected current-task samples under the same optimization budget?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.