acceptodds
Under review as a conference paper at ICLR 2027

CTSC: Cross-Task Score Calibration for Class-Incremental Learning

12 views

Abstract

Class-incremental learning asks a model to acquire new classes over a sequence of tasks while recognizing all seen classes without task identity at test time. This global prediction setting depends on comparable scores across task blocks, yet sequential training can leave those blocks on incompatible scales. We study this cross-task score problem in hybrid ETF/classifier models and propose CTSC, which combines progressive equiangular tight frame (ETF) anchors and feature preservation with classifier standardization before score fusion and additive taskbias fitting afterward. Pre-fusion standardization controls the classifier contribution relative to ETF evidence, whereas post-fusion task offsets adjust cross-task competition while preserving within-task rankings. Across Seq-CIFAR-10, SeqCIFAR-100, and Seq-TinyImageNet with replay budgets of 200 and 500, CTSC improves Class-IL recognition and reduces forgetting under matched local evaluation. Fixed-checkpoint interventions show that task offsets recover cross-task errors without changing within-task predictions, and that pre-fusion standardization and post-fusion correction are not interchangeable. Calibration-pool and taskprior interventions characterize when replay-fitted offsets transfer to the test distribution.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Class-Incremental Learning, Continual Learning, Cross-Task Score Calibration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2PpgOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work tries to assert that sequential learning in class incremental learning scenarios can leave the task blocks on incompatible scales resulting in a cross task score problem. The paper claims to study this problem and goes on to combine progressive Equiangular Tight Frame (ETF) and feature preservation with classifier standardisation before score fusion and additive task bias fitting afterwards. The pre fusion standardisation tries to control classifier contribution relative to ETF evidence. While post fusion task offsets adjust the cross task competition along with preserving within task rankings. The paper establishes their claims through experiments across many contemporary baseline works and also does a detailed ablation study.

Critical strengths
  1. The gap between Class Incremental Learning and Task Incremental learning accuracy, i.e the scores that are not comparable across tasks is a recognised bottleneck. Targeting this explicitly is a worthwhile contribution even though the relevant diagnostics is missing.
  2. The authors have compared across broad set of baselines spanning replay, contrastive, neural collapse methods, etc. This wide range of comparisons allows to get a better picture of the proposed approach.
  3. The standardisation, ETF based alignment and post fusion task offset etc are light weight components introduced for CIL framework.
  4. The combination of training time ETF alignment and post training corrections is a good design to target the cross task score problem.
Critical weaknesses
  1. Eventhough the cross task score problem is asserted as the motivation in the paper, proper diagnostics or results supporting this is not provided. This would provide with better insight into the problem in hand.
  2. The paper shows improvement over CIL and TIL across multiple baselines, but this alone is not a sufficient metric to show that the cross task score problem is mitigated. Evidences like error decomposition in Class Incremental Learning, that is within task error and cross task error would indicate how much improvement has been achieved. Standard calibration improvement measures like ECE etc would also help strengthen the calibration claims in the paper.
  3. The main table (Table 1) shows many values in bold especially under CIFAR-10 which are not the best numbers but are still shown in bold. (SeqCIFAR10 shows 66.86 (24.83) in bold against CTSC eventhough the best number corresponds to LODE 68.01 (24.63), other similar discrepancies also exist in the Table).
  4. Proposed CTSC shares the ETF alignment framework which is a significant part of the approach with ProNC. This suggests that the main contributions of this work is how this alignment approach is combined with post fusion task offset and classifier standardisation. Given this a direct plugin of the suggested novel component over the ProNC approach would have been more insightful. The ablation results in Figure 4 further strengthen this as the major influence on the accuracy seem to be coming from the ETF and Feature alignment part.
  5. Related to the above observation, the accuracy metric used in the paper as the major evidence sits very close to the accuracy numbers of the ProNC framework. Given this standard deviation across multiple runs would help us understand how much real improvement has been achieved with respect to the ProNC framework.
  6. The method uses ETF progressively by appending the new directions to the existing basis. This in turn would change the old anchors. The paper claims to use feature alignment to align the anchors. But this raises the natural question that aligning with a previous checkpoint would only result in the features aligning with the old anchors corresponding to the old ETF. How does the change in the ETF upon addition of new directions to the basis is accounted for while aligning the anchors.
Questions for authors
  1. Can authors provide diagnostics tat substantiate the cross-tasks core problem and its mitigation? For example decompose CIL errors into within task and cross task errors together with a task confusion matrix for the proposed CTSC and a representative baseline.
  2. As the paper also claims calibration improvement, can the authors provide standard calibration measures such as ECE for CTSC and main baselines?
  3. Please correct the Table 1 bold entries to match correctly with the actual results in the table.
  4. Can the authors also apply the proposed components as a plug in on top of ProNC under identical settings? Further can they also compare the task offset step against established bias correction methods such as BiC and WA? This would help us to isolate the contribution beyond the gains from ETF and feature alignment components.
  5. Please report the standard deviation over multiple seeds for CTSC and ProNC, so that the significance of the improvement over ProNC can be properly assessed.
  6. When new directions are amended to the basis, the old anchors change. How are the old class features realigned to the updated anchor, given that the alignment to previous checkpoint keeps features consistent with old ETF? Can th cosine similarity between the old class features to their respective current anchors be reported to properly judge the suitability of the alignment approach used.
1 comment

I thank the authors for their targeted and detailed response. Most of my concerns have been addressed, and I believe this warrants an upgrade of my score.

I have a few minor follow up questions with regard to the section "6. Realignment when the ETF expands".

  1. The CTSC gain is fairly consistent and at the same time the deterioration across the methods after the tasks also look almost at the similar rate. ProNC drops about 0.19 from Task 2 to Task 10, and CTSC drops about 0.175. If the replay current anchor term is actively performing realignment after each expansion, wouldn't this decline also noticeably slower when compared across the methods? The CTSC gain looks like the ProNC shifted up by an offset, which could also potentially indicate some advantage established early in the learning process.
Reviewer_A6iyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper discusses the class-incremental learning (CIL) problem, specifically, how the performance of a model in this setting relies on cross-task score compatibility, since score drift across task blocks causes newer classes to dominate the global prediction. The paper proposes a cross-task score calibration (CTSC) approach for class-incremental learning that decouples representation maintenance from cross-task score calibration. CTSC targets hybrid ETF - classifier scores by normalizing the classifier branch before ETF fusion and fitting offsets for all seen tasks after fusion.

Critical strengths

The proposed approach freezes the model and adds only per-task offsets, while guaranteeing that within-task rankings remain unchanged.

The proposed method has been evaluated and compared across three benchmark datasets, where class incremental learning performance is better than the compared methods.

Thorough ablation and hyperparameter analysis have been carried out.

Thorough stress testing of the calibration step.

Critical weaknesses

Limited overall novelty of the proposed approach.

Evaluations were not conducted on larger backbones or full ImageNet-scale IL tasks.

The current formulation of CTSC assumes known class-to-task assignments and balanced task blocks during training and calibration. As noted in the limitations, extending this calibration mechanism to streams with ambiguous task boundaries or unknown task groupings remains unexplored.

Writing is not clear, e.g., "In hybrid ETF/classifier models, score operations before and after fusion are not interchangeable.", "We write τ(c) for the known task assignment of class c."

Questions for authors

Primary questions:

  1. Specify which components in the proposed approach are unique and which are reused from existing approaches.

  2. Why were evaluations on larger backbones and full ImageNet-scale IL tasks not carried out

Reviewer_qk3fOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies cross-task score misalignment in replay-based class-incremental learning. Building on ProNC-style ETF anchors and feature preservation, CTSC standardizes classifier logits within each task block before fusion with ETF scores, fits additive task biases on replay samples, and uses age-balanced cross-entropy weighting. Table 1 reports average Class-IL gains of 1.81 and 1.45 percentage points over locally reproduced ProNC across three datasets at buffer sizes 200 and 500, respectively.

Critical strengths

The paper addresses a meaningful distinction between within-task discrimination and cross-task competition. The fixed-checkpoint intervention provides useful evidence: task offsets improve Class-IL accuracy from 26.64% to 38.57%, while Task-IL accuracy remains 84.95% (Table 6). The main calibrator is lightweight, using one scalar per task and 80 optimization steps. Experiments on fusion placement, preprocessing, and calibration sources help characterize the method, including cases where additional correction hurts performance.

Critical weaknesses
  1. The diagnostic figures and reported statistics need reconciliation. Section 4.3 and Appendix B.3 state that Fig. 3(a) uses an illustrative CTSC trajectory for Tasks 2-9 and that measured values are below ProNC. The plotted trajectory instead places CTSC on or above ProNC, without disclosure in the caption. Fig. 4 describes multi-run summaries but reports Class-IL FF of 27.32, versus 27.64 in Table 1. Shared-replay Task-IL accuracy is 85.38 ± 0.52 in Appendix A.5 but 85.71 ± 0.52 in Table 4, despite the stated reuse of reliability checkpoints. These issues weaken confidence in the empirical presentation.
  2. The training and calibration protocols are inconsistent. Section 3.5 uses previous-task offsets in the hybrid training loss, while the main method and Appendix C specify raw-input fitting after each task. Appendix B.4 instead describes normalized intermediate fits for the runs supplying the CIFAR-100 buffer-200 main result, followed by raw refitting at saved checkpoints. The distinction matters because offsets used during training can affect later representations, not just final score calibration.
  3. The contribution beyond existing correction methods and the comparison fairness remain insufficiently isolated. Table 3 provides a useful placement comparison, and Table 7 already tests an ETF baseline with task bias, but the latter uses normalized fitting inputs and 200 steps rather than CTSC's raw-input, 80-step protocol. Table 13 varies loss weights across dataset-buffer settings without explaining selection or comparable ProNC tuning. Table 15 changes Class-IL FAA from 43.62 to 44.42 when the feature-preservation weight increases from 170 to 255. This point-estimate comparison cannot decompose the main-table gain, but motivates matched calibration and tuning controls.
Questions for authors
  1. Can you replace Fig. 3(a) with measured trajectories and reconcile the statistics above by identifying the underlying runs? Please also justify the interpretation of near-zero LSE margins in Fig. 3(b,c): score scales differ across models, and Eq. (22) matches average task probabilities rather than forcing true-task margins to zero.
  2. Which offsets were used during training, and how do the Appendix B.4 runs correspond to the stated raw-input protocol? In Table 2, reported raw/normalized gains are +2.52/+0.77 points with shared replay and +2.54/+2.65 with held-out calibration. Given this change in the ordering of the mean gains, what evidence supports raw fitting across all main configurations?
  3. Can you compare ProNC with the identical CTSC calibrator and a matched BiC-style alternative, using comparable tuning budgets? Please specify the validation procedure, selection criterion, and seeds and uncertainty definitions beyond the documented three-seed CIFAR-100 buffer-200 CTSC result. Please also discuss Online Bias Correction for Task-Free Continual Learning (Chrysakis and Moens, ICLR 2023). Resolving the reporting and protocol issues and demonstrating gains under matched controls could materially change my assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.