acceptodds
Under review as a conference paper at ICLR 2027

What Continual Learning Preserves: An Interventional Study of Vision Models

Abstract

Continual learning requires retaining prior knowledge while acquiring new information. Beyond accuracy, prior work examines what continual learners preserve through representations, explanations, learned features, and important components. We extend this analysis to causal organization, using activation patching to measure component influences on final representations (causal profiles) and one another (effective networks). In controlled visual learning and continual self-supervised learning, preservation of this organization tracks retained performance across methods, preservation strengths and preservation targets. Two simple regularizers that preserve causal profiles or effective networks improve retention over fine-tuning. Preserving shuffled or reversed profiles can instead impair performance even when representations remain similar to their learned state. Controls further show that larger representation changes need not accompany larger accuracy losses. Decomposing performance changes across current, past, and future tasks reveals different learning–retention trade-offs. Strong weight regularization retains knowledge mainly by limiting further learning, whereas representation and causal-profile preservation allow continued acquisition. On Split-CIFAR-100, continued learning also improves previously learned tasks, allowing causal-profile preservation to outperform freezing and transferred weight-regularization baselines. On Split-ImageNet-R, causal-profile and effective-network preservation reduce forgetting relative to fine-tuning. These findings establish an intervention-based analysis framework and identify causal organization as a useful preservation target for retaining knowledge without unnecessarily restricting new learning.

Primary area:
interpretability and explainable AI
Keywords:
continual learning, mechanistic interpretability, self-supervised learning, activation patching, causal organization, effective network, catastrophic forgetting

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_rwe4OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates knowledge preservation in continual learning through intervention-defined causal organization. Using activation patching, the authors introduce causal profiles and effective networks to measure component influences and propose corresponding preservation objectives. Target-perturbation experiments and task-wise accuracy decomposition investigate the importance of component correspondence and the trade-off between retention and new knowledge acquisition.

Critical strengths

An interesting intervention-based perspective on knowledge preservation. The paper goes beyond conventional parameter and representation analyses by investigating how internal component influences evolve during continual learning. The controlled target-perturbation experiments provide an interesting starting point for understanding the relationship between internal organization and forgetting.

Critical weaknesses
  1. The empirical improvements are limited and do not convincingly justify the proposed complexity. Causal-profile preservation achieves almost identical accuracy to standard representation preservation on both Split-CIFAR-100 from baseline 89.45% to Causal profile 89.46% and Split-ImageNet-R from 70.40% to 70.42%, while effective-network preservation performs worse. On Split-ImageNet-R, even simple L2 regularization and the frozen pretrained encoder outperform the proposed methods. Given that intervention-based regularization requires substantially more complicated component-level computations, the absence of clear performance benefits or quantitative computational cost comparisons makes its practical value questionable.

  2. The experimental evaluation is insufficient to establish the generality of the proposed approach. The main continual-learning experiments rely on a single pretrained DINOv2 backbone and two class-split benchmarks, with only 200 training updates per task. Moreover, evaluation relies on separately trained all-class linear probes rather than a classifier that must itself learn incrementally, while the pretrained encoder already provides very strong performance. It remains unclear whether the proposed preservation mechanisms would remain effective in more challenging settings involving substantial knowledge acquisition, longer task sequences, or significant domain shifts, such as multi-domain task incremental learning (MTIL)[1].

  3. The novelty and empirical competitiveness of the proposed approach are insufficiently established. PNR [2] also addresses the stability–plasticity trade-off in continual self-supervised learning through regularization, using overlapping evaluation settings such as CIFAR-100. However, the authors neither clarify the novelty of their approach relative to PNR nor provide performance comparisons against it.

[1] Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models. ICCV 2023.

[2] Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning. ICML 2024

Questions for authors

Please refer to the critical weaknesses above.

Reviewer_cEENOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces an intervention-based analysis framework to study what continual learning preserves in vision models. By employing activation patching, the authors measure component influences on final representations (causal profiles) and interactions between layers (effective networks). The experiments demonstrate that preserving these causal structures improves continual learning performance over standard fine-tuning, whereas preserving shuffled or reversed targets reduces performance. Furthermore, taskwise performance decomposition reveals that weight regularization largely limits future adaptation, whereas causal-profile preservation successfully balances knowledge retention with continued learning acquisition.

Critical strengths

These are the paper's strengths:

  • the paper extends traditional continual learning analysis by introducing an intervention-based approach (activation patching) to measure causal profiles and effective networks.
  • the authors evaluate their approach on both synthetic objects dataset (Zeroverse) as well as standard public benchmarks like Split-CIFAR-100 and Split-ImageNet-R
  • the experiments are validated across different model architectures, including CORnet-S and DINOv2-B, demonstrating applicability across both convolutional and transformer models.
  • the results reveal that strong weight regularization merely limits adaptation, whereas causal-profile preservation successfully preserves past knowledge while allowing continued learning and acquisition
  • the taskwise performance decomposition clealy separates final performance gains into current-, past-, and future-task contributions, clarifying the trade-offs between stability and plasticity
Critical weaknesses

These are the paper's weaknesses:

  • the empirical validation is restricted only to SSL frameworks. Given that a large proportion of practical continual learning applications operate under supervised or semi-supervised settings, it remains an open question whether preserving causal profiles and effective networks yields similar benefits outside of SSL
  • the performance gains from continued training on public benchmarks are relatively modest
  • regarding takswise decomposition, it reflects the performance changes across tasks, but it does not directly identify the underlying internal causes driving those shifts.
  • the evaluation lacks in many aspects: a) in terms of datasets, the evaluation is limited to only two benchmark families (Split-CIFAR-100 and Split-ImageNet-R) alongside synthetic data b) in terms of models, only a single vision transformer family (DINOv2-B) and a single convolutional model (CORnet-S) are considered, whch reduces significantly the generalizability of the results c) in terms of continual learing methods evaluated, while CaSSLe is a recent one, the other continual learning strategies consists of older baselines like EWC and standard ER
  • finding the correct balance between objective scales and relative gradient strength introduces significant tuning overhead, requiring exhaustive validation sweeps for each method and dataset
Questions for authors

These are my concerns:

  • can the authors present a motivation why the considered only the SSL case? I encourage the authors to extend their analysis to supervised and semi-supervised cases
  • the authors should extend their experiments to additional datasets and model architectures to further assess the generalizability of their proposed method
  • the authors should also consider some more recent parameter-efficient fine-tuning (PEFT) methods like LoRA-based Adapters
  • computing causal profiles and effective networks requires activation patching across image pairs during training. What is the computational overhead (training time and memory footprint) of your intervention-based regularizers compared to standard rehearsal or weight-penalty baselines?
  • your evaluation demonstrates an effective short-term acquisition. But how does strict causal-profile preservation scale over extended task sequences (e.g., 20 to 50 tasks)? Would be possible a loss of plasticity in this case?
  • table 1 does not clearly specify which architecture was used to obtain the reported results: DINOv2 or CORNet-S
1 comment
Reviewer_cEENOpenReview ↗

Final decision

The authors did not present any rebuttal, therefore I maintain my initial rating.

Reviewer_CjsyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the preservation of causal organization in Continual Learning (CL). It introduces two intervention-based measures called causal profile and effective network, which quantify components’ causal influences on the model’s final outputs and component’s effective connection between layers, respectively. Based on these measures, the paper further proposes intervention-based regularizers to retain the network’s causal organization during CL. Controlled experiments support the claim that the proposed measures can serve as tools for analyzing retained functions and as candidate targets for preservation.

Critical strengths
  • This paper introduces intervention-based measures to quantify the causal organization of the model’s internal computation during continual learning and propose regularizers that use these measures as a preservation target.
  • The claim is supported by well controlled comparisons across different methods and benchmarks including variations of experiments settings that isolating the role of the intervention-defined causal organizations.
Critical weaknesses
  • W1: The paper provides limited motivations for studying causal organizations as a measure and preservation target during continual learning. Although the paper claims the inspiration from the brain, it does not clearly explain how activation patching connects to that motivation.
  • W2: The experiments provide convincing evidence for the effectiveness of the proposed measures and regularizers. However, the paper provides limited explanation on why preserving causal organization produces these empirical gains in continual learning.

That said, I am willing to revise my rating based on the discussion.

Questions for authors

The evaluation is limited to continual self-supervised learning, with a new linear probe fitted on frozen features after each stage (Q1-Q2).

  • Q1: Can the proposed approach be applied to supervised continual learning?

  • Q2: Can the proposed approach also be evaluated under protocols that do not refit a linear probe on frozen features after each stage?

  • Q3: What labeled data are used to train the linear probe after each stage, and what is the architecture of the classification head?

  • Q4: How sensitive are the results to the choice of γ\gamma and number of tasks for λt\lambda_t in Eq. (2)?

  • Q5: Could the authors discuss the computational overhead of computing the two measures and training with the proposed intervention-based regularizers?

1 comment

As the discussion deadline approaches and the authors have not responded, I will maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.