acceptodds
Under review as a conference paper at ICLR 2027

Class-Incremental Learning with Zero-Shot Unlearning via Activation Disentanglement

Abstract

We study class-incremental learning (CIL) with zero-shot unlearning, where previously learned task-specific knowledge must be removed without access to past data. Existing CIL methods reduce forgetting during sequential learning, but they do not ensure safe unlearning: deleting a task-specific component can still perturb later-task activations and degrade retained performance. Motivated by this activation-space entanglement challenge, we introduce Anti-Entanglement of Activation (AEA), which enforces inter-task orthogonality in representation space to enable unlearning by parameter deletion. Building on this idea, we propose CAD, a buffer-free class-incremental learning framework with dual-branch LoRA adapters that separate shared and removable knowledge, allowing zero-shot unlearning through simple adapter detachment rather than retraining. The framework further facilitates unlearning by selectively deleting forgotten-task basis vectors from gradient-projection memory. Across multiple benchmarks, CAD achieves stronger retention–removal trade-offs and lower membership-inference risk than existing baselines. Our code is available https://anonymous.4open.science/r/DaCIL-115D/here.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Unlearning, Continual Learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_pmTKOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This work is the first to address continual learning with unlearning (CLU) under the replay-free and task-agnostic settings. The main techinical contributions in the proposed CAD framework are three-fold: dual-branch LoRA architecture, anti-entanglement of activation (AEA) and basis-deletion strategy. Extensive results on cifar100 and ImageNet-R validate the effectiveness of all three components. In addition, the proposed CAD retains efficient computations and storage.

Critical strengths
  1. The paper first provides a comprehensive review on previous related works, including continual learning and machine unlearning. Based on these, it becomes easy to understand the limitations which are difficult to reslove in current mehtods.

  2. The authors carefully describes the method with implementation details and theretical proofs, which help to understand the working mechnisms behind the CAD framework.

  3. The experiments are comprehensive, with the averaged results over three taks sequences. The evaluation metrics are diverse and convincing. More experiments in the appendix further shed light on the advantage of the proposed method.

Critical weaknesses

Although the proposed method has been clearly introduced with text and formulations. It lacks an illustration to depict its overview architecture, which should be a good addition to this work.

The experiment lacks an ablation study on the dual-branch LoRA architecture.

Questions for authors

The proposed CAD is built on top of InfLoRA. It is interesting to discuss whether CAD might be applied to other methods as well. I do not expect new experiments due to limited rebuttal time, but would like to see more clarifications about this point further.

In Table 2, the BWT_f results are not the best, when compared to UniCLUN. It is suggested to clarify the reason for this. I can expect Buffer might be one reason, but it should not be the only one.

Reviewer_zjKgOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies class-incremental continual learning with zero-shot unlearning (CLU). In this setting, a model receives a stream of learning and unlearning commands, cannot store past data, and has no access to task identifiers at test time. The authors argue that orthogonal weight updates (GPM/InfLoRA) are not enough for safe structural deletion. Removing an earlier task's adapter can still perturb the activations of later tasks, a phenomenon they call activation-space entanglement.

Critical strengths
  1. The CLU protocol combines a buffer-free requirement, task-ID-free inference, independent unlearning, and arbitrary learning/unlearning orders. This is more realistic than most prior CLU work, which assumes task IDs or replay buffers. Table 1 clearly positions the setting.

2 Unlearning in the task-exclusive branch requires no data and no optimization; it amounts to discarding an adapter. This gives negligible unlearning cost (App. F.1) and makes the method easy to deploy.

  1. The paper covers:

    • multiple command sequences, including a longer 30-step sequence;
    • three datasets and two backbones;
    • a joint AEA/BDS ablation and a LoRA-rank sensitivity study;
    • a memory accounting and a task-ID-available evaluation;
    • a superclass split testing tasks with shared semantics.

    Results are averaged over three seeds in the main table.

  2. Several appendix sections acknowledge limitations clearly.

  3. Hyperparameters, baseline adaptations, and the MIA procedure are documented in detail, and code is provided.

Critical weaknesses

I have two main concerns that I don't think the rebuttal can address, so I am highly reserved, leaning toward a strong accept.

My first issue is that the paper mostly reassembles existing pieces.

Please see the following novelty issues I find and try to justify from the readers' point of view: what contribution does your method make beyond combining existing pieces?

a. GPM-constrained LoRA, with task adapters summed so no task ID is needed>>>>>> InfLoRA (Liang & Li, 2024). The “task-ID-free” property comes from InfLoRA, not from CAD.

b. AEA loss Σ‖A_iᵀA_t‖_F>>>>> Essentially the O-LoRA penalty (Wang et al., EMNLP Findings 2023). That paper is not cited, which is a significant omission.

c. Shared early layers, task-specific deeper layers>>>> Same design as DualPrompt (G-prompt early, E-prompt late) and other layer-partitioned methods.

d.Unlearning by detaching an adapter>>> Already used by CLPU-style per-task modules. Their own “InfLoRA” baseline also does this. Basis deletion Bookkeeping on GPM: tag each basis with the tasks that use it, then prune bases tagged only with the forgotten task.

My second issue is paper overclaims:

e. "Unlearning by parameter deletion" applies to only half of the network. The abstract and contributions present CAD as achieving zero-shot unlearning "through simple adapter detachment." However, Eq. 3 merges every task's update into the base weights of the first L=6L = 6 of 12 blocks. These updates are never deleted. Removal in the shared branch depends on catastrophic forgetting during subsequent training after BDS releases the GPM constraints .The claim should therefore be restricted to the task-exclusive branch.

f. "Safe" and "reliable" deletion is not established. The paper claims that AEA "enables reliable structural deletion without disrupting retained tasks." Yet Sec. 3.3 itself states that decision sufficiency "does not guarantee prediction preservation under adapter deletion." The Ethics Statement also concedes that the reported measures "should not be interpreted as formal guarantees." The analysis preserves only the projected proxy Zj+1\mathbf{Z}_{j+1}, not Hj+1\mathbf{H}_{j+1}, and Fig. 3(b) still shows non-zero degradation with AEA.

g. "Exact unlearning" (Sec. 4.2, Fig. 4(a)) holds only by construction. JSD =0= 0 at L=0L = 0 is shown only for C3\mathbb{C}_3, where all forgotten tasks follow all retained tasks. In that ordering, deleting the tail adapters trivially recovers the earlier checkpoint. No result covers the case where retained tasks are trained after the forgotten task (e.g., C1\mathbb{C}_1).

h. "Consistently improves over baselines" (Table 2 caption) overstates the margins.

  • ACC$_f$ is 0.00 for CAD, InfLoRA, and CLPU alike.
  • On CIFAR-100, CAD's MR (27.10±1.9827.10 \pm 1.98) is within one standard deviation of InfLoRA (27.67±1.5227.67 \pm 1.52).
  • CAD does not have the best BWT$_f$.
  • In Table 11, CODA-Prompt has better BWT$_r$ on ImageNet-R (−1.89-1.89 vs. −5.13-5.13).
  • The joint ablation (Table 12) attributes only about +2.3 points of ACC$_r$ to AEA and about +0.4 points to BDS.

"Consistently" and "strongest trade-off" should be replaced by a statement of which metrics improve, and by how much.

i. The effect of AEA in representation space is visually exaggerated (Fig. 6). The color bar is clipped at 0.025. Off-diagonal cosine similarities before AEA are therefore already at most about 0.025, meaning the adapter outputs are nearly orthogonal without AEA. The figure presents this as substantial entanglement. Please report the raw values on a scale that includes the diagonal.

j. Efficiency and memory claims are framed favorably.

  • Unlearning speed. The "120× faster unlearning" (App. F.1) compares deleting a matrix against a method that retrains. This speedup follows from the design and is not an empirical finding.
  • Memory savings from BDS. BDS is said to improve "memory efficiency without sacrificing representational integrity." It saves about 0.2–0.45 MB on a total of about 354 MB (around 0.1%, Tables 12 and 16), and "representational integrity" is never measured.
  • Storage overhead. The claim of "low storage overhead" rests on totals dominated by the backbone. Adapter storage grows linearly with the number of retained tasks (Eq. 35).

Other issues that can be considered major issues are:

Inconsistency in Algebra:

k. AEA does not reduce the deletion perturbation; it only rotates it (Eqs. 14–18).

Deleting ΔWj\Delta W_j changes the activation of task j+1j+1 by the full term ΔWjRj+1\Delta W_j R_{j+1}. Eq. 2 constrains only ΔWiRj=0\Delta W_i R_j = 0 for i>ji > j. An earlier adapter acting on later-task inputs is therefore unconstrained.

The AEA condition Aj+1⊤Aj=0A_{j+1}^\top A_j = 0 makes ΔWjRj+1\Delta W_j R_{j+1} orthogonal to Sj+1\mathcal{S}_{j+1}, but it leaves the norm ∥ΔWjRj+1∥\lVert \Delta W_j R_{j+1} \rVert unchanged. Hence only the projected proxy Zj+1Z_{j+1} is preserved, while Hj+1H_{j+1} still changes through Ej+1E_{j+1}.

The authors acknowledge this in Section 3.3: "Decision sufficiency alone does not guarantee prediction preservation under adapter deletion." With that acknowledgment, Section 3.3 serves as motivation rather than as a guarantee.

Question: Can the authors bound ∥Hj+1(−j)−Hj+1∥\lVert H_{j+1}^{(-j)} - H_{j+1} \rVert, or the resulting change in risk Rj+1\mathcal{R}_{j+1}, under AEA?

L. Decision sufficiency (Def. 3.2) is not the property that is needed. Def. 3.2 compares Ri(Zi)\mathcal{R}_i(\mathbf{Z}_i) with Ri(Hi)\mathcal{R}_i(\mathbf{H}_i) for the original model. After deletion, however, the activation is Zi+Ei(−j)\mathbf{Z}_i + \mathbf{E}_i^{(-j)}, not Zi\mathbf{Z}_i, and nothing in the analysis bounds its risk. The empirical support (App. F.7) is limited:

  • it covers a single task (task 9) and a single backbone (ViT-Tiny);
  • it uses single-layer interventions, whereas deletion removes adapters in layers 7–12 simultaneously, so the errors compound across layers.

Moreover, F.7 reports that removing E\mathbf{E} reduces error by up to 6.19 points. This indicates that earlier-adapter contributions actively hurt task 9, which is at odds with treating E\mathbf{E} as a benign residual.

m. Si\mathcal{S}_i has dimension at most rr. Since col(ΔWiRi)⊆col(Ai)\mathrm{col}(\Delta\mathbf{W}_i\mathbf{R}_i) \subseteq \mathrm{col}(\mathbf{A}_i), we have dim⁡Si≤r=10\dim \mathcal{S}_i \le r = 10. The claim that the decision-driving signal "flows primarily through" a subspace of dimension at most 10 in a 192- or 768-dimensional layer is asserted rather than demonstrated. The decomposition in Eq. 9 is also asymmetric: W0Ri\mathbf{W}_0\mathbf{R}_i is left unprojected, while earlier-adapter terms are projected. Please justify this choice.

n. I suggest adding other baselines.

Missing baselines

MU baselines

  • SalUn: C. Fan, J. Liu, Y. Zhang, E. Wong, D. Wei, S. Liu. SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation. ICLR, 2024. arXiv:2310.12508
  • SCRUB: M. Kurmanji, P. Triantafillou, J. Hayes, E. Triantafillou. Towards Unbounded Machine Unlearning. NeurIPS, 2023. arXiv:2302.09880
  • SSD: J. Foster, S. Schoepf, A. Brintrup. Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening. AAAI, 2024. arXiv:2308.07707

CL baselines

  • DCNet: T. Wang, J. Guo, D. Li, Z. Chen. On the Discrimination and Consistency for Exemplar-Free Class Incremental Learning. arXiv:2501.15454
  • O-LoRA: X. Wang, T. Chen, Q. Ge, H. Xia, R. Bao, R. Zheng, Q. Zhang, T. Gui, X. Huang. Orthogonal Subspace Learning for Language Model Continual Learning. Findings of EMNLP, 2023. arXiv:2310.14152
  • SNV: Mohammad Ali Vahedifar, Abhisek Ray, Qi Zhang. Shapley Neuron Values for Continual Learning: Which Neurons Matter Most? ICML, 2026. https://arxiv.org/abs/2605.15877
Questions for authors

Please see weaknesses.

Reviewer_QR2EOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies class-incremental learning where previously learned tasks can be removed without accessing their data. The proposed method, CAD, combines shared adapters in earlier layers with task-specific adapters in later layers. It introduces an orthogonality loss to reduce interference when removing task-specific adapters, and deletes task-exclusive bases from the gradient projection memory. The experiments show improved retained-task performance compared with several baselines after sequences of learning and unlearning operations.

Critical strengths
  1. The paper addresses a meaningful problem. Preventing new tasks from damaging old tasks does not necessarily prevent damage to later tasks when an earlier adapter is removed. This distinction gives a reasonable motivation for the method.

  2. The method does not require data or additional optimisation at deletion time. The improvements over the adapter-deletion baseline and the ablation results provide useful evidence that AEA helps retained-task performance.

  3. The paper includes several analyses of its components, including the effects of adapter separation, basis deletion, and the choice of deletion layers.

Critical weaknesses
  1. My main concern is whether the method removes task information sufficiently, rather than mainly making the original classifier unable to predict the forgotten classes. Updates from forgotten tasks remain in the shared layers, and basis deletion only changes constraints on future learning. Therefore, better retained accuracy may partly come from removing less information.

  2. The membership inference evaluation is difficult to interpret. The paper only reports how often forgotten training samples are identified as members, without showing how often unseen samples from the same classes are also identified as members. We need both to assess whether the attacker can distinguish training samples from unseen samples. A low member rate alone is therefore not enough to show effective unlearning.

  3. Some adapted continual-learning baselines are relatively weak as unlearning methods. Removing replay samples, Fisher information, or distillation constraints does not itself remove information from the current inference weights. These comparisons are useful as reference points, but the main conclusions should rely more on competitive deletion and unlearning baselines.

Questions for authors

See weaknesses

Comments on the paper

1 comment

My concerns are partially addressed. I hope the authors not only agree with my comments but also take action to improve the paper's quality.

Regarding baselines, I understand you didn't have time to compare with them during the rebuttal. I would like to see them in the related work, with a more detailed comparison between your paper and them. Please also add your response to weakness 1 to the related work or introduction so readers understand your paper and how it differs from others.

The authors acknowledge that the main components of CAD are inherited from existing methods: GPM-constrained LoRA from InfLoRA, the orthogonality condition underlying AEA from O-LoRA, shared/task-specific architectural separation from prior methods such as DualPrompt, and structural unlearning through adapter detachment from modular unlearning approaches. The authors argue that CAD's novelty lies in identifying deletion-induced interference and providing a deletion-specific interpretation of activation-space orthogonality. While I recognize the relevance of this problem and the potential practical value of the proposed integration, I remain unconvinced that applying an existing orthogonality regularizer to an existing continual-learning framework constitutes a substantial methodological advance. This prevents me from giving strong support; however, due to your effort during the rebuttal and your problem formulation, I am raising my score from 2 to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.