I have two main concerns that I don't think the rebuttal can address, so I am highly reserved, leaning toward a strong accept.
My first issue is that the paper mostly reassembles existing pieces.
Please see the following novelty issues I find and try to justify from the readers' point of view: what contribution does your method make beyond combining existing pieces?
a. GPM-constrained LoRA, with task adapters summed so no task ID is needed>>>>>> InfLoRA (Liang & Li, 2024). The “task-ID-free” property comes from InfLoRA, not from CAD.
b. AEA loss Σ‖A_iᵀA_t‖_F>>>>> Essentially the O-LoRA penalty (Wang et al., EMNLP Findings 2023). That paper is not cited, which is a significant omission.
c. Shared early layers, task-specific deeper layers>>>> Same design as DualPrompt (G-prompt early, E-prompt late) and other layer-partitioned methods.
d.Unlearning by detaching an adapter>>> Already used by CLPU-style per-task modules. Their own “InfLoRA” baseline also does this.
Basis deletion Bookkeeping on GPM: tag each basis with the tasks that use it, then prune bases tagged only with the forgotten task.
My second issue is paper overclaims:
e. "Unlearning by parameter deletion" applies to only half of the network.
The abstract and contributions present CAD as achieving zero-shot unlearning "through simple adapter detachment." However, Eq. 3 merges every task's update into the base weights of the first L=6 of 12 blocks. These updates are never deleted. Removal in the shared branch depends on catastrophic forgetting during subsequent training after BDS releases the GPM constraints .The claim should therefore be restricted to the task-exclusive branch.
f. "Safe" and "reliable" deletion is not established.
The paper claims that AEA "enables reliable structural deletion without disrupting retained tasks." Yet Sec. 3.3 itself states that decision sufficiency "does not guarantee prediction preservation under adapter deletion." The Ethics Statement also concedes that the reported measures "should not be interpreted as formal guarantees." The analysis preserves only the projected proxy Zj+1, not Hj+1, and Fig. 3(b) still shows non-zero degradation with AEA.
g. "Exact unlearning" (Sec. 4.2, Fig. 4(a)) holds only by construction.
JSD =0 at L=0 is shown only for C3, where all forgotten tasks follow all retained tasks. In that ordering, deleting the tail adapters trivially recovers the earlier checkpoint. No result covers the case where retained tasks are trained after the forgotten task (e.g., C1).
h. "Consistently improves over baselines" (Table 2 caption) overstates the margins.
- ACC$_f$ is 0.00 for CAD, InfLoRA, and CLPU alike.
- On CIFAR-100, CAD's MR (27.10±1.98) is within one standard deviation of InfLoRA (27.67±1.52).
- CAD does not have the best BWT$_f$.
- In Table 11, CODA-Prompt has better BWT$_r$ on ImageNet-R (−1.89 vs. −5.13).
- The joint ablation (Table 12) attributes only about +2.3 points of ACC$_r$ to AEA and about +0.4 points to BDS.
"Consistently" and "strongest trade-off" should be replaced by a statement of which metrics improve, and by how much.
i. The effect of AEA in representation space is visually exaggerated (Fig. 6).
The color bar is clipped at 0.025. Off-diagonal cosine similarities before AEA are therefore already at most about 0.025, meaning the adapter outputs are nearly orthogonal without AEA. The figure presents this as substantial entanglement. Please report the raw values on a scale that includes the diagonal.
j. Efficiency and memory claims are framed favorably.
- Unlearning speed. The "120× faster unlearning" (App. F.1) compares deleting a matrix against a method that retrains. This speedup follows from the design and is not an empirical finding.
- Memory savings from BDS. BDS is said to improve "memory efficiency without sacrificing representational integrity." It saves about 0.2–0.45 MB on a total of about 354 MB (around 0.1%, Tables 12 and 16), and "representational integrity" is never measured.
- Storage overhead. The claim of "low storage overhead" rests on totals dominated by the backbone. Adapter storage grows linearly with the number of retained tasks (Eq. 35).
Other issues that can be considered major issues are:
Inconsistency in Algebra:
k. AEA does not reduce the deletion perturbation; it only rotates it (Eqs. 14–18).
Deleting ΔWj changes the activation of task j+1 by the full term ΔWjRj+1. Eq. 2 constrains only ΔWiRj=0 for i>j. An earlier adapter acting on later-task inputs is therefore unconstrained.
The AEA condition Aj+1⊤Aj=0 makes ΔWjRj+1 orthogonal to Sj+1, but it leaves the norm ∥ΔWjRj+1∥ unchanged. Hence only the projected proxy Zj+1 is preserved, while Hj+1 still changes through Ej+1.
The authors acknowledge this in Section 3.3: "Decision sufficiency alone does not guarantee prediction preservation under adapter deletion." With that acknowledgment, Section 3.3 serves as motivation rather than as a guarantee.
Question: Can the authors bound ∥Hj+1(−j)−Hj+1∥, or the resulting change in risk Rj+1, under AEA?
L. Decision sufficiency (Def. 3.2) is not the property that is needed.
Def. 3.2 compares Ri(Zi) with Ri(Hi) for the original model. After deletion, however, the activation is Zi+Ei(−j), not Zi, and nothing in the analysis bounds its risk. The empirical support (App. F.7) is limited:
- it covers a single task (task 9) and a single backbone (ViT-Tiny);
- it uses single-layer interventions, whereas deletion removes adapters in layers 7–12 simultaneously, so the errors compound across layers.
Moreover, F.7 reports that removing E reduces error by up to 6.19 points. This indicates that earlier-adapter contributions actively hurt task 9, which is at odds with treating E as a benign residual.
m. Si has dimension at most r.
Since col(ΔWiRi)⊆col(Ai), we have dimSi≤r=10. The claim that the decision-driving signal "flows primarily through" a subspace of dimension at most 10 in a 192- or 768-dimensional layer is asserted rather than demonstrated. The decomposition in Eq. 9 is also asymmetric: W0Ri is left unprojected, while earlier-adapter terms are projected. Please justify this choice.
n. I suggest adding other baselines.
Missing baselines
MU baselines
- SalUn: C. Fan, J. Liu, Y. Zhang, E. Wong, D. Wei, S. Liu. SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation. ICLR, 2024. arXiv:2310.12508
- SCRUB: M. Kurmanji, P. Triantafillou, J. Hayes, E. Triantafillou. Towards Unbounded Machine Unlearning. NeurIPS, 2023. arXiv:2302.09880
- SSD: J. Foster, S. Schoepf, A. Brintrup. Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening. AAAI, 2024. arXiv:2308.07707
CL baselines
- DCNet: T. Wang, J. Guo, D. Li, Z. Chen. On the Discrimination and Consistency for Exemplar-Free Class Incremental Learning. arXiv:2501.15454
- O-LoRA: X. Wang, T. Chen, Q. Ge, H. Xia, R. Bao, R. Zheng, Q. Zhang, T. Gui, X. Huang. Orthogonal Subspace Learning for Language Model Continual Learning. Findings of EMNLP, 2023. arXiv:2310.14152
- SNV: Mohammad Ali Vahedifar, Abhisek Ray, Qi Zhang. Shapley Neuron Values for Continual Learning: Which Neurons Matter Most? ICML, 2026. https://arxiv.org/abs/2605.15877
My concerns are partially addressed. I hope the authors not only agree with my comments but also take action to improve the paper's quality.
Regarding baselines, I understand you didn't have time to compare with them during the rebuttal. I would like to see them in the related work, with a more detailed comparison between your paper and them. Please also add your response to weakness 1 to the related work or introduction so readers understand your paper and how it differs from others.
The authors acknowledge that the main components of CAD are inherited from existing methods: GPM-constrained LoRA from InfLoRA, the orthogonality condition underlying AEA from O-LoRA, shared/task-specific architectural separation from prior methods such as DualPrompt, and structural unlearning through adapter detachment from modular unlearning approaches. The authors argue that CAD's novelty lies in identifying deletion-induced interference and providing a deletion-specific interpretation of activation-space orthogonality. While I recognize the relevance of this problem and the potential practical value of the proposed integration, I remain unconvinced that applying an existing orthogonality regularizer to an existing continual-learning framework constitutes a substantial methodological advance. This prevents me from giving strong support; however, due to your effort during the rebuttal and your problem formulation, I am raising my score from 2 to 3.