- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Neural Collapse Alignment Regularization (NCAR), a retain-side regularizer that can be added to existing gradient-based unlearning methods. It constructs simplex ETF targets for the remaining classes, aligns them with the current classifier, and fits their translation and scale to retained features. The resulting penalty encourages within-class concentration and structured class separation while preserving the base forgetting objective. The paper provides a geometric characterization of the regularizer and evaluates it across three datasets, ResNet and ViT architectures, and random-subset and full-class deletion. The reported results show improved accuracy-based agreement and retained-feature similarity to retrained models across several base methods, supported by component ablations and neural-collapse diagnostics.
Critical strengths
The method has a clear intuition and a coherent design. Using neural-collapse geometry to regularize retained representations is a reasonable way to limit disruption during unlearning. Adapting the targets to the current classifier and feature scale makes the regularizer flexible, and it can be added to existing methods without replacing their forgetting objectives.
The experiments show benefits across several base methods, two architectures, and both random-subset and full-class deletion. Improvements in retained-feature similarity and accuracy-based agreement with retraining suggest that the regularizer has practical value beyond a single configuration, although the gains vary substantially by method.
Critical weaknesses
I like the intuition, but the paper presents a relatively simple regularizer as a more substantial contribution than it is. NCAR combines established ideas in within-class concentration, simplex ETF geometry, and feature–classifier alignment. Its main addition is fitting classifier-aligned targets for retained features during unlearning. The geometric analysis explains this construction, but the novelty over existing NC-guided training and CMF-based unlearning remains limited. Direct comparisons with these approaches and simpler alternatives, such as center loss and feature distillation, are needed to establish the value of this particular design.
The model-selection procedure weakens the empirical evidence. The authors select hyperparameters and checkpoints using agreement with an exactly retrained model, and the selection criterion includes accuracy on the same test set used for reporting. This leaves performance without access to retraining outcomes unresolved and compromises the independence of the test evaluation. Results under a validation-based selection procedure, without access to the retrained reference during selection, are needed to assess whether the reported improvements hold in practice.
Questions for authors
Clarification requests:
- What is the main technical contribution beyond existing NC-guided training and CMF-based unlearning? In particular, why should orienting the simplex using the current classifier be preferable to fitting it to retained class means?
- The stated selection procedure uses retraining-relative Avg. Gap, which includes test accuracy. Were the same test examples used for checkpoint selection and final reporting? How would the authors select and the stopping point without an exactly retrained reference?
Additional evidence needed for the current claims:
- Do NCAR’s gains persist when hyperparameters and checkpoints are selected using validation data without access to retraining outcomes?
- Under comparable tuning and computational budgets, how does NCAR compare with center loss, retained-feature distillation, and existing NC- or CMF-based approaches? These comparisons would determine whether its additional construction provides a meaningful benefit.
We thank the reviewer for the thoughtful questions.
Technical contribution and classifier alignment
For the main contribution and distinction from prior NC-based methods, we kindly refer to our response to Reviewer AU12.
Regarding classifier alignment, the intention is to anchor the structural target to the current model's classification directions rather than determine its orientation solely from retained features. The −CA ablation directly compares these alternatives. Under full-class unlearning, classifier alignment reduces Avg. Gap from 3.85 to 2.17 and improves CKA from 0.780 to 0.925.
Hyperparameter and checkpoint selection
We used retraining-relative Avg. Gap, a commonly adopted evaluation criterion in machine unlearning, to select hyperparameters and checkpoints. The base methods were first tuned using this criterion. When applying NCAR, we kept the selected base hyperparameters fixed and selected only the regularization coefficient (\beta) and checkpoint using the same criterion. This allowed us to compare the base methods and their NCAR variants under consistent selection conditions.
The statement that NCAR does not require exact retraining refers specifically to the construction of its regularization targets. NCAR constructs these targets from the current classifier and retained features, with translation and scale fitted analytically. The retrained model is used as a reference for experimental selection and evaluation, but is not involved in constructing the regularization targets.
We recognize the reviewer's concern regarding the inclusion of test accuracy in Avg. Gap. The reported results are therefore comparisons under a common reference-guided selection protocol, rather than an independent validation-only evaluation. We hope this distinction clarifies both the rationale for our experimental procedure and the intended scope of NCAR's retraining-free target construction.