- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper presents RECAST, which stands for Retain-Guided Contraction via Alignment Scaled Transformation. RECAST is an unlearning method that measures the alignment between gradients of the retain loss and model weights. Based on this alignment, it contracts weight groups more strongly when they are less aligned. The unlearning procedure consists of two stages. In the first stage, weight groups are contracted according to their measured alignment, determined through cosine similarity. This contraction is bounded by a contraction hyperparameter. In the second stage, called consolidation, the model is fine-tuned again on the retain set only. The authors provide experiments on class-level unlearning as well as concept unlearning in a generative model setting.
Critical strengths
- The paper presents results on class-level unlearning. The reported results are favorable, as the proposed method consistently ranks among the two best-performing methods across all metrics. It is also valuable that the authors evaluate the proposed method in a generative model setting, demonstrating promising results and utility in a setting other than class-level unlearning.
- The mathematical analysis provided in the appendix is a valuable addition, and the proofs and propositions appear sound. Furthermore, the extensive experimental protocols provided in the appendix are beneficial for reproducibility.
Critical weaknesses
- Presentation and Overall Readability
The most significant weakness of the paper is its presentation. In its current form, the manuscript appears unfinished and not yet ready for submission. While many of the issues listed below would be relatively minor in isolation, their accumulation substantially affects the overall quality and readability of the paper. In particular, the paper does not guide the reader well through its content. Several abbreviations are used without being properly introduced or explained, sometimes requiring the reader to search through the paper or appendix to understand their meaning. The following issues should be addressed:
- Figure and table formatting: Figure 3 and Table 4 are poorly integrated into the surrounding text, leaving awkward white space behind or overflowing the page limit. Minor point: the column separator in Table 1 is misaligned between the behavioral and internal profile sections.
- Unexplained abbreviations: e.g.: FT, NCC, NC3, LMC, and SH,… Please introduce abbreviations when you first use them. Also, is ROFT the same as FT?
- Broken hyperlinks: Every occurrence of "CIFAR-10/preactivation ResNet-18" appears to contain a broken hyperlink associated with the word "preactivation."
- Minor notation issue: The repeated use of "FLUX.1 [dev]" instead of simply "FLUX" appears somewhat unnatural and could be reconsidered.
- Minor citation issue: The citation for the Random Label method appears to be incorrect and should refer to [1] to credit the original idea.
- Incomplete Appendix
- Table 11 refers to ablation studies, but it is unclear where these results are presented in the paper or whether they are included at all.
- Table 12 is entirely empty.
- Table 14 contains annotations such as "(seq.; deferred)" which appear to be leftover annotations.
- Table 15 contains a "Correlation" row that is almost entirely empty across the reported settings.
- Table 18 similarly contains a "Corrected" row that is almost entirely empty.
- Fig. 4 is just a screenshotted Table from another paper which is never referenced in the text.
- Baseline Selection
- The selection of baselines is inconsistent across Tables 1–3. For instance, SSD, NegGrad, and SHs are missing from Table 2, while other baselines are absent from Tables 1 or 3. Why are not all baselines evaluated throughout? A consistent selection of baselines is essential for a transparent and fair evaluation. Otherwise, it raises concerns about potential cherry-picking of results.
- Some relevant baselines are not considered. Although mentioned in the limitations, at least L1 sparsity-based unlearning methods should be included due to their similarity in dampening model weights, and potentially SCRUB as well. For the FLUX experiment, ConceptPrune [2] would be a relevant comparison, as it identifies critical model parameters and zeros them out (instead of using contraction as you propose).
- Incremental Novelty The main novelty appears to be the application of the direction-aware principle of DASH to the unlearning setting. Using only the retain set during unlearning has been explored before (e.g., FT, EU-k, CF-k, L1-Sparse), while other methods have also proposed weight dampening followed by a fine-tuning phase, as acknowledged in the paper. Therefore, the main contribution appears to be a new variant of existing approaches rather than a fundamentally new method. However, this would be less concerning if the proposed method consistently outperformed closely related baselines, which further emphasizes the importance of the previous point (see Baseline Selection).
[1] Graves, Laura, Vineel Nagisetty, and Vijay Ganesh. "Amnesiac machine learning." Proceedings of the AAAI conference on artificial intelligence. Vol. 35. No. 13. 2021.
[2] Chavhan, Ruchika, Da Li, and Timothy Hospedales. "Conceptprune: Concept editing in diffusion models via skilled neuron pruning." arXiv preprint arXiv:2405.19237 (2024).
Questions for authors
- Line 203: You claim that "The contraction itself is agnostic to whether the forget subset is class- or sample-defined." Where is this demonstrated? This also ties into the debate regarding class- vs. instance-level unlearning. How does your method perform in the more common instance-level unlearning setting?
- What does measure? What is meant by "confidence" in this context?
Withdrawal of Submission
Dear Reviewers,
We sincerely thank you for the time and effort you have dedicated to reviewing our submission, as well as for your valuable and constructive feedback.
We have decided to withdraw our submission from ICLR 2027.
We have also uploaded a revised version of the manuscript incorporating corrections and improvements, which is available on OpenReview for anyone interested in the updated version.
We appreciate your comments and will take them into account as we continue improving our work.
Best regards, The Authors