- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates machine unlearning under heterogeneous forgetting difficulties and proposes GSUO, a two-stage framework consisting of Signal-Guided Optimization (SGO) and Compensatory Optimization (CO). For random-subset forgetting, the method divides forget samples into Normal and Boundary groups based on prediction confidence and assigns different target distributions for optimization. For class-wise forgetting, it introduces intra-class dispersion and feature alignment objectives to reshape representations of forgotten classes. The subsequent compensation stage aims to preserve model utility. Experiments across several image classification datasets and architectures demonstrate competitive forgetting performance, computational efficiency, and improvements over existing baselines on several evaluation metrics.
Critical strengths
-
The paper addresses the heterogeneity of forgetting behavior across samples and proposes task-specific guidance signals. The resulting framework is relatively simple and achieves promising efficiency while maintaining competitive model utility.
-
The experiments cover multiple datasets, model architectures, and forgetting scenarios, with comparisons against numerous established baselines. The ablation studies and privacy evaluations provide useful empirical evidence supporting the proposed approach.
Critical weaknesses
-
The novelty of the proposed method could be further clarified, as several components appear closely related to existing machine unlearning techniques. A more comprehensive discussion of the differences from prior work would help better establish its contributions.
-
The theoretical analysis could be strengthened to provide more convincing support for the proposed method. Some assumptions and design choices would benefit from further justification.
-
The evaluation of unlearning effectiveness could be more comprehensive. Additional evidence would help demonstrate that the proposed method reliably removes the influence of forgotten data.
-
Although the experimental results are generally promising, the improvements over existing methods are not consistently significant across different settings. A more detailed analysis of the performance trade-offs would strengthen the empirical conclusions.
-
The experimental evaluation could be further expanded to better demonstrate the robustness and generalizability of the proposed method. Additional experiments under more diverse settings would help support its practical applicability.
Questions for authors
-
Could the authors further clarify the key differences between the proposed framework and existing machine unlearning methods, particularly regarding the novelty of the guidance signal design?
-
Could the authors provide additional justification for the assumptions and design choices underlying the proposed method?
-
How do the authors ensure that the evaluation metrics adequately reflect the effectiveness of machine unlearning? Could additional evidence be provided to support the reliability of the forgetting results?
-
Could the authors further explain the performance differences between GSUO and competing methods across different experimental settings?
-
Could the authors provide additional evidence regarding the robustness and generalizability of GSUO under different forgetting configurations and hyperparameter settings?
7 comments
[W1/Q1 — Part 2/2: Experimental and Theoretical Evidence]
To further substantiate these distinctions, we compare GSUO with five recent methods—MU-Mis [1], MUNBa [2], SULI [3], FaLW [4], and LTD [5]—on CIFAR-10 with ResNet-18 under 10% random-subset forgetting. The results are summarized below.
| Method | Acc() ↑ | Acc() ↑ | Acc() | ↓ |
|---|---|---|---|---|
| MU-Mis [1] | 95.94 | 80.89 | 95.72 | 14.83 |
| MUNBa [2] | 94.54 | 77.16 | 78.62 | 1.46 |
| SULI [3] | 72.58 | 61.16 | 71.06 | 9.90 |
| FaLW [4] | 98.62 | 80.32 | 75.20 | 5.12 |
| LTD [5] | 99.99 | 82.68 | 84.14 | 1.46 |
| GSUO | 99.47 | 82.11 | 82.12 | 0.01 |
These methods differ from GSUO in their unlearning mechanisms. MU-Mis [1] suppresses forgotten-sample contributions without accessing retained data; MUNBa [2] coordinates forgetting and retention through Nash bargaining; SULI [3] employs layered iterative unlearning; FaLW [4] addresses sample heterogeneity through adaptive loss reweighting; and LTD [5] approximates local retraining through a teacher model trained on retained neighbors. In contrast, GSUO explicitly constructs differentiated predictive targets for Normal and Boundary samples without requiring additional teacher training.
GSUO achieves the smallest of 0.01%, compared with 5.12% for FaLW, 1.46% for LTD and MUNBa, 14.83% for MU-Mis, and 9.90% for SULI. Although LTD achieves slightly higher retain and test accuracies, GSUO provides substantially closer alignment between forget-set and test-set accuracy while maintaining comparable model utility. These findings provide additional empirical support for the proposed differentiated guidance signal design. We will incorporate these additional experimental comparisons and their corresponding results into the revised manuscript.
Finally, the theoretical analysis and ablation studies support the effectiveness of this differentiated design. Proposition 1 and Observation 1 in Appendix A characterize heterogeneous sample-wise parameter responses and explain the potential limitations of applying uniform forgetting objectives to samples with different forgetting difficulties. Furthermore, Table 4 shows that applying only the Normal or Boundary guidance signal results in values of 13.03% and 15.07%, respectively, whereas combining both signals reduces to 0.01%. For class-wise forgetting, removing either the dispersion or alignment signal also reduces retained-class performance. Table 3 further verifies the complementary contributions of SGO and CO.
In summary, the key novelty of GSUO lies in the coordinated construction of differentiated guidance signals that explicitly specify the desired post-unlearning behavior of different samples and forgetting tasks. By integrating heterogeneous predictive targets for random-subset forgetting, complementary feature-redistribution signals for class-wise forgetting, and subsequent utility compensation, GSUO establishes a unified signal-guided optimization framework. The additional comparisons with five recent machine unlearning methods, together with the theoretical analysis and component-level ablations, further support the effectiveness of this differentiated design.
[1] X. Cheng, Z. Huang, W. Zhou, Z. He, R. Yang, Y. Wu, and X. Huang, "Remaining-data-free machine unlearning by suppressing sample contribution," in Proc. International Conference on Learning Representations (ICLR), 2026.
[2] J. Wu and M. Harandi, "MUNBa: Machine Unlearning via Nash Bargaining," in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2025.
[3] H. Lyu, X. Zhang, H. Hu, S. Wang, C. He, and L. Qi, "Fine-Grained and Efficient Self-Unlearning with Layered Iteration," in Proc. International Joint Conference on Artificial Intelligence (IJCAI), 2025, pp. 7643–7651.
[4] L. Yu, Z. Zhao, Y. Wang, P. Wang, X. Cao, B. Wang, and Y. Wang, "FaLW: A forgetting-aware loss reweighting for long-tailed unlearning," in Proc. International Conference on Learning Representations (ICLR), 2026.
[5] P. Dolgova and S. U. Stich, "Forgetting has neighbors: Localized collateral forgetting in machine unlearning," arXiv:2605.31317, 2026.
[W2/Q2] Regarding the assumptions and applicability of GSUO, our framework does not rely on a specific neural network architecture or require the data to follow a particular parametric distribution.
Instead, GSUO is primarily designed for deep learning-based classification tasks, where predictive distributions and intermediate feature representations can be obtained. For random-subset forgetting, we consider the standard setting in which forget samples and unseen test samples are drawn from the same underlying data distribution. For class-wise forgetting, we consider the different characteristics of entire forgotten classes and design corresponding feature redistribution-based guidance signals. Therefore, GSUO accommodates both random-subset and class-wise forgetting without requiring the same guidance mechanism to be applied across different tasks.
To evaluate the applicability and effectiveness of GSUO, we conduct extensive experiments on four datasets, including CIFAR-10, CIFAR-100, Tiny-ImageNet, and Lacuna-10, using different model architectures such as ResNet-18, ResNet-50, VGG-11, VGG-16, and ViT. We further evaluate the framework using multiple metrics covering forgetting effectiveness, model utility, generalization, privacy protection, and computational efficiency. The experimental results demonstrate that GSUO achieves favorable performance across the evaluated datasets, model architectures, and forgetting tasks, supporting its applicability to different deep classification settings.
Regarding the motivations behind our design choices, first, the confidence-based grouping in random-subset forgetting is motivated by the heterogeneous forgetting difficulty among forget samples. As analyzed in Appendix A, prediction confidence and gradient magnitude exhibit consistent trends in characterizing sample-wise forgetting difficulty. Under our definition of relative forgetting difficulty as the inverse of the gradient norm, high-confidence Normal samples generally exhibit smaller gradient responses and greater forgetting difficulty, whereas low-confidence Boundary samples exhibit larger gradient responses and lower relative forgetting difficulty. The empirical results in Appendix A further support this relationship, showing substantial differences in curvature-aware influence between confidence-based sample groups.
Based on these observations, GSUO distinguishes Normal and Boundary samples and designs different guidance targets according to their predictive characteristics. For high-confidence Normal samples, directly assigning uniform targets may introduce excessive perturbations to the learned predictions. Therefore, we combine the original predictive distribution with a uniform distribution to gradually reduce prediction confidence while retaining part of the original predictive information. For low-confidence Boundary samples, assigning arbitrary incorrect labels may unnecessarily disrupt retained-class decision boundaries. Instead, we use the predictive distribution of the nearest alternative retained-class prototype as the guidance target, providing a forgetting direction informed by the feature structure of retained data. These differentiated targets aim to mitigate over-unlearning and under-unlearning while preserving model utility.
Second, for class-wise forgetting, our guidance signal design is motivated by the compact feature distributions of forgotten classes and the feature changes observed after retraining. Unlike random-subset forgetting, removing an entire class requires disrupting its learned feature structure rather than merely adjusting individual prediction distributions. Therefore, we introduce an intra-class dispersion loss to reduce the compactness of forgotten-class representations and an alignment loss to guide their feature centers toward the retained-data region. These two complementary losses jointly reshape the feature distribution of forgotten classes, aiming to achieve effective forgetting while limiting the impact on retained-class discrimination.
Third, the two-stage optimization framework is designed to balance forgetting effectiveness and model utility. Signal-Guided Optimization (SGO) first guides the forgetting process using task-specific signals, while Compensatory Optimization (CO) subsequently enhances model utility. For random-subset forgetting, CO fine-tunes the model on retained data. For class-wise forgetting, CO reinforces retained-class classification while suppressing forgotten-class predictions, using different learning rates for the feature extractor and classifier to maintain the feature redistribution achieved during SGO.
These design choices are further supported by the ablation studies in Tables 3 and 4.
We thank the reviewer for these valuable comments.
[W1/Q1 — Part 1/2: Methodological Novelty]
We would like to further clarify the key differences between GSUO and existing machine unlearning methods, particularly the motivation and novelty of its guidance signal design. While individual operations such as soft-target construction, feature manipulation, and retained-data fine-tuning have been explored in prior work, our contribution lies in the differentiated construction and coordination of guidance signals according to the expected post-unlearning behavior of different forget samples and forgetting tasks. Rather than simply combining existing optimization techniques, GSUO explicitly specifies distinct target behaviors within a unified optimization framework.
First, GSUO constructs heterogeneous target distributions according to sample characteristics in random-subset forgetting. As analyzed in Appendix A, forget samples exhibit different forgetting difficulties, meaning that applying a uniform forgetting objective may lead to under-unlearning for some samples and unnecessary perturbations for others. To address this issue, GSUO distinguishes high-confidence Normal samples from low-confidence Boundary samples and constructs different guidance signals based on their expected post-unlearning behavior. For Normal samples, GSUO combines the original predictive distribution with a uniform distribution to achieve controlled confidence reduction. For Boundary samples, it identifies the nearest alternative retained class based on feature-prototype distances and uses its soft predictive distribution as the target for unlearning. Unlike conventional hard-label relabeling, this retained-class-informed soft target preserves predictive relationships among classes and provides a forgetting direction guided by retained-data feature geometry. Thus, GSUO constructs distinct target predictive behaviors rather than merely adjusting forgetting strength.
Second, GSUO extends this principle to task-specific feature-space guidance for class-wise forgetting. Unlike random-subset forgetting, class-wise forgetting requires disrupting the representation structure of entire forgotten classes. Accordingly, GSUO jointly designs an intra-class dispersion loss and an alignment loss. The former disrupts compact forgotten-class feature clusters, while the latter guides their feature centers toward the retained-data region. These complementary signals jointly reshape forgotten-class representations. Therefore, GSUO constructs prediction-space or feature-space guidance according to the structural characteristics of each forgetting task rather than applying a fixed heuristic across tasks.
Third, GSUO coordinates signal-guided forgetting with subsequent utility compensation. The SGO stage first guides forget samples toward their task-specific target states, while the CO stage subsequently enhances retained-data performance. In class-wise forgetting, CO further adopts a layer-wise learning rate strategy to help preserve the feature redistribution established during SGO while adapting the classifier to the modified representations. The contribution therefore lies not in retained-data fine-tuning itself, but in its coordinated role within the overall signal-guided optimization framework.
Additional experimental comparisons, theoretical analysis, and ablation evidence are provided in Part 2 below.