acceptodds
Under review as a conference paper at ICLR 2027

GSUO: Machine Unlearning under Signal-Guided Optimization

Abstract

Current machine unlearning methods predominantly rely on global, coarse-grained intervention strategies. They lack precise pilot signals to guide the unlearning process and fail to provide differentiable guidance across different unlearning tasks. Due to the varying difficulty of forgetting across different samples, this uniform strategy leads to two problems: when Boundary samples have not yet been forgotten, the entire model remains in an under-unlearned state, leaving residual information that can be exploited by privacy attacks; when normal samples have already been over-forgotten, the entire model falls into an over-unlearned state, thereby harming the model's utility. In this paper, we propose GSUO, a guidance-signal-aware unlearning optimization framework that designs task-specific fine-grained guidance signals to steer the unlearning process and is applicable to both random-subset and class-wise forgetting tasks. Extensive experiments demonstrate that GSUO outperforms 14 baselines in terms of both unlearning effectiveness and generalization, while achieving high efficiency and significant speedups, validating its effectiveness for reliable machine unlearning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine Unlearning, Data Privacy, Privacy Preservation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cX9dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates machine unlearning under heterogeneous forgetting difficulties and proposes GSUO, a two-stage framework consisting of Signal-Guided Optimization (SGO) and Compensatory Optimization (CO). For random-subset forgetting, the method divides forget samples into Normal and Boundary groups based on prediction confidence and assigns different target distributions for optimization. For class-wise forgetting, it introduces intra-class dispersion and feature alignment objectives to reshape representations of forgotten classes. The subsequent compensation stage aims to preserve model utility. Experiments across several image classification datasets and architectures demonstrate competitive forgetting performance, computational efficiency, and improvements over existing baselines on several evaluation metrics.

Critical strengths
  1. The paper addresses the heterogeneity of forgetting behavior across samples and proposes task-specific guidance signals. The resulting framework is relatively simple and achieves promising efficiency while maintaining competitive model utility.

  2. The experiments cover multiple datasets, model architectures, and forgetting scenarios, with comparisons against numerous established baselines. The ablation studies and privacy evaluations provide useful empirical evidence supporting the proposed approach.

Critical weaknesses
  1. The novelty of the proposed method could be further clarified, as several components appear closely related to existing machine unlearning techniques. A more comprehensive discussion of the differences from prior work would help better establish its contributions.

  2. The theoretical analysis could be strengthened to provide more convincing support for the proposed method. Some assumptions and design choices would benefit from further justification.

  3. The evaluation of unlearning effectiveness could be more comprehensive. Additional evidence would help demonstrate that the proposed method reliably removes the influence of forgotten data.

  4. Although the experimental results are generally promising, the improvements over existing methods are not consistently significant across different settings. A more detailed analysis of the performance trade-offs would strengthen the empirical conclusions.

  5. The experimental evaluation could be further expanded to better demonstrate the robustness and generalizability of the proposed method. Additional experiments under more diverse settings would help support its practical applicability.

Questions for authors
  1. Could the authors further clarify the key differences between the proposed framework and existing machine unlearning methods, particularly regarding the novelty of the guidance signal design?

  2. Could the authors provide additional justification for the assumptions and design choices underlying the proposed method?

  3. How do the authors ensure that the evaluation metrics adequately reflect the effectiveness of machine unlearning? Could additional evidence be provided to support the reliability of the forgetting results?

  4. Could the authors further explain the performance differences between GSUO and competing methods across different experimental settings?

  5. Could the authors provide additional evidence regarding the robustness and generalizability of GSUO under different forgetting configurations and hyperparameter settings?

7 comments

We thank the reviewer for these valuable comments.

[W1/Q1 — Part 1/2: Methodological Novelty]
We would like to further clarify the key differences between GSUO and existing machine unlearning methods, particularly the motivation and novelty of its guidance signal design. While individual operations such as soft-target construction, feature manipulation, and retained-data fine-tuning have been explored in prior work, our contribution lies in the differentiated construction and coordination of guidance signals according to the expected post-unlearning behavior of different forget samples and forgetting tasks. Rather than simply combining existing optimization techniques, GSUO explicitly specifies distinct target behaviors within a unified optimization framework.
First, GSUO constructs heterogeneous target distributions according to sample characteristics in random-subset forgetting. As analyzed in Appendix A, forget samples exhibit different forgetting difficulties, meaning that applying a uniform forgetting objective may lead to under-unlearning for some samples and unnecessary perturbations for others. To address this issue, GSUO distinguishes high-confidence Normal samples from low-confidence Boundary samples and constructs different guidance signals based on their expected post-unlearning behavior. For Normal samples, GSUO combines the original predictive distribution with a uniform distribution to achieve controlled confidence reduction. For Boundary samples, it identifies the nearest alternative retained class based on feature-prototype distances and uses its soft predictive distribution as the target for unlearning. Unlike conventional hard-label relabeling, this retained-class-informed soft target preserves predictive relationships among classes and provides a forgetting direction guided by retained-data feature geometry. Thus, GSUO constructs distinct target predictive behaviors rather than merely adjusting forgetting strength.
Second, GSUO extends this principle to task-specific feature-space guidance for class-wise forgetting. Unlike random-subset forgetting, class-wise forgetting requires disrupting the representation structure of entire forgotten classes. Accordingly, GSUO jointly designs an intra-class dispersion loss and an alignment loss. The former disrupts compact forgotten-class feature clusters, while the latter guides their feature centers toward the retained-data region. These complementary signals jointly reshape forgotten-class representations. Therefore, GSUO constructs prediction-space or feature-space guidance according to the structural characteristics of each forgetting task rather than applying a fixed heuristic across tasks.
Third, GSUO coordinates signal-guided forgetting with subsequent utility compensation. The SGO stage first guides forget samples toward their task-specific target states, while the CO stage subsequently enhances retained-data performance. In class-wise forgetting, CO further adopts a layer-wise learning rate strategy to help preserve the feature redistribution established during SGO while adapting the classifier to the modified representations. The contribution therefore lies not in retained-data fine-tuning itself, but in its coordinated role within the overall signal-guided optimization framework.
Additional experimental comparisons, theoretical analysis, and ablation evidence are provided in Part 2 below.

[W1/Q1 — Part 2/2: Experimental and Theoretical Evidence]
To further substantiate these distinctions, we compare GSUO with five recent methods—MU-Mis [1], MUNBa [2], SULI [3], FaLW [4], and LTD [5]—on CIFAR-10 with ResNet-18 under 10% random-subset forgetting. The results are summarized below.

MethodAcc(DrD_r) ↑Acc(DtD_t) ↑Acc(DfD_f)∣Diff∣\lvert\mathrm{Diff}\rvert ↓
MU-Mis [1]95.9480.8995.7214.83
MUNBa [2]94.5477.1678.621.46
SULI [3]72.5861.1671.069.90
FaLW [4]98.6280.3275.205.12
LTD [5]99.9982.6884.141.46
GSUO99.4782.1182.120.01

These methods differ from GSUO in their unlearning mechanisms. MU-Mis [1] suppresses forgotten-sample contributions without accessing retained data; MUNBa [2] coordinates forgetting and retention through Nash bargaining; SULI [3] employs layered iterative unlearning; FaLW [4] addresses sample heterogeneity through adaptive loss reweighting; and LTD [5] approximates local retraining through a teacher model trained on retained neighbors. In contrast, GSUO explicitly constructs differentiated predictive targets for Normal and Boundary samples without requiring additional teacher training.
GSUO achieves the smallest ∣Diff∣|\mathrm{Diff}| of 0.01%, compared with 5.12% for FaLW, 1.46% for LTD and MUNBa, 14.83% for MU-Mis, and 9.90% for SULI. Although LTD achieves slightly higher retain and test accuracies, GSUO provides substantially closer alignment between forget-set and test-set accuracy while maintaining comparable model utility. These findings provide additional empirical support for the proposed differentiated guidance signal design. We will incorporate these additional experimental comparisons and their corresponding results into the revised manuscript.
Finally, the theoretical analysis and ablation studies support the effectiveness of this differentiated design. Proposition 1 and Observation 1 in Appendix A characterize heterogeneous sample-wise parameter responses and explain the potential limitations of applying uniform forgetting objectives to samples with different forgetting difficulties. Furthermore, Table 4 shows that applying only the Normal or Boundary guidance signal results in ∣Diff∣|\mathrm{Diff}| values of 13.03% and 15.07%, respectively, whereas combining both signals reduces ∣Diff∣|\mathrm{Diff}| to 0.01%. For class-wise forgetting, removing either the dispersion or alignment signal also reduces retained-class performance. Table 3 further verifies the complementary contributions of SGO and CO.
In summary, the key novelty of GSUO lies in the coordinated construction of differentiated guidance signals that explicitly specify the desired post-unlearning behavior of different samples and forgetting tasks. By integrating heterogeneous predictive targets for random-subset forgetting, complementary feature-redistribution signals for class-wise forgetting, and subsequent utility compensation, GSUO establishes a unified signal-guided optimization framework. The additional comparisons with five recent machine unlearning methods, together with the theoretical analysis and component-level ablations, further support the effectiveness of this differentiated design.
[1] X. Cheng, Z. Huang, W. Zhou, Z. He, R. Yang, Y. Wu, and X. Huang, "Remaining-data-free machine unlearning by suppressing sample contribution," in Proc. International Conference on Learning Representations (ICLR), 2026.
[2] J. Wu and M. Harandi, "MUNBa: Machine Unlearning via Nash Bargaining," in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2025.
[3] H. Lyu, X. Zhang, H. Hu, S. Wang, C. He, and L. Qi, "Fine-Grained and Efficient Self-Unlearning with Layered Iteration," in Proc. International Joint Conference on Artificial Intelligence (IJCAI), 2025, pp. 7643–7651.
[4] L. Yu, Z. Zhao, Y. Wang, P. Wang, X. Cao, B. Wang, and Y. Wang, "FaLW: A forgetting-aware loss reweighting for long-tailed unlearning," in Proc. International Conference on Learning Representations (ICLR), 2026.
[5] P. Dolgova and S. U. Stich, "Forgetting has neighbors: Localized collateral forgetting in machine unlearning," arXiv:2605.31317, 2026.

[W2/Q2] Regarding the assumptions and applicability of GSUO, our framework does not rely on a specific neural network architecture or require the data to follow a particular parametric distribution.
Instead, GSUO is primarily designed for deep learning-based classification tasks, where predictive distributions and intermediate feature representations can be obtained. For random-subset forgetting, we consider the standard setting in which forget samples and unseen test samples are drawn from the same underlying data distribution. For class-wise forgetting, we consider the different characteristics of entire forgotten classes and design corresponding feature redistribution-based guidance signals. Therefore, GSUO accommodates both random-subset and class-wise forgetting without requiring the same guidance mechanism to be applied across different tasks.
To evaluate the applicability and effectiveness of GSUO, we conduct extensive experiments on four datasets, including CIFAR-10, CIFAR-100, Tiny-ImageNet, and Lacuna-10, using different model architectures such as ResNet-18, ResNet-50, VGG-11, VGG-16, and ViT. We further evaluate the framework using multiple metrics covering forgetting effectiveness, model utility, generalization, privacy protection, and computational efficiency. The experimental results demonstrate that GSUO achieves favorable performance across the evaluated datasets, model architectures, and forgetting tasks, supporting its applicability to different deep classification settings.
Regarding the motivations behind our design choices, first, the confidence-based grouping in random-subset forgetting is motivated by the heterogeneous forgetting difficulty among forget samples. As analyzed in Appendix A, prediction confidence and gradient magnitude exhibit consistent trends in characterizing sample-wise forgetting difficulty. Under our definition of relative forgetting difficulty as the inverse of the gradient norm, high-confidence Normal samples generally exhibit smaller gradient responses and greater forgetting difficulty, whereas low-confidence Boundary samples exhibit larger gradient responses and lower relative forgetting difficulty. The empirical results in Appendix A further support this relationship, showing substantial differences in curvature-aware influence between confidence-based sample groups.
Based on these observations, GSUO distinguishes Normal and Boundary samples and designs different guidance targets according to their predictive characteristics. For high-confidence Normal samples, directly assigning uniform targets may introduce excessive perturbations to the learned predictions. Therefore, we combine the original predictive distribution with a uniform distribution to gradually reduce prediction confidence while retaining part of the original predictive information. For low-confidence Boundary samples, assigning arbitrary incorrect labels may unnecessarily disrupt retained-class decision boundaries. Instead, we use the predictive distribution of the nearest alternative retained-class prototype as the guidance target, providing a forgetting direction informed by the feature structure of retained data. These differentiated targets aim to mitigate over-unlearning and under-unlearning while preserving model utility.
Second, for class-wise forgetting, our guidance signal design is motivated by the compact feature distributions of forgotten classes and the feature changes observed after retraining. Unlike random-subset forgetting, removing an entire class requires disrupting its learned feature structure rather than merely adjusting individual prediction distributions. Therefore, we introduce an intra-class dispersion loss to reduce the compactness of forgotten-class representations and an alignment loss to guide their feature centers toward the retained-data region. These two complementary losses jointly reshape the feature distribution of forgotten classes, aiming to achieve effective forgetting while limiting the impact on retained-class discrimination.
Third, the two-stage optimization framework is designed to balance forgetting effectiveness and model utility. Signal-Guided Optimization (SGO) first guides the forgetting process using task-specific signals, while Compensatory Optimization (CO) subsequently enhances model utility. For random-subset forgetting, CO fine-tunes the model on retained data. For class-wise forgetting, CO reinforces retained-class classification while suppressing forgotten-class predictions, using different learning rates for the feature extractor and classifier to maintain the feature redistribution achieved during SGO.
These design choices are further supported by the ablation studies in Tables 3 and 4.

Reviewer_KfxROpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes a heuristic approximate unlearning method called GSUO, where the first stage assigns targets to the forget samples and then in the second phase it fine-tunes on the retain samples.

The paper also proposes different methodologies for unlearning randomly selected samples and for class unlearning. For randomly selected samples, if the model has high confidence, the loss function tries to make the model unconfident about that sample by forcing its prediction closer to a uniform distribution over the predicted classes. Other way around, if the model has low confidence, the sample is forced to align with another class that it is close to.

For class-wise unlearning, the method first separates the samples inside the class and then assigns them to the corresponding classes they are close to.

Critical strengths

None

Critical weaknesses
  • [W1] The approach to the metrics provides wrong intuition about unlearning performance. They consider the performance is better when the retain and test set accuracies is high and the difference between forget and test accuracy is small. However the correct interpretation should be closeness to the retraining from scratch model.
  • [W2] The methodology resembles the already existing literature on heuristic methods on unlearning (relabeling to a close class, smoothing toward a uniform distribution, fine tuning on retain samples).
  • [W3] Forgetting difficulty is defined as the inverse of the norm of the gradient which implies the more confidence of the sample (if confidence is defined as the closeness to the one-hot vector) then more difficult to unlearn. However then the samples sampled by the confidence with raises questions about the soundness of the theoretical claims.
Questions for authors
  • [Q1] Can you explain what is the novelty considering the already existing papers?
  • [Q2] Can you explain the forgetting diffuculty and why the reported result is given by the samples sampled by confidence? Is there a difference between the confidence and the gradient or is there a direct relationship?
  • [Q3] Why the metrics are not analyzed from the perspective comparing it with the retraining from scratch?
5 comments

We thank the reviewer for these valuable comments.
[W2/Q1] We acknowledge that operations such as prediction smoothing, similarity-based target assignment, and retained-data fine-tuning have been explored in prior studies. However, the novelty of GSUO lies not in these individual operations, but in a signal-guided optimization framework that explicitly constructs differentiated post-unlearning targets according to sample characteristics and forgetting-task requirements.
First, unlike conventional smoothing or relabeling, GSUO assigns different predictive targets to Normal and Boundary samples. For high-confidence Normal samples, it combines the original prediction with a uniform distribution to achieve controlled confidence reduction. For low-confidence Boundary samples, it constructs a soft target distribution from the nearest alternative retained class via feature-prototype distances. This retained-class-informed guidance preserves inter-class predictive relationships rather than simply assigning a new hard label. Consequently, GSUO specifies distinct desired forgetting behaviors instead of applying a common operation uniformly. Second, GSUO introduces task-specific representation guidance for class-wise forgetting. An intra-class dispersion loss disrupts compact forgotten-class representations, while an alignment loss guides their feature centers toward retained-data regions. These complementary objectives address both the internal structure and overall redistribution of forgotten-class representations.
Third, GSUO coordinates forgetting and utility recovery through the sequential SGO–CO framework. SGO guides forgotten samples toward their designated target states, while CO restores retained-data utility. For class-wise forgetting, CO additionally employs forgotten-class prediction suppression and layer-wise learning rates to preserve the representation changes achieved during SGO. Thus, retained-data fine-tuning acts as a coordinated recovery stage.
To further substantiate these distinctions, we compare GSUO with five recent methods—MU-Mis [1], MUNBa [2], SULI [3], FaLW [4], and LTD [5]—on CIFAR-10 with ResNet-18 under 10% random-subset forgetting. The results are summarized below.

MethodAcc(DrD_r) ↑Acc(DtD_t) ↑Acc(DfD_f)∣Diff∣\lvert\mathrm{Diff}\rvert ↓
MU-Mis [1]95.9480.8995.7214.83
MUNBa [2]94.5477.1678.621.46
SULI [3]72.5861.1671.069.90
FaLW [4]98.6280.3275.205.12
LTD [5]99.9982.6884.141.46
GSUO99.4782.1182.120.01

These methods adopt different unlearning mechanisms. MU-Mis [1] suppresses forgotten-sample contributions without accessing retained data; MUNBa [2] balances forgetting and retention through Nash bargaining; SULI [3] employs layered iterative unlearning; FaLW [4] uses adaptive loss reweighting; and LTD [5] relies on a teacher model trained on retained neighbors. In contrast, GSUO constructs differentiated predictive targets for Normal and Boundary samples based on their expected post-unlearning behaviors, without requiring additional teacher training.
GSUO achieves the smallest ∣Diff∣|\mathrm{Diff}| of 0.01%, compared with FaLW (5.12%), LTD(1.46%), MUNBa (1.46%), MU-Mis (14.83%), and SULI (9.90%). Although LTD achieves slightly higher accuracies, GSUO provides substantially closer alignment between forget-set and test-set accuracy while maintaining comparable utility. These results support the effectiveness of differentiated guidance signals beyond simple combinations of existing heuristics. We will incorporate these comparisons into the revised manuscript. In summary, the novelty of GSUO lies in constructing and coordinating differentiated forgetting guidance according to sample characteristics and task-specific requirements, rather than simply combining existing heuristic operations. Together with the component-level ablations, the additional comparisons provide further empirical support for the effectiveness of the proposed signal-guided optimization framework.
[1] X. Cheng et al., "Remaining-data-free machine unlearning by suppressing sample contribution," in Proc. International Conference on Learning Representations (ICLR), 2026.
[2] J. Wu and M. Harandi, "MUNBa: Machine Unlearning via Nash Bargaining," in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2025.
[3] H. Lyu et al., "Fine-Grained and Efficient Self-Unlearning with Layered Iteration," in Proc. International Joint Conference on Artificial Intelligence (IJCAI), 2025, pp. 7643–7651.
[4] L. Yu et al., "FaLW: A forgetting-aware loss reweighting for long-tailed unlearning," in Proc. International Conference on Learning Representations (ICLR), 2026.
[5] P. Dolgova and S. U. Stich, "Forgetting has neighbors: Localized collateral forgetting in machine unlearning," arXiv:2605.31317, 2026.

[W3/Q2] We would like to further clarify the relationship between prediction confidence and gradient-based forgetting difficulty, as well as the motivation for using confidence to distinguish Normal and Boundary samples. Our choice is supported by the theoretical analysis and empirical verification in Appendix A, while also considering computational complexity and unlearning efficiency.
First, prediction confidence and gradient magnitude exhibit consistent trends in characterizing sample-wise forgetting difficulty, as analyzed in Appendix A. Under our definition of relative forgetting difficulty, high-confidence Normal samples tend to exhibit smaller gradient responses and greater forgetting difficulty, whereas low-confidence Boundary samples tend to exhibit larger gradient responses and lower forgetting difficulty. This relationship provides the theoretical motivation for confidence-based sample grouping.
Importantly, this consistency is further supported by the empirical results in Appendix A. We partition forget samples into Normal and Boundary groups based on prediction confidence and compare their curvature-aware influence distributions. As shown in Table 6 and Figure 3, the mean curvature-aware influence of Boundary samples is 1007.8, compared with 19.1 for Normal samples, representing a 52.8× difference. The KS statistic of 0.954 and Cliff's δ of 0.994 further indicate a pronounced separation between the two groups. Moreover, the rankings induced by gradient magnitude and curvature-aware influence exhibit strong agreement, with a Spearman correlation of 0.997 and a Kendall correlation of 0.954 on CIFAR-10 with ResNet-18. These findings are consistent with our theoretical analysis, demonstrating that the confidence-based grouping effectively captures differences in sample-wise parameter responses.
Second, confidence-based grouping is computationally more efficient than directly computing gradient-based forgetting difficulty. Although gradient magnitude provides a useful characterization of sample-wise forgetting difficulty, calculating the full parameter gradient for each forget sample requires additional backward propagation, introducing substantial computational and memory overhead, particularly for large models and forget sets. In contrast, prediction confidence can be obtained directly from the original model's forward predictions, without requiring per-sample gradient computation. Given the theoretical relationship and empirical consistency demonstrated in Appendix A, using confidence as a practical proxy allows GSUO to distinguish samples with different forgetting characteristics at a much lower computational cost.
Third, this design is also beneficial to unlearning efficiency. By avoiding additional per-sample gradient computation during sample partitioning, GSUO reduces the computational overhead associated with identifying different types of forget samples. The overall efficiency of GSUO is further reflected in the experimental results. For example, on CIFAR-10 with ResNet-18, GSUO completes unlearning in 4.77 seconds, compared with 230.10 seconds for retraining from scratch, achieving approximately a 48× speedup. Although this speedup reflects the efficiency of the complete framework, confidence-based grouping avoids additional gradient-computation overhead and supports the efficiency-oriented design of GSUO.
In summary, prediction confidence and gradient magnitude are not identical measures, but they exhibit consistent trends in characterizing relative forgetting difficulty, as supported by both the theoretical analysis and empirical results in Appendix A. Therefore, using prediction confidence to distinguish Normal and Boundary samples is consistent with our gradient-based analysis, while offering a more computationally efficient approach to sample grouping. This enables GSUO to account for heterogeneous forgetting characteristics without introducing the additional computational cost of per-sample gradient estimation.

[W1/Q3] We would like to clarify the objective of machine unlearning and the motivation behind our evaluation metrics.
The fundamental goal of machine unlearning is to remove the influence of target training samples from a learned model while preserving its utility on the remaining data. Ideally, after unlearning, the model should behave on the forgotten samples as if they had never been included in its training data, while maintaining comparable predictive performance on the retained data distribution. For random-subset forgetting, where the forgotten samples and unseen test samples are drawn from the same underlying distribution, this means that the unlearned model should no longer exhibit a training-set advantage on the forgotten samples. Therefore, the accuracy on the forget set is expected to approach that on the unseen test set.
We formally define the post-unlearning state of the model and its constraints from the three aspects of Forgetting, Accuracy, and Efficiency:
Let DfD_f and DrD_r denote the forget set and retain set, respectively. Let Θu\Theta_u represent the parameters of the unlearned model, and Θr\Theta_r those of a model retrained from scratch using only DrD_r.

  • Forget: the influence of DfD_f does not exist in Θu\Theta_u;
  • Accuracy: the prediction capability of Θu\Theta_u on DrD_r is comparable to that of Θr\Theta_r;
  • Efficiency: the computational costs (e.g., running time) are cheaper than retraining.

While retraining from scratch is widely regarded as the gold-standard reference for machine unlearning, these objectives do not imply that the unlearned model must reproduce exactly the same accuracy values as a particular retrained model on the retain, forget, and test sets. Furthermore, generalization performance on unseen data is particularly important. Accuracy on the retained training set primarily reflects performance on previously seen samples and cannot, by itself, adequately characterize the model's generalization ability.
Regarding the evaluation metrics, we report Acc(Dr)\mathrm{Acc}(D_r), Acc(Df)\mathrm{Acc}(D_f), Acc(Dt)\mathrm{Acc}(D_t), and time, enabling direct comparisons with the Original and Retrain. In addition, we report ∣Diff∣=∣Acc(Df)−Acc(Dt)∣|\mathrm{Diff}|=|\mathrm{Acc}(D_f)-\mathrm{Acc}(D_t)|, which measures the performance difference between the forget set and the unseen test set. This metric is motivated by the fundamental objective of random-subset unlearning: after forgetting, the model should no longer exhibit an additional training-set advantage on the forgotten samples.
The relationship between our metric and retraining-based evaluation can be further illustrated mathematically. Let R(D)R(D) and U(D)U(D) denote the accuracies of the retrained and unlearned models on dataset DD, respectively. Since both the forget set and the test set are unseen by the retrained model and are drawn from the same underlying distribution, we expect R(Dt)≈R(Df)R(D_t)\approx R(D_f). Under the ideal condition that R(Dt)=R(Df)=rR(D_t)=R(D_f)=r, and considering the case where U(Dt)≤r≤U(Df)U(D_t)\leq r\leq U(D_f), the average accuracy deviation between the retrained and unlearned models on these two sets can be derived as follows:

ΔR,U=12(∣R(Dt)−U(Dt)∣+∣R(Df)−U(Df)∣)=12(U(Df)−U(Dt))=12∣Diff∣.\Delta_{R,U} = \frac{1}{2}\left(|R(D_t)-U(D_t)|+|R(D_f)-U(D_f)|\right) = \frac{1}{2}(U(D_f)-U(D_t)) = \frac{1}{2}|\mathrm{Diff}|.

This derivation establishes a direct relationship between ∣Diff∣|\mathrm{Diff}| and the average accuracy deviation from the retrained model under the stated conditions. It also provides further motivation for using the forget–test accuracy gap to evaluate unlearning effectiveness, as this metric captures whether the unlearned model exhibits comparable performance on forgotten and unseen samples, consistent with the expected behavior of retraining. In other words, its performance on the forgotten samples should be comparable to that on unseen test samples.
Furthermore, we introduce the Acc-Index, Facc=Acc(Dt)−∣Diff∣F_{\mathrm{acc}} = \mathrm{Acc}(D_t) - |\mathrm{Diff}|, which jointly considers the generalization performance of the unlearned model and its forgetting effectiveness. Specifically, Acc(Dt)\mathrm{Acc}(D_t) reflects the model's predictive performance on unseen data, while ∣Diff∣|\mathrm{Diff}| captures the residual training-set advantage on forgotten samples. For the retrained model, since ∣Diff∣|\mathrm{Diff}| is expected to approach zero, FaccF_{\mathrm{acc}} approaches Acc(Dt)\mathrm{Acc}(D_t).
As discussed above, generalization performance on unseen data is more informative for assessing model utility than accuracy on the retained training set alone. Therefore, even when comparing the unlearned model against the retrained model, it is not necessary to require identical accuracy on DrD_r. Instead, we place greater emphasis on preserving test-set performance while ensuring that forgotten samples no longer exhibit an additional training-set advantage. This provides a more comprehensive assessment of the trade-off between forgetting effectiveness and model generalization.

Reviewer_Qe4nOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GSUO, a two-step machine unlearning framework consisting of Signal-Guided Optimization (SGO) and Compensatory Optimization (CO). For random-subset forgetting, SGO divides forget samples into Normal and Boundary groups and aligns them with different target distributions—smoothed original predictions for Normal samples and the prediction distribution of the most similar retained class for Boundary samples—while CO fine-tunes on the retain set to preserve utility. For class-wise forgetting, SGO uses intra-class dispersion and alignment losses to reshape forget-class features, and CO applies positive cross-entropy on retain data and negative cross-entropy on forget data with a layer-wise learning-rate strategy. The authors evaluate GSUO on CIFAR-10, CIFAR-100, Tiny-ImageNet, and Lacuna-10 with ResNet, VGG, ViT, and AllCNN models against 14 baselines, reporting strong forgetting efficacy, high retain/test accuracy, competitive U-LiRA/MIA privacy results, and substantial speedups, supported by ablations and sensitivity analyses.

Critical strengths
  1. Task-specific guidance signals for heterogeneous forgetting difficulty. The Normal/Boundary split gives GSUO an interpretable mechanism to avoid uniform under-/over-unlearning by assigning different target distributions according to sample confidence and expected post-unlearning behavior. This is consequential because it directly addresses a core weakness of coarse-grained unlearning and makes the method conceptually motivated rather than merely heuristic.
  2. A unified two-stage formulation with broad empirical support. GSUO separates guided forgetting from utility restoration and adapts the guidance signal across forgetting regimes, giving the framework a coherent structure for balancing forgetting efficacy and model utility. Its broad evaluation across tasks, datasets, architectures, and baselines strengthens the case that this design is practically effective and generalizable.
Critical weaknesses
  1. While the paper provides an extensive evaluation, the selected baselines lack the most recent unlearning methods, making it difficult to fully contextualize the proposed method's relative advantages. Furthermore, despite the claims of consistent superiority, GSUO does not achieve the best performance across all evaluated metrics in the provided tables. This indicates that the method may not uniformly outperform existing approaches, and its overall empirical advantage is less definitive than suggested.
  2. The Retrain baseline appears consistently weak across multiple tables, achieving lower test or retain-test accuracy compared to GSUO and other baselines. Since Retrain conceptually represents the theoretical upper bound for retain-set generalization, this counterintuitive trend weakens the conclusiveness of the comparative results.
Questions for authors
  1. Given that the Retrain baseline consistently underperforms GSUO in terms of test and retain-test accuracy across multiple tables, could the authors clarify the training protocol and computational budget (e.g., number of epochs, learning rate schedule, early stopping criteria) allocated to the Retrain baseline?
  2. The current baseline comparisons primarily include methods up to 2023/2024. Could the authors include comparisons with more recent methods (e.g., 2025–2026) to better validate the relative advantages of GSUO?
  3. The paper claims that GSUO consistently achieves the best performance, yet the empirical results show that it does not dominate all evaluated metrics. Could the authors elaborate on the potential reasons for this inconsistency? Specifically, does this reflect an inherent trade-off or a limitation of the proposed framework?
3 comments

We thank the reviewer for these valuable comments.
[W2/Q1]: We appreciate the reviewer's concern regarding the performance and training protocol of the Retrain baseline. We would like to clarify that Retrain serves as the gold-standard reference for unlearning behavior rather than a theoretical upper bound on generalization accuracy, and that its relative performance compared with GSUO varies across experimental settings. We also provide the detailed training protocol and computational budget below.
First, Retrain is a reference for unlearning behavior, rather than a theoretical upper bound on test accuracy. The objective of machine unlearning is to make the unlearned model behave similarly to a model trained without the forgotten data. However, this does not imply that Retrain must achieve the highest possible generalization performance. Retrain is trained exclusively on the retain set DrD_r, whereas the original model is trained on the complete dataset DD. For example, our random-subset forgetting experiments remove approximately 10% of the training samples, corresponding to 5,000 samples on CIFAR-10 and CIFAR-100. Moreover, deep neural network training is subject to stochastic optimization trajectories and finite training budgets. More generally, in practical machine unlearning scenarios, the original model's complete training history or detailed training configurations may not be available, making it difficult to reproduce the original training procedure. In such cases, retraining may be conducted under different initialization conditions, potentially including random initialization, which can introduce additional variations in optimization trajectories and generalization performance. Although this situation does not apply to our experimental setting, it illustrates an additional source of variability in retraining outcomes and further highlights why a single Retrain realization should not be regarded as a theoretical upper bound on generalization accuracy. Consequently, GSUO achieving higher test accuracy than Retrain in certain settings is not inherently inconsistent with the objective of machine unlearning.
Second, GSUO and Retrain follow different training and optimization procedures. Retrain trains a model exclusively on the retain set DrD_r, whereas GSUO operates on the original model Θo\Theta_o, which has already been trained on the complete dataset DD. Specifically, GSUO first performs Signal-Guided Optimization (SGO) to guide forgotten samples toward their desired post-unlearning states, followed by Compensatory Optimization (CO) to restore model utility. This targeted optimization procedure allows GSUO to preserve useful task-specific representations while adjusting the model's behavior on forgotten data. Consequently, their distinct training procedures and objectives may lead to different generalization performance.
Third, the experimental results do not support the claim that Retrain consistently underperforms GSUO. As shown in Table 1, on CIFAR-100 with ViT, Retrain achieves 90.20% test accuracy, exceeding GSUO's 85.06%. On Tiny-ImageNet with ResNet-50, their test accuracies are nearly identical (78.34% versus 78.65%). Conversely, GSUO achieves higher test accuracy on CIFAR-10 with ResNet-18 (82.11% versus 80.61%) and CIFAR-100 with VGG-16 (61.30% versus 58.63%). These results demonstrate that the relative generalization performance of Retrain and GSUO varies across datasets, architectures, and forgetting configurations, rather than exhibiting a universally consistent trend.
Fourth, regarding the training protocol and computational budget of Retrain, we adopt validation-loss-based early stopping and use fixed learning rates without a learning-rate schedule. Specifically, validation loss is evaluated each epoch and training is terminated when it fails to improve for a specified patience period. The checkpoint with the highest validation accuracy is selected as the final model. For CIFAR-10 with ResNet-18, we use a learning rate of 0.001 and a patience of 20 epochs. Validation loss reaches its minimum at epoch 2, and training stops at epoch 22. For CIFAR-100 with VGG-16, we use a learning rate of 0.0001 and a patience of 15 epochs, with training stopping at epoch 18 after the minimum validation loss is reached at epoch 3. For CIFAR-100 with ViT, we use a learning rate of 0.0001 and a patience of 15 epochs, with training stopping at epoch 17 after the minimum validation loss is reached at epoch 2. For Tiny-ImageNet with ResNet-50, we use a learning rate of 0.0001 and a maximum training budget of 20 epochs; early stopping is not triggered, and all 20 epochs are completed.
In summary, differences in training data and optimization procedures can lead to different generalization outcomes, and our experimental results do not indicate that Retrain consistently underperforms GSUO. We will provide its detailed training protocol in the revised manuscript.

[W1/Q2]:We agree that comparisons with more recent machine unlearning methods are important for evaluating the relative advantages of GSUO. To provide a more comprehensive evaluation, we have added five recent methods—MU-Mis [1], MUNBa [2], SULI [3], FaLW [4], and LTD [5]—as additional baselines and evaluated them on CIFAR-10 with ResNet-18 under the 10% random-subset forgetting setting. The results are summarized below.

MethodAcc(DrD_r) ↑Acc(DtD_t) ↑Acc(DfD_f)∣Diff∣\lvert\mathrm{Diff}\rvert ↓
MU-Mis [1]95.9480.8995.7214.83
MUNBa [2]94.5477.1678.621.46
SULI [3]72.5861.1671.069.90
FaLW [4]98.6280.3275.205.12
LTD [5]99.9982.6884.141.46
GSUO99.4782.1182.120.01

The results show that GSUO achieves the smallest ∣Diff∣|\mathrm{Diff}| of 0.01%, compared with 14.83% for MU-Mis, 1.46% for MUNBa, 9.90% for SULI, 5.12% for FaLW, and 1.46% for LTD. While LTD achieves slightly higher retain and test accuracies, GSUO achieves more desirable unlearning effects while maintaining comparable model utility. Unlike FaLW's adaptive loss reweighting and LTD's local teacher distillation, GSUO explicitly constructs differentiated guidance signals for Normal and Boundary samples without requiring additional teacher training.
Overall, these additional comparisons further support the effectiveness of GSUO in balancing forgetting efficacy and model utility.

[1] X. Cheng, Z. Huang, W. Zhou, Z. He, R. Yang, Y. Wu, and X. Huang, "Remaining-data-free machine unlearning by suppressing sample contribution," in Proc. International Conference on Learning Representations (ICLR), 2026.
[2] J. Wu and M. Harandi, "MUNBa: Machine Unlearning via Nash Bargaining," in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2025.
[3] H. Lyu, X. Zhang, H. Hu, S. Wang, C. He, and L. Qi, "Fine-Grained and Efficient Self-Unlearning with Layered Iteration," in Proc. International Joint Conference on Artificial Intelligence (IJCAI), 2025, pp. 7643–7651.
[4] L. Yu, Z. Zhao, Y. Wang, P. Wang, X. Cao, B. Wang, and Y. Wang, "FaLW: A forgetting-aware loss reweighting for long-tailed unlearning," in Proc. International Conference on Learning Representations (ICLR), 2026.
[5] P. Dolgova and S. U. Stich, "Forgetting has neighbors: Localized collateral forgetting in machine unlearning," arXiv:2605.31317, 2026.

[W2/Q3]: We would like to clarify that the superior performance of GSUO refers to its overall effectiveness in balancing forgetting quality, model utility, and computational efficiency, rather than achieving the best result on every individual metric.
First, machine unlearning inherently involves multiple competing objectives, and optimizing one metric does not necessarily lead to optimal performance on others. Therefore, we evaluate unlearning methods from a comprehensive perspective, with particular emphasis on forgetting effectiveness. For random-subset forgetting, GSUO consistently achieves the smallest ∣Diff∣|\mathrm{Diff}| among the evaluated approximate unlearning methods across different datasets and model architectures, while maintaining competitive test accuracy and computational efficiency. For class-wise forgetting, GSUO achieves effective forgetting while preserving strong retained-class performance. These results demonstrate the effectiveness of our guidance signal design in balancing forgetting and model utility.
Second, the Pareto trade-off analysis in Figures 2 and 4 further supports the overall advantages of GSUO. We introduce the Acc-Index, Facc=Acc(Dt)−∣Diff∣F_{\mathrm{acc}}=\mathrm{Acc}(D_t)-|\mathrm{Diff}|, to jointly measure forgetting effectiveness and generalization, and evaluate it against computational speedup relative to retraining. The results show that GSUO achieves a favorable position in the empirical trade-off between unlearning quality and efficiency, rather than prioritizing either objective in isolation. This demonstrates that the performance advantages of GSUO should be assessed comprehensively rather than through individual metrics alone.
In summary, the observed variations across individual metrics primarily reflect the inherent trade-offs among the objectives of machine unlearning, rather than an inconsistency in the effectiveness of GSUO. Although further improvements remain possible in certain settings, GSUO consistently demonstrates strong forgetting effectiveness and a favorable overall balance among unlearning quality, model utility, and computational efficiency. We will clarify that the claim of superior performance refers to this comprehensive evaluation rather than dominance across every metric.

Dear Reviewer Qe4n,
We would like to follow up on our response to your review. We have provided detailed clarifications on the Retrain baseline's training protocol and computational budget, added comparisons with five recent unlearning methods (2025–2026), and further explained the trade-offs among forgetting effectiveness, model utility, and computational efficiency.
In particular, the additional comparisons show that GSUO achieves the smallest |Diff| (0.01%) among the evaluated methods, supporting its effectiveness in balancing forgetting and model utility.
When you have a chance, we would appreciate your thoughts on whether these additional results and clarifications address your concerns. We would be happy to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.