acceptodds
Under review as a conference paper at ICLR 2027

Label Smoothing Improves Gradient Ascent in LLM Unlearning

Abstract

LLM unlearning has emerged as a promising approach, aiming to enable models to forget hazardous/undesired knowledge at low cost while preserving as much model utility as possible. Among existing techniques, the most straightforward method is performing Gradient Ascent (GA) w.r.t. the forget data, thereby forcing the model to unlearn the forget dataset. However, GA suffers from severe instability, as it drives updates in a divergent direction, often resulting in drastically degraded model utility. To address this issue, we propose Smoothed Gradient Ascent (SGA). Theoretically, we analyze how the smoothing rate affects the update direction and derive a local condition for reducing its component along the pure GA direction. Intuitively, this extends GA by combining gradient contributions from forget and normal data, with the smoothing rate controlling their relative weights. Empirically, we evaluate SGA on three benchmarks: TOFU, Harry Potter, and MUSE-NEWS. Experimental results show that, with an appropriate smoothing rate, SGA can improve the forgetting–utility trade-off over GA and achieve competitive performance against existing baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Unlearning, Label Smoothing, Responsible AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XzrqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Smoothing Gradient Ascent (SGA), which adapts label smoothing to LLM unlearning by incorporating normal data into the forgetting objective. The authors provide a gradient-based analysis of the smoothing mechanism and evaluate SGA on three unlearning benchmarks (TOFU, Harry Potter, and MUSE-News) across multiple LLMs. The results demonstrate that SGA can improve forgetting effectiveness and model utility under appropriately selected smoothing rates.

Critical strengths
  1. Clear presentation and meaningful methodological exploration. The paper is well written and presents a reasonable adaptation of label smoothing to LLM unlearning. The gradient-based analysis provides useful insights into how smoothing may influence the forgetting process.

  2. Comprehensive experimental evaluation. The authors evaluate SGA across multiple models and unlearning tasks, comparing it against several established baselines. The results demonstrate the potential effectiveness of the proposed approach.

Critical weaknesses
  1. Strong sensitivity to the smoothing rate. The effectiveness of SGA depends heavily on the choice of r, with substantially different preferred values across models and tasks. For example, Tables 1 and 2 show that positive smoothing rates can be effective in some settings, whereas negative rates are preferred in others. Moreover, SGA does not consistently outperform competing methods across all evaluation metrics. These observations make its overall advantage less conclusive.

  2. Unclear parameter selection and generalizability. Although the authors provide a gradient-based analysis, the paper lacks a practical criterion for selecting the sign and magnitude of r. More importantly, it remains unclear whether a smoothing rate tuned on one forget set can generalize to unseen forget sets. Since real-world unlearning requests can vary substantially, requiring extensive tuning for each new forget set would significantly limit the method’s practical applicability.

Therefore, my recommendation is Weak Reject, but I am open to reconsideration if the rebuttal provides convincing explanations.

Questions for authors

Q1. Can the authors provide a practical criterion for selecting the sign and magnitude of r without extensive hyperparameter tuning? In particular, can the gradient-based analysis in Section 4.3 and Figure 2 be used to guide this selection?

Q2. Can a smoothing rate selected on one forget set generalize to other unseen forget sets? For example, how does SGA perform when using a fixed r across different forget sets without additional tuning?

Q3. Given the variations across metrics in Table 1, can the authors clarify whether SGA consistently provides a better forgetting–utility trade-off than strong baselines such as DPO-RT, rather than outperforming them only on selected metrics or under carefully tuned settings?

5 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer Xzrq

We thank you for your time and valuable feedback. Our responses to your comments are provided below:

Weakness 1. Strong sensitivity to the smoothing rate

Please refer to “Response to Common Concerns Raised by Reviewers: 1. Sensitivity to the Smoothing Rate.”


Weakness 2. Unclear parameter selection and generalizability

Referring to Equation (1) in Section 3.1, “Formulation,” the baselines we consider and our motivation are based on the assumption that achieving forgetting on the forget set fulfills the unlearning objective. Regarding the reviewer’s concern, SGA’s use of an external LLM to generate specifically tailored, semantically similar normal data gives it an advantage in generalizability over the baselines discussed in this paper. During training, the model simultaneously forgets the forget data and learns from highly similar normal data, where the key target may remain unchanged while the content differs. For example: “Jaime Vasquez was born on the 17th of April in the year 1962” and “Jaime Vasquez was born on the 8th of March in the year 1955.” By learning to distinguish between the forget data and the normal data, the model learns modified knowledge associated with the specific unlearning target.

The reviewer raises a meaningful question about whether a smoothing rate tuned on one forget set can generalize to new forget sets. However, this work focuses on unlearning with a given forget set. In future work, we may explore ideas from continual learning to address this issue.


Question

1. Please refer to Weakness1.

2 and 3. Appendix D provides the detailed hyperparameter configurations for each experiment. We would also like to clarify that SGA uses exactly the same experimental settings, including all shared hyperparameters, as the baselines in the corresponding experiments. The only difference is the method-specific smoothing rate rr, which can be computed using the criterion in Section 4.3. Upon acceptance, we will release the complete experimental code to ensure reproducibility.


We look forward to your response and would be happy to discuss any aspect of the paper with you.

Reviewer_XzrqOpenReview ↗

Maintain rejection

Thank you for the clarification. I acknowledge that smoothing can improve the forgetting–utility trade-off with an appropriately chosen rate. However,

  1. the response does not resolve my main concern:
  • how to select the sign and magnitude of rr without relying on final unlearning metrics, or
  • whether a rate selected on one forget set remains effective on unseen forget sets. The local geometric analysis provides useful insight, but does not establish such a selection criterion or demonstrate its predictive validity.
  1. I do not expect a single rate to work universally across models and datasets. However, evidence of transfer across forget sets within a fixed model would be important for establishing practical applicability. At present, the results support a promising empirical finding under suitable tuning, while the reliability and reusability of that tuning remain unverified.

I therefore maintain my weak reject recommendation.

In Section 4.3, we provide a criterion for estimating the range of suitable smoothing rates rr. Applying this criterion to TOFU, we estimate that r>0r > 0 is suitable for Llama2-7B and that r<0r < 0 is suitable for OPT-2.7B and Phi-1.5B. We tested additional values of rr within the recommended range for each model and observed consistently good performance across the tested values. We report these additional results in Appendix E, and they are broadly consistent with our analysis in Section 4.3.

Llama2-7B (Recommended r<0r<0)OPT-2.7B (Recommended r>0r>0)Phi-1.5B (Recommended r>0r>0)
MethodFQ ↑MU ↑F-RL ↓R-RL ↑MethodFQ ↑MU ↑F-RL ↓R-RL ↑MethodFQ ↑MU ↑F-RL ↓R-RL ↑
GA (r=0r=0)0.00680.59900.48170.9204GA (r=0r=0)0.05410.48510.44900.6440GA (r=0r=0)0.05410.50580.49140.8012
SGA (r=0.2r=0.2)0.00680.60180.47770.9229SGA (r=−0.2r=-0.2)0.09710.48510.39730.6148SGA (r=−0.2r=-0.2)0.09710.51000.48310.8062
SGA (r=0.4r=0.4)0.00680.60270.47920.9163SGA (r=−0.4r=-0.4)0.09710.48590.40210.6939SGA (r=−0.4r=-0.4)0.09710.50930.48510.8009
SGA (r=−2r=-2)0.26570.47840.39590.6012SGA (r=−2r=-2)0.09710.51200.48160.8043
SGA (r=−4r=-4)0.40460.47470.38630.5890SGA (r=−4r=-4)0.09710.50950.48910.7953

We hope these results address your concern. Thank you for taking the time to review our work and for your thoughtful feedback.

Reviewer_aCQzOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work explores label smoothing to improve the training stability of gradient ascent (GA) in LLM unlearning, which is vulnerable to unbounded gradients and weights that in turn cause network divergence, severely degrading model utility. The authors present a smoothed unlearning objective by incorporating gradients from closely related examples from a "normal" set, which is aimed at mitigating the instability caused by unbounded gradients in GA. For each example in the forget set, authors carefully curate K-1 examples for the normal set which are semantically similar or safe template responses.

Critical strengths
  1. Soundness

    • The idea of smoothing unbounded gradients in GA is reasonably well motivated and addresses an important failure point in unlearning.
    • The two terms are balanced using a smoothing rate parameter r, for which the authors present a local geometric condition, which can potentially be extended to adapt r for each example.
  2. Experiments

    • Authors experiment with 3 open source models and 3 distinct unlearning benchmarks, providing reasonable coverage.
  3. Results

    • The presented results show promise for specific values of the smoothing rate parameter r.
Critical weaknesses
  1. Novelty

    • While the motivation for gradient smoothing is reasonable, the objective presented here is structurally very similar to the well known Gradient Difference (GD) objective, which similarly combines retain and forget loss terms with independently tunable weights.
    • The authors include GD as one of their baselines, but appear to use the default data mixes offered by the benchmarks, while their SGA objective uses carefully curated "normal data" with semantically similar examples, held at a 1:K-1 ratio.
    • It is unclear if applying this this same curated normal set as the retain set in vanilla GD, with carefully tuned weighting terms would perform similarly to their proposed SGA objective.
  2. Mixed Results

    • The presented results are quite mixed: Table 1 shows SGA performing similarly to GD and other prior baselines on Llama2-7B.
    • In Table 2, perplexity increases significantly for SGA in both models, far more than baselines other than GA. For OPT, SGA with r=-0.2 leads to perplexity even higher than GA.
    • Similarly in Table 3, the KnowMem metric on the retain set shows a major drop with SGA, suggesting substantial utility degradation.
  3. Unverified baselines

    • A number of baselines reported in this draft appear to be quoted directly from Wang et al.
    • This raises significant concerns about the validity of these findings, since it is unclear whether all other conditions are held identical to draw a fair comparison.
Questions for authors

None

2 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer aCQz

We thank you for your time and valuable feedback. Our responses to your comments are provided below:

1. Novelty

  1. In terms of motivation, SGA aims to shift the model’s learning direction when predicting the next-token distribution relative to GA, thereby avoiding substantial degradation in model utility. From this perspective, both SGA and GD are improvements upon GA.

  2. A core contribution of SGA is to select normal data that are as semantically similar as possible to the forget data, with the aim of avoiding large differences between their gradient directions during label smoothing. GD does not emphasize this gradient correction for each individual forget example.


2. Mixed Results

  1. In Section 5.5, we reran the TOFU experiments using normal data generated by an external LLM. As shown in Table 4, SGA’s performance improved significantly.

  2. The effectiveness of SGA depends on an appropriate choice of the smoothing rate rr. We analyze this choice in Section 4.3 and validate the analysis in Figure 2. Selecting rr within the range suggested by our analysis enables SGA to perform effectively.

Regarding the overall experimental results, SGA builds on GA and improves its stability through label smoothing. It uses tailored normal data rather than retain data, making it more aligned with practical application scenarios. However, owing to the inherent limitations of GA-based methods, SGA does not achieve state-of-the-art performance across all aspects.


3. Unverified Baselines

We detail the settings for each experiment in Appendix D, “Experimental Setup.” Since our work follows Wang et al. [1], these settings are largely consistent with their hyperparameter configurations, and some baseline results are consistent with those reported in prior work. Upon acceptance, we will release all code used in this paper to ensure the reproducibility of our results and the fairness of the comparisons.

We also provide additional FLAT results across different random seeds. Although these differ from those reported in the original FLAT paper, their consistency across seeds supports the reliability of our experimental results.

SeedFQ ↑MU ↑R-RL ↑F-RL ↓
400.00680.62410.91800.4384
1230.00300.62870.89040.4394
2340.00680.62550.89350.4227
3450.01430.62120.90700.4183
420.00300.63040.92840.4825

We look forward to your response and would be happy to discuss any aspect of the paper with you.

[1] LLM Unlearning via Loss Adjustment with Only Forget Data, ICLR2025

Dear Reviewer,

The authors have kindly provided detailed responses to your comments. We would appreciate it if you could take a moment to review their rebuttal and share any further feedback or updates to your evaluation.

Thank you for your time and thoughtful consideration.

Reviewer_be8qOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to reduce the instability and utility collapse of GA by constructing semantically related normal examples and then combining the forget and normal losses using a smoothing rate r, named as SGA. The methods is evaluated on various unlearning benchmarks and SGA exhibits an improvement on the forgetting–utility trade-off over GA and competitive performance against existing baselines.

Critical strengths
  1. The method is simple and does make GA less prone to immediate collapse across several models and benchmarks.
  2. The manuscript is well written and easy to follow, and the authors evaluated across extensive benchmarks and perform a comprehensive experiments to demonstrate the effectiveness of SGA.
Critical weaknesses
  1. The work stabilizes a questionable objective. A retrained model need not have high loss on the forget examples, since the same knowledge may come from pretraining or correlated retain data. Making GA more stable therefore does not by itself move the model toward the correct retraining endpoint. Could the author further clarify this motivation ?
  2. In the case of r<0, Eq.3 is a weighted GA on forget data and a descent on normal data. I think this paper is not really towards forget-only unlearning regime, rather, it preserves model utility by generating some data, like SCAR [1] does, which introduces OOD data as a surrogate for retain data. Therefore, I am afraid the proposed SGA is more like introducing more retain data rather than moving towards true forget-only unlearning. This authors should add comparisons with baselines that utilize these custom normal data.

[1] Is retain set all you need in machine unlearning? restoring performance of unlearned models with out-of-distribution images, ECCV, 2024.

Questions for authors
  1. Whether r could be selected without access to final unlearning metrics? Can the proposed local statistic predict its sign and magnitude before training?
1 comment

Dear Reviewer,

The authors have kindly provided detailed responses to your comments. We would appreciate it if you could take a moment to review their rebuttal and share any further feedback or updates to your evaluation.

Thank you for your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.