acceptodds
Under review as a conference paper at ICLR 2027

Unlearning Can Hurt When There Is Nothing to Forget

Abstract

Unlearning is meant to remove one piece of knowledge from a language model and leave the rest intact. Its side effects are measured by how much the model's scores on other topics drop, and the whole drop is charged to the removal. We show that for two widely used methods, RMU and NPO, most of that drop does not come from removing that knowledge. We fine-tune two copies of a model the same way, one on the documents to be forgotten and one on unrelated documents, and give both the identical unlearning run. If the damage came from removing what the model learned, the copy that never learned it would be spared. Under RMU it is not: across three datasets and fictitious authors, five models from four families, 14 fine-tuning seeds and four measures of damage, it loses about as much as the copy that did, including on the closely related topics where damage is largest, and unlearning statements a model never saw costs as much as unlearning what it knows from pretraining. At least 74% of RMU's collateral damage on MMLU, and over 90% on the other two datasets, occurs without the model having learned what was removed, and NPO on MMLU, at the same amount of damage, behaves like RMU. The control detects dependence where it exists: under GradDiff and SimNPO, a copy that learned other documents about the forget topic is damaged less than the taught copy, about half as much under GradDiff, and damage aimed at chosen topics, by gradient ascent or by RMU itself, is detected in nearly every draw. Unlearning evaluations should report collateral damage next to the damage the same run does to a model that never learned the forget documents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, large language models, unlearning evaluation, collateral damage, experimental control, RMU, GradDiff, NPO, SimNPO

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_4H7zOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies whether collateral damage from LLM unlearning depends on the model having learned the forget documents. It compares two fine-tuned copies of a model: one trained on the forget documents and another trained on different documents. Both undergo the same unlearning procedure. The paper reports similar damage under RMU and, in some settings, NPO, while GradDiff and SimNPO show greater dependence on prior learning. It recommends including this control in unlearning evaluations.

Critical strengths
  1. The proposed control addresses a reasonable evaluation question. Applying unlearning to a model that never learned the target documents can reveal side effects that a standard before-and-after comparison cannot separate. Matching the fine-tuning recipe also helps account for the difference between pretrained and fine-tuned models.
  2. The RMU observation is tested across several models, datasets, and evaluation measures. The authors also examine alternative pairings, neutral fine-tuning data, and generated answers. These checks make the reported pattern more credible within the studied settings, even though they do not fully support the broader interpretation.
Critical weaknesses
  1. The novelty is limited. The basic finding that an unlearning update can damage a model even when the target documents were never learned is unsurprising. RMU pushes activations toward a random target without checking whether the input was memorized. More importantly, Section 2 already discusses closely related observations in Kim et al. and SALMUBench. The paired design offers a different measurement, but the paper does not establish a substantial advance in understanding why the effect occurs or when it should disappear. The three contributions largely amount to one control and two sets of observations obtained with it. Extensive repetition strengthens confidence in an observation, but does not by itself create a deeper contribution.
  2. The central comparison does not identify the quantity claimed in the paper’s framing. The difference between taught and untaught damage measures the net effect of prior fine-tuning on the outcome of unlearning. It does not separate damage caused by removing knowledge from changes in the model’s general susceptibility to an update. These effects could offset each other, so equal total damage does not establish that knowledge removal contributed little to the taught model’s damage. Appendix A explicitly acknowledges this limitation, but the abstract still states that most damage does not come from removing knowledge. The untaught model also retains pretrained knowledge about the forget topic; it lacks the newly taught false statements, not all related knowledge. The TOFU check helps with this distinction, but does not resolve the causal interpretation.
  3. The claimed separation between methods is highly dependent on the control design and metric. In Appendix H, GradDiff’s log-probability damage ratio is around one half in the split-topic design, but rises to 2.55 at 80 steps in the main design. At 160 and 320 steps, its main-design intervals include one. The same appendix states that multiple-choice accuracy does not separate the methods in the split-topic setting. On WMDP, GradDiff’s ratio is 0.29 in log-probability but 1.40 on the contrastive score, reversing the conclusion about which copy is harmed more. These are not minor numerical variations. They show that the result is not yet a stable property of the unlearning method. The paper needs to explain these reversals rather than mainly report them.
  4. Much of the evidence comes from settings with degraded generation, making the practical interpretation unclear. Appendix E reports that removing every run where either copy produces degenerate output leaves only 40 of 224 MMLU runs. In those remaining runs, damage is too small to estimate the accuracy ratio reliably. The nondegenerate results on other datasets provide some support, but do not establish the headline quantitative conclusion for useful unlearning settings. The cross-method comparison has a related problem: GradDiff causes roughly five to eight times RMU’s log-probability damage on MedMCQA and WMDP, while the damage-matched RMU–GradDiff test on MMLU is inconclusive. Matching damage after running the experiments also does not ensure comparable forgetting. Results at independently selected settings that achieve meaningful forgetting while preserving usable generation would be more informative than pooling a large sweep.
  5. The presentation is poor and unnecessarily repetitive. Section 5 repeatedly lists ratios, intervals, run counts, and additional experiments, with little explanation connecting them. Important qualifications are scattered across long appendices, including the failure to separate methods under accuracy and the uncertainty in the damage-matched comparison. The extensive seed-by-seed reporting adds further length without resolving these central questions. Repeated runs are useful for checking stability, but their results should be consolidated rather than repeatedly presented as new evidence. Overall, the manuscript reads more like an experiment log than a focused scientific argument. A much shorter presentation should distinguish the robust RMU observation, the setting-dependent findings, and the conclusions that remain unsupported.
Questions for authors
  1. What assumptions allow the taught–untaught damage difference to separate harm caused by knowledge removal from changes in general susceptibility to unlearning? How does the interpretation in the abstract follow despite the limitation acknowledged in Appendix A?
  2. Why does GradDiff’s result change substantially between the main and split-topic designs, and sometimes reverse across metrics? Which conclusion about the method survives these changes?
  3. Can the authors establish the main quantitative claims at independently selected operating points with meaningful forgetting and nondegenerate generation? Do the differences between methods persist at comparable forgetting levels within the same control design?
  4. Beyond quantifying an already documented failure mode, what new explanation or prediction does this control provide? Can the authors demonstrate a concrete case where it changes a conclusion that existing evaluations would otherwise get wrong?
Reviewer_Nv2nOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines whether collateral damage in LLM unlearning depends on prior learning of the forget documents. It compares two model copies fine-tuned with the same recipe on different content: one learns the forget documents, while the other does not. Both undergo unlearning with the same documents and settings. Across multiple models, datasets, and metrics, RMU causes similar overall damage to both copies. NPO shows a similar pattern in log-probability damage on MMLU at a damage level matched to GradDiff. When the copies learn separate document sets about the same topic, GradDiff and SimNPO damage the copy trained on the forget documents more on some metrics, suggesting that this dependence varies across methods and evaluation settings. The paper recommends reporting damage to both copies, together with their difference and ratio, to help distinguish overall performance loss from its dependence on prior learning of the targeted documents.

Critical strengths
  1. A useful control design for evaluating collateral damage. This paper provides a practical way to assess whether collateral damage depends on prior learning by applying the same unlearning procedure to a model that never learned the forget documents.
  2. Strong experimental support for the main RMU finding. The authors test their main RMU finding across multiple models, datasets, fine-tuning seeds, and evaluation metrics, consistently showing similar overall damage in both copies. They also perform additional checks, such as changing topic pairings, using neutral fine-tuning, and excluding held-out topics with degenerate generations. These experiments make the main finding more convincing.
Critical weaknesses
  1. It is unclear whether collateral damage mainly comes from knowledge removal. Appendix A explains that the comparison measures the overall effect of having learned the forget documents, but cannot tell whether the damage comes from removing knowledge or from changes in the model's sensitivity to other damage. However, the abstract claims that most collateral damage does not come from knowledge removal. The similar damage in both copies shows that prior learning has little effect on the overall damage, but does not directly prove what causes it. The authors should conduct further experiments to verify this claim and better understand the source of the damage.

  2. GradDiff results are inconsistent across experiments. The finding that GradDiff causes more damage to the taught copy depends on the experimental design and evaluation metric. The split-topic experiments support this finding using log-probability and some generated-answer metrics, but the results are less consistent in the main MMLU design and with multiple-choice accuracy. The WMDP contrastive-score result even shows the opposite trend. Although the authors discuss these limitations, they should be made clearer in the main conclusions.

Questions for authors
  1. How can the authors tell whether the damage comes from knowledge removal? Appendix A states that the comparison only measures the overall effect of prior learning. Are there any additional assumptions that would justify concluding that most of RMU's collateral damage does not come from removing the learned target knowledge?

  2. The split-topic experiment shows a clearer difference between the taught and untaught copies, but it also changes the training data and forget documents. How can the authors show that the difference comes from what the untaught copy learned rather than these other changes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.