acceptodds
Under review as a conference paper at ICLR 2027

Unlearning Can Remember the Test and Forget the Rest

Abstract

Unlearning should remove specific facts from a model and leave everything else intact. Evaluations check the second part with a retain set: facts the model should still know afterwards. But many unlearning methods train on that same retain set. A method could therefore damage much of what the model knows, restore exactly the facts that will be tested, and look nearly as good as retraining from scratch. We show that this happens. We train language models of 0.6B to 1.24B parameters from two families on synthetic biographies, so we can also test people the model knows but no method targets or trains on. A method that resets parameters and then fine-tunes on the retain set can remove the targets while keeping retain accuracy high, yet in all 17 configurations of our main setting these untouched people lose 44 to 104% as much accuracy as the targets; where we retrained a matched model without the targets, the untouched people lose only 6 to 15% as much. The loss appears in both model families and equally for untouched facts whose questions were never trained. Prompting with their own biographies does not bring most of their facts back. The loss persists even when the untouched people are learned exactly like the targets, and only people in the retain set are spared. On the TOFU benchmark, a standard method, gradient difference with a strong retain weight, shows the same pattern: retain questions it trains on stay at the retrained level, while untouched questions of the same kind fall 20 points below it. A retain score therefore cannot show that unlearning was selective, and evaluations should also test known facts that no method trains on.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, LLM unlearning, unlearning evaluation, retain set, collateral damage, holdout evaluation, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zZ17OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper conducted a series of experiments to test the limitations of existing benchmarks and their evaluation methods. A typical benchmark setting contains a set of target facts to remove (target) and a set of facts that need to be preserved (retain). However, as many unlearning methods retrain the model on the “retrain set” to preserve or restore the knowledge of the facts in the retain set, certain facts that are not in the target set might have been forgotten as collateral damage in the original unlearning process. As a result, the authors argue that reporting the effectiveness of an unlearning method on these two sets is not sufficient, and should report the “untargeted-holdout drop” against matched retraining. In addition to validating this core argument, the authors also show that on the widely used benchmark, TOFU, one of the “standard” methods (Gradient Difference) expresses the same issue of forgetting untargeted facts.

Critical strengths
  • The problem is clearly defined, with an operational test of "selectivity": forget the targets, keep the retain set, and damage untargeted facts no more than retraining would.
  • The authors conducted a great number of experiments testing different configurations of unlearning methods, using their synthetic data and a number of public benchmarks.
Critical weaknesses
  • The presentation of this paper is far from ideal. For example, the structure and flow make it difficult to follow the important settings and the significance of the results of each experiment. Some terminology is introduced before formally defining it (e.g., Forget gap, Protocol U) and the use is not always consistent.
  • The main conclusion, while interesting, is not too surprising. Some aspects of the key message have also been reported previously. For example, Peleg et al. (2026) highlights the “over-forgetting” issue and Ko et al. (2025) reports that unlearned models fail up to 98.7% of test cases that were answerable by the original pretrained models.
Questions for authors
  • It appears that the paper only uses the ROUGE-L score from TOFU experiments. What do other metrics proposed in TOFU show?
  • How practical is the proposed evaluation protocol (Section 6)? For example, if the targeted forget set of facts exists in pretraining data, does this mean that the model pretraining is needed?
Reviewer_qDirOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper's main contribution is showing that typical reset-and-repair unlearning methods can forget knowledge that is neither in the unlearn set nor the retain set, despite matching retraining performance on the retain set and unlearn set. Consequently, existing unlearning evaluations that focus solely on the unlearn set and the retain set (which can be used for training) largely overlook the method's negative impact on useful knowledge outside the retain set. As a solution, the authors propose to add another held-out "untargeted" evaluation set that cannot be used for training, containing knowledge that is neither in the retain set nor the unlearn set. To support this observation, the authors performed experiments for standard gradient-based unlearning methods on transformer models trained on synthetic wiki biographies, Llama-2-7B fine-tuned on the real MUSE-News corpus, and TOFU.

Critical strengths
  • The paper's main contribution is pointing out a flaw in the existing unlearning evaluation that solely focus on a retain set and an unlearn set, and systematically showing this phenomenon across a range of benchmarks and unlearning methods. The proposed conceptual fix is clear, and requires evaluating the performance change on a third untouched set, and small changes imply a more desirable unlearning method.
Critical weaknesses
  • Novelty: Evaluating the unintended effects of unlearning on data outside the forget and retain sets is not new. For example, TOFU (Maini et al., 2024) explicitly introduces two additional evaluation sets, Real Authors and World Facts, to measure collateral effects on knowledge not included in either the forget or retain set. Similarly, the NeurIPS 2023 Machine Unlearning Challenge evaluates utility on held-out test data in addition to the retain set. Thus, the main novelty appears to lie in demonstrating the severity of this issue across methods and benchmarks, rather than in the proposed three-set evaluation framework itself. However, an important class of unlearning methods is missing from this empirical investigation, as discussed below.

  • Missing evaluation and discussion of provable unlearning methods: The paper overlooks methods that provide formal guarantees of closeness between the unlearned model and a hypothetical retrained model, such as Zhang et al. (ICML 2024) and Koloskova et al. (ICML 2025). Such guarantees, when sufficiently strong, constrain the behavior of the unlearned model on any evaluation set, including the proposed untargeted set, relative to the retrained model. Therefore, the evaluation issue identified in this paper may be less problematic for these methods. I encourage the authors to include suitable certified-unlearning methods in their experiments, or at least discuss whether they suffer from the same issue.

  • Soundness: The paper proposes evaluating performance on a third, untargeted set containing knowledge that belongs to neither the retain set nor the forget set, and considers an unlearning method desirable only if performance on the untargeted set remains nearly unchanged. However, this criterion could itself be misleading, as the expected performance change should depend on the relationship between the untargeted knowledge and the forget and retain sets.

On the one hand, if the untargeted knowledge has little semantic similarity or topical relevance to the retain and forget sets, its performance may change very little after unlearning, making it difficult to reveal the impact on the most affected untouched knowledge. On the other hand, if the untargeted knowledge is closely related to the retain or forget set, maintaining unchanged performance after unlearning may not always be possible or even desirable.

For example, suppose the model is asked to forget the fact that A is B's father while retaining the fact that B is C's father. If the untargeted set contains the fact that A is C's grandfather, its performance need not remain unaffected by unlearning, as this fact is logically dependent on the forgotten knowledge.

The paper largely overlooks the subtle issue of designing an appropriate untargeted set and determining what constitutes desirable performance on that set, particularly given its potential relationships with the retain and forget sets. In particular, it is unclear why minimizing performance changes relative to the original model should be the preferred criterion, rather than matching the behavior of a model retrained without the forget set.

Questions for authors
  • Does provably unlearning methods suffer from this evaluation issue as well? This would be helpful for understanding the scope of this evaluation issue, and whether achieving unlearning guarantee is an alternative way to addressing this evaluation issue.

  • How to design a valid untargeted set? Specifically, how similar should it be to the retain/unlearn set in topics? How to ensure it does not contain any knowledge that are in the retain/unlearn set? How to take the knowledge correlation into account when setting the success threshold for untargeted set?

Reviewer_xc1nOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper argues that standard unlearning evaluations cannot detect collateral damage, because many unlearning methods train on the same retain set that is used for evaluation. To show this, the authors construct reset-and-repair, which resets parameters selected by a Fisher-based score to their pretrained values and then fine-tunes on the retain set. In controlled experiments on synthetic biographies (0.6B–1.24B parameters), this method removes the targets and keeps retain accuracy high, while people outside both the forget and retain sets (the “untargeted holdout”) lose much of their accuracy. The authors recommend that evaluations score such a holdout against a retrained model.

Critical strengths

The paper studies an interesting and important problem: how to detect collateral damage from unlearning, i.e., knowledge the model loses beyond the targets. It shows that current methods offer only a partial fix, restoring the retain set they train on while leaving other knowledge damaged.

Critical weaknesses
  1. Lack of Clarity. The paper is very hard to follow. Even the list of contributions is hard to understand, because it relies on terms that have not yet been introduced, such as Inserted, Mixed, Two-stage, and Protocol U. Many sentences also combine several separate results; for example, contribution 2 packs seven findings from different experiments into one sentence. Overall, the reader has to jump to tables and appendices to understand the main claims. The paper needs substantial rewriting so that each claim is clear on its own.

  2. Limited novelty. The paper’s central observation is that a retain score can hide damage, because methods that train on the retain set naturally perform well on it. This effect is already known and addressed in literature. MUSE (Shi et al., ICLR 2025) uses one retain set for regularizing methods and a separate, disjoint retain set for evaluation, precisely so that methods cannot optimize toward the data they are tested on. Thaker et al. (SaTML 2025) argue that unlearning benchmarks need held-out queries, because evaluating on the queries used during unlearning overstates progress. Peleg et al. (2026) show the same effect directly: when retain performance is measured on the training queries, over-forgetting is hidden, and methods that train on the retain set suffer most on held-out retain questions. Methods that use no retain set at all, such as gradient ascent, NPO without a retain term, or task vectors, report retain scores on data they never train on by construction.

  3. The method does not address the holdout. Reset-and-repair is built from existing components (PerTA’s per-parameter scores, Fisher-based resetting, and fine-tuning on the retain set) and is designed only to protect the retain data.

  4. Small models. The results use small models (0.6B–1.24B) on synthetic biographies. Scaling to Qwen3-1.7B already fails (Appendix D.6), and the 7B check has no holdout. The authors should test larger models in standard fine-tuning setups.

Questions for authors

How does the proposed untargeted holdout differ from MUSE’s disjoint retain split and from RWKU’s neighbor set, beyond being learned the same way as the targets? Can the authors restate contributions I had difficulty following them? I would thing that the efect persist but have you experimented with larger models and with finetuned knowledge?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.