acceptodds
Under review as a conference paper at ICLR 2027

Retrained Models Can Relearn in Context

Abstract

Machine unlearning aims to make a language model behave as if it had never been trained on certain data, and audits check whether it does. A common audit, in-context relearning, counts a fact as not forgotten if related context brings the answer back. A model retrained without the fact fails this audit too when the fact can be inferred from what the model kept. We build synthetic facts in which a person's sector follows from their employer and ask models to forget each person's sector: in seven combinations of Llama and Qwen models and fact sets, a model retrained without the sectors selects the forgotten sector from ten candidates 54–91% of the time once told the employer, and retrained OLMo models do the same. In real pretraining, a model from the Hubble suite never shown a set of biographies infers a person's nationality from their birthplace as often as the model that saw them (98%). Recovery alone therefore does not show that unlearning failed; what matters is whether the unlearned model behaves differently from a retrained one. Judged this way across 145 hyperparameter settings of 11 unlearning methods, open-ended generation fails to detect most unlearned models that still prefer the forgotten answer, and a score based on how much context raises the model's preference for the answer rates 45 of 48 as more forgotten than the retrained model; a multiple-choice question without context detects most of them. For facts that can be inferred, we add conflicting context: a prompt that names a wrong employer. A retrained model follows it; an unlearned model that keeps the original answer departs from retraining. This test flags RMU models on Qwen3 and Llama that pass the multiple-choice question, provided they still infer the sector from the employer. For facts that cannot be inferred, such as random identifiers inserted into Hubble's pretraining data, recovery from context remains a valid test: seven of nine 8B models unlearned with SimNPO or WGA that pass our health checks choose the forgotten identifier from 11 options 89–98% of the time, against 11% for a model pretrained without it. Unlearning audits should run a retrained model through the same prompts and, for facts that can be inferred, add conflicting context.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, unlearning evaluation, in-context relearning, retrained reference, knowledge conflicts, large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_X7KLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines the reliability of context-based evaluations of language-model unlearning. Experiments with synthetic relational facts show that models retrained without target facts can recover them from related context, motivating comparisons against retrained references under identical prompts. The authors evaluate 11 unlearning methods and introduce conflicting-context probes that reveal behavioral differences between selected unlearned models and retrained references, complementing generation and no-context candidate scoring. Additional experiments on Hubble demonstrate recovery of non-inferable random identifiers from biographical context after unlearning, and measure collateral effects on identifiers outside the forget set.

Critical strengths
  1. The paper provides a concrete experimental demonstration that contextual recovery alone can misidentify successful unlearning as failure. Retrained models recover inferable target facts with 54–91% accuracy when given the employer, compared with 8–13% without context (Figure 2; Table 3). This supports the practical recommendation to evaluate recovery against a retrained reference under identical prompts, helping distinguish reconstruction from retained knowledge from evidence of failed unlearning.

  2. The Hubble experiments provide useful complementary evidence for non-inferable targets. Seven healthy 8B SimNPO/WGA settings recover random identifiers with 89–98% accuracy from biographical context, compared with 11% for the no-exposure reference (Table 19). Evaluating identifiers outside the forget set also exposes collateral suppression (Figure 7; Table 20). These results strengthen the paper’s distinction between inferable facts and arbitrary identifiers and highlight the importance of evaluating selective forgetting.

Critical weaknesses
  1. The incremental diagnostic value of conflicting context remains insufficiently established. In the main sweep, no-context scoring combined with free generation already flags all 48 healthy models, matching the coverage obtained by combining no-context scoring with conflicting context (Table 10). Moreover, three of the five RMU settings missed by the one-sided no-context test already differ significantly from the retrained reference in the negative direction (Table 12), making them distinguishable through a two-sided comparison. The additional RMU experiments in Table 16 provide useful evidence, but the benefit beyond existing tests remains supported by a limited set of configurations. This limits the demonstrated significance of the proposed audit.

  2. Unresolved evaluation and replication issues weaken confidence in the empirical results. Appendix A.3 reports that the Llama sweep used an incorrect attention mask and that the unlearned checkpoints were unavailable for rescoring. Correcting the mask changes the stored original model’s wrong-employer result from 77% to 84% while leaving the retrained result unchanged, so using the same faulty scorer does not ensure cancellation of its effects. Separately, rebuilding the Qwen3 original model and repeating the RMU settings reduces true-sector retention under conflicting context from 63–73% to 50–57%, with three of five reruns failing the health checks (Appendix C.4). Corrected evaluations and an explanation of this variability are needed to establish the reliability of the reported findings.

Questions for authors
  1. What additional diagnostic value remains under comparable baselines? Table 10 shows that no-context scoring combined with free generation already flags all 48 healthy settings. For the key RMU cases, could you compare conflicting-context detection with no-context accuracy and two-sided margin tests, using matched candidate sets where possible? Please identify which settings exhibit additional detection or more reliable detection under these comparisons. Evidence of a consistent benefit would strengthen my assessment of the proposed test’s contribution.

  2. How sensitive are the Llama findings to the scoring correction? Could you provide corrected evaluations for a small, representative subset of the Llama settings supporting the main conclusions, including settings near the health or detection thresholds? If rerunning is necessary because the checkpoints are unavailable, applying both scorer versions to the same new checkpoints would help separate scoring effects from training variability. Please report whether the health classifications and differences from retrained references remain stable. Such evidence would substantially reduce my concern about the affected results.

  3. How reproducible is the key RMU phenomenon? Appendix C.4 reports substantial changes after rebuilding the Qwen3 original model and repeating the RMU configurations. Can the authors explain these changes through a comparison of the actual training configurations and checkpoints, and provide a small number of independent repetitions of the key five-epoch settings in Table 16? Reporting health checks and detection outcomes for each repetition would help establish how consistently the phenomenon occurs. Reproducing the effect under a fixed protocol would increase my confidence in the reliability of the findings.

Reviewer_6bGcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a way to distinguish between unlearned models (that have been trained with various methods to forget facts) and retrained models (that have never been trained on the facts to forget). In particular, the authors show that when an unlearned model was provided with wrong information about a forgotten fact, it would still frequently recover it. On the other hand, a model trained without the fact would not. This could provide a way to distinguish between models that unlearned facts and models that were never trained on them.

Critical strengths

Strenghts

  • Using counterfactuals to distinguish between retrained models and unlearned models is a nice idea.
Critical weaknesses

Weaknesses

  • The paper is very hard to read due to its writing style, and the exposition is far from meeting the bar for publication. The main thesis itself is hard to parse from the text. The results sections mostly list the numerical findings without providing insightful explanations, and the paper lacks flow and storytelling, making it hard to read and understand. The experimental setup is spread across the whole paper, making it complicated to compare how each setup differs from the others. For example, the wrong employer test sometimes uses 2 options and sometimes uses 10 options. There are also too many figures in the main paper, frequently hard to parse, sometimes even with text overlapping with visual elements of the panels (for example, figure 2c).
  • The authors only play with a single synthetic task/relation for most of the main results, without different categories of facts/relations. There are many datasets of counterfactuals that could have been used instead, and it is not clear why the authors chose a single specific toy-ish category. This is specifically problematic in the context of this paper given the authors’ own words in the limitations (”we chose the retrained model as the reference and the wrong-employer prompt as a test after seeing results”), which suggests results might not extend to broader settings and are more of a cherry-picked artifact of the setup.
  • The fact that unlearned models can still recover the information even if provided wrong information is not surprising, and the fact that retrained models cannot do it is also not surprising. Furthermore, it is not clear why the findings should matter.

Overall, all the weaknesses have heavily affected my judgement, and I believe the paper would need major rewriting and experimental extensions to convince me otherwise.

Questions for authors
  • Is it right to leave “unhealthy” models aside?
  • It is not clear to me why it is important to distinguish between retrained models and models that have unlearned.
  • (see weaknesses)
Reviewer_tmK2OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper is a contribution to assessing unlearning methods. The core insight behind the paper is that a model can successfully unlearn a fact but still behave as though it knows that fact because it follows as an inference from its knowledge or from its context. Thus, the key analytic steps are to compare with a model that was never exposed to the forget data (to see if the fact are still recoverable) and with contexts that convey the wrong answer (to see whether the model has lingering effects of the original answer). Using these methods, the authors remove some reasons to doubt unlearning methods while also painting an overall picture in which unlearning methods are not every sucessful.

Critical strengths

The findings in this paper are a genuine update for me in terms of my thinking about how to evaluate unlearning methods. In effect, the paper points out that I think we've been making some analytic errors as a community when we think about what unlearning methods should achieve. The paper also nicely organizes this into a proper protocol for evaluation and uses it to provide what I think is a better overall assessment of unlearning methods.

The paper is quite thorough in terms of analyses, and I like that the experiments are grounded in the Hubble framework. A lot of Section 5 was pretty compressed and seemed sort of hard to digest into a single story, but I can puzzle out the results with some work.

Critical weaknesses

The idea that retrained models can re-learn in context isn't actually new to this paper. It's there in older unlearning work, and it is very prominent in the presentation of the BLUR benchmark (https://arxiv.org/abs/2506.15699). BLUR should be cited but is not. This is fairly minor, though, and I think current paper is more systematic in its use of this idea. The conflicting context idea is new as far as I know.

There is one methodological point I feel unsure about: the models listed in section 2 were chosen because the retained model recovers the forget data reliably. This makes sense in terms of having something to investigate, but it seems like it is a strange bias that affects the value of this paper when it comes to auditing unlearning methods.

Questions for authors

See "weaknesses" above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.