acceptodds
Under review as a conference paper at ICLR 2027

When Deletion Writes: Target Imprints in LLM Editing and Unlearning

Abstract

Machine unlearning and model editing aim to remove specific knowledge from language models while preserving unrelated knowledge. A common approach to evaluating deletion is to test whether the deleted knowledge can still be recovered, where successful recovery is interpreted as evidence that the knowledge has survived. In this work, we show that such an interpretation can be misleading, because the deletion process itself can create a recoverable trace of the target it suppresses. We demonstrate this by applying the same deletion procedure to random target tokens that the model has never associated with the subject, and we find that these tokens become recoverable after deletion. Since these tokens had no prior association with the subject, the signals that newly emerge for them cannot represent surviving knowledge. In other words, a deletion can write the very thing that it deletes. We refer to this phenomenon as a loss-target imprint. Across the mechanisms we study, the imprint appears when the deletion loss directly references the target token, increases with deletion strength, and remains detectable by probes adapted to the deletion method, with the target often ranking first or second among all vocabulary tokens. These findings expose a fundamental limitation of recovery-based evaluation: failure to recover a deleted target does not establish that the target has been removed, whereas successful recovery does not establish that the recovered signal survived deletion. Our results highlight the need for unlearning evaluations that distinguish knowledge surviving deletion from signals introduced by the deletion process itself.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine Unlearning, Knowledge Deletion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PsnAOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Previous approaches to evaluating knowledge deletion in machine unlearning and model editing typically examine whether the deleted knowledge can still be recovered. However, the authors argue that the deletion process itself may introduce recoverable traces of the targeted knowledge, which they refer to as loss-target imprint. They further show that such imprints are closely related to how explicitly the deletion objective specifies the target. This paper highlights a fundamental limitation of recovery-based evaluation and emphasizes the need for unlearning evaluation methods that can distinguish between knowledge retained after deletion and new signals introduced by the deletion process itself.

Critical strengths
  1. The paper identifies and clearly defines an interesting and fundamental limitation of recovery-based evaluation in machine unlearning. I believe this is a highly important research problem, not only for machine unlearning but also for model editing and AI safety more broadly.

  2. The paper is well-organized and clearly written, allowing even readers unfamiliar with the problem to understand the main motivation and logical flow of the study.

  3. The experimental setup and analysis are appropriate and provide sufficient evidence to support the authors' claims. In particular, the design of the control experiments effectively demonstrates the reproducibility of the observed phenomenon and its relationship to the proposed explanation.

Critical weaknesses
  1. The main findings are limited to single-token factual associations and single-fact deletion, with some methods evaluated under modified settings. Further experiments are needed to verify whether Loss-Target Imprint generalizes to multi-token, multi-relation, and large-scale unlearning scenarios.

  2. The random-target provenance control effectively demonstrates that unlearning can introduce new recovery signals. However, it remains unclear how these signals can be distinguished from retained knowledge in actual unlearning scenarios. This raises concerns about the practical applicability of the proposed analysis to existing recovery-based evaluations.

  3. Although the paper examines several mitigation strategies, systematic experimental comparisons with existing unlearning methods are relatively limited in the main body. Furthermore, while the analysis of the underlying causes is sufficiently detailed, the experimental validation of approaches to mitigate or prevent Loss-Target Imprint remains limited.

Questions for authors

How do weight tying between input embeddings and output unembeddings, and the choice of updated parameters, affect the strength and recoverability of Loss-Target Imprint? In particular, does the imprint still occur when only the input embeddings are updated while the untied output unembeddings remain frozen?

3 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part I)

Thank you for recognizing the problem's importance.

Extend the findings beyond single-token, single-fact deletion and modified method settings.

Our new experiments cover complementary settings: batched GA tests identification after output suppression, WMDP-cyber tests domain-level traces, and TOFU tests multi-token, multi-relation answers.

TOFU contains questions about fictitious authors with multi-sentence answers. We use its fine-tuned phi-1.5 model and full-parameter NPO recipe. The forget10/05/01 splits target 10%/5%/1% of authors.

We compare unlearning true answers with unlearning perturbed wrong answers the model was not trained to produce. The differential lens selects among six answers by the decrease in their distinctive tokens' logit-lens scores, meaning vocabulary logits decoded from hidden states. This uses directional suppression; GA and MEND use absolute score changes. Uniform selection gives 16.7%.

SplitQuestions / authorsWrong answer identified when targetedSame answer when not targetedPaired increase
forget10400 / 2030.0%13.3%+16.8 percentage points
forget05200 / 1023.5%8.5%+15.0 percentage points

The increases are significant when accounting for shared authors (p=0.00002 and p=0.004). Forget01 gives 22.5% versus 7.5% on 40 questions, but is not significant (p=0.11).

Forgetting is partial: true answers still rank first among six candidates by mean token log-probability on 92%/84%/65% of questions for forget10/05/01, respectively, versus 100% initially. These results establish multi-token target identification during unlearning, rather than after complete knowledge removal.

For identification after successful output suppression, full-parameter GA on GPT-2 XL removes all 195 previously top-5 targets. At batch sizes eight and 32, random-target lens top-10 rates remain 78% and 47%, with utility of 90% and 82% in the corresponding real-target runs.

For larger batches, MEMIT-deletion removes 99% of targets from the output top-5 at both 40 and 720 facts. Probe recovery increases over the unmodified-model probe by 33 and 26 points, with utility of 83% and 46%.

Clarify how the random-target control distinguishes new signals from retained knowledge in practice.

The control provides a per-fact reference, not a definitive source label. We separately delete 19 frequency-matched random targets outside the base output top-5. The reference p-value is (1 + number of random scores ≥ real-target score) / 20, with a minimum of 0.05.

A twentieth random target checks calibration: under GA, it is flagged on 6/103 facts at the 5% threshold and 11/103 at 10%, close to nominal rates.

Real targets are recovered on 67% of facts, but 34% of all facts are recovered with scores that are not significantly above the random-target reference at the 10% significance level. Falling below this reference does not rule out retained knowledge; exceeding it does not prove survival. Frequency matching does not fully control initial probabilities, semantics, or optimization responses. The audit tests whether recovery exceeds the constructed reference, without claiming an exact provenance decomposition.

Provide more systematic comparisons with existing unlearning methods in the main paper.

We add full-parameter GA and NPO, gradient difference, RMU, and IDK alongside the editors, and will include the main comparisons in the paper. Gradient difference combines forget-data ascent with retain-data training; IDK trains an “I don't know” response.

On Zephyr-7B-β, the base model of WMDP's released RMU checkpoint, NPO and low-rate gradient difference achieve similar WMDP-cyber forgetting accuracy: 32.3% and 32.6%. Yet retained general accuracy on MMLU is 53.4% and 57.4%.

To audit traces, we repeat unlearning with cyber-specific tokens replaced by frequency-matched general words, comparing their log-probabilities between runs after subtracting untargeted-control changes. NPO produces 0.53 natural-log probability units stronger suppression (paired p=0.004 across 689 token types). Thus, similar forgetting scores can hide differences in both introduced traces and retained utility.

AuthorsOpenReview ↗

Official Comment by Authors (Part II)

Provide stronger evidence on how to reduce or prevent loss-target imprints.

Early stopping reduces probe recovery while preserving output suppression, but does not remove all detectable traces. On the same 103-fact evaluation, reducing GA from eight steps to two keeps every target outside the output top-5, lowers random-target probe recovery from 46% to 11%, and improves utility from 86% to 94%. One step gives 98.5% forgetting, 95% utility, and 7% recovery, not significantly above chance.

In a separate test, 59 targets are outside the base model's output top-5. After one GA step, 76% rank in the differential-lens top-10, versus 2% for untargeted controls. The lens ranks absolute before/after changes in logit-lens scores; this is separate from the model's output ranking.

Other interventions have tradeoffs. Projection leaves recovery at 42%. LEACE, which removes an object-related hidden-state subspace, reduces utility to 27% while recovery remains above the unmodified-model probe. Four decoys reduce recovery to 6%, but forgetting falls to 94.5%.

IDK, KL-to-uniform (flattening output probabilities), and RMU (changing hidden representations without naming an output token) achieve 97–98.5% forgetting and 94–95% utility, with much weaker lens signals. However, relearning tests that fine-tune on some deleted facts and test others show that weaker traces do not guarantee knowledge removal.

Does the imprint depend on weight tying or which parameters are updated, especially with input-only updates and frozen, untied output embeddings?

Weight tying is not required. On Pythia-2.8B, we freeze the separate output embedding matrix and update only subject tokens' input-embedding rows. Random-target probe recovery increases by 15 percentage points over the unmodified-model probe (p=0.004), with 99% utility; the signal appears in early layers.

MLP updates give a 21-point increase appearing later. Input-only updates achieve weaker suppression within the tested learning-rate range, so these magnitudes are not a matched-strength comparison. Nevertheless, the requested input-only, frozen-output setting clearly produces an imprint.

Thank you for the detailed rebuttal and additional experiments. My concerns and questions regarding W1, W3, and Q1 have been adequately addressed through the additional experiments and explanations.

Well.. W2, I understand that it may be difficult to fully resolve this issue at this stage.

Since the authors have satisfactorily addressed most of my concerns, I have decided to increase my rating.

Reviewer_CMEMOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper shows that knowledge deletion can introduce recoverable target-specific signals, termed loss-target imprints. Random-target controls demonstrate that recovery need not indicate surviving knowledge. Experiments across editing and unlearning methods, architectures, and datasets evaluate these signals using adapted probes and differential logit-lens analysis.

Critical strengths

The paper shows that recovering a deleted target does not necessarily mean the model retained its original knowledge. Experiments with randomly assigned targets provide convincing evidence that deletion itself can create recoverable information.

Critical weaknesses
  1. Baseline coverage. AlphaEdit [1] and MEND [2] are not evaluated. Including these methods would test whether the proposed loss-target rule holds across different editing mechanisms.
  2. Limited practical scope. The paper does not evaluate target recovery after domain-level unlearning, as studied on WMDP by Chen et al. [3]. Its single-token experiments leave this application untested. Evaluation in the WMDP setting, with forgetting measured on WMDP and retained utility on MMLU, would strengthen practical relevance.
  3. Incomplete mechanism analysis. The gradient analysis explains why deletion can leave a recoverable target signal, but the explanation of why this signal reappears after removing the target-aligned direction remains limited. A deeper analysis of this behavior would strengthen the paper's mechanistic contribution.

[1] Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models. ICLR, 2025.

[2] Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast Model Editing at Scale. ICLR, 2022.

[3] Yiwei Chen, Soumyadeep Pal, Yimeng Zhang, Qing Qu, and Sijia Liu. Unlearning Isn’t Invisible: Detecting Unlearning Traces in LLMs from Model Outputs. ICLR, 2026.

Questions for authors
  1. Can the authors evaluate AlphaEdit and MEND, separately measuring recovery of installed and removed tokens?
  2. Can the authors test the deletion-induced imprint phenomenon in the WMDP setting used by Chen et al. [3]?
  3. Why does the target signal reappear in later layers after the target-aligned direction is removed? Can the authors provide a more detailed explanation supported by experiments?
2 comments

Thank you for the suggestions. We have added the requested editors and broader experiments.

Evaluate AlphaEdit and MEND, measuring recovery of installed and removed tokens separately.

Both editors leave stronger signals for the installed token named by the objective.

AlphaEdit achieves 99% edit success. Adapted-probe recovery increases over the unmodified-model probe by 65 percentage points for installed tokens, versus nine for removed tokens (p=0.11). Random installation targets become recoverable at 76%.

MEND updates layers 45–47. Its probe increase is not significant, but the differential lens identifies installed targets on all 103 test facts, including failed installations; the same holds for random targets. Removed tokens are identified on 22%, versus 0% for untargeted controls. Here, identification means ranking in the top-10 by absolute before/after changes in logit-lens scores (vocabulary logits decoded from hidden states) or final output logits.

In 21 random-target edits, the target is outside the edited model's output top-10. For 19 of these 21, it ranks in the differential-lens top-10 using layer 46 or 47 alone. This uses hidden-layer changes without output logits; it does not mean the target enters the model's output top-10.

Test target recovery after domain-level unlearning on WMDP, with MMLU for retained utility.

Our WMDP experiments show that domain-level unlearning introduces target-specific traces.

We use Zephyr-7B-β, the base model of WMDP's released RMU checkpoint, with 800 WMDP-cyber forget documents and 200 held-out audit passages. WMDP-cyber measures forgetting; MMLU measures retained general knowledge. Gradient difference combines ascent on forget data with training on retain data; it and NPO update all parameters.

We repeat unlearning after replacing 715 cyber-specific tokens with frequency-matched general words. On unchanged held-out passages, we compare replacement-word log-probabilities between runs, subtracting changes to frequency-matched untargeted controls. Neither word is the correct next token. Negative values mean stronger suppression in natural-log probability units (nats).

MethodWMDP-cyber accuracyMMLU accuracyReplacement-word log-probability change
NPO32.3%53.4%−0.70 nats
Gradient difference, learning rate 1e-625.6%57.8%−0.31 nats
RMU27.4%57.9%+0.01 nats

Accuracies are from original-corpus runs; base accuracies are 42.5% and 58.8%. Across 689 token types, traces are significant for NPO and gradient difference (p<0.0001 and p=0.04), but not RMU.

For target identification, we add TOFU, a benchmark about fictitious authors with multi-token answers. The lens selects among six answers by the decrease in their distinctive tokens' logit-lens scores. On 400 questions, it identifies a wrong answer on 30.0% when targeted by unlearning, versus 13.3% when not targeted (author-aware p=0.00002). WMDP supports domain-level traces; TOFU supports multi-token target identification.

Explain why the target signal reappears after removing the target-aligned direction, with experimental support.

Projection removes the edited MLPs' direct target-aligned contribution, but other representation changes remain. Downstream blocks process these changes and rebuild the target component.

Decomposing hidden-state changes by block confirms this: the edited MLPs' direct contribution is approximately zero after projection, yet downstream blocks rebuild a target-aligned shift of −27.4 at layer 36, versus −31.9 without projection.

We then constrain both the target's output score and aligned hidden-state change to remain near their pre-deletion values. The aligned magnitude falls from 12.4 to 0.09. Deletion instead raises competing output scores, but needs 48 rather than eight steps, reaches 95% rather than 100% output-level forgetting, and reduces utility from 82% to 75%. Removing the local contribution alone therefore leaves a downstream route for the signal to return.

Thanks for your response, I will maintain my score.

Reviewer_te8XOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper shows that deletion can introduce recoverable target-specific signals, so post-deletion recovery does not necessarily indicate that the original knowledge survived.

Critical strengths

The random-target control is intuitive and convincing, exposing an important ambiguity in recovery-based unlearning evaluation.

Critical weaknesses
  1. Evidence for practical gradient-based unlearning remains limited: recovery gains become statistically insignificant with just eight jointly deleted facts under GA, and full-parameter GA shows weaker effects than localized updates.

  2. The practical value of the proposed audit is insufficiently demonstrated. Beyond identifying a provenance confound, the paper provides limited evidence that the control changes conclusions about existing unlearning methods or informs better method selection.

Questions for authors

See Weaknesses

1 comment

Thank you for recognizing the value of the random-target control.

Evidence for practical gradient-based unlearning is limited. Recovery gains become insignificant with eight jointly deleted facts, and full-parameter GA shows weaker effects.

Our new experiments show target identification after successful output suppression, including batched deletion. Using full-parameter GA on GPT-2 XL, we delete 1, 8, or 32 facts together, fixing the learning rate and ten update steps. We repeat each setting with randomly assigned targets to test signals introduced by deletion.

The adapted probe measures top-5 recovery among candidate objects; its increase is relative to a probe trained on the unmodified model. The differential lens ranks all 50,257 tokens by absolute before/after changes in logit-lens scores, meaning vocabulary logits decoded from hidden states. Lens recovery means a top-10 rank and requires both checkpoints.

Facts deleted togetherPreviously top-5 targets removedRetained utilityRandom-target probe increaseRandom-target lens top-10
1195/19595%+20 percentage points98%
8195/19590%+4 points, not significant78%
32195/19582%+3 points, not significant47%

Of 1,064 evaluated facts, 195 targets were initially in the output top-5. Recovery uses 193 held-out facts; untargeted controls reach the lens top-10 on only 1–3%. Utility measures retained top-1 accuracy on 40 unrelated known facts at each of the first six group checkpoints.

Thus, batching weakens the probe, but the lens still identifies random targets. For real targets under single-fact full-parameter GA, moving the probe from layers 28/32/36 to 40/44/47 increases the recovery gain over the unmodified-model probe from four to 12 percentage points (p=0.002).

The audit's practical value is unclear. Show whether it changes conclusions about existing methods or helps choose between them.

The control changes the interpretation of existing recovery results. On the same GPT-2 XL/CounterFact evaluation, probe recovery of real targets is 66% after GA and 30% after RMU. Read as retained knowledge, this would suggest GA retains more. Yet random-target recovery is also much higher after GA: 46% versus 6%. The audit therefore shows that higher recovery alone cannot establish that GA retains more original knowledge. Which method retains more knowledge still becomes an open question, while the difference in recoverability itself still stands.

It also helps select methods. We add Zephyr-7B-β, the base model of WMDP's released RMU checkpoint. WMDP-cyber measures knowledge being removed; MMLU measures retained general knowledge. Gradient difference combines ascent on forget data with training on retain data.

For each method, we repeat unlearning with cyber-specific tokens replaced by frequency-matched general words. We compare these words' log-probabilities between runs on the same held-out passages, subtracting changes to untargeted controls. Negative values mean stronger suppression, measured in natural-log probability units (nats).

MethodWMDP-cyber accuracyMMLU accuracyReplacement-word log-probability change
NPO32.3%53.4%−0.70 nats
Gradient difference, learning rate 3e-732.6%57.4%−0.16 nats

At similar benchmark forgetting, gradient difference retains four points more MMLU accuracy and produces 0.53 nats less suppression (paired p=0.004 across 689 token types). The audit reveals a difference that forgetting accuracy alone misses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.