acceptodds
Under review as a conference paper at ICLR 2027

Suppression or Relocation? Safe-Anchored Relocation for LLM Unlearning

Abstract

Large language model unlearning seeks to remove unwanted knowledge without costly retraining. Prevailing unlearning methods typically rely on suppression-based approaches that lower the probability of specific responses. However, our diagnostic analysis across five baselines reveals that much of the suppressed probability mass consistently flows toward the model's original top- preferences, which may continue to reveal forgotten knowledge through similar expressions. We identify that the root cause is the normalization in softmax, whereby suppressed probability must redistribute but current objectives provide no mechanism to direct this flow. To address this, we propose Safe-Anchored Relocation (SAR), which moves beyond suppression by explicitly relocating probability mass at the token level. SAR constructs dynamic reference branch and current model token sets at each answer position, then employs an InfoNCE-like loss to relocate probability toward reference defined anchors. Both theoretical analysis and experiments on diverse benchmarks confirm the effectiveness of our approach.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Unlearning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_odt3OpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies probability redistribution in suppression-based LLM unlearning, arguing that suppressing target tokens can increase the probability of alternative tokens that still reveal the knowledge to be forgotten. It proposes Safe-Anchored Relocation (SAR), which dynamically constructs reference-only and current-model-only token sets and optimizes a contrastive probability-mass objective to redirect probability toward reference anchors. The paper provides a local theoretical analysis and evaluates SAR on TOFU and MUSE, reporting improved forgetting with competitive utility compared with several unlearning baselines.

Critical strengths

Insightful problem analysis. The paper highlights how probability redistribution in suppression-based unlearning can lead to residual knowledge leakage and provides a systematic analysis of this behavior in existing methods.

Well-developed theoretical analysis. The paper derives the local logit-update behavior of SAR and establishes conditions under which probability mass shifts toward reference anchors, providing a clear mathematical foundation for the proposed objective.

Critical weaknesses
  1. From the perspective of novelty: The method treats candidate tokens unique to the reference branch as safe anchors, but there is currently insufficient evidence showing that these candidates will not retain or leak the target knowledge. The existing theoretical analysis characterizes the direction of probability redistribution, but does not establish a connection between the reference anchors and semantic safety. Therefore, the core claim of "safe relocation" still needs further support.

  2. From the perspective of writing: The introduction emphasizes that the same token can play different semantic roles in different contexts, but the subsequent introduction of SAR does not clearly explain how the reference-guided dynamic sets address this issue. Although the set construction depends on the current context, it still lacks an explanation of why the prediction difference between the reference and the current model can distinguish knowledge leakage from harmless use. It is suggested to clarify the correspondence between the motivation and the method design, and to add a comparative analysis of the same token in target and non-target contexts.

  3. From the perspective of layout: Table 1 is placed near the method section, causing the experimental results to appear before the corresponding experimental setup and interrupting the reading of the method section.

Questions for authors

How do the authors establish that reference-only top-k tokens are safe anchors? Can the authors provide direct evidence of anchor safety and analyze SAR’s behavior when the reference branch retains the target knowledge? How does SAR’s context-dependent token-set construction address the concern raised in the introduction that the same token may be harmful in one context but harmless in another? Can the authors provide a targeted evaluation of this distinction?

6 comments

Thank you for your recognition and valuable suggestions. Please find our response below:

1. [W1/Q1] Semantic knowledge leakage of Sref(h)S_{\mathrm{ref}}(h)

First, it should be clarified that SAR does not directly use the reference model's top-kk tokens. Let hh denote the question and answer prefix, R(h)R(h) and C(h)C(h) denote the top-kk token sets of the reference model and the current model, respectively, and yy denote the true next token. The actual receiving set is Sref(h)=(R(h)∖C(h))∖{y}.S_{\mathrm{ref}}(h) = (R(h)\setminus C(h))\setminus\{y\}. Next, we examine the semantic knowledge leakage of Sref(h)S_{\mathrm{ref}}(h) through the following experiments. We use Llama-3.2-3B with top-kk = 4. An LaaJ identifies 67 of the 200 QA pairs in TOFU Forget 5% as containing a target-leaking subject. For example, for the question “In which city was Hina Ameen born?” and the answer “Hina Ameen was born in Karachi, Pakistan,” the extracted subject is “Karachi, Pakistan.” For each selected QA pair, we perform separate interventions at two positions: the beginning of the answer (P1) and the beginning of the target-leaking subject (P2). At each position, we insert each candidate token from Sref(h)S_{\mathrm{ref}}(h) individually and let the original, non-unlearned model continue generating the response. In the single-token setting, after inserting the candidate, the same original model continues. In the three-token setting, the candidate is used as the first token, the reference model continues generating two tokens, and then the original model continues. Subsequently, LaaJ is used to evaluate whether the complete response leaks the extracted target subject.

SettingReferenceReplacement positionSrefS_{\mathrm{ref}} (%)Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\} (%)Scur,tS_{\mathrm{cur},t} (%)
Single-tokenInstructP133.4665.9274.63
P227.9548.4664.23
NPO RefP172.8266.4178.21
P232.8344.9563.01
Self-promptP166.3667.2782.12
P233.0139.8766.50
Three-tokenInstructP117.0459.9567.04
P223.2132.5635.90
NPO RefP167.4468.2172.18
P222.6032.3239.90
Self-promptP164.5570.0083.64
P230.3940.8566.83

For leakage-positive P2 continuations, we further identify whether the target is already explicit within the intervention segment or becomes explicit only during subsequent generation.

(Continuation of W1/Q1)

SettingReferenceCandidate setIntervention segment leakage (%)Leakage after intervention (%)
Single-tokenInstructSrefS_{\mathrm{ref}}4.0096.00
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}20.0080.00
Scur,tS_{\mathrm{cur},t}16.0683.94
NPO RefSrefS_{\mathrm{ref}}1.7598.25
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}24.5675.44
Scur,tS_{\mathrm{cur},t}17.7482.26
Self-promptSrefS_{\mathrm{ref}}9.5290.48
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}21.4378.57
Scur,tS_{\mathrm{cur},t}14.7485.26
Three-tokenInstructSrefS_{\mathrm{ref}}16.6783.33
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}48.8951.11
Scur,tS_{\mathrm{cur},t}51.4348.57
NPO RefSrefS_{\mathrm{ref}}8.3391.67
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}47.3752.63
Scur,tS_{\mathrm{cur},t}48.6551.35
Self-promptSrefS_{\mathrm{ref}}22.2277.78
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}51.8548.15
Scur,tS_{\mathrm{cur},t}47.8752.13

These results support two observations. 1. Across different reference models and intervention positions, our anchor set generally reduces continuation leakage compared with the control sets. 2. Attribution analysis shows that most remaining leaks emerge during subsequent generation by the original, non-unlearned model. This is consistent with the model retaining the target knowledge and reintroducing it during continuation. Together, these results provide empirical support for the relative safety of our anchor set.

2. [W2/Q2] Same token, different token sets

We first wish to clarify that the token set selection in SAR is dynamic and empirical. We use the prediction disagreement between the reference model and the current model as the signal for selection. The intervention experiments above have already shown that the receiving sets under all three reference settings produce lower target leakage rates than the two control sets. At the same time, this set construction is dynamic and context-dependent. Let hh denote the current question and answer prefix, R(h)R(h) and C(h)C(h) denote the top-kk token sets of the reference model and the current model under this context, respectively, and yy denote the true next token. SAR constructs at each prediction position

\qquad S_{\mathrm{cur},t}(h)=(C(h)\setminus R(h))\cup\{y\}.$$ Therefore, SAR does not fix in advance which tokens are safe or harmful, but rather re-determines set membership based on the predictions of the two models in the specific context. In addition, by definition, $$S_{\mathrm{ref}}(h)\cap S_{\mathrm{cur},t}(h)=\varnothing.$$ This ensures that the same token will not simultaneously receive two opposite optimization requirements, acceptance and suppression, at the same prediction position. A practical example is given below. The common candidate token is `village`, and square brackets mark this token. | Set | Question | Prefix and candidate token | | ----------------------- | ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | | $S_{\mathrm{ref}}(h_1)$ | What is a brief summary of Xin Lee Williams' book "The Village That Vanished"? | `The [village]` | | $S_{\mathrm{cur}}(h_2)$ | What is one of the fictional books written by Xin Lee Williams that follows the theme of "The Town That Drowned"? | `One other book written by Xin Lee Williams is "The [village]` | This case demonstrates how SAR makes different choices for different uses of the same token depending on the specific context. # 3. [W3] Reading coherence Thanks for your comments. We have fixed this issue in the revised paper.
Reviewer_WCAwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Safe-Anchored Relocation (SAR), a token-level unlearning objective that redirects probability toward reference-model candidates while suppressing current-model candidates and the target token. It analyzes the local logit updates of SAR and BS-T, and evaluates forgetting and utility on TOFU and MUSE News. SAR improves the aggregate TOFU results across the reported models and forget splits, with additional experiments on reference choices and training configurations.

Critical strengths
  • I find the motivation compelling: unlearning should consider not only which outputs to suppress, but also where the displaced probability should go. The analysis of BS-T makes this concern concrete, showing that including a candidate in the soft forget target does not necessarily lower its logit. Explicitly defining a receiving set is a natural response to this issue and gives SAR a clear rationale.

  • The empirical evaluation is extensive. The paper covers three model sizes and three forget ratios on TOFU, alongside experiments on MUSE News, and standard SAR consistently improves the aggregate TOFU score over the five existing baselines. The reference-model comparisons, full-answer versus prefix ablation, and hyperparameter studies also help establish how the method behaves beyond its best-performing configuration.

Critical weaknesses
  • The role of the reference model needs clearer justification. SAR treats reference-only top-k tokens as suitable destinations for the displaced probability, but disagreement with the current model does not establish that these candidates are unrelated to the knowledge being forgotten. An instruct reference may already know the target facts, particularly on MUSE News, while an NPO reference may retain some of them. The self-prompt variant changes the requested behavior without necessarily removing that knowledge either. I appreciate the comparisons across reference choices, but they do not directly establish what makes the receiving set “safe.” This matters because SAR could successfully redirect probability without reliably redirecting it away from expressions of the target fact.

  • A plot with forgetting performance on the horizontal axis and utility on the vertical axis would be a useful addition. Showing different configurations for each method would make their forgetting–utility trade-offs easier to compare.

Questions for authors

See Weakness.

A minor presentation question: could Table 1 be moved closer to its discussion in Section 5.2? Its current placement within the method section makes it harder to follow the main results.

9 comments

Thank you for your positive and constructive feedback. Please find our response below:

1. [W1/Q1] Semantic knowledge leakage of Sref(h)S_{\mathrm{ref}}(h)

We examine the semantic knowledge leakage of Sref(h)S_{\mathrm{ref}}(h) through the following experiments. We use Llama-3.2-3B with top-kk = 4. An LaaJ identifies 67 of the 200 QA pairs in TOFU Forget 5% as containing a target-leaking subject. For example, for the question “In which city was Hina Ameen born?” and the answer “Hina Ameen was born in Karachi, Pakistan,” the extracted subject is “Karachi, Pakistan.” For each selected QA pair, we perform separate interventions at two positions: the beginning of the answer (P1) and the beginning of the target-leaking subject (P2). At each position, we insert each candidate token from Sref(h)S_{\mathrm{ref}}(h) individually and let the original, non-unlearned model continue generating the response. In the single-token setting, after inserting the candidate, the same original model continues. In the three-token setting, the candidate is used as the first token, the reference model continues generating two tokens, and then the original model continues. Subsequently, LaaJ is used to evaluate whether the complete response leaks the extracted target subject.

SettingReferenceReplacement positionSrefS_{\mathrm{ref}} (%)Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\} (%)Scur,tS_{\mathrm{cur},t} (%)
Single-tokenInstructP133.4665.9274.63
P227.9548.4664.23
NPO RefP172.8266.4178.21
P232.8344.9563.01
Self-promptP166.3667.2782.12
P233.0139.8766.50
Three-tokenInstructP117.0459.9567.04
P223.2132.5635.90
NPO RefP167.4468.2172.18
P222.6032.3239.90
Self-promptP164.5570.0083.64
P230.3940.8566.83

For leakage-positive P2 continuations, we further identify whether the target is already explicit within the intervention segment or becomes explicit only during subsequent generation.

(Continuation of W1/Q1)

SettingReferenceCandidate setIntervention segment leakage (%)Leakage after intervention (%)
Single-tokenInstructSrefS_{\mathrm{ref}}4.0096.00
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}20.0080.00
Scur,tS_{\mathrm{cur},t}16.0683.94
NPO RefSrefS_{\mathrm{ref}}1.7598.25
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}24.5675.44
Scur,tS_{\mathrm{cur},t}17.7482.26
Self-promptSrefS_{\mathrm{ref}}9.5290.48
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}21.4378.57
Scur,tS_{\mathrm{cur},t}14.7485.26
Three-tokenInstructSrefS_{\mathrm{ref}}16.6783.33
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}48.8951.11
Scur,tS_{\mathrm{cur},t}51.4348.57
NPO RefSrefS_{\mathrm{ref}}8.3391.67
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}47.3752.63
Scur,tS_{\mathrm{cur},t}48.6551.35
Self-promptSrefS_{\mathrm{ref}}22.2277.78
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}51.8548.15
Scur,tS_{\mathrm{cur},t}47.8752.13

These results support two observations. 1. Across different reference models and intervention positions, our anchor set generally reduces continuation leakage compared with the control sets. 2. Attribution analysis shows that most remaining leaks emerge during subsequent generation by the original, non-unlearned model. This is consistent with the model retaining the target knowledge and reintroducing it during continuation. Together, these results provide empirical support for the relative safety of our anchor set.

2. [W2/Q2] Results of different configurations of different methods

We provide image comparisons in Figure 7 in the revised paper. Currently, we provide a tabular performance comparison of all baseline methods and our method under different hyperparameter combinations for llama-3.1-8B on TOFU Forget 5%. The results of GradDiff is:

λ\lambdaAgg ↑Mem ↑Util ↑
111.2798.125.98
256.0151.3061.67
548.8837.6569.65
1038.3126.2371.00

The results of NPO is:

β\betaλretain\lambda_{retain}Agg ↑Mem ↑Util ↑
0.1162.5056.2970.26
0.1262.6254.7473.14
0.1561.6352.3075.01
0.5151.9740.0773.93
0.5251.9239.9174.27
0.5550.7238.5574.14

The results of SimNPO is:

β\betaϵ\epsilonAgg ↑Mem ↑Util ↑
2.50.12534.9823.0572.54
2.50.1535.5323.4972.86
2.50.2035.5923.5772.65
2.50.2537.9325.6073.19
3.50.12533.7321.9672.70
3.50.1535.0423.0373.24
3.50.2035.1823.1972.81
3.50.2537.1324.8973.09
4.50.12533.9622.0873.50
4.50.1533.8021.9972.97
4.50.2035.2523.2772.67
4.50.2536.7024.5173.00

The results of WGA is:

α\alphaλretain\lambda_{retain}Agg ↑Mem ↑Util ↑
0.05160.9366.1256.50
0.05257.7149.4669.26
0.05547.4835.3772.21
0.1163.7162.3665.11
0.1258.9050.3071.06
0.1546.8834.6572.46
0.5168.6962.2376.64
0.5259.3849.0475.25
0.5542.4329.8873.14
1.0168.0063.8972.66
1.0255.6544.5874.04
1.0539.7027.1873.61
Reviewer_tDsiOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates probability redistribution during LLM unlearning and argues that existing suppression-based methods, including BS-T, cannot fully address the squeezing effect. It proposes Safe-Anchored Relocation (SAR), which uses a reference model to identify alternative tokens and encourages probability mass to move toward these reference anchors. The paper provides local theoretical analysis and experiments on TOFU and MUSE News.

Critical strengths
  1. The paper studies an interesting problem: reducing the probability of a forget target does not specify where the remaining probability mass should go. Examining this behavior can help explain why some unlearning methods produce semantically similar answers even after suppressing the original response.
  2. The proposed method is relatively simple and provides an intuitive way to control token probabilities using a reference model. The experiments show promising results on TOFU, and the authors include comparisons with BS-T, alternative reference models, and local theoretical analysis. These are useful contributions, although I have several concerns about their interpretation and generality.
Critical weaknesses
  1. The experimental coverage is too limited. All evaluated models belong to the Llama family, including Llama-3.2-1B/3B and Llama-2-7B. There are no experiments on more recent architectures such as Qwen3.5 or Gemma 4, or even Llama-3-8B. The model scale is also limited to 1B–7B. Furthermore, the paper only evaluates TOFU and MUSE News. It lacks harmful knowledge unlearning experiments on WMDP, where selecting genuinely safe alternatives may be much more challenging.
  2. The robustness evaluation is insufficient. The paper mainly evaluates forgetting and utility on standard benchmark queries. Figure 3(c) tests further fine-tuning on Alpaca, but this does not measure whether forgotten knowledge can be recovered. There are no targeted relearning attacks, jailbreaks, or adversarial extraction tests. Since SAR claims to reduce semantic leakage through probability relocation, its robustness against alternative ways of eliciting forgotten knowledge is particularly important.
  3. The theoretical criticism of BS-T is overstated. Proposition 3.1 shows that non-target top-k logits can still increase when the mixing coefficient is smaller than the top-k probability mass. However, this does not mean BS-T fails to mitigate the squeezing effect. Compared with standard gradient ascent, BS-T still reduces the increase of these logits through its belief-based correction. The proposition establishes a limitation in absolute logit suppression, not a failure of relative suppression. The paper also does not evaluate BS-S, which extends belief-based unlearning to sequence-level responses. Therefore, the claim that suppression-based approaches are fundamentally insufficient is not adequately supported.
  4. The main diagnostic metric does not directly measure spurious unlearning. Figure 2 measures the probability mass assigned to the original model's top-k tokens. However, high-probability tokens are not necessarily semantically related to forgotten knowledge. Moreover, BS-T uses dynamically updated top-k beliefs during training, whereas Figure 2 evaluates a fixed top-k set from the original model. A lower probability mass on this set does not necessarily imply less knowledge leakage. The semantic evaluation in Figure 3(b) covers only one model and one forgetting setting, which is insufficient to support the broader claim.
  5. The paper does not provide publicly accessible code. This makes it difficult to verify the reported results, which are significantly improved compared with baselines, e.g., in Table 1.
Questions for authors
  1. Can the authors extend the experiments to Qwen3.5, Gemma 4, or Llama-3-8B, and evaluate WMDP to demonstrate that SAR works beyond small Llama models and factual memorization tasks?
  2. Can the authors evaluate robustness against targeted fine-tuning, jailbreaks, paraphrased queries, and other adversarial attempts to recover forgotten knowledge?
  3. Proposition 3.1 only establishes that BS-T does not guarantee absolute suppression of every top-k logit. How does this demonstrate that BS-T fails to mitigate squeezing relative to gradient ascent?
  4. How strongly does the original model's top-k probability mass correlate with actual semantic knowledge leakage? Can the authors provide direct evidence that the proposed reference anchors are safe, especially when the reference model also knows the forget targets?
  5. Can the authors release the anonymous code during the review period, including the exact selected hyperparameters, random seeds, and reproduction scripts for all methods?
6 comments

Thank you for your insightful and detailed comments. Please find our response below:

1. [W1/Q1] Results of Different backbones and WMDP benchmark

The results of Qwen3.5-2B on TOFU Forget 5% is:

MethodAgg ↑Mem ↑Util ↑
GradDiff44.4736.4956.91
NPO42.9834.3357.46
SimNPO30.3720.2061.17
WGA59.5360.4958.61
BS-T44.9945.8244.18
SAR63.1178.7652.65

The results of Gemma-4-E2B-it on TOFU Forget 5% is:

MethodAgg ↑Mem ↑Util ↑
GradDiff60.2068.1853.89
NPO50.8539.2872.10
SimNPO37.1724.4477.59
WGA60.2179.2048.56
BS-T50.1638.2672.80
SAR58.4354.0163.63

The results of Llama-3.1-8B-Instruct on TOFU Forget 5% is:

MethodAgg ↑Mem ↑Util ↑
GradDiff56.0151.3061.67
NPO62.6254.7473.14
SimNPO37.9325.6073.19
WGA68.6962.2376.64
BS-T56.5850.5264.30
SAR75.1177.7672.63

The results of Llama-3.2-3B-Instruct on WMDP is:

MethodBio ↓ (%)Cyber ↓ (%)MMLU ↑ (%)
GradDiff24.4324.3626.42
NPO38.2628.1334.06
SimNPO39.0430.4530.74
WGA26.5524.5625.51
BS-T26.5524.5625.51
SAR (self prompt)35.3528.9442.69

2. [W2/Q2] Harmful relearning attacks

Following the setting in SSIUU[1], we mixed 10% and 30% of the data in TOFU Forget 5% into the alpaca fine-tuning data, respectively. The Agg results after fine-tuning are as follows:

MethodHarmlessHarmful 10%Harmful 30%
GradDiff47.8147.0645.51
NPO63.4762.9362.03
SimNPO37.6537.0335.19
WGA56.4654.6652.41
BS-T47.3046.5545.10
SAR67.5066.3564.93

3. [W3/Q3] The deny of BS-T

Our Proposition 3.1 shows that whether BS-T can successfully suppress the selected topk tokens depends on the hyperparameters set before training. Since the training process is dynamic, no fixed hyperparameter can satisfy the dynamic changes during training. Therefore, we believe that BS-T does not fully solve the spurious forgetting caused by the squeezing effect that it proposes. However, we do not deny BS-T's contribution, nor do we state that BS-T cannot alleviate the squeezing effect compared with GA. Could you clarify which sentence in the original text caused your misunderstanding?

4. [W4] More Laaj results of spurious unlearning

We used LLM-as-a-Judge (LaaJ) to re-evaluate llama-3.2-1B and llama-3.2-3B on TOFU Forget 1% and 5%. The results of llama-3.2-1B on TOFU Forget 1% are as follows:

MethodNaturalness ↑5 − Similarity ↑
GradDiff2.883.00
NPO4.102.47
SimNPO4.411.27
WGA3.662.63
BS-T4.132.25
SAR3.904.40

The results of llama-3.2-1B on TOFU Forget 5% are as follows:

MethodNaturalness ↑5 − Similarity ↑
GradDiff2.233.68
NPO4.593.46
SimNPO4.482.17
WGA3.493.53
BS-T2.513.56
SAR3.764.75

The results of llama-3.2-3B on TOFU Forget 1% are as follows:

MethodNaturalness ↑5 − Similarity ↑
GradDiff3.542.53
NPO4.472.65
SimNPO4.481.30
WGA3.852.40
BS-T3.982.19
SAR4.143.17

The results of llama-3.2-3B on Forget forget 5% are as follows:

MethodNaturalness ↑5 − Similarity ↑
GradDiff3.143.06
NPO4.583.72
SimNPO4.511.59
WGA3.653.10
BS-T3.423.05
SAR3.464.64

5. [Q4] Semantic knowledge leakage of Sref(h)S_{\mathrm{ref}}(h)

We examine the semantic knowledge leakage of Sref(h)S_{\mathrm{ref}}(h) through the following experiments. We use Llama-3.2-3B with top-kk = 4. An LaaJ identifies 67 of the 200 QA pairs in TOFU Forget 5% as containing a target-leaking subject. For example, for the question “In which city was Hina Ameen born?” and the answer “Hina Ameen was born in Karachi, Pakistan,” the extracted subject is “Karachi, Pakistan.” For each selected QA pair, we perform separate interventions at two positions: the beginning of the answer (P1) and the beginning of the target-leaking subject (P2). At each position, we insert each candidate token from Sref(h)S_{\mathrm{ref}}(h) individually and let the original, non-unlearned model continue generating the response. In the single-token setting, after inserting the candidate, the same original model continues. In the three-token setting, the candidate is used as the first token, the reference model continues generating two tokens, and then the original model continues. Subsequently, LaaJ is used to evaluate whether the complete response leaks the extracted target subject.

SettingReferenceReplacement positionSrefS_{\mathrm{ref}} (%)Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\} (%)Scur,tS_{\mathrm{cur},t} (%)
Single-tokenInstructP133.4665.9274.63
P227.9548.4664.23
NPO RefP172.8266.4178.21
P232.8344.9563.01
Self-promptP166.3667.2782.12
P233.0139.8766.50
Three-tokenInstructP117.0459.9567.04
P223.2132.5635.90
NPO RefP167.4468.2172.18
P222.6032.3239.90
Self-promptP164.5570.0083.64
P230.3940.8566.83

For leakage-positive P2 continuations, we further identify whether the target is already explicit within the intervention segment or becomes explicit only during subsequent generation.

(Continuation of Q4)

SettingReferenceCandidate setIntervention segment leakage (%)Leakage after intervention (%)
Single-tokenInstructSrefS_{\mathrm{ref}}4.0096.00
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}20.0080.00
Scur,tS_{\mathrm{cur},t}16.0683.94
NPO RefSrefS_{\mathrm{ref}}1.7598.25
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}24.5675.44
Scur,tS_{\mathrm{cur},t}17.7482.26
Self-promptSrefS_{\mathrm{ref}}9.5290.48
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}21.4378.57
Scur,tS_{\mathrm{cur},t}14.7485.26
Three-tokenInstructSrefS_{\mathrm{ref}}16.6783.33
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}48.8951.11
Scur,tS_{\mathrm{cur},t}51.4348.57
NPO RefSrefS_{\mathrm{ref}}8.3391.67
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}47.3752.63
Scur,tS_{\mathrm{cur},t}48.6551.35
Self-promptSrefS_{\mathrm{ref}}22.2277.78
Scur,t∖{y}S_{\mathrm{cur},t}\setminus\{y\}51.8548.15
Scur,tS_{\mathrm{cur},t}47.8752.13

These results support two observations. 1. Across different reference models and intervention positions, our anchor set generally reduces continuation leakage compared with the control sets. 2. Attribution analysis shows that most remaining leaks emerge during subsequent generation by the original, non-unlearned model. This is consistent with the model retaining the target knowledge and reintroducing it during continuation. Together, these results provide empirical support for the relative safety of our anchor set.

6. [W5/Q5] Reproducibility

Thanks for your interest in our work. However, as we stated in the "REPRODUCIBILITY STATEMENT," all code, along with configuration files and scripts for reproducing figures and tables, will be released upon publication. Thank you again for your interest in our work, and please wait patiently.


[1] Yang, N., Kim, D., Kwon, J., Kim, M., Jung, K., & Cha, M. (2025). Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning. ArXiv, abs/2509.22263.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.