acceptodds
Under review as a conference paper at ICLR 2027

WADE: Wasserstein Distributional Alignment for Knowledge Erasure in LLMs

Abstract

Machine unlearning for large language models aims to remove the influence of specified forget data while preserving retained knowledge. Retraining on the retained set is the gold standard, but is prohibitively expensive. Existing approximate methods typically optimize proxy forget losses or steer forget activations toward a single target vector, which can suppress forgotten answers without matching the distribution of forget-query behavior. We propose WADE, a closed-form activation-editing method that casts LLM unlearning as distributional alignment in activation space. WADE edits selected MLP down-projection matrices so that the mean and covariance of forget-query activations match a target distribution induced by unknown-entity prompts, under a second-order Wasserstein objective with a retain-key regularizer. A Bures–Procrustes reduction yields a non-iterative, ridge-regularized weight update that provably does not increase the surrogate objective. On TOFU across three LLM families, WADE attains the lowest KS distance to the retrain-only reference among the evaluated baselines at their published settings; under per-method hyperparameter sweeps it provides the low-KS end of the alignment–utility frontier, while NPO attains higher utility and K-FADE reaches a neighboring point at much higher cost. Membership-inference probes show retrain-comparable aggregate leakage, with a remaining gap under Min-K++.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, AI security, LLM

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DZmtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Based on my understanding, this paper is about machine unlearning in large language models and tries not only to forget and retain, but also to make the latent space of forgotten data in the network similar to the network's activations for questions it has never been trained on. It makes limited adjustments to the selected MLP layers to make sure the retain data is not changed that much (!). They test their method against several other well-known methods (like RMU) and on two common unlearning datasets: TOFU and WMDP (Cyber).

Critical strengths
  1. The problem is very interesting, and I think the authors cared about something that is really important, which is: what should the network do after unlearning? Should it answer questions related to the forgotten data with gibberish? Should it say "I do not know"? Should it give a false fact? Getting the response closer to how the network answers questions about data it was never trained on is a good intuition.
  2. The authors provided a closed-form edit. This will help a lot with computational costs and avoid the need for complex optimization steps.
Critical weaknesses
  1. The method works with mean and covariance activations of the forget, retain, and target sets. That is not necessarily a problem, but based on personal experience, activations can be highly complex in cases where we have multimodal data and complex concepts. Changing those mean and covariance activations can damage other objectives we have, and this can result in behaviors of the network that cannot be simply explained, since these activations are highly entangled and it gets worse as the problem gets more complex.
  2. Based on what I understand, this method only works with a few layers for weight editing. I can understand that the authors wanted to make sure their method remains efficient by choosing a few layers and I understand they evaluate different editing windows and choose layers based on that. If this method is to be used on another problem, how should we find the layers to change? it is not efficient to do this for all the time. Also, in unlearning problems, even after unlearning, if we look at early and middle layers, we can see that features of the forgotten data are still present and, in this case, are different from unknown-information representations in our network. So, while the network behaves as if it has never seen that data, its internal representations can show signs of forgotten data.
Questions for authors
  1. Based on the first weakness, can the authors justify why they think using mean and covariance activations is not problematic, and do they have an alternative way of getting the activations for more complex settings?
  2. Based on the second weakness, can the authors propose a way to detect which layers should be selected for weight editing? Also, can we look at t-SNE, PCA, or any other method that tells us about the internal representations of early and middle layers (especially the layers that were not selected for weight editing) of the network on forgotten data versus retain and target data?
6 comments
AuthorsOpenReview ↗

Response to Reviewer DZmt: W1/Q1 & W2/Q2 — two moments, layer selection, internal representations

Thank you for the careful and precise review. Both of your concerns pointed to analyses the paper was missing, and each led to new experiments that we report below.

W1 & Q1: Why mean and covariance, and what to use in more complex settings

Two moments are what a linear weight edit can control, and the solve restricts where the edit acts. Because the edited map is linear, the mean and covariance of the keys determine the mean and covariance of its outputs, and no further statistic can be set independently. The update also has limited support: every row of ΔW lies in the span of the forget pseudo-keys and the retain keys, so an input whose key is orthogonal to this span is left unchanged, and the retain term penalizes changes on the retain keys inside it (Appendix C).

The entanglement you describe does appear for inputs close to the forget set. On TOFU forget10 with Llama-2-7B, the probability of the correct answer on retained fictitious authors, who share the distribution of the forget set, is 0.497 after the edit (Retrain: 0.989), while ROUGE on real authors and world facts is 0.822 and 0.849 (Retrain: 0.915 and 0.902). The edit is therefore safe for knowledge outside the forget distribution, and its cost falls on inputs entangled with the forget set. We will add this to the limitations (Sec. 5).

For more complex settings, in which the forget keys form several modes, one mean and covariance is too coarse a summary. The natural extension is a mixture: cluster the forget keys, apply the same closed-form solve to each component and add the updates; with one component it reduces to WADE. We have not implemented it, and our experiments are text-only.

W2 & Q2: Selecting the layers, and internal representations in early and middle layers

How to find the layers to edit. The window should be re-selected for a new model, and we propose to restrict the search to roughly the first third of the layers. The basis is prior work and our own check. Causal tracing in ROME and MEMIT locates factual associations in early-to-middle MLP layers. On Llama-2-7B (32 layers) we verified this in two ways. We ran causal tracing ourselves: the five layers with the highest single-layer effect are 4, 8, 5, 7 and 9. We also swept edit windows over the whole depth (Fig. 1(a)): the gain is concentrated in the shallow layers. Layers 4–8 give the largest average reductions in KS-D to Retrain (above 0.20), and later layers give smaller gains. This restriction makes the search cheaper, and each candidate is a closed-form edit of 7.1 s per layer (Table 5).

Internal representations. Your expectation is confirmed, and it holds for every method we probed: features of the forgotten data remain. To examine the representation, we read the residual stream at the last subject token in every layer, including the layers that are not edited, and trained a linear probe to separate forget-set authors from holdout authors that no model was trained on (5-fold cross-validation grouped by author). An AUC of 0.5 means that the two groups cannot be told apart. Retrain has seen neither group, so its AUC is the no-trace level.

Probe AUC (forget vs. holdout authors)Layer 4Layer 8Layer 12Layer 16Layer 24Layer 32
Retrain0.4560.5070.6280.6520.4590.596
Finetuned0.4950.8630.9040.9000.7060.726
NPO0.5841.0001.0001.0000.9820.988
SimNPO0.6320.9641.0001.0000.9911.000
RMU0.9401.0000.9630.8750.7020.725
WADE0.4950.8960.8980.8680.7160.781

None of the methods reaches Retrain's level: from layer 8 on, the forget authors remain linearly separable after unlearning with WADE, NPO, SimNPO and RMU alike. The methods differ in one respect. WADE stays close to Finetuned (layer 4, before the edit window, is identical by construction), whereas NPO and SimNPO make the forget authors more separable than before unlearning (0.96–1.00 from layer 8 on), and RMU does so in the early layers (0.940 at layer 4, 1.000 at layer 8). Behavioral indistinguishability, which the paper claims (Sec. 3.1), therefore does not imply that the representation is erased, for WADE or for the baselines: WADE changes how these states are mapped to outputs, and the outputs are what move to Retrain (KS-D 0.048, Table 1). We will add the probe to the appendix and state this in the limitations (Sec. 5).

Reviewer_DZmtOpenReview ↗

Rebuttal response

Thank you authors for your rebuttal. First point is valid. for the second point; I expected your method to have less probe accuracy in hidden layers for forget vs retained since your method is pushing the responses to forget set towards space of answers the network gives for data that never trained on. Can authors clarify why this did not show itself in probe accuracy? What is the difference between retrained network and WADE that stopped your method to be closer to probe accuracy that retrained gained. Thank you again for your rebuttal.

AuthorsOpenReview ↗

Response to Reviewer DZmt (follow-up)

Why the probe accuracy does not decrease

One clarification first: the probe separates forget authors from holdout authors that no model was trained on, not from retained authors.

WADE matches the mean and covariance of the MLP outputs at the edited layers to those of the reference prompts. This constrains two statistics of the forget set in one component of the residual stream. It does not move each forget example to the state of an unseen author, so the forget authors remain linearly separable from the holdout authors.

Difference from Retrain. Retrain was never trained on the forget authors, so their states stay close to those of holdout authors in every layer. WADE starts from a model that has learned them and relocates their MLP outputs at the edited layers. This moves the output distribution to Retrain's (KS-D 0.048, Table 1) without returning the states to where unseen authors lie. The two are compatible because the output depends only on the directions that later layers read, whereas a probe can use any direction.

WADE therefore reaches Retrain in behavior, and we do not claim that it does so in representation. We will extend the representation analysis with further experiments.

Reviewer_c7c7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies LLM unlearning, where the goal is to make a model behave like one retrained on the retain set. Existing unlearning methods typically steer forget representations toward a single target vector rather than matching the distribution of forgetting behavior. To address this, the paper proposes WADE, which aligns the distribution of forget activations with a target activation distribution associated with unknown-answer behavior elicited by unknown-entity prompts. It then edits the MLP down-projection weights so that the mean and covariance of forget activations match those of the target distribution. Experiments show that WADE achieves the lowest KS distance to the retrained model among the evaluated baselines on TOFU. It also reduces WMDP-Cyber accuracy while preserving MMLU performance.

Critical strengths

The idea of matching a distribution is simple and easy to follow, and the update is closed-form and computationally inexpensive.

Critical weaknesses
  1. The paper defines the unlearning goal as matching the behavior of a retrained model. However, where does this retrain-like behavior come from? Why should matching the covariance of activations produce a retrain-like distribution? The target is an unknown-entity proxy rather than the retrained model. It would be useful to explain why aligning activations with the distribution associated with unfamiliar names should produce a truth-ratio distribution resembling that of a retrained model. Otherwise, there appears to be a mismatch between the stated unlearning goal and the optimization target.

  2. The novelty is limited, and several relevant works are missing [3]. The update follows MEMIT's normal equation. Matching the mean and covariance of activations, or transporting one activation distribution to another, has already been explored in the steering literature [1, 2], but these works are not discussed. Thus, the main new technical contribution appears to be applying these ideas to unlearning, with unknown-entity prompts defining the target distribution. The novelty should be further clarified. Moreover, the gains do not clearly make up for the limited novelty.

[1] Singh, Shashwat, et al. “Representation surgery: Theory and practice of affine steering.” arXiv preprint arXiv:2402.09631 (2024).

[2] Rodriguez, Pau, et al. “Controlling language and diffusion models by transporting activations.” International Conference on Learning Representations. Vol. 2025. 2025.

[3] Yamashita, Tomoya, et al. “Sparse-Autoencoder-Guided Internal Representation Unlearning for Large Language Models.” arXiv preprint arXiv:2509.15631 (2025).

Questions for authors
  1. Table 1 uses bold formatting only for FQ and KS-D. WADE does not achieve the best MU on any model, and its TR on Qwen3-8B is not the closest to that of the retrained model.

  2. In Table 1, GD on Qwen3-8B has the same FQ and KS-D as the Finetuned model, but different TR and MU values. Is this correct?

  3. In Table 11, the MU of the mean-only variant drops to 0.223, even though its update norm is much smaller than WADE's. Why does a smaller edit hurt utility so much?

  4. What does the unlearned model actually output in response to forget questions? Does it say "I don’t know", or does it fabricate new facts? Also, since the activations used for the edit are collected at the author-name token, does forgetting still hold for paraphrased questions or questions that do not mention the name directly?

  5. How would the authors choose lambda for a new model without a retrained reference model, since the sweep in Table 10(a) relies on KS-D to the retrained model?

  6. Where are the full results, including those of the baselines, for TOFU-forget01 and TOFU-forget05? Since the first step is to construct the distribution, the size of the forget set may affect performance.

3 comments
AuthorsOpenReview ↗

Response to Reviewer c7c7 (1/2): W1 & W2 — why the target gives Retrain-like behavior; novelty and gains

Thank you for the careful review.

W1: Why matching activation statistics gives a Retrain-like TR distribution

The optimization target and the evaluation reference are two different objects, and the paper's claim concerns the second: WADE is judged by how close its outputs are to Retrain's, not by reproducing Retrain's activations (Sec. 3.2).

Where the Retrain-like behavior comes from. Retrain was never trained on the forget authors, so for Retrain they are unknown entities, and its answers to the forget questions are those of a model asked about someone it has not learned. The reference is collected from the same model that is being edited, on the same question templates with the author replaced by an unknown name. It therefore records how this model processes a question about an author it has not learned. Moving the forget activations to the reference makes the edited model process the forget questions in the same way, which is the behavior Retrain shows on the forget set. Whether the resulting outputs match Retrain is then checked directly: on TOFU forget10 with Llama-2-7B, the KS-D between the TR distributions of WADE and Retrain is 0.048 (FQ 0.758), against 0.373 for Finetuned (Table 1).

Why the covariance. Two sets of results isolate its contribution on Llama-2-7B forget10 (Tables 2 and 11).

Target (same prompts, layers and λ)KS-D to Retrain ↓MU ↑
Mean and covariance (WADE)0.0480.578
Mean only0.2300.223
Mean only, scaled to WADE's update norm0.3450.000
Covariance only0.2680.574
Mean and covariance with the same spectrum, randomly rotated0.0820.442
Mean and covariance with shuffled eigenvalues0.1510.374

The mean alone leaves KS-D at 0.230, and enlarging the mean-only update to WADE's norm makes it worse (0.345). The covariance alone gives 0.268. Keeping the spectrum and changing only the directions of the target covariance raises KS-D from 0.048 to 0.082 and lowers MU from 0.578 to 0.442. The gain therefore comes from the mean and the target's own covariance directions together, not from the size, rank or spectrum of the update. The same ordering holds on the other four models in Table 2, where mean-only alignment gives KS-D 0.215–0.238 and WADE 0.088–0.160.

The objective does not by itself guarantee a Retrain-like TR distribution. It moves the forget set to the state that Retrain is in with respect to these authors, and the output-level match is an empirical result that we verify against Retrain. We will make this distinction explicit at the start of Sec. 3.

W2: Novelty, related work and the size of the gains

We agree on the sources and will cite and discuss [1–3]: matching or transporting activation distributions has been explored for steering [1, 2], and the normal equation is MEMIT's (Sec. 3.3). [3] is the closest in idea: it also moves the internal activations of the forget entity toward those of unknown entities. It does so with a hinge loss on selected sparse-autoencoder latents, minimized by gradient descent for each target entity. WADE needs no autoencoder and no training: it matches the mean and covariance of the MLP outputs over the whole forget set, under a retain penalty, and solves for the weights in closed form.

WADE's contribution is the formulation that turns such a distributional target into a weight edit. Steering transforms activations at inference. WADE instead solves for a weight matrix that maps the forget keys to the target distribution under a retain penalty. The covariance term of this problem is not quadratic in the weights, so the normal equation does not apply to it directly. WADE rewrites the term in its Procrustes form, fixes the orthogonal coupling from the pre-edit weights, and obtains r+1 pseudo key–value pairs that the normal equation solves, where MEMIT inserts one optimized value per fact. This is a single step rather than a joint solve (Appendix C), and it never increases the surrogate objective (Proposition 1).

Gains. On TOFU forget10 with Llama-2-7B, WADE has KS-D 0.048 to Retrain, against 0.100 for NPO and 0.113 for LUNAR (Table 1), with a three-layer edit of 21.3 s (7.1 s per layer) against 497.8 s of training for NPO (Appendix B, Table 5). With the same solver and a mean-only target, KS-D is 0.230 (Table 2), so the gain comes from the distributional constraints, not from the normal equation.

AuthorsOpenReview ↗

Response to Reviewer c7c7 (2/2): Q1–Q6 — Table 1, mean-only utility, model outputs, choice of λ, forget05/01

Q1: Bold formatting in Table 1

We will bold the best value in every column of Table 1, including TR and MU.

Q2: GD on Qwen3-8B has the same FQ and KS-D as Finetuned

These values are consistent. KS-D is the largest gap between two empirical CDFs, so two different TR distributions can have the same KS-D, and with 400 forget examples it only takes values on a grid of 1/400. At a fixed sample size FQ is a function of KS-D, so equal KS-D gives equal FQ. TR is a mean and MU is computed on other data, so both can differ.

Q3: Why the smaller mean-only edit hurts utility more

Utility is not determined by the size of the update. In Table 11, the covariance-only edit has an update norm of 28.96 and keeps MU 0.574, while the mean-only edit has a norm of 7.12 and MU 0.223; scaling the mean-only edit to WADE's norm (32.67) brings MU to 0.000. What differs is the target: the mean-only edit sends every forget key toward one output vector. Its damage is not refusal. The mean-only model refuses 2.8% of retain questions, and its retain answers become long and inaccurate (72.5 words on average and ROUGE-L recall 0.185, against 24.9 words and 0.975 for Retrain). We do not have a mechanistic account beyond this, and we will not attribute utility to update size in the ablation discussion.

Q4: What the model outputs, and forgetting under rewording

Outputs. The model fabricates, as Retrain does. WADE refuses 0.7% of the forget questions (Retrain: 2.0%) and otherwise gives a fluent answer that is not the gold answer. Example: "What genre is the author Raven Marais particularly noted for?" Gold: film. Retrain: "…noted for their contributions to the Gothic genre." WADE: "…noted for their engaging and dynamic contributions to the broader LGBTQ+ community." This is the behavior the reference encodes: it is collected from the fine-tuned model on unknown-author prompts, and a model fine-tuned on TOFU answers questions about an author it has not learned in the same confident format, which is what Retrain does on the forget set. The refusal-style "target-state response" in the Appendix A box is an illustration, not a model output; we will replace it with real outputs such as the one above.

Paraphrased questions and questions that do not mention the name. Forgetting holds in both cases. Each is evaluated against Retrain on the same questions (forget10, Llama-2-7B). On the official TOFU paraphrases of the forget questions, WADE's KS-D to Retrain is 0.065 and its forget probability is 0.152 (Retrain: 0.124). On questions that replace the author's name with a description of the author, KS-D is 0.120 and the forget probability is 0.093 (Retrain: 0.081). Finetuned still answers both kinds of question (forget probability 0.677 and 0.627), so the knowledge can be reached without the original wording or the name, and the edit removes it in both cases.

Q5: Choosing λ without a Retrain reference

λ is not selected with the Retrain model. We fixed λ = 0.01 a priori and use it for all models, all TOFU splits and WMDP-Cyber (Sec. 4.1). Table 10(a) is a sensitivity analysis around this value, not the selection procedure; for a new model we would use the same default.

Q6: Full results on TOFU forget05 and forget01, and the effect of forget-set size

The table gives the results for all methods on Llama-2-7B; Retrain is the official reference of each split.

Methodforget05 (200 examples): FQ ↑forget05: KS-D ↓forget05: MU ↑forget01 (40 examples): FQ ↑forget01: KS-D ↓forget01: MU ↑
WADE0.2700.1000.6140.9190.1250.635
NPO0.4660.0850.6000.9190.1250.643
SimNPO0.1120.1200.6330.1650.2500.622
LUNAR0.1120.1200.2630.0290.3250.556
RMU4.8e-050.2300.3600.0540.3000.622
AlphaEdit2.6e-140.3950.6290.00130.4250.628

GD and GA collapse on both splits (KS-D ≥ 0.95). The size of the forget set does affect WADE: its KS-D is 0.048, 0.100 and 0.125 on forget10, forget05 and forget01. With fewer forget examples the mean and covariance are estimated from fewer keys, and with 40 examples the covariance has rank at most 39. WADE nevertheless stays among the methods closest to Retrain on every split: on forget05, NPO (0.085) and WADE (0.100) are the two closest, and on forget01 they tie at 0.125; with 40 examples the KS-D grid is 0.025, and FQ rejects neither. WADE reaches this without gradient training. Its closed-form solve does not grow with the forget set, and only the forward passes that collect the keys do, whereas the cost of NPO grows with the number of forget examples times the number of epochs; on forget10 the three-layer edit takes 21.3 s against 497.8 s for NPO (Table 5).

Thank you for the detailed rebuttal and the newly added experiments. Most of my questions are answered. However, a few concerns remain.

After the clarification, the novelty still looks moderate. If I understand correctly, the main contribution is turning the mean and covariance target into a closed-form weight edit, and the key new technical step is the Procrustes rewriting that turns the covariance term into extra key–value pairs for the MEMIT-style solve. Similar ideas, such as matching activation distributions and using unknown entities as the target, have been explored in prior work.

Besides, the new forget05 and forget01 results show that the KS-D advantage mostly holds on forget10. On forget05 NPO is better (FQ 0.466 vs 0.270), and on forget01 the two are tied. So on smaller forget sets, the main benefit of WADE seems not to be better unlearning. The gains do not clearly make up for the limited novelty.

Based on this, I still think weak reject is a reasonable score, and I will keep my score.

Reviewer_FHkvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a closed-form activation-editing method for LLM unlearning, WADE. Instead of suppressing forgotten knowledge through optimization-based losses or steering activations toward a single target vector, WADE aligns the mean and covariance of forget-query activations with a target distribution constructed from unknown-entity prompts. The authors formulate this alignment using a second-order Wasserstein objective and derive a closed-form weight update with retain-key regularization. Experiments on TOFU and WMDP-Cyber demonstrate promising retrain-reference alignment and forgetting performance across several LLMs.

Critical strengths
  1. The paper addresses an important limitation of existing machine unlearning approaches by focusing on matching retrain-like behavior rather than merely suppressing forgotten answers.

  2. The distributional alignment formulation provides an interesting alternative to point-target activation steering. The closed-form editing solution also avoids iterative gradient-based optimization.

  3. The experiments cover multiple LLM families and include comparisons with recent unlearning methods, ablation studies, hyperparameter analyses, and membership-inference evaluations.

  4. The paper provides detailed mathematical derivations and implementation information. The authors also include code in the supplementary materials, which supports reproducibility.

Critical weaknesses
  1. The method constructs its target activation distribution using synthetic unknown-entity prompts. However, it remains unclear how closely this distribution approximates the behavior of a model retrained without the forget data. Further analysis of this relationship would strengthen the motivation.

  2. Although WADE achieves strong retrain-reference alignment, its model utility is lower than that of some competing methods, particularly NPO. Further discussion of this trade-off would help clarify the practical advantages of WADE.

  3. The experiments primarily focus on entity-level forgetting using TOFU, with an additional evaluation on WMDP-Cyber. Evaluation on more diverse forgetting scenarios, such as selective association-level forgetting, would strengthen the generalizability of the method.

  4. WADE aligns only the first two moments of activation distributions, which may not capture higher-order differences. The membership-inference results also show remaining leakage under Min-K++. Further discussion of these limitations would help clarify the extent to which WADE approximates retraining.

Questions for authors
  1. Could the authors further discuss how well the activation distributions constructed from synthetic unknown-entity prompts approximate those of a retrain-only model? How sensitive is WADE to potential differences between these distributions?

  2. WADE achieves strong retrain-reference alignment but does not consistently outperform methods such as NPO in model utility. Could the authors clarify in which unlearning scenarios WADE offers the most favorable trade-off between forgetting quality and retained capabilities?

  3. Could the authors discuss how WADE is expected to generalize to selective association-level forgetting, where only specific relations about an entity should be removed while preserving other knowledge about the same entity?

  4. Since WADE matches only the first two moments of activation distributions, could the authors further discuss the implications of higher-order distributional differences for unlearning effectiveness, particularly in light of the remaining membership-inference leakage under Min-K++?

2 comments
AuthorsOpenReview ↗

Response to Reviewer FHkv: W1–W4 & Q1–Q4 — reference, trade-off with NPO, association-level forgetting, Min-K++

Thank you for the careful review.

W1 & Q1: How well the reference approximates Retrain, and sensitivity to the difference

We do not claim that the reference activations are close to those of Retrain, and WADE does not rely on it: Sec. 3.2 defines the reference as an intervention state, and agreement with Retrain is evaluated at the output. The reference is nevertheless chosen for a reason. Retrain was never trained on the forget authors, so it answers the forget questions as a model that has not learned these authors. The reference is collected from the model being edited, on the same question templates with the author replaced by an unknown name, so it records how this model processes a question about an author it has not learned. Moving the forget activations to this state gives outputs whose TR distribution has KS-D 0.048 to Retrain on TOFU forget10 with Llama-2-7B, against 0.373 for Finetuned (Table 1).

On sensitivity: WADE uses only the mean and covariance of the reference activations over many substituted names, so it is not tied to any individual prompt: across five disjoint sets of fictitious names, KS-D varies within 0.010 and MU within 0.0062 (Appendix A). What it is sensitive to is the kind of state the reference induces. A reference that induces a different state would move the forget set elsewhere, and this would appear as a larger KS-D to Retrain, which is why we evaluate against Retrain at the output and not in activation space.

W2 & Q2: Where WADE offers the most favorable trade-off

NPO retains more utility than WADE (MU 0.621 against 0.578 on forget10 with Llama-2-7B, Table 1). WADE is the better choice in three situations.

When closeness to Retrain matters most and compute is limited. WADE has KS-D 0.048 against 0.100 for NPO (Table 1), with a closed-form three-layer edit of 21.3 s (7.1 s per layer) against 497.8 s of training for NPO (Appendix B, Table 5).

When forgetting must hold for paraphrased questions and questions that do not mention the name. On the official TOFU paraphrases, WADE's KS-D to Retrain is 0.065 and NPO's is 0.078. On questions that replace the author's name with a description, it is 0.120 for WADE and 0.148 for NPO.

When the knowledge to keep lies outside the forget distribution. WADE's decrease in MU comes from the retained fictitious authors, who share the distribution of the forget set. On real authors and world facts its ROUGE is 0.822 and 0.849 (Retrain: 0.915 and 0.902).

W3 & Q3: Selective association-level forgetting

The formulation covers it, and we have not evaluated it. To forget an association (s, r, o) while keeping other facts about s, the forget prompts are the queries about (s, r), the target prompts are the same queries with s replaced by a synthetic entity, and queries about the other relations of s are added to the retain set, so the retain term in the same closed-form solve protects them (Sec. 3.2, Appendix A). We make no performance claim for this variant.

W4 & Q4: Higher-order differences and the leakage under Min-K++

Matching two moments fixes the location and spread of the forget activations at the edited layers and leaves their shape and tails free. For a linear edit this is also the limit: once the mean and covariance of its outputs are matched, no further statistic can be set independently. The consequence shows in what WADE does and does not match. The TR distribution is close to Retrain's (KS-D 0.048), and averaged over the four attacks the membership signal is at Retrain's level (mean |AUC − 0.5| of 0.099 against 0.101, Table 3). Min-K++ is the exception: its AUC is 0.787 for WADE and 0.502 for Retrain (Appendix D). Min-K++ normalizes each token's log-probability by the mean and variance of the log-probability over the vocabulary, so it responds to the shape of the next-token distribution, which a two-moment match of activations does not control. We regard this as the likely source of the gap, as Sec. 5 notes, and we do not claim that WADE removes it.

Reviewer_FHkvOpenReview ↗

Response to authors' rebuttal

Thank you for the detailed response and additional analyses. I appreciate the authors' clarifications and will maintain my original score.

Comments on the paper

1 comment

The authors have posted their rebuttal, and the discussion period will end soon. Could you please read it and reply at your earliest convenience, stating whether your concerns have been addressed and whether your rating changes? Thank you so much for your active participation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.