- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Nonmember-Law matching (NLM) as an algorithm for performing machine unlearning in large language models guided by the principle that an unlearned model should treat the forget set no different than a held out test set of the same distribution that was not trained on. The algorithm achieves unlearning by balancing score-matching inspired optimization using a reference set and the forget set alongside distilling the retain set from the original model to preserve model utility. The authors evaluate their method on two unlearning benchmarks, and achieve state-of-the-art performance on a variety of metrics over a number of baselines.
Overall this paper presents a novel and interesting approach to unlearning, which is sufficiently justified. The work is well written and well presented, but lacks in substantial areas and raises nontrivial questions. Given sufficient clarification to the points outlined below, I would be very open to increasing my score.
Critical strengths
- The work positions itself well within the larger literature and has comprehensive evaluation against baselines which are clearly outlined and well presented. NML seems to outperform most state-of-the-art methods for LLM unlearning through its novel procedure.
- The unlearning method in question is intuitive and interesting, and the energy-based score-matching approach is a powerful technique to leverage for unlearning.
Critical weaknesses
- While I am happy that the authors have provided code alongside their submission - the code does not include evaluation scripts or scripts for reference data generation. The second in particular is in my opinion a critical part of the work.
- All evaluation experiments are performed on the Llama family of models, which calls into question how well NML works on other model families. Can the authors provide such an experiment?
- The method relies greatly on the prerequisite that the distribution of the reference set is approximately equivalent to the distribution of the forget set. This is only outlined in Section 2, and the presentation would be improved if this was stated in the introduction.
- It seems unlikely to me that in practical settings a held-out reference set of the same distribution as the forget set is available, and therefore I argue that the reference set construction case is a far more interesting exploration than when a held-out set is available. The performance of NML on the reference-set construction case is lacking sufficient empirical evidence.
- The authors note that the Retrained model does not obtain zero gaps indicating that forget and held-out records remain distinguishable. This calls into question the validity of the reference set, and the target of the optimization performed in NML. Can this be chalked up to randomness, in which case can the authors report retraining across seeds, or does this suggest that reference sets must be more carefully chosen or constructed. I would welcome discussion from the authors on this point.
- A milder point is that the title is misleading since this technique is tested exclusively for LLMs.
Questions for authors
- In Figure 1, NPO and SimNPO cross the 0.5 threshold and go below it while NML remains close to 0.5 the entire time. This makes me ask, had the baselines achieved unlearning when 0.5 was reached?
- When the reference set is not of the same distribution as the forget set, how does the method behave? It seems to me as if it is hard to verify if the distributions are the same if that is not known apriori.
8 comments
Response to Reviewer Beoj (2/3): W4 & W5 — constructed references; Retrain's nonzero gaps
W4: evidence for constructed references
We agree that the constructed case is important in practice. We add evidence in three parts: constructed references on a second, larger model (TOFU); the LLM-constructed news references, which remain weaker; and, beyond construction, public human-written text as another source of reference.
Constructed references on two models (TOFU). The table compares NML trained with the held-out reference and with a constructed one (3 seeds per row):
| Model | Reference used by NML | Loss gap | Min-K% gap | MU |
|---|---|---|---|---|
| Llama-3.2-1B | Held-out (Table 4) | 0.052 | 0.044 | 0.600 |
| Llama-3.2-1B | Constructed, references 1–3 (Table 4) | 0.077–0.110 | 0.069–0.102 | 0.600–0.601 |
| Llama-2-7B | Held-out (Table 1) | 0.049 | 0.236 | 0.631 |
| Llama-2-7B | Constructed, reference 1 (new) | 0.055 | 0.185 | 0.632 |
On both models, NML with a constructed reference stays close to NML with the held-out reference, at the same utility, and far below SimNPO's loss gap (0.237 on 1B and 0.410 on 7B, Table 1). We will add the 7B row to Table 4.
LLM-constructed news (MUSE-News). These references are coarser, because a language model has difficulty generating long text that is close enough to human-written news. NML's loss gap with them (0.263–0.331, Appendix B) is still below those of NPO (0.498), UnDIAL (0.498) and SimNPO (0.455). Our answer to Q2 gives a check that identifies such a reference before unlearning.
Beyond construction: public text (MUSE-News). This is not a constructed reference, and it applies only when public text of the same genre exists. With BBC articles (the source of MUSE-News) published after the benchmark's period and filtered for overlap with MUSE (3 seeds), NML reaches a loss gap of 0.052, the same as Retrain, against 0.123 with the held-out reference (Table 2).
Taken together, constructed references are supported on TOFU at two model scales and remain weaker for long-form news; where public text of the same genre exists, it is an alternative.
W5: Retrain's nonzero gaps, and what they mean for the reference and the target
It is not randomness. We retrained five models with different seeds on TOFU 1B. For the loss attack, the AUC is 0.396 ± 0.001 across these models and 0.388 for the official Retrain of Table 1, corresponding to gaps of 0.104 and 0.112. The gap reflects a systematic difference between the forget and held-out authors.
What this means for the target. NML's target, the original model's NLLs on the reference, can differ from the ideal target, Retrain's NLLs on the forget set, for two reasons. The first is the model: the original model has seen the forget set. This effect is small: on the same held-out examples, the original model and Retrain are at a KS distance of 0.030–0.045 on TOFU and 0.047–0.055 on MUSE-News. The second is the data: under Retrain, the held-out and forget sets are at 0.215–0.245 on TOFU and 0.133–0.141 on MUSE-News. Retrain's nonzero gaps show this second difference. NML moves the forget NLLs to the reference, so they differ from Retrain's by about this amount, and its gap against held-out examples can fall below Retrain's; we do not read this as better than Retrain. The difference shrinks as the reference becomes more comparable to the forget set, and two results indicate that it is small enough in practice: NML has the lowest KS-D to Retrain on TOFU 1B (0.061, Table 1), and references that do not come from the held-out distribution (constructed QA, public news) still give low gaps against held-out examples.
How much the choice of reference matters. References within a KS distance of 0.30 of the ideal target (Sec. 4.1) give loss gaps of 0.049–0.123 after NML, in no particular order of distance. The LLM-constructed news, at 0.375–0.422, gives 0.263–0.331. The distance thus separates a clearly mismatched reference from acceptable ones and does not rank them. Our answer to Q2 gives a check that needs no Retrain.
Response to Reviewer Beoj (3/3): Q1 & Q2 — Figure 1, mismatched references
Q1: had the baselines achieved unlearning when the AUC reached 0.5?
Not fully: at the crossing, forgetting is not yet complete. We tracked Min-K% AUC and KS-D around the crossings in Figure 1 (TOFU 1B, 3 seeds).
| Step | SimNPO: KS-D / AUC | NPO: KS-D / AUC | NML: KS-D / AUC |
|---|---|---|---|
| 60 | 0.133 / 0.508 | 0.193 / 0.597 | 0.103 / 0.557 |
| 72 | 0.097 / 0.357 | 0.180 / 0.390 | 0.088 / 0.544 |
| 84 | 0.088 / 0.296 | 0.168 / 0.281 | 0.090 / 0.578 |
SimNPO is closest to chance at step 60 (AUC 0.508), where its KS-D is 0.133. Its KS-D reaches 0.088 at step 84, where its AUC has fallen to 0.296, and 0.076 at the final step (Table 1). NPO crosses between steps 60 and 72 with a KS-D of 0.180–0.193, against 0.151 at the final step. For both baselines, the step with a near-chance AUC and the step with the best agreement with Retrain are different steps. The crossing is also brief: within 12 updates, SimNPO's AUC falls from 0.508 to 0.357 and NPO's from 0.597 to 0.390. NML's AUC stays within 0.544–0.578 over the same steps, and its KS-D at the final step is 0.061 (Table 1).
Q2: behavior under a mismatched reference, and how to verify a reference
Behavior. NML degrades gradually, not abruptly. On TOFU 1B we replaced a growing share of the held-out reference with QA about real well-known authors (different difficulty) or about fictional companies (different domain), over 3 seeds:
| Share replaced | Real authors: Loss / Min-K% gap | Fictional companies: Loss / Min-K% gap |
|---|---|---|
| 0% | 0.052 / 0.044 | 0.052 / 0.044 |
| 20% | 0.079 / 0.075 | 0.083 / 0.074 |
| 50% | 0.120 / 0.114 | 0.115 / 0.109 |
| 100% | 0.255 / 0.250 | 0.216 / 0.211 |
MU stays at 0.598–0.601. At 50% replacement, both gaps remain well below SimNPO's (0.237 / 0.252). Full replacement brings them close to that level.
Verification. A mismatch can be detected before unlearning, without a retrained model. We score the forget set and the candidate reference with a model that was trained on neither, and compute the KS distance between the two NLL distributions. On MUSE-News, with three scoring models (the checkpoint before fine-tuning, Phi-1.5 and Llama-3.2-1B), the distance is 0.05–0.14 for public news, 0.09–0.27 for the held-out set and 0.98–1.00 for LLM-constructed news, which is also the reference with the largest final loss gap (0.263–0.331, against 0.052 and 0.123). On TOFU, under three scoring models, the distance is 0.30–0.32 for the held-out reference, 0.43–0.59 when the reference is fully replaced by QA about real authors, and 0.57–0.65 when it is replaced by QA about fictional companies; the loss gaps after NML are 0.052, 0.255 and 0.216. The distance is read relative to a comparable reference of the same task, not against a fixed threshold. We regard the check as a screening test: it separates mismatched references from comparable ones, and we do not claim that it predicts the final loss gap among comparable references. We will add these results to a new appendix on reference validity.
Response to Reviewer Beoj (1/3): W1–W3 & W6 — code, model family, introduction, title
Thank you for the careful review.
W1: code for reference generation and evaluation
Since you identify reference generation as the critical part, we prepared an anonymous repository for it: https://anonymous.4open.science/r/ref-construction-release-C364/
For each benchmark, it includes the constructor script, inputs, prompts, generation settings, acceptance rules and thresholds, three constructed references, and reports listing rejected generations. Neither constructor reads the benchmark's held-out split or the retrained model, and examples are accepted by fixed rules on their text, never by their score under the model to be unlearned. We regenerated all three TOFU references and reran training and evaluation. The resulting means differ from Table 4 by at most 0.004 at the table's precision. For evaluation, we follow the protocol of the OpenUnlearning benchmark and use its public implementations of the metrics and attacks (Sec. 3.1); what we add is the split of held-out nonmembers into a reference fold and an attack fold, described in the same section. We will add the prompts to Appendix A and release the training and evaluation code with the final version.
W2: another model family
NML also works outside the Llama family. We added Phi-1.5 on TOFU Forget10 (3 seeds, same setting as Table 1):
NML has the lowest KS-D and the lowest gap under each attack among the unlearning methods shown, with the same MU as NPO.
We also ran the larger Phi-3.5-mini (3.8B) with the hyperparameters of the Llama setting unchanged (3 seeds). NML has the lowest gap under three of the four attacks (Loss 0.099, zlib 0.031 and Min-K%++ 0.013, against 0.118, 0.152 and 0.100 for NPO, the best baseline there); on Min-K%, NPO is slightly lower (0.079 against 0.089). We will add both tables to the appendix. On a third family, Gemma-7B-it, we have first results under the same unchanged hyperparameters. NML reaches Loss, zlib and Min-K% gaps of 0.045, 0.157 and 0.034 (3 seeds), at or below Retrain's (0.070, 0.162, 0.080), whereas NPO overshoots (Loss gap 0.41–0.49 in the two seeds completed so far). Given the time available, the remaining baselines on Gemma and further model families are still running; we will add them to the appendix.
W3 & W6: the prerequisite in the introduction; the title
We agree with both. The third paragraph of the introduction will state the prerequisite directly after the sentence on where the reference comes from:
If the venue permits a title change, the title will become "Nonmember-Law Matching for Suppressing Unlearning Signatures in Language Models"; in either case, the abstract and introduction will state that the method and its evaluation are for language models.