acceptodds
Under review as a conference paper at ICLR 2027

What Survives Unlearning? Calibrated Subject-Level Auditing of Residual Knowledge in LLMs

Abstract

Unlearning for large language models (LLMs) is evaluated almost exclusively through forget-set averages, which cannot tell whether each forgotten subject has become indistinguishable from a model that never saw it. We introduce ReCAUL, a post-hoc, method-agnostic audit that answers this subject-level question at an unchanged query budget. ReCAUL compares an unlearned checkpoint with deletion references trained without the forgotten subjects, calibrates each relation and wording against a reference-only null, subtracts a matched retained subject, and cross-fits two wordings so that one selects each subject's strongest relation and the other confirms it; its randomization test repeats the selection in every draw. On TOFU with Llama-3.1-8B, the resulting statistic, CF-PMAX, exposes 2.6 times the residual found by uniform aggregation of the same ten queries (6.52 vs. 2.54; ) in a task-vector checkpoint that passes the aggregate screen, with positive residual in all 24 audited subject blocks. The result replicates for a gradient-difference checkpoint, on the same relations (6.24; 24/24), holds for negative preference optimization (NPO; 4.79; 22/24), and stays positive under recalibration, re-pairing, and single references, while reference-only pseudo-candidates trigger it at a near-nominal 4.9% rate. Prompted with the audited relation, the NPO checkpoint regenerates more of the forgotten answer than its deletion references, while its retained subjects show no such gap. Code and frozen score rows that reproduce the primary audit are provided in the supplementary material.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, large language models, unlearning evaluation, auditing, privacy

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_onqzOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel unlearning auditing method to evaluate whether machine unlearning methods achieve subject-level unlearning, made the model unlearns information about a specific subject. The core concept involves pairing each checkpoint with three split-mathced deletion references as benchmarks for the audit. The paper performs large-scale experiments, a comprehensive review of related work and a clearly structured description of the proposed method.

Critical strengths
  1. The experiments are thorough, and the results are reported in detail, providing sufficient data to support the conclusions.
  2. The review of related work is comprehensive, clearly situating the present study within the context of existing research.
  3. The methodology section is well-structured, with each paragraph serving a clear purpose, making it easy for readers to follow the author’s line of reasoning.
Critical weaknesses
  1. Existing approaches, such as RWKU and LUCID, also focus on subject-level unlearning audits. However, the paper does not provide comparison between the proposed audit method and these existing ones, making it difficult to substantiate the advantages of the new approach.
  2. This method requires three ideal retraining models that have never seen the data to be unlearned as references for each checkpoint. This not only renders the evaluation process prohibitively expensive but also creates an excessive reliance on specific datasets. Furthermore, if such ideal reference models were readily available, the necessity for performing unlearning would be called into question.
  3. The meaning of Figure 1 is unclear (while the subject profile on the left is visible, the process flow on the right is difficult to interpret). In Figure 2, the reading order for the lower section is counterintuitive. While the experimental plots (e.g., Figure 5) are aesthetical, they lack clarity in conveying information.
  4. In sections such as 5.3, the paper lacks an interpretation of the underlying meaning of the reported figure. For instance, what is specifically meant by "behaviorally meaningful," and how is the term "meaningful" defined? Readers are likely to become overwhelmed by the raw numerical data, struggling to grasp its actual implications.
Questions for authors
  1. Have previous evaluation efforts not considered subject-level auditing? What are the essential differences and advantages of your audit method compared with RWKU, LUCID and others that also focus on subject-level unlearning audits? Why are there no direct comparative experiments?
  2. The audit requires three split-matched deletion references. What is the cost of obtaining this reference model, and does this strong dependence mean that the evaluation method is overly coupled to the data set and difficult to generalize to new scenarios?
  3. What does the right part of Figure 1 want to express?
  4. The lower part of Figure 2 uses reverse reading order. Is this intentional?
  5. What are the actual meanings of the various indicators in the experimental graph (Figure 5)? Can you explain the results in a more intuitive way (such as specific behavioral examples)?
  6. Section 5.3 mentions that unlearning has "behaviorally meaningful" - how is the "behaviorally meaningful" here defined and quantified? What counts as "meaningful"? Can an interpretive framework be provided that goes beyond a list of numbers?
7 comments
AuthorsOpenReview ↗

Response to Reviewer onqz (W1 & Q1)

Thank you for the detailed review. We answer each question below.

W1 and Q1. How does the audit differ from RWKU, LUCID, and others, and why no direct comparison?

RWKU and LUCID do study what remains of specific targets, but they report averages or counts over many targets, and they read the answers the model generates. They do not say, for each person, whether that person is still remembered, with a known error rate. Our audit does, with a yes/no answer (a verdict) for each person. It checks whether the unlearned model still picks the person's true answer out of about 100 candidates more clearly than models that never learned the person. The closest per-person method, U-LiRA+ (Du et al.), trains about 100 shadow models, while ours reuses the same three retrained models for every person.

We agree that a direct comparison was missing. These audits give no decision for each person, so in the revision we computed their scores for each person on our models, with the same threshold for all audits (Section 5.2). AUROC measures how well a score separates NPO's forgotten people from retrained models without any threshold.

Table 1. Share of forgotten people found still remembered (%) on Llama-3.1-8B at one common threshold, with AUROC and cost in forward passes per person.

AuditTaskVecGradDiffNPOAUROC (NPO)Forward passes
Greedy recall (RWKU)96100330.70238
Exact recall (LUCID)6763330.67238
Leak@32100100630.959,851
Beam-10 recall96100420.793,670
Ours100100920.99942
Ours, at the cost of greedy recall9696830.98234

TaskVec and GradDiff forget little, and most audits find nearly all of their people, so the audits agree where they should. NPO forgets the most, and there ours finds 92% where the others find 33% to 63%, at a tenth of the cost of Leak@32. With the stricter threshold of the verdict, ours still finds 79%.

The gap holds in two harder cases, NPO on 30 held-out people and IdkDPO, which answers "I don't know" to 58% of the questions about forgotten people.

Table 2. Share of forgotten people found still remembered (%) in two harder cases, with the same threshold for all audits.

AuditNPO, 30 held-out peopleIdkDPO, 24 people
Greedy recall (RWKU)304
Exact recall (LUCID)2333
Exact match5050
Beam-10 recall2750
Ours8792

In both cases, the existing audits find at most 50% of the forgotten people, and ours finds 87% and 92%. Six of NPO's 24 audited people are missed by every other audit, and ours flags all six. No model that never learned the people is flagged (Table 2 of the correction comment), so the unlearning method alone does not explain what our audit finds.

Existing audits read the model's answers and miss many people that the model still remembers. Our audit finds them, at lower cost, with an answer for each person.

AuthorsOpenReview ↗

Response to Reviewer onqz (W2, W3, Q2, Q3 & Q4)

W2 and Q2. What does a retrained model cost? Is the audit tied to one dataset? And if retrained models exist, why unlearn?

The cost is a one-time setup for a pool of people (everyone whose deletion might be requested), the audit is not tied to one dataset, and it does not replace unlearning.

Table 1. Cost of the audit.

StepHow oftenCost
Train a retrained modelonce per pool of peopleabout 2 GPU-hours
Apply unlearning to a retrained modelonce per method25 to 50 training steps
Score the unlearned modelevery request942 forward passes per person

A retrained model is trained once for a pool of people, not for each checkpoint. We trained three on one A800 GPU, and two such models are enough for the audit. If retraining is impossible, a quick check compares the unlearned model with the public model it started from and needs no retrained model.

Table 2. Quick check without any retrained model. Share of forgotten people flagged (%). The last row is a test on models that should not be flagged.

ModelFlagged
TaskVec100
GradDiff100
NPO75
Retrained models0

The quick check flags 75% to 100% of the forgotten people and no retrained model.

The audit is not tied to TOFU. For facts about people, it needs candidate answers of the same type as the true one. For free text such as news, it needs only the text and one retrained model, and it reads how likely the unlearned model finds each forgotten article. In the revision, we ran it on real news (MUSE-News) with MUSE's public retrained model and no training. Three of five public unlearned models get verdicts. For the other two, which MUSE rates as over-forgotten, the audit makes no claim.

Table 3. Share of forgotten articles still remembered (%) on MUSE-News. Models are named by method and uploader.

ModelStill remembered
SimNPO (OPTML)71
NPO (LV)83
RMU (LV)89

These three models still remember 71% to 89% of the forgotten articles.

Retrained models do not make unlearning unnecessary. A retrained model is only an evaluation tool, as in TOFU and MUSE. It is built once to test an unlearning method, and each deletion request is then handled by unlearning, without retraining.

The audit needs a one-time setup plus cheap scoring per request, works on new data such as news, and checks unlearning without replacing it (Sections 5.7, 5.8).

W3 and Q3. What does the right part of Figure 1 express?

It showed that the same NPO model can look forgotten on average but still remember a specific person. In the revision, Figure 1 shows this with real numbers. The average loss on forgotten answers rises from 0.01 before unlearning to 1.74 after NPO, close to the 1.92 of retrained models, so NPO looks forgotten. Yet for one forgotten author, Bao Nguyen, two of the three NPO models (trained with three random seeds) give the true main genre in all ten generated answers, and no retrained model ever does.

W3 and Q4. Is the reverse reading order in the lower part of Figure 2 intentional?

It was, but it was confusing. In the revision, Figure 2 reads from top to bottom and left to right, and follows one forgotten author through the audit.

AuthorsOpenReview ↗

Response to Reviewer onqz (W3, W4, Q5 & Q6)

W3 and Q5. What do the indicators in Figure 5 mean?

Panels (a) to (f) are robustness checks, and all pass. Panels (a) to (e) each change one choice we fixed in advance and ask whether the score of the submitted audit (6.52) stays positive. Panels (g) and (h) compare our original score with a simple average. In the revision, we moved the figure to the appendix as Figure 14 and gave each panel a plain title.

Table 1. The robustness checks of Figure 14.

PanelChangeResult
(a)calibration people redrawn 6,024 timesalways positive
(b)pairs redrawn 5,000 timesalways positive
(c)one retrained model at a timelowest score 6.12
(d)controls removed from the scorerlowest score 5.48
(e)only the closest pairslowest score 6.55
(f)false alarms on retrained modelsrate 0.049

Panels (a) to (e) leave the score positive, and panel (f) shows false alarms at the designed rate of 0.05. Panels (g) and (h) show our original score next to a simple average of the same questions, 6.52 against 2.54. A model that was never unlearned shows a similar gap, so the revision no longer reads this gap as evidence (see our correction comment). For a concrete example of what the score means in behavior, see the forgotten author Chiamaka Adebayo under Q6 below, and Bao Nguyen in Figure 1.

W4 and Q6. What does "behaviorally meaningful" mean, and how is it measured?

In the revision, we replaced the phrase with three clear levels of what a model still knows about a forgotten person. A level counts as meaningful when the unlearned model is above models that never learned the person.

Table 2. The three levels. Likelihood is a standardized score, recognition a share of questions, and generation the recall above retrained models.

LevelQuestionTaskVecGradDiffNPORetrained models
LikelihoodIs the true answer more likely?3.102.700.960
RecognitionIs it ranked first among about 100?35%38%24%6%
GenerationIs it said when asked?+0.45+0.45+0.080

The gaps between levels show where unlearning acts. NPO lowers its confidence in all answers, so its likelihood falls, yet it still picks the true answer first far more often than retrained models (24% against 6%). For example, asked for the main genre of the forgotten author Chiamaka Adebayo, NPO gives the true genre (Queer), while all three retrained models answer Young Adult Fiction.

The verdict uses recognition, because a general loss of confidence cannot fake it. To help readers with the many numbers, the paper now has a reading guide (Table 3 of the paper) that says what each number means (Sections 3.3, 5.5).

We hope these answers are clear. If you have any further questions, please let us know, and we will answer as soon as possible. If the revision addresses your concerns, we would be grateful if you could take it into account in your final assessment. Thank you again.

Reviewer_92kwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper focuses on the problem that current LLM unlearning is evaluated using forget-set averages, which cannot determine whether each subject has been unlearned. The authors propose ReCAUL, a post-hoc audit that checks whether an unlearned LLM's behavior on each forgotten subject still differs from that of retrained reference models. Experiments on TOFU with Llama-3.1-8B show that CF-PMAX identifies 2.6 times as much residual information as uniform averaging on a TaskVec checkpoint, and this finding is replicated on GradDiff.

Critical strengths

The problem is interesting. Unlearning requests are made on a per-person basis, so evaluating unlearning at the subject level is useful.

Critical weaknesses
  1. The experiments are narrow, covering only one model, one synthetic dataset, and three unlearning methods. It remains unclear whether ReCAUL works on other models or benchmarks such as MUSE [1] and RWKU [2].

  2. The evidence that average scores mask unlearning failures is insufficient. The abstract states that CF-PMAX exposes 2.6 times as much residual information as uniform averaging. However, a maximum naturally exceeds an average. Even in a model that has not undergone unlearning, the maximum exceeds the mean by 2.32 points. Thus, the 2.6x figure may overstate what the audit actually reveals.

The paper is very difficult to read and follow. My understanding is that it proposes a fine-grained method for evaluating unlearning at the person and fact levels, but the empirical study is narrow. In my view, the contribution is insufficient for an ICLR paper.

[1] MUSE: Machine Unlearning Six-Way Evaluation for Language Models.

[2] RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models.

Questions for authors
  1. The NPO results were analyzed after the authors had already seen its outputs. What does this reanalysis add to the paper's main contributions?

  2. The pairs are matched on 15 length features. Why do these features capture subject-level difficulty? Did the authors check whether the matched pairs had similar scores on the original model before unlearning?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 92kw (W2)

Thank you for the review. We answer each point below.

W2. "A maximum naturally exceeds an average", so 2.6× may overstate what the audit reveals, and the evidence that averages mask failures is insufficient.

You are right about the 2.6× number, and we dropped it. The 2.32 you cite came from a wrongly chosen control model (see our correction comment). With the correct model, a model with no unlearning shows a gain of 4.12, about the same as an unlearned one (TaskVec, 3.98), so the gain cannot show that knowledge is left.

In the revision, we replace it with a yes/no answer (a verdict) for each forgotten person that uses no maximum. We check whether the unlearned model picks the person's true answers out of about 100 candidates more clearly than two groups of models that never learned the person. One group is models retrained without the person. The other is the same retrained models after the unlearning method is applied to them, which shows what the method does to a model that never knew the person. A person is flagged only if the score is above 99% of the scores in each of the two groups. A model gets verdicts only if its estimated share of false alarms is at most 10%. We then tested the verdict where it could not have been tuned. Its score was fixed in advance for 30 held-out people and for Qwen2.5-7B-Instruct, and the results hold there as well (see our answer to Q1).

On models for which we know what the verdict should be, it behaves as it should.

Table 1. The verdict on Llama-3.1-8B models for which we know what it should be. Share of forgotten people flagged (%).

ModelShould be flagged?Flagged
Retrained, then unlearned (9 models)no0
Original Llama chat modelno0
New retrained models (3)no0
TaskVecyes88
GradDiffyes100
NPOyes79

The first three rows are 13 models that never learned the people. They are the retrained models after each unlearning method was applied to them, the original Llama chat model, and three retrained models trained after the reviews. None of them is flagged, and the unlearned models are. The same holds on Qwen2.5-7B-Instruct, where none of 10 such models is flagged. Retrained models that we deliberately taught some facts are flagged too, more often the more they are taught.

Averages do hide failures. With NPO, the usual forget-set average moves 91% of the way from the model before unlearning to the retrained models, yet NPO still remembers most forgotten people.

Table 2. Share of NPO's forgotten people found still remembered (%). The last row uses the stricter two-group threshold of the verdict, the other rows one common threshold.

Check on NPOFound
Average likelihood38
Greedy recall (RWKU)33
Exact recall (LUCID)33
Leak@3263
Beam-10 recall42
Ours92
Ours, stricter threshold79

The average and the existing audits find at most 63% of NPO's forgotten people still remembered. Ours finds 92%, and 79% with the stricter threshold. On a second NPO run, which looks forgotten on average, our verdict still finds 50% of the forgotten people.

The average is fooled because NPO lowers its confidence in all answers. For people it was not asked to forget, the drop is 2.08 nats per token, almost as large as the 2.61 for forgotten people. Our verdict compares answers with each other, so a general drop does not fool it. Our second model family, Qwen2.5-7B-Instruct, shows the same. There NPO moves the average 57% of the way, while it still picks the true answer first as often as before unlearning.

The 2.6× ratio is gone, the new verdict gives no false alarms on known cases, and it finds much of what averages miss (Sections 3.3, 5.1 to 5.4, 5.6).

AuthorsOpenReview ↗

Response to Reviewer 92kw (W1, Contribution & Readability)

W1. Only one model, one synthetic dataset, and three methods. Does it work on other models and on benchmarks such as MUSE and RWKU?

Yes. In the revision, we added a second model family (Qwen2.5-7B-Instruct), real news articles (MUSE-News, five public unlearned models, no training needed), and three more unlearning methods (SimNPO, IdkDPO, RMU).

Table 1. Share of forgotten people still remembered (%) in the added settings. For MUSE-News, the share is of forgotten articles, and models are named by method and uploader.

Qwen2.5-7B-InstructStill remembered
TaskVec88
GradDiff92
NPO88

MUSE-NewsStill remembered
SimNPO (OPTML)71
NPO (LV)83
RMU (LV)89

On Llama-3.1-8B, the new method SimNPO leaves 83% of the forgotten people remembered. Three models get no verdict, and for them the audit makes no claim. They are RMU on Llama-3.1-8B, which recalls less than the retrained models, and two MUSE-News models that MUSE rates as over-forgotten. On Qwen2.5-7B-Instruct, no model that never learned the people is flagged.

RWKU unlearns knowledge that the pretrained model already has, so no model that never saw this knowledge exists to compare with. We therefore ran RWKU's own score on our models, with the same threshold as ours. The comparison includes IdkDPO, one of the three new methods.

Table 2. Share of forgotten people found still remembered (%), same threshold for both scores.

Unlearned modelRWKU's scoreOurs
NPO3392
IdkDPO492

IdkDPO answers "I don't know" to 58% of the questions about forgotten people, so a score that reads its answers finds almost nothing, while ours still finds 92%. The audit thus works on another model family, on real news, and on other methods (Sections 5.2, 5.6, 5.7, Appendices K and L).

"In my view, the contribution is insufficient for an ICLR paper."

Beyond fine-grained evaluation, the paper offers three things.

  1. A yes/no answer for each forgotten person, with a known error rate. Existing benchmarks report only averages or counts.
  2. A finding. Averages and answer-based audits miss knowledge that the model still has.
  3. A practical audit. It needs a one-time setup, cheap scoring per request, and a quick version with no retraining at all.

"The paper is very difficult to read and follow."

In the revision, we rewrote the paper around one question and one test and added a reading guide (Table 3 of the paper). We also redrew Figures 1 and 2 to follow one forgotten person with real numbers.

AuthorsOpenReview ↗

Response to Reviewer 92kw (Q1 & Q2)

Q1. The NPO results were analyzed after seeing its outputs. What does this reanalysis add to the main contributions?

It shows two things that TaskVec and GradDiff cannot, and both hold on data we had not seen. First, most of NPO's apparent forgetting is lost confidence. The likelihood of correct answers drops almost as much for people NPO was not asked to forget (2.08 nats per token) as for forgotten people (2.61). Second, NPO still recognizes answers it no longer says. When all ten generated answers are wrong, it still picks the true answer first 22% of the time, against 6% for retrained models.

In the revision, we tested both findings where we had not seen the outputs, each with a plan fixed in advance.

Table 1. Share of forgotten people still remembered after NPO (%), by our verdict, on data we had not seen.

New dataStill remembered
Llama-3.1-8B, 30 held-out people73
Llama-3.1-8B, second NPO run50
Qwen2.5-7B-Instruct, 24 people88

In each new test, NPO still remembers half or more of the forgotten people. The second NPO run forgets only 24 of the 54 people. We also briefly trained NPO and the retrained models in the same way on one fact per forgotten person, and measured the facts we did not teach.

Table 2. Recall of facts we did not teach (0 to 1), before and after the brief training.

ModelBeforeAfter
NPO0.330.61
Retrained models0.250.24

NPO recovers facts we did not teach, and the retrained models do not. On Qwen2.5-7B-Instruct, the likelihood again drops almost as much for people NPO was not asked to forget (2.72 nats per token) as for forgotten people (2.92), and NPO still picks the true answer first 27% of the time, against 28% before unlearning. The same unlearning applied to models that never learned the people does not make them recognize the true answers.

Both findings hold on new data, and the paper still labels the original NPO analysis as done after seeing its outputs (Sections 5.3, 5.5, 5.6).

Q2. The pairs are matched on 15 length features. Why do these capture subject-level difficulty, and were the pairs similar on the model before unlearning?

Length is a simple stand-in for difficulty that we could fix before seeing any model output, since matching on model scores would have used the outcome. It gives the two people in a pair questions and answers of similar length. A coin flip then decided which person in each pair was forgotten, so the test stays valid however well the pairs match.

In the revision, we ran the check you asked for. On the model before unlearning, the score of the forgotten person minus the score of the partner is −0.14 on average (95% interval −0.42 to 0.14), so the pairs show no systematic difference. After we re-paired the people by their scores on that model, the audit results stay significant (p < 0.001).

The pairs are balanced, so matching on length was enough here. The new verdict looks at each forgotten person alone and needs no pairs at all (Appendix A).

We hope the revision answers your concerns. If you have any further questions, please let us know, and we will answer as soon as possible. If the revision resolves your concerns, we would be grateful if you could reconsider your assessment. Thank you again.

Comments on the paper

6 comments
AuthorsOpenReview ↗

Response to Reviewer 38tn (W1, W2 & Q2)

Thank you for the positive and specific review. In our answers, a forgotten person (in TOFU, a fictitious author) is "still remembered" when the unlearned model recognizes the person's true answers better than models that never learned the person. Recognizing means preferring the true answer over about 100 other candidate answers of the same type.

W1. Do the findings hold beyond TOFU, one model family, 24 pairs, and five relations?

Yes. In the revision, we repeated the audit on a second model family, on all 54 forgotten people instead of 24, and on real news articles without fixed relations.

Table 1. Share of forgotten people still remembered (%).

MethodLlama-3.1-8B, 24 peopleLlama-3.1-8B, 30 held-out peopleQwen2.5-7B-Instruct, 24 people
TaskVec8810088
GradDiff10010092
NPO797388

On MUSE-News, we audited five public unlearned models, named below by method and uploader, against MUSE's public retrained model, without training anything. Three of them get verdicts. News has no candidate answers, so here the audit reads how likely the model finds each forgotten article.

Table 2. Share of forgotten articles still remembered (%) on MUSE-News.

ModelStill remembered
Model before unlearning97
SimNPO (OPTML)71
NPO (LV)83
RMU (LV)89
Pretrained model, never saw the articles0

The first and last rows are two reference points. The model before unlearning is MUSE's public target model. It reaches 97% and not 100%, because it scores 19 of the 697 articles no better than articles it never saw, so it shows no sign of having learned them. The pretrained model, which never saw the articles, is never flagged. The other two public models, SimNPO (LV) and GradDiff (LV), get no verdict. MUSE rates them as over-forgotten, and for them the audit makes no claim. The finding holds on another model family, on more people, and on real news (Sections 5.1, 5.6, 5.7).

W2 and Q2. Retrained models are expensive, especially for large models. Which reference-training and calibration costs are reused across deletion requests, and which are paid again?

Reference training and calibration are done once for a pool of people (everyone whose deletion might be requested) and reused for every deletion request in that pool. Only scoring the unlearned model is paid again for each request. A request for people outside the pool needs new retrained models.

Table 3. Cost of each part of the audit.

StepHow oftenCost
Train a retrained modelonce per pool of peopleabout 2 GPU-hours
Apply unlearning to a retrained modelonce per method25 to 50 training steps
Score the unlearned modelevery request942 forward passes per person

We trained three retrained models on one A800 GPU, and two such models are enough for the audit. Calibration only scores the retrained models once. For large models, a retrained model is the same kind of model that TOFU and MUSE already train for their own evaluation. It costs one fine-tuning run and is paid once per pool. When a benchmark releases a retrained model, as MUSE does, nothing needs training.

In the revision, we also tested two ways to save cost. First, we reused the old retrained models for a new deletion request, in which NPO forgets only 24 of the 54 people, with no new training. The audit finds that 50% of its forgotten people are still remembered. Second, if retraining is impossible, a quick check compares the unlearned model with the public model it started from.

Table 4. Quick check without any retrained model. Share of forgotten people flagged (%). The last row is a test on models that should not be flagged.

ModelFlagged
TaskVec100
GradDiff100
NPO75
Retrained models0

The quick check flags 75% to 100% of the forgotten people and no retrained model. The audit therefore costs a one-time setup plus cheap scoring per request, and it can run with no retraining at all (Section 5.8).

AuthorsOpenReview ↗

Response to Reviewer 38tn (W3 & Q1)

W3 and Q1. How should the size of the score be read, and do people with larger scores recover more information?

Larger scores go with more recovery. The link is strong across models and positive but weaker across people and facts. In the revision, we measured how often the forgotten answers are recovered when the model generates text, and computed the rank correlation of this recovery with the score. Across the 14 models we scored on Llama-3.1-8B, the correlation is 0.94. Table 1 gives it within each unlearning method, for the 30 held-out people of the submitted generation experiment and for facts on Qwen2.5-7B-Instruct, where the analysis was fixed before training.

Table 1. Rank correlation between the score and recovery.

LevelTaskVecGradDiffNPO
Across the 30 held-out people0.360.290.30
Across facts on Qwen2.5-7B-Instruct0.330.340.31

To read the size of the score, we put the likelihood score on a scale from retrained models (0) to the model before unlearning (1).

Table 2. Knowledge kept after unlearning on Llama-3.1-8B, on the scale from 0 to 1.

MethodKnowledge kept
TaskVec0.93
GradDiff0.81
NPO0.29

For NPO, generation understates what is left, because NPO stops the model from saying answers while the model still knows many of them. When all ten generated answers are wrong, NPO still picks the true answer first 22% of the time, against 6% for retrained models.

In the revision, we tested whether this hidden knowledge is real. We briefly trained NPO and the retrained models in the same way on one fact per forgotten person, and measured the facts we did not teach.

Table 3. Recall of facts we did not teach (0 to 1), before and after the brief training.

ModelBeforeAfter
NPO0.330.61
Retrained models0.250.24

NPO gets these facts back and the retrained models do not. A model that looks forgotten when we only read its answers can still give the facts back, so generation alone underestimates what is left (Section 5.5).

We hope these answers address your points. If you have any further questions, please let us know, and we will answer as soon as possible. If the revision addresses your concerns, we would be grateful if you could take it into account in your final assessment. Thank you again.

AuthorsOpenReview ↗

Correction to one control, and what the revision shows

Thank you to all three reviewers. Before our replies, we correct one mistake in the paper.

The paper uses a control model to show what our audit reports for a model that still knows everything. This control should be the model before unlearning. By mistake, we used the original Llama chat model, which never saw the TOFU data. This affects every place where the submitted paper reports this control, including Table 1 and Figures 3 to 5.

In the revision, we scored the control with the correct model. Only this control's numbers change.

Table 1. Gain of our original score over a simple average of the same questions (score minus average).

ModelGain
Control, as submitted2.32
Control, corrected4.12
Unlearned model (TaskVec)3.98

With the correct control, a model that was never unlearned shows about the same gain as an unlearned one. Our headline number (2.6×) was the same comparison written as a ratio, 6.52 against 2.54 for TaskVec. This number therefore cannot show that knowledge is left, as Reviewer 92kw suspected, and we no longer use it.

In the revision, we replace it with a yes/no answer (a verdict) for each forgotten person, which says whether the unlearned model still remembers that person.

Table 2. Share of forgotten people still remembered (%), by the new verdict.

MethodLlama-3.1-8BQwen2.5-7B-Instruct
TaskVec8888
GradDiff10092
NPO7988
Control models00

The control row covers 23 models that never learned the people, 13 on Llama-3.1-8B and 10 on Qwen2.5-7B-Instruct. The verdict flags no person in any of them, while each unlearned model in the table still remembers most forgotten people. Our replies to each review and a summary for the ACs follow.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.