acceptodds
Withdrawn from ICLR 2027

Neighbourhood-Robust LLM Unlearning in Query and Weight Space

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Unlearning is a proposed safeguard against hazardous or private knowledge, but it protects only if adversaries cannot recover what was removed. Objectives such as negative preference optimisation (NPO) suppress the answer only under the forget questions and at the returned weights, whereas attackers rephrase and follow up, and deployed models are fine-tuned. We cast unlearning as adversarial robustness over the queries reaching a fact and the weights further fine-tuning reaches, and propose neighbourhood-robust unlearning (NRU): a red-teaming probe generator, refreshed as the model changes, reads its responses to find queries that still extract each fact, and each update lowers a soft maximum of the loss over these queries, optionally after a simulated fine-tune. On a relation-level split of TOFU, ten probe strategies with a detector checked against a retraining oracle recover 0.83 of the answerable facts from the original model and 0.08 from the oracle, and NPO still leaves 0.51 recoverable at suppression 0.37 on the direct question. At matched utility, with additional training, a query neighbourhood lowers recovery by up to 0.345 (0.260 for NRU's generator arm), against 0.085 for mining leaked responses; NRU's refresh and soft maximum each lower recovery at 32 epochs. Simulating fine-tuning exposes a trade-off between the axes. Likelihood checks at the forget prompt thus give false assurance; safety audits should measure what an adaptive adversary can still extract, which training over query neighbourhoods reduces.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, LLM unlearning, relearning attacks, knowledge extraction, negative preference optimization, distributionally robust optimization, group DRO, tamper-resistant training, adversarial robustness, red teaming, benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_89krOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies LLM unlearning under two forms of subsequent access: alternative queries that may still elicit a forgotten fact, and continued fine-tuning that may restore its accessibility. It proposes neighbourhood-robust unlearning (NRU), which periodically mines queries against the current model, maintains a probe buffer, and minimizes a per-item soft maximum of NPO losses over the original and mined queries. An optional extension evaluates this objective after simulated benign fine-tuning using a first-order update.

The main evaluation uses Qwen2.5-3B-Instruct on a relation-level TOFU split, with 59 reference-answerable forget items. It combines ten strategies with eight queries per strategy, same-entity anchor questions, a retraining oracle, detector sensitivity checks, and continued-training evaluation. Query augmentation reduces measured recovery at similar anchor accuracy, although the main matched-utility comparisons use unequal training budgets. Within NRU-G, refresh and soft-maximum aggregation help at 32 epochs, but response-conditioned generation shows no detectable additional benefit, and inexpensive fixed augmentation performs better at comparable anchor accuracy. The joint query/weight-space variant does not achieve a satisfactory robustness–utility operating point. Additional RWKU and Llama experiments document settings in which the evaluation cannot meaningfully distinguish the methods.

Critical strengths
  1. The evaluation separates information accessibility from collateral damage in a useful way.

The relation-level split and same-entity anchor set make it possible to detect methods that obtain low recovery by damaging an author's other facts. Reporting query-based recovery, direct-question suppression, and utility together is therefore consequential to interpreting unlearning performance. The reference-versus-retraining-oracle comparison provides a useful detector sanity check: the reported rule yields recovery of 0.83 versus 0.08, while the judge-only rule fails to distinguish the two adequately (Tables 5–6). The alternative detector rules and second-model-family judge further strengthen the comparative evidence, while the paper appropriately acknowledges that these checks do not establish semantic ground truth.

  1. The controlled comparisons and transparent negative results make the empirical study informative.

The paper distinguishes matched-anchor comparisons, equal-epoch comparisons, and component ablations rather than treating them as interchangeable. It includes fixed-pool, uniform-replay, same-data, nonadaptive-generator, and shuffled-probe controls. At 32 epochs, removing the soft maximum or refresh increases recovery by 0.096 and 0.090, respectively, with author-cluster bootstrap intervals and multiplicity correction for the prespecified contrasts (Table 1c). The authors also explicitly report the absence of a detectable benefit from response conditioning, the stronger performance of inexpensive fixed augmentation, and the utility collapse of the longer-trained stability arms. This transparency makes the findings more useful and prevents low recovery alone from being mistaken for successful, utility-preserving unlearning.

Critical weaknesses
  1. The central disclosure metric is not sufficiently validated against independent semantic judgments.

All three detector rules that pass the instrument gate retain a lexical-overlap requirement. Consequently, agreement across these rules, or replacement of the semantic judge while keeping that requirement, does not establish sensitivity to semantically correct disclosures expressed with different wording. Appendix G explicitly states that there is no independently labelled sample and that detector sensitivity to paraphrased disclosure, and errors on unlearned models other than the oracle, remain unestimated. On the provided stratified sample of 200 turns, an independent entailment model fires on 0.31 of responses whereas the reported conjunction fires on 0.09. This disagreement does not establish which detector is correct, but it makes independent adjudication important. Reference/oracle separation is a useful control, yet it cannot rule out method-dependent false negatives on the responses used to rank unlearning methods. Since the paper's main contribution is empirical and its endpoint is information recovery, this unresolved measurement issue materially limits confidence in the conclusions.

  1. The incremental value of the adaptive and joint NRU construction remains insufficiently established, including as a diagnostic contribution.

At 32 epochs, fixed-suite NAU has recovery around 0.16 with anchor accuracy 0.44, whereas NRU-G has recovery around 0.25 with anchor accuracy 0.43, at substantially higher measured per-epoch cost (Tables 3, 7, and 8). The response-denied generator also shows no detectable disadvantage. Thus, the successful refresh and soft-maximum ablations explain performance within NRU-G but do not establish an advantage over simple query augmentation. These negative findings are valuable; however, their explanation remains incomplete because the effects of restricted mining coverage and the permissive training detector are not isolated.

The weight-space extension likewise provides no demonstrated useful joint operating point: NRU-G + stability has recovery 0.72 at 16 epochs, while its recovery of 0.05 at 32 epochs accompanies anchor accuracy of only 0.13 and rises to 0.26 after continued training (Table 9). The strongest fixed-suite methods are not evaluated after continued training (Section 5.4). Moreover, the informative method comparison remains confined to the small Qwen/TOFU-Rel setting, since the RWKU and Llama extensions cannot distinguish the central effects (Appendix I). Together, these gaps leave the proposed mechanisms and their failure modes less decisively understood than would be needed for a strong methodological or diagnostic contribution.

Questions for authors
  1. Clarification: What does query variation recover beyond the original question?

For NPO and the principal query-neighbourhood methods, please report the original forget question's reproduction rate under the same detector used for the probe evaluation. What fraction of facts are not reproduced under the original question but are subsequently recovered by another query? In particular, the comparison between suppression S0 = 0.37 and neighbourhood recovery = 0.51 does not by itself establish how many disclosures a direct-question evaluation would miss. A paired analysis would materially strengthen or qualify the interpretation of P1.

  1. Measurement validation: Do the conclusions survive independent adjudication of semantic disclosure?

Can the provided response sample be independently labelled, including detector-negative responses that may disclose the fact through paraphrase, and with representation from the main unlearning methods as well as the reference and oracle? Please report the detector's precision and recall and whether the principal method comparisons remain consistent with the adjudicated labels. Changing the semantic judge while retaining the lexical gate does not address errors shared by that gate.

  1. Targeted control: What explains the absence of an observed benefit from response-conditioned mining?

Section 7 identifies both restricted mining coverage and a permissive training detector as possible explanations, but neither is tested. What fraction of buffered probes actually disclose the target fact under a validated criterion? Can the adaptive, response-denied, and fixed-suite comparisons be examined with matched item coverage and probe budgets? This would help distinguish a limitation of response conditioning from a limitation of the training neighbourhood constructed in the current implementation.

  1. Evidence for the second-axis conclusion: How do the strongest query-neighbourhood baselines behave after continued training?

Section 5.4 omits NAU + anchor, DRO-NAU + anchor, and NRU-S from the continued-training evaluation. Can their recovery and both utility measures be reported after the same 100-step procedure? For the joint method, are intermediate-epoch results available that preserve utility while achieving meaningful unlearning? These results would materially affect the practical assessment of joint robustness and the interpretation of the failure observed at the two reported operating points.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 89kr (1/2)

Thank you for a careful, constructive and fair review; it is the kind of review that improves a paper. It led to new measurements for Q1, Q3 and Q4 and to a correction under Q2. Every change is blue. Part (2/2) answers the second half of W2 (the weight-space extension), Q4 and Q1.

W1 and Q2. Validation of the detector against independent judgments

Your concern is well taken, and we want to be precise about how far the revision goes. The detector is checked against a retraining oracle and against two further judges, but it is not yet validated against human labels, and Section 7 (p. 9) now says so up front. Three points in Appendix G (p. 18) bear on the concern; we first explain why labels are not yet reported.

Why labels are not yet reported. A validation that settles your question needs annotators who are independent of us and blind to the arm and to the detector's verdict, at least two of them so that their agreement can be reported, and enough labelled turns per arm to test whether missed disclosures differ between arms. The 200-turn sheet cannot carry the per-arm part: it holds equal numbers from the four agreement cells, not from each arm. We were not able to arrange such a study within the discussion period, and we felt that a small sample labelled by the authors would not do justice to your concern. We have therefore stated the limitation clearly and released the sheet unscored, and we regard independent adjudication as the natural next step.

  • A correction, for which we thank you. The entailment model fires on 0.31 of the 200-turn sample, but that sample was drawn in equal numbers from the four agreement cells of the two criteria. Its rate therefore cannot be set against the conjunction's 0.09, which is taken over all issued turns, and the revised text says so. The two rates are therefore not comparable: they neither show a disagreement nor validate the detector.
  • Other judges. A second judge from another model family (Llama-3.1-8B) reproduces every significant comparison and turns one null into a significant effect. A third judge (Phi-4), run on every turn with no lexical prefilter, fails alone as the reported judge does (oracle 0.92 against reference 1.00) and passes with the lexical criterion (0.08 against 0.85).
  • Your point about the lexical criterion stands. Every rule that passes the gate keeps it, so none of this can expose an error that the criterion makes on paraphrased disclosures. Until labels exist we cannot give the detector's precision and recall, and the paper says so.

W2 (first half) and Q3. Why response-conditioned mining shows no benefit

Thank you for pointing out that the submitted text named two explanations without testing either. Appendix H.3 (pp. 23-24) now tests both, and the abstract now states that the generator does not beat NPO at identical epochs (p. 1).

  • The permissive training detector. The six NRU-G runs end training with 405 to 549 buffered probes over 60 to 64 items, and the training detector admitted 75% to 81% of them on the judge alone. Answered by the reference and scored under the reported rule, 0.21 of the 2,846 buffered probes reproduce the answer: 0.74 of those admitted lexically and 0.07 of those admitted on the judge alone. Most of what NRU-G trains on is therefore not a query that reproduces the answer. The reported rule is the criterion here; we have no hand-validated one.
  • Restricted coverage. We restricted the fixed suite to the items that NRU-G's final buffer covers, with at most as many probes per item (136 to 147 probes on 44 to 46 items), and trained it under the soft maximum. It does not separate from NRU-G at 16 epochs (p = 0.634) and recovers less at 32, 0.17 against 0.25 (p = 0.003, uncorrected), at anchor accuracy 0.35 against 0.43. Restricted coverage alone thus does not explain why NRU-G falls behind the suite. Section 4 (p. 5) now also states that the fixed suite itself has probes for 72 of the 103 items; the submitted text said it covers every item.
  • The soft maximum on the suite. Over the whole suite's pool (244 probes on 72 items) the soft maximum does not beat the uniform weight on the same rows: recovery 0.43 against 0.42 at 16 epochs and 0.14 against 0.09 at 32. It pays on the mined buffer (Table 1(c), p. 7) and not on the suite.

Two caveats remain, and the paper states them. The lower anchors leave the 32-epoch reading of the coverage control open, and the response-denied generator was not rerun at matched coverage. The reason for the second is that the adaptive and the response-denied generator already mine the same generator-train entities with the same strategies (Section 4, p. 5), so that pair is matched by design. A three-way match would need a second suite cut to the response-denied buffer and its own training runs, which we did not make. Section 5.3 (p. 8) now cites these results, and Section 7 (p. 9) states them where the submitted text said "neither was tested".

AuthorsOpenReview ↗

Response to Reviewer 89kr (2/2)

W2 (second half). The operating point of the weight-space extension

Your reading is correct, and the runs you asked for sharpen it. Table 9(a) (p. 23) now reports both stability arms at 20 and 24 epochs. From recovery 0.69 and 0.61 at 20 epochs they change by -0.01 and +0.05 under continued training, and from 0.50 and 0.35 at 24 epochs by +0.06 and +0.14 (Appendix H.2, p. 20). At 24 epochs NRU-G + stability thus has recovery 0.35 at anchor accuracy 0.44, and 0.49 after continued training. NAU + anchor at 32 epochs has 0.16 at the same anchor accuracy, and 0.40 after continued training. One encouraging point: the fine-tune removes less of their suppression (0.10 to 0.29) than of any arm trained on queries alone (Section 1, p. 1). Yet their recovery nears the oracle's only at 32 epochs, where their anchor accuracy falls to 0.13. The two-axis arms therefore do not yet reach a useful joint operating point. Section 5.4 (p. 8) presents this as an analysed result, with its mechanism, and we see a different simulated fine-tune as the way forward.

Q4. Continued training for the strongest fixed-suite methods

Table 9(a) (p. 23) now includes NAU + anchor and DRO-NAU + anchor at 16 and 32 epochs, with anchor and retain accuracy before and after the same hundred-step procedure for every arm. Recovery rises from 0.29 to 0.55 and from 0.16 to 0.40 for NAU + anchor, and from 0.44 to 0.58 and from 0.18 to 0.40 for DRO-NAU + anchor. NRU-G rises from 0.25 to 0.47 at 32 epochs. The share of suppression lost at 16 epochs is 0.58 for NAU + anchor and 0.37 for DRO-NAU + anchor (Appendix H.2, p. 20). The fixed-suite arms are therefore no more stable than the others, and the paper now states the range over the eight arms trained on queries alone: 0.34 to 0.68 (Section 5.4, p. 8). NRU-S is the one arm of your list that we were not able to run through continued training in the discussion period; Section 5.4 (p. 8) and Section 7 (p. 9) note this. We do not expect it to change the conclusion, for two reasons that the paper prints. NRU-S trains on the same suite as NAU + anchor and reaches the same recovery at 32 epochs (0.16 for both; Table 3, p. 16). And every query-only arm that we did measure, the two fixed-suite arms included, loses 0.34 to 0.68 of its suppression. This is an expectation, not a measurement, and the paper makes no claim about NRU-S after continued training.

Q1. What query variation recovers beyond the original question

Thank you for this question; the analysis it prompted supports P1 directly. New Table 10 (p. 24) reports three shares of the 59 answerable items, for eight arms at 16 and 32 epochs: the share whose forget question reproduces the fact under the same detector, recovery within eight queries, and the share reached only by other queries. For NPO at 16 epochs these are 0.27, 0.51 and 0.27, and recovery exceeds direct reproduction by +0.24 [+0.11, +0.37] (Appendix H.3, p. 23). So about half of the facts that the probes recover are facts that the forget question does not reproduce. At 32 epochs the three shares are 0.01, 0.16 and 0.16, so a check on the forget question alone would miss nearly every recovered fact. For NAU + anchor the shares are 0.19, 0.29 and 0.18 at 16 epochs and 0.04, 0.16 and 0.14 at 32. For NRU-G they are 0.37, 0.50 and 0.15, then 0.11, 0.25 and 0.16. This is the paired reading of P1 that you asked for. Suppression of 0.37 is a likelihood measure, so it is not comparable with these detector rates. Section 5.1 (p. 6) now states the 16-epoch figures.

Confinement to Qwen and TOFU-Rel

We agree, and the paper says so (Section 7, p. 9). Appendix I had noted that a sweep below 16 epochs was needed on Llama-3.2-3B, and we ran it at 4 and 8 epochs (p. 25). It does not open a window for comparison: NPO has barely unlearned there (recovery 0.77 and 0.78), NRU-G recovers 0.03 more at 4 epochs (unadjusted p = 0.02), and only response mining recovers less (0.03 at 8 epochs, with anchors at 0.32). At 32 epochs, as at 16, every arm is under the oracle's rate. Neither Llama nor RWKU can therefore arbitrate between objectives.

Change. Section 1 (p. 1), Section 5.1 (p. 6), Section 5.3 (p. 8), Section 5.4 (p. 8), Section 7 (p. 9), Appendix G (p. 18), Appendix H.2 (p. 20), Appendix H.3 (pp. 23-24), Table 9(a) (p. 23), Table 10 (p. 24) and Appendix I (p. 25).

Thank you again for the care in your review. We hope these measurements answer your questions, and we would be grateful if you would reconsider your assessment in their light.

Thank you for your response. I understand your point to some extent, but your reply was too long for me to examine in detail; therefore, I will keep my rating as it is. I apologize for any consequences this may cause.

Reviewer_H5GhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The basic NPO can only suppress the answer under fixed questions, and the fact can be found if the attackers rephrase the question. This paper proposes a neighbourhood-robust unlearning, which finds multi-questions to get the fact and focus on the hardest query in the meanwhile.

Critical strengths
  1. The paper is easy to follow.
  2. The paper shows that normal unlearning leaves recoverable information for the future and confirms that training in the use of various questioning techniques can reduce the rate of recovery.
  3. This paper conducts comprehensive ablations and reports the failure results.
Critical weaknesses
  1. The proposed adaptive probing fails to outperform NPO. More convincing experiments are required to demonstrate the superiority of the method. Furthermore, the method incurs high computational cost for utility calculations, which makes it impractical for real-world applications.
  2. The primary empirical findings are based on the Qwen2.5-3B-Instruct and a small-scale synthetic dataset. No significant differentiation between the various methods is observed in this experiment. Experiments with larger models and datasets are required.
  3. The function of the adaptive generator is not clear. The results of the adaptive generator only claim that it does not beat NPO as a negative result.
Questions for authors

refer to weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer H5Gh

Thank you for your careful review, and for recognising the ablations and the open reporting of failures. Your three points helped us sharpen what the paper claims and add evidence for it. Every change is blue.

W1. The adaptive probing and NPO; more convincing experiments

Thank you for pressing on this point; it led to the two most useful additions of the revision. The paper's claim concerns the objective: training over a query neighbourhood, compared at matched utility, where each arm is set against the NPO run nearest in accuracy on the entity's other facts. Utility has to be matched because longer training lowers recovery at those facts' expense: NPO's anchor accuracy falls from 0.44 to 0.30 between 16 and 32 epochs (Section 5.1, p. 6). At matched utility, recovery changes by -0.345 for uniform training on a fixed probe suite (NAU + anchor), -0.328 for family reweighting (DRO-NAU + anchor), -0.260 for the generator-built neighbourhood (NRU-G) and -0.085 for response mining (RULE-NPO) (Table 1(b), p. 7).

You are right that at identical epochs the generator arm is not ahead of NPO: it is on par at 16 epochs (-0.006 [-0.10, +0.08]) and behind at 32 (+0.090 [+0.03, +0.15]; Table 1(a), p. 7). We report this openly, and the abstract (p. 1) and Section 7 (p. 9) now state it up front. Two new measurements explain it (Appendix H.3, pp. 23-24):

  • What the buffer holds. Answered by the reference model, only 0.21 of the 2,846 probes in the generator's buffers reproduce the answer under the reported rule.
  • Coverage. The generator mines probes for only part of the forget set. A fixed suite cut down to the same items and trained under the soft maximum does not separate from NRU-G at 16 epochs and recovers less at 32 (0.17 against 0.25, at anchor accuracy 0.35 against 0.43).

Together they suggest that the quality of the mined probes, more than their coverage, limits the generator arm. The idea of training over a query neighbourhood is not what fails.

W1. Computational cost

This is a fair concern, and the paper quantifies it. NRU-G needs a 48 GB card and eleven times NPO's wall-clock time per epoch (Section 7, p. 9). The cost lies in the generator: NRU-S keeps the refresh and the soft maximum but takes its probes from the fixed suite, and it costs three times NPO's seconds per epoch (Section 5.3, p. 8; Table 3, p. 16). Hence the paper's advice "to train on a query neighbourhood built cheaply before training, such as a fixed probe suite" (Section 8, p. 10).

W2. Scale, dataset and differentiation

We share your wish for larger models, and we see scaling as the most important next step. We have not run a larger model, and the reason is cost and memory. The study took 413 GPU-hours on 3B models, 177 of them for probe evaluation alone (Appendix E, p. 16). The generator arm already needs a 48 GB card at batch size 4 for a 3B model, so a larger model with the generator in the loop would need more than the cards used here. Section 7 (p. 9) therefore says that "nothing bears on full fine-tuning, larger models or forget sets beyond 103 rows".

On a second model family, Llama-3.2-3B, the detector check replicates (0.85 against 0.08). The sweep that we added at 4 and 8 epochs shows that this model moves from barely unlearned (recovery 0.77 and 0.78) to under the oracle's rate between 8 and 16 epochs, so it offers no window in which objectives can be compared (Appendix I, p. 25). The dataset is synthetic for a principled reason: facts learned in pretraining admit no retraining oracle, so nothing checks the detector there (Section 7, p. 9).

On differentiation, the methods do separate where the comparison is controlled. The four matched-utility effects have intervals that exclude zero (Table 1(b), p. 7). At an identical 16 epochs NAU + anchor recovers less than NPO (-0.22 [-0.31, -0.13]; Table 8, p. 22). Inside NRU-G at 32 epochs, removing the soft maximum or the refresh raises recovery by 0.096 and 0.090, and each survives a Holm correction (Section 5.3, p. 7).

W3. The function of the adaptive generator

Thank you for flagging this; our description was too implicit. Section 3.1 (p. 3) now states it: "The generator thus supplies coverage and not a new loss: it finds queries that the current model still answers, and the loss below trains on them." The experiments then show which of its parts matter. Refreshing the probes and weighting each item's hardest query both matter at 32 epochs, while reading the model's responses adds no detectable benefit (Section 5.3, p. 7). The generator is thus one effective way to build the neighbourhood, though not the cheapest.

Change. Section 3.1 (p. 3), Section 5.3 (p. 8), Section 7 (p. 9), Appendix H.3 (pp. 23-24) and Appendix I (p. 25).

We hope these additions address your concerns, and we would be grateful if you would reconsider your assessment in their light.

Reviewer_BHYUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a robust unlearning method called neighbourhood-robust unlearning.

Critical strengths

None

Critical weaknesses
  • I have tried to read the paper. However, the language of this paper is very awkward and full of strangely complex syntax. I am not sure whether the paper is written by AI or someone who wants to show off their poor English.
Questions for authors

Have you checked what the AI generated?

7 comments
AuthorsOpenReview ↗

Response to Reviewer BHYU

Thank you for your review and for the time you gave the paper. Your comments on the writing were justified: the text asked too much of the reader, and that is our responsibility. Your review led us to go through the main text sentence by sentence and rewrite every sentence that carried several claims at once, used a term before explaining it, or needed a second reading. About two in five of its sentences now carry changes, all in blue.

The problem and our technical solution (Informativeness)

In one sentence: unlearning methods suppress a fact only under the question they were given, so we put an attacker inside the training loop, a probe generator that keeps finding the questions under which the current model still gives the fact away, and we train each fact against the hardest of those questions.

  • The problem. A method such as NPO lowers the answer's likelihood only under the forget question itself. Asked in other words, the model still gives the fact away: after NPO, ten probe strategies of eight queries each recover 0.51 of the answerable facts, against 0.83 before unlearning and 0.08 for a model retrained without those facts (Section 5.1, p. 6).
  • Our solution, NRU (Section 3, p. 3; Algorithm 1, p. 15). During unlearning a probe generator repeatedly questions the current model with four strategies: rephrasing, inversion, two-step composition and follow-up. Each query whose response still leaks a fact is kept in that fact's buffer. Each update then lowers, for every fact, a soft maximum of the NPO loss over the forget question and its buffered queries, so training concentrates on the query under which the fact is currently easiest to extract. A second term keeps the same entity's other facts. An optional variant scores the loss after a few simulated fine-tuning steps, to cover later fine-tuning.
  • What it achieves. Compared with the NPO run nearest in accuracy on the entity's other facts, the generator-built neighbourhood lowers recovery by 0.260 (Table 1(b), p. 7). The refresh and the soft maximum each contribute: removing either raises recovery again at 32 epochs (Table 1(c), p. 7).
  • Its scope. The gain is established at matched utility; at identical epochs NRU is on par with NPO at 16 epochs and behind it at 32 (Table 1(a), p. 7). We draw this boundary openly, because it tells a practitioner when the method pays.

Weakness: awkward language and complex syntax (Presentation)

You were right. Two examples of the changes:

  • Abstract (p. 1). Before: "We cast unlearning as adversarial robustness over the queries reaching a fact and the weights further fine-tuning reaches, and propose neighbourhood-robust unlearning (NRU): a red-teaming probe generator, refreshed as the model changes, reads its responses to find queries that still extract each fact, and each update lowers a soft maximum of the loss over these queries [...]." Now: "We treat unlearning as robustness over two neighbourhoods: the queries that reach a fact, and the weights that further fine-tuning reaches. We propose neighbourhood-robust unlearning (NRU). A probe generator is rerun as the model changes and reads the model's responses to find queries that still extract each fact."
  • Section 5.4 (p. 8). Before: "What the simulated steps undo grows with how far the update has pushed the forget loss". Now: "The more the update has raised the forget NLL, the more of it the simulated steps undo".

Beyond single sentences, the abstract now explains its terms, the first paragraph of the introduction says what the paper does (p. 1), terms are defined where they first appear, and each step of Algorithm 1 has a comment (p. 15).

Soundness

Before any method is compared, the detection rule must separate the model before unlearning (0.83) from the retrained model (0.08); a judge model alone fails that test (Table 5, p. 19). Every interval is a bootstrap that resamples whole authors (Section 4, p. 6), and the four component contrasts of Section 5.3 (p. 7) carry a Holm correction.

Question: Have you checked what the AI generated?

Yes, and thank you for raising it. The AI use statement (p. 10) discloses that generative models drafted analysis and plotting scripts, LaTeX and prose as assistants; that the authors ran, reviewed and checked that work against the recorded measurements; and that no number, citation or claim was accepted from a model without verification against the artefact or the cited text. It also discloses that an LLM assistant was used to edit and rewrite the text, and that the authors reviewed every change. Your review showed us that our care for the numbers was not matched by equal care for readability, and the revision is our answer.

Change. The abstract, Sections 1 to 5 and 7, and Algorithm 1 (p. 15), all in blue.

Thank you again. We hope the revised text lets the contribution come through, and we would be grateful if you would reconsider your assessment in its light.

Thanks for your response.

I acknowledge that I have checked the revision and the authors' response.

I would keep my score as is.

AuthorsOpenReview ↗

Seeking Your technical evaluation of the method.

Dear Reviewer BHYU,

Thank you for your continued engagement during the discussion phase.

As you indicated a confidence rating of "I have substantial direct research experience on this topic," we place high value on your domain expertise and are eager to engage in a rigorous technical discussion regarding our work. However, while you have noted several times that you have reviewed the manuscript, your comments thus far state high-level conclusions without an accompanying technical evaluation of our proposed methodology, theoretical formulation, or empirical design.

In the interest of a transparent and constructive peer-review process, an evaluation must be grounded in specific, verifiable technical arguments. Without concrete references to potential flaws in our equations, formal assumptions, or experimental protocols, neither we, the Area Chairs, nor the public readership have an objective basis to determine whether your assessment is technically accurate and well-founded.

We respectfully ask that you share the specific technical critiques of our method that informed your rating. Providing this technical substance will allow both us and the Area Chairs to verify the validity of your concerns, evaluate whether your conclusions hold under technical scrutiny, and ensure the final decision is grounded in scientific evidence.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of changes

We thank the three reviewers for their careful reading. Their comments made the paper clearer and its evidence more complete. Every new or changed word in the revised PDF is blue. The statements still end on page 10; the main text now runs five lines onto page 10. Tables 1 to 9 keep their numbers; the new Table 10 moves the former Tables 10 to 12 to Tables 11 to 13.

The paper in brief. Unlearning methods suppress a fact only under the forget question, so the fact often returns when the question is rephrased or after a short benign fine-tune. Our method, NRU, puts an attacker inside the training loop. A probe generator repeatedly questions the current model, each query that still extracts a fact is kept in that fact's buffer, and each update lowers a soft maximum of the loss over these queries, so training concentrates on each fact's hardest query. At matched utility, training over such a query neighbourhood lowers recovery by 0.260 with the generator and by up to 0.345 with a fixed probe suite (Table 1(b), p. 7).

What the revision adds, with the reviewers it answers

  1. (89kr, Q1) New Table 10 (p. 24), with Section 5.1 (p. 6) and Appendix H.3 (p. 23). For NPO at 16 epochs the forget question itself reproduces 0.27 of the answerable facts under the reported detector, the probes recover 0.51, and 0.27 are reached only by other queries. This supports the paper's first property directly.
  2. (89kr, Q4 and W2) Table 9(a) (p. 23) now includes NAU + anchor and DRO-NAU + anchor at 16 and 32 epochs, both stability arms at 20 and 24 epochs, and retain accuracy for every arm. Benign fine-tuning removes 0.34 to 0.68 of the suppression of each of the eight arms trained on queries alone (Section 5.4, p. 8; by arm in Appendix H.2, p. 20).
  3. (89kr, Q3; H5Gh, W1) Appendix H.3 (pp. 23-24) explains why adaptivity adds no detectable benefit: only 0.21 of the 2,846 buffered probes make the reference reproduce the answer under the reported rule. It also adds two comparisons on the fixed suite, one at the generator's coverage. Section 4 (p. 5) had said that the fixed suite covers every item; it has probes for 72 of the 103 items, and the text now says so.
  4. (89kr, W1 and Q2) Appendix G (p. 18) corrects a comparison: the 200-turn sample was drawn in equal numbers from the four agreement cells of the two criteria, so its entailment rate of 0.31 cannot be set against the conjunction's 0.09 over all issued turns. It also adds a judge from a third model family.
  5. (H5Gh, W2; 89kr) The sweep below 16 epochs that Appendix I called for was run at 4 and 8 epochs (p. 25).
  6. (H5Gh, W3) Section 3.1 (p. 3) states what the generator is for.
  7. (BHYU) About two in five sentences of the main text carry changes: the abstract explains its terms, terms are defined where they first appear, and each step of Algorithm 1 (p. 15) has a comment.

Scope of the claims. To keep the claims precise, Section 7 (p. 9) now states their scope up front. The gain is established at matched utility; at identical epochs the generator arm is on par with NPO at 16 epochs and behind it at 32 (Table 1(a), p. 7), which is why the paper recommends the cheaper fixed suite. Training on both axes at once exposes a trade-off, which Section 5.4 (p. 8) analyses. The detector is checked against a retraining oracle and two further judges; human adjudication is the natural next step, and the 200-turn sheet is released for it. NRU-S after continued training and larger models are left to future work, the latter because the generator arm already needs a 48 GB card for a 3B model.

Removed or moved text, which the PDF cannot show in blue. The pages are full, so each addition is paid for by removing text that another part of the paper states.

  • Section 1: "We ask whether an unlearned model can resist a probe generator [...]"; the list of three prior approaches, which Section 6 gives with its citations; the descriptions after the first, third and fourth contributions; and "Components are removed one or two at a time [...]".
  • Section 2: "We test this as an observed property" (P1 to P4 are now labelled "Property"), and the symbol for the reproduction probability, used once.
  • Sections 3.3 and 4: one clause each, repeated elsewhere. Section 6: its last sentence, which Section 1 states.
  • Sections 5.1 and 5.2: two supporting sentences, moved to Appendix G (p. 19) and Appendix H.1 (p. 20).
  • Appendices H and I: sentences that repeated Table 9 or the reproducibility statement, and one rhetorical clause.
  • Statements that the new measurements replace: that NAU and DRO-NAU were not measured after continued training; "Whether an intermediate number of epochs [...] is untested"; "neither was tested"; and "we did not run one", said of the sweep below 16 epochs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.