acceptodds
Under review as a conference paper at ICLR 2027

What Does It Mean to Unlearn? Indistinguishability and Admissibility

Abstract

LLM unlearning is increasingly required to remove sensitive information without retraining large models from scratch, and is often considered successful once the model stops revealing that information. However, a model that no longer outputs this information may still reveal it through its preferences among other answers, while refusal or degenerate outputs give up useful behavior. A few studies detect such failures, and none defines or achieves success. We argue that a truly unlearned model should leave no trace of having known the target information, while still acting naturally without it. We therefore require indistinguishability of outputs across worlds that differ only in the target information, leaving no evidence of the target. To keep responses appropriate, we further require admissibility, specified by an admissible region of response distributions shared across worlds. Based on this region, we introduce WorldLeak and prove that bounding it certifies -indistinguishability at a chosen resolution without a retrained reference model. For repair, the closest distribution within a WorldLeak budget gives a unique target. WorldRepair realizes it by reweighting response likelihoods and adapting erasure strength to residual leakage under a retain constraint. On TOFU, MUSE, and WMDP, WorldRepair matches strong baselines on native metrics with the lowest WorldLeak and cross-world distinguishability. Our framework defines successful unlearning in a form that can be both certified and optimized.

Primary area:
interpretability and explainable AI
Keywords:
LLM Unlearning, Indistinguishability, Unlearning Certification

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_iaUcOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies what successful LLM unlearning should mean beyond suppressing target answers. It proposes a specification combining indistinguishability across protected worlds with a shared admissible region of response distributions. WorldLeak measures residual risky behavior and deviations from admissible response proportions. Under the stated assumptions, bounds on this quantity imply indistinguishability at the chosen audit resolution. The paper also derives a unique closest behavior target under a WorldLeak budget. WorldRepair approximates this target through adaptive erasure and likelihood-based behavior correction under a retain constraint. Experiments on TOFU, MUSE News, and WMDP show reduced behavioral risk and cross-world distinguishability with competitive retained utility.

Critical strengths
  1. The paper offers a compelling and precise account of the unlearning objective. I particularly appreciate the move beyond target-answer suppression to asking whether observable behavior still carries evidence of the protected information. The examples of indirect clues and shifted candidate preferences make this distinction concrete. Combining indistinguishability with admissibility also addresses the need to specify appropriate behavior after forgetting. This conceptual contribution substantially improves how the problem can be understood and evaluated, and is a central reason for my positive assessment.

  2. The connection between theory, evaluation, and training is coherent and supported by convincing experiments. The shared admissible region provides a common reference for WorldLeak, an indistinguishability guarantee, and the projection target used by WorldRepair. The paired-world experiments are particularly valuable because the worlds are repaired independently without directly optimizing their discrepancy. In the TOFU audit, the reported mean per-seed maximum cross-world TV is 0.040, compared with 0.28 for the strongest baseline on this metric (Figure 5). The results across TOFU, MUSE News, and WMDP, together with the component ablations, provide persuasive evidence that the framework improves the measured behavior while maintaining competitive utility.

Critical weaknesses
  1. The practical coverage of the guarantee depends on the predefined behavior taxonomy and audit resolution. Similar distributions over a small number of behavior classes do not necessarily rule out distinguishable patterns within those classes, such as differences in wording or more subtle indirect clues. The paper explicitly recognizes this issue and gives an event-approximation term for extending label-level guarantees to full observations (Appendix A.6). However, the experiments do not establish a uniform bound on this term for unrestricted responses. Consequently, the empirical certification supports the specified audit channels, and the extent to which it captures richer forms of residual leakage remains uncertain. I view this as the main limitation on the scope of the contribution.
Questions for authors

Clarification of audit coverage: Beyond the admissible-region bound sensitivity study in Table 23, do you have evidence on whether the observed improvements remain stable when the behavior taxonomy is refined or the audit looks for distinguishing patterns within an admissible class? In particular, how should a practitioner assess whether the chosen resolution captures the residual leakage relevant to their application? This would help determine how broadly the reported certification and experimental gains can be interpreted.

3 comments

Thank you very much for your thoughtful review and positive assessment. We appreciate your recognition of our contribution and the connection between our theory, evaluation, and training method. Your questions on audit coverage help us add an experiment and clarify how users can choose the audit classes.

1. Additional evidence on audit coverage

To check for distinguishing patterns within classes, we added an independent classifier that predicts which world a response comes from using its text. We use a frozen all-mpnet-base-v2 encoder with logistic regression to predict world A or B from each response. Training and validation use separate batches of newly generated responses. For testing, we use 840 TOFU and 1,280 WMDP responses per method. We report balanced accuracy (BA) on all responses and the safe non-selection subset. The chance level is 50%.

SettingMethodAll responses: BA (%)Safe subset: BA (%)
TOFUBefore unlearning92.26—
TOFUGradDiff73.21—
TOFUNPO68.9352.72
TOFUSimNPO66.5555.16
TOFURMU62.7451.79
TOFURT67.7455.93
TOFUWorldRepair52.5053.12
WMDPBefore unlearning88.52—
WMDPGradDiff76.0975.00
WMDPNPO66.0956.64
WMDPSimNPO63.4454.22
WMDPRMU62.8956.67
WMDPRT62.1151.34
WMDPWorldRepair51.8052.81

Several baselines also approach 50% within the safe subset, but their all-response BA remains above 60%, showing that their responses are still more distinguishable overall. For WorldRepair, this classifier's BA stays close to 50% on both the full response sets and the safe subsets. These results provide additional response-text evidence.

2. Choosing and checking the classes

Users should start with the information they need to protect. For example, when forgetting a person's hometown, the audit should distinguish directly naming the hometown from giving an indirect clue that narrows down its location. It can then separate uncertainty, helpful answers based on retained knowledge, and degenerate outputs according to the task's requirements.

To check whether the classes capture relevant leakage, users can inspect new responses within each class. When paired-world models are available, a response-text classifier can help detect remaining differences, as in our added experiment. If new clues are found, users can refine the classes and test them on another set of responses. The goal is to capture leakage relevant to the application.

Thank you again for your careful reading and constructive suggestions.

Thank you for the thoughtful response and the additional experiments. I appreciate the independent response-text classifier evaluation, particularly the results on the safe non-selection subset, which directly address my concern about potential distinguishability within predefined behavior classes.

While the theoretical guarantee necessarily remains dependent on the chosen audit resolution, I believe the new evidence and clarifications have satisfactorily addressed my concerns within the paper's stated scope.

I am satisfied with the response and will raise my positive rating to 4.

Thank you very much for your support and positive reassessment! We truly appreciate the time and care you put into reviewing our work and discussing it with us.

We are glad to have this constructive exchange. Thank you again for helping us improve the paper.

Reviewer_jFeWOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

Spoiler: My review style is to read passages of the paper and deconstruct them with comments and/or questions. So sometimes you might find my raw thoughts there. There might be questions in the weakness section as well. I believe that an AI assistant, if fed my entire review, can reorganize it better. Maybe the authors can use it to do this task.

Summary

The paper tackles unlearning from a privacy-leakage perspective and, in my opinion, wrongly tries to answer what unlearning actually means. The authors claim that the unlearned model must be indistinguishable from not having had the forgotten knowledge in the first place, which was already a desideratum since the birth of the field. Next, they state that the unlearned model should lie within an admissible region, defined ad hoc for each forgotten target. The authors propose WorldLeak and WorldRepair to support these claims.

Critical strengths

Since we're encouraged by the ACs and SACs to express our crude meanings, I'm doing so. Therefore, I plead with the authors to read my comments without offense, as none is intended.

None identified, but I give credit to the authors for trying to tackle a highly unregulated field, such as "undefined" (i.e., everybody is doing what they want), by shedding light on its meaning, which, unfortunately for the authors, has been cemented as the trade-off between forget quality and model utility, and has been accepted as such since 2022 at least. I understand that this field seems like the Wild West; however, your stating "What does it Mean to Unlearn?" in the title actually backfires, since you're just proposing a Privacy Leakage (PL) mitigation framework while maintaining SoTA performance, or sometimes underperforming by a huge margin.

That's why I don't see any strengths in this paper. I'm extra willing to discuss with the authors about this central claim, which to me is anticlimactic, especially once you get past the math-fumes in the central part of the paper and move onto the experiments. Then you have a "meh" moment, where you figure out that the paper is just another PL metric proposal, in its essence. Please change my mind without referring to the math for justification. What are you doing here?

Critical weaknesses
  1. No code. Can't reproduce! Huge red flag for me.
  2. There is no reason to define the admissible region as you are doing. Why are you doing it like this? It doesn't feel like a theory-driven paper. It feels like you tried it empirically and then tried to write up the theory/math part, which is nothing wrong per se, but it's missing the "why." I can't take these definitions for granted and trust you.
  3. It feels like you did a lot of build-up and then the climax under-delivers. If the refusal option is completely indistinguishable, by refusing to respond, who says that, from the user's perspective, it's a non-admissible model? I mean, when you have a model that has forgotten zz, and it always says I don't know who zz is (TOFU-like question), that's a perfectly unlearned model. The admissible region feels like steering here, and that comes with trade-offs in neighboring knowledge.
  4. Stating that "neighboring protected worlds are generally unavailable" is false, since the RWKU benchmark [1] gives you questions regarding the neighborhood of the forget target.
  5. The usage of the terminology "world" to indicate the target to be forgotten misaligns the authors from the rest of the field, especially if they're trying to respond, which they don't by the way, to the question "What does it mean to unlearn?"
  6. The authors say, I don't remember where (probably in Section 3), that refusing to respond when given a question from the forget set is still leaking privacy, and that they need to fix this. Yet, in the experiments (see Figures 6 and 7), they refuse to respond and say "I don't know" in the majority of cases. Huh!?
Questions for authors

Please note that I'm willing to change my scores, if you clearly state that this paper isn't actually responding to the question "What Does it Mean to Unlearn?" (it's very dangerous to use a question for your title if you're not directly responding to it), but that it's proposing a new PL metric, and that you need to acknowledge it from the get-go.

Q1: The example of Table 1 is just confusing. I do understand what a world is in your terminology after paying more attention, but when you introduce it here, you're alienating yourself from the rest of the literature. Here, the world is just a concept to be forgotten. So, say so. It doesn't justify the claim that your method is WorldRepair or that your proposed PL metric is called WorldLeak. I'd suggest you not confuse the reader. Here's my original question before I read the entire paper:

What is a world!? What are the columns saying here? Are they examples that you're testing to see if you reveal your forgotten target z?

I feel like this doesn't give a motivation for neighboring knowledge to resurface the forgotten target. Unless I'm not understanding something

Q2: The questions of weakness #2, which I'm reporting here:

There is no reason to define the admissible region like you are doing. Why are you doing it like this?

Q3: Who says that measuring the deviation of the forgotten concept zz from the shared region determines the drift from the common knowledge? Are you measuring model utility here (probably not) or the deviation from the neighboring knowledge?

Small suggestion: *It's very risky to claim something is happening if you're defining stuff yourself and then don't use the same definition to measure how the SoTA *behaves.

Q4: How did you generate the 100 candidate questions for behavioral auditing? What is behavioral auditing here?

Q5: In the baselines, why didn't you compare against ICU [2] and RT [1]? They are basically rejection-based methods, which, if you perform better against them, would support your thesis that only refusal isn't good. These experiments on TOFU should be run during the rebuttal period.

Q6: In Table 6, why are the classes different for each dataset? Did you define them yourself?

Q7: Why is privacy leakage negative in these tables for the base model and you (see MUSE News)?

Q8: In the majority of cases, you're still refusing to respond (Figures 6 and 7). So, why are you better than straight refusal? To answer this question, run ICU [2] with a system prompt that says "Say I don't know when asked for Target: {target}". I'm expecting this to have similar results to yours. Or just use straight Refusal (check the paper by Zaradoukas et al. [3]).

[1] Jin et al. Rwku: Benchmarking real-world knowledge unlearning for large language models. NeurIPS'24

[2] Pawelczyk et al. In-context unlearning: Language models as few-shot unlearners. ICML'24


General feeling while reading the first 3 sections of the paper. The authors should ask themselves why.

I had this gut feeling that the climax would underdeliver at the beginning of the paper, and it actually does. You're not forgetting more than SoTA methods, and you're not achieving better model utility (see Tables 3 and 5). You're just trying to minimize Privacy Leakage (PL).

It seems like WorldLeak is just another PL measure... whereas the paper's central section describes it as a method! Of course, if you define your own method and measure against WorldLeak, you're going to score the best; otherwise, what's the purpose of a published paper? My main critique is that you're not better than SoTA anywhere besides in your metric and privacy leakage, since your method is just designed around this "auditing" protocol. In this light, I'd suggest you run RWKU on 20 targets (same as PURGE [3]) and check MIA metrics. I believe that you might be better in these metrics, which is fair, but your claims and statements in the introduction are just far-fetched.

[3] Zaradoukas et al. Reinforcement Unlearning via Group Relative Policy Optimization. ICLR'26

Reviewer_kBJxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper argues that current LLM unlearning methods are judged by the wrong criterion: a model can stop producing the forget target yet still reveal it through related alternative answers it prefers, and refusing everything removes the leak at the cost of usefulness.

The paper proposes two requirements. First, indistinguishability: the model's behavior should look the same whether or not it ever learned the protected fact. Second, admissibility: the remaining behavior should fall within a set of acceptable response types (helpful, uncertain, balanced among candidates) rather than collapsing to refusal or nonsense. Responses are categorized by an LLM judge, and a metric, WorldLeak, scores how far a model's response mix is from the acceptable set. The paper proves that a low WorldLeak score implies indistinguishability at the level of these categories, and that there is a unique closest acceptable target to steer toward.

A training method, WorldRepair, drives the model to that target by reducing the likelihood of the forget content in proportion to measured leakage and reshaping the response mix, while constraining drift on retained data. Experiments on TOFU, MUSE News, and WMDP report standard metrics, WorldLeak, a paired-world test comparing two models that differ only in the protected fact, and a finite-sample certificate for one such pair.

Critical strengths
  • The diagnosis is interesting and well illustrated.
  • The paired-world evaluation protocol is reusable, and the reported numbers are internally consistent.
Critical weaknesses

1. The certification result is vacuous at the reported operating points and holds only at label resolution.

Instantiating Theorem 1 with the reported TOFU WorldLeak (η=2.39\eta = 2.39) and the stated bounds (cH∈[0.1,0.5]c_H \in [0.1, 0.5], so max⁡ilog⁡(ui/ℓi)=log⁡5\max_i \log(u_i/\ell_i) = \log 5) yields ε≈6.4\varepsilon \approx 6.4, δ=ητ=0.12\delta = \eta\tau = 0.12; the baselines give ε∈[19,26]\varepsilon \in [19, 26]. e6.4≈600e^{6.4} \approx 600 is not a privacy guarantee in any useful sense, and the log⁡(u/ℓ)\log(u/\ell) term is a property of the region, not the model, so ε≥1.6\varepsilon \ge 1.6 on TOFU and MUSE for any model. The only non-vacuous certificate is Table 6, (0,0.13)(0, 0.13) at n=256n = 256, but this is over M=4M = 4 classes, for a model pair A.6 never identifies (the only paired WMDP worlds described, A.7.3–A.7.4, are built on synthetic entities such as Neralis-7), and the bound reduces to max⁡{r^A,r^B}+hn\max\{\hat r_A, \hat r_B\} + h_n because both empirical risks fall below the Hoeffding radius. The measured label TV (≤0.005\le 0.005) does not enter it.

More fundamentally, all guarantees are stated for the induced label channel TM(Oγ)T_M(O_\gamma), not for OγO_\gamma. The paper's own Example 3.1 (an event with P±(A)=12±νP_\pm(A) = \tfrac12 \pm \nu) can be realized within a single admissible class, so two worlds with secret-dependent hedging satisfy WorldLeak ≈0\approx 0 while remaining distinguishable in BB samples. The authors concede this (end of Section 4.1; Proposition 4) but never bound ζzz′\zeta_{zz'}, so nothing is certified about responses. The abstract's "certifies (ε,δ)(\varepsilon,\delta)-indistinguishability at a chosen resolution" is carried entirely by the qualifier, and the introduction's "a finite-sample WMDP audit certifies the evaluated pair" overstates Table 6. Note also that A.7.5 admits risky mass determines the reported TOFU scores; the same holds on MUSE and WMDP (every reported WorldLeak equals r/τr/\tau). Operationally the metric is a judge-labeled risky-response rate scaled by a per-benchmark τ\tau.

2. The empirical comparison is circular, and the realized behavior is the refusal outcome the paper argues against.

WorldRepair optimizes against an LLM judge with the same class definitions as the audit judge, on prompts drawn from the audit pool (TOFU: K=30K = 30 per block from the same 510 queries; WMDP: 16 candidate + 16 indirect prompts per block in the audit format, with no statement that these are disjoint from the audit set). Winning on WorldLeak against baselines that never saw this objective is uninformative. The paired-world runs use separate repair/audit templates (A.7.3) but the same facts and candidate format. There are no paraphrased or reordered probes, and none of the cited robustness evaluations (Leak@k, relearning, adversarial auditing) is run. Judge validity rests on one author inspecting a fifth of the labels; no agreement statistics are reported.

The admissible region is specified with cU∈[0.5,0.9]c_U \in [0.5, 0.9] against cH∈[0.1,0.5]c_H \in [0.1, 0.5], and the models track it. WorldRepair's helpful mass exceeds all baselines (15.8% TOFU, 13.2% MUSE), but the dominant mode is refusal: U=85.6%U = 85.6\% on MUSE, and all sixteen WorldRepair responses in Tables 8–11 are refusals (Tables 20a and 21a are the only helpful examples). This is close to the δrefuse\delta_{\text{refuse}} case Section 3.2 presents as the failure mode. The remaining differentiator is flattened candidate distributions (Table 19, mean TV to uniform 3.94%), which is not "acting naturally without knowing": a model that never acquired the fact retains a prior over candidates, and enforced uniformity is itself an output-level unlearning trace of the kind Chen et al. (2026) detect.

The baselines also appear mis-tuned. Hyperparameters were "tuned per benchmark toward stronger forgetting," producing a collapsed GradDiff (TOFU Model Utility 0.063) and an RMU with MMLU 48.0 and 20.3% degeneration. The RMU paper reports 31.2 / 28.2 / 57.1 (Bio / Cyber / MMLU) for the same Zephyr-7B base, and this paper's "Full" row reproduces that base exactly; modulo evaluation-harness differences, published RMU ties WorldRepair on Bio and beats it on Cyber (31.1) and MMLU (50.85). The two closest prior methods, Yuan et al. (2025b) and Gu et al. (2026), both of which prescribe post-unlearning target behavior, are cited but not compared.

Post rebuttal updates

The rebuttal has resolved most of my concerns. I decided to raise the score to Weak Accept.

Questions for authors

Please address the weaknesses.

Comments on the paper

3 comments

Thank you for the careful follow-up, for recognizing our additional experiments, and for reconsidering your assessment. Your suggestion is especially helpful, and we have now completed experiments.

1. Audit resolution and certification

"the practical significance of certifying it depends critically on audit resolution."

Re: We provide a practical way to audit model behavior and an indistinguishability guarantee for the chosen classes. When we identify new differences between worlds or have specific requirements for how the model should behave after unlearning, we can refine the classes accordingly. WorldLeak provides a practical way to audit each model against a shared specification without requiring a paired reference model.

Since models from both worlds are not always available in practice, our method offers a way to check the conditions required for each world in Theorem 1. We also welcome better approaches.

Following your suggestion, we have now added a separate classifier that predicts which world a response comes from using its response text.

2. Independent response-text evaluation

"An independent cross-world discriminator operating directly on held-out generated responses, particularly those classified as admissible, would provide much stronger evidence for the paper's central indistinguishability claim."

Re: We use a frozen all-mpnet-base-v2 encoder with logistic regression to predict world A or B from each response. Training and validation use separate batches of newly generated responses. The 840 TOFU and 1,280 WMDP responses per method are used for testing. We report balanced accuracy (BA), with a chance level of 50%, on all responses and the safe non-selection subset.

SettingMethodAll responses: BA (%)Safe subset: BA (%)
TOFUBefore unlearning92.26—
TOFUGradDiff73.21—
TOFUNPO68.9352.72
TOFUSimNPO66.5555.16
TOFURMU62.7451.79
TOFURT67.7455.93
TOFUWorldRepair52.5053.12
WMDPBefore unlearning88.52—
WMDPGradDiff76.0975.00
WMDPNPO66.0956.64
WMDPSimNPO63.4454.22
WMDPRMU62.8956.67
WMDPRT62.1151.34
WMDPWorldRepair51.8052.81

Several baselines also approach 50% within the safe subset, but their all-response BA remains above 60%, showing that their responses are still more distinguishable overall. For WorldRepair, this classifier's BA stays close to 50% on both the full response sets and the safe subsets. These results provide additional response-text evidence.

3. Meaning of admissible bounds

"it does not independently justify which bounds represent appropriate post-unlearning behavior."

Re: The admissible region provides a shared reference when paired-world models are unavailable. If models from different worlds meet the same constraints, then we can bound how much their behavior can differ at the chosen class level.

Users set the bounds based on their task requirements. At a minimum, they can treat degenerate responses as unacceptable, even if those responses do not reveal the forgotten information.

Table 23 shows the effect of these choices. Shifting the MUSE bounds toward helpful responses increases the helpful share from 11.88% to 25.16%, while risk rises from 1.22% to 1.47%. This shows the trade-off users can choose; it does not establish one correct set of bounds for every task. We clarify that these bounds express task requirements.

4. Comparison with refusal baselines

"A comparison with a simple, explicitly controlled refusal policy would help clarify the additional value of the proposed behavioral specification."

Re: We added ICU and Rejection Tuning (RT) comparisons on TOFU. RT uses supervised fine-tuning on questions about the forget targets paired with refusal responses.

MethodD (%) ↓C (%) ↓I (%) ↓G (%) ↓SC (%)H (%)U (%)Model Utility ↑Forget Q-A ROUGE ↓
ICU17.83.730.012.917.96.810.80.5350.491
RT5.01.117.95.420.52.447.70.1950.026
WorldRepair1.31.95.13.719.715.852.60.5600.120

ICU retains substantial leakage. RT achieves low answer overlap but leaves indirect clues and loses utility. Our separate checks suggest that its learned refusal is less reliable on fill-in-the-blank or adversarial prompts than on direct questions. WorldRepair reduces direct and indirect leakage while preserving more helpful responses and higher utility than both baselines.

Thank you for your reply. I am satisfied with the rebuttal. I have raised the score to Weak Accept.

Thank you very much for your careful review and positive reassessment. We truly appreciate the time you devoted to this discussion.

Your suggestions helped us clarify our claims and add several important experiments. Thank you again for helping us improve the paper!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.