acceptodds
Under review as a conference paper at ICLR 2027

Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Models

Abstract

Existing attacks on unlearned diffusion models assume that the erased concepts are known in advance and focus on recovering them. In practice, however, model providers may not disclose which concepts have been removed, and even with access to the original base model, an adversary may still lack a clear target to attack. In this paper, we aim to answer the following critical but overlooked questions: ***which concepts have been erased from the model, and how many have been erased in total?*** To this end, we present TRACER, a framework that rapidly and accurately identifies erased concepts and estimates their number. TRACER efficiently identifies erased concepts without generating and classifying images. Using lightweight spectral analysis of weight footprints, it enables efficient search over large candidate vocabularies. To distinguish multiple erased concepts, we introduce a footprint coverage objective that guides sequential discovery. TRACER estimates the number of erased concepts by detecting a sharp decline in candidate confidence as the selected concepts account for the erasure footprint, without requiring labeled examples for calibration. The framework requires only lightweight linear algebra and limited forward probes, with no prior knowledge of the unlearning algorithm. Experiments across text-to-image and text-to-video backbones and diverse unlearning methods demonstrate that TRACER identifies erased concepts and estimates their number *in seconds*, achieving **150–137,000×** and **133–20,000×** speedups over MIA and brute-force search on image and video models, respectively, with substantially higher identification accuracy.

Primary area:
generative models
Keywords:
Machine Unlearning, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_YU55OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the identification of unknown erased concepts in diffusion models. It proposes TRACER, which compares original and unlearned weights to construct erasure footprints, then progressively selects concepts from a candidate vocabulary based on subspace coverage. The number of erased concepts is estimated from drops in candidate confidence. Experiments across image and video models show high recovery accuracy and substantial speedups over generation-based probing and membership inference baselines.

Critical strengths
  1. The paper investigates an interesting problem that whether model weights reveal undisclosed erasure targets and their number, extending the study of unlearning vulnerabilities beyond recovering known erased concepts.

  2. This paper proposes a coherent approach that combines weight-difference analysis, candidate representation processing, and progressive subspace coverage. Avoiding image generation makes searches over large candidate vocabularies potentially practical. The performance looks good while keeping speed fast.

Critical weaknesses
  1. Since even the ablated variants substantially outperform the existing baselines, the current experiments do not establish how much of the advantage comes from the proposed components rather than directly exploiting weight differences.

  2. Under the evaluation of 50 checkpoints with five erased concepts each, Recall@5 must change in increments of 0.4 percentage points. However, Table 3 reports 16.2%, 36.6%, and 48.2%, may indicating that some baselines are evaluated under different settings.

  3. The erased classes are sampled from Imagenette-20, which extends Imagenette’s ten classes (or possibly Imagenette-50, I am not sure since Section 4.1 says Imagenette-20 while Section 4.2 says Imagenette-50). This may limit the evaluation of the method. Given that the method can identify almost all erased classes, Imagenette-20 may not be challenging enough to distinguish performance differences. The authors should evaluate a larger set of classes as erasure targets.

Questions for authors

See weaknesses, I will raise my score if the authors clarify the evaluation details and provide further ablations to attribute the performance gains.

3 comments

We thank the reviewer for the careful reading and constructive suggestions. We address each point below.

W1: Attributing the performance gains to the proposed components

We agree that removing one component at a time does not isolate the effect of simply using weight differences. We therefore add a cumulative ablation (Table A) that starts from a naive weight-difference baseline and adds the components one by one. The naive baseline scores each candidate by ∥ΔWxc∥2\|\Delta W x_c\|^2, the energy of its text embedding projected onto the cross-attention weight change, and ranks all candidates once. The "+ Background suppression" row corresponds to the "w/o progressive selection" variant in Table 5.

Table A. Cumulative ablation in Recall@5 (%) on SD v1.5, using the same checkpoints as Table 5. Baselines from Table 3 are included for reference.

VariantESDUCESPEEDScaPre
MIA4.814.042.448.0
Brute-force16.036.438.852.0
Naive ΔW\Delta W33.257.259.664.0
+ Multi-imprint integration41.661.666.069.6
+ Background suppression56.478.482.881.2
+ Progressive selection (Full TRACER)100.0100.0100.0100.0

The naive baseline already outperforms both baselines on all four methods (33.2–64.0%), confirming that weight differences carry a strong signal of erasure targets, a core finding of this work. However, it remains far from full recovery, and each component brings consistent gains. Progressive selection contributes the largest gain on every method (17.2–43.6 points), as it separates jointly erased concepts that a single static ranking cannot distinguish. We will add this table to Section 4.4.

W2: Recall@5 values in Table 3

Thank you for this careful check. The reviewer is correct that Recall@5 must change in increments of 0.4 percentage points. The three values are transcription errors; the correct values are 16.0 (ESD, Brute-force), 36.4 (UCE, Brute-force), and 48.0 (ScaPre, MIA), and these corrections do not affect any conclusion. We confirm that all methods in Table 3, including the baselines, are evaluated under the identical setting: the same 50 checkpoints per method and erased-set size, the same 1,000-entry candidate vocabulary, and the same true KK provided to all approaches (Section 4.3). We will correct the table in the revision.

W3: Choice and scale of the erased concept set

We apologize for the confusion. "Imagenette-50" in Section 4.2 is a typo for "Imagenette-20"; all object-erasure experiments use the same Imagenette-20 set.

Why Imagenette. Imagenette is the standard object benchmark on which concept-erasure methods such as ESD and UCE validate their object erasure, which ensures that erasure is effective on the evaluated checkpoints. For arbitrary classes, many unlearning methods are not validated and may fail to erase the targets or damage unrelated generation, so identification results would conflate identification failures with erasure failures. To support larger erased-set sizes, we extend the ten Imagenette classes with ten common ImageNet-1k categories. As the community develops new unlearning methods and benchmarks, we will continue to update our code repository and evaluate TRACER on each new method and its corresponding dataset.

Difficulty of the task. The difficulty of identification lies mainly in the candidate vocabulary rather than the size of the target pool. TRACER searches over all 1,000 ImageNet class names, and over 73,406 entries in the expanded vocabulary, without knowing which concepts or how many were erased. This vocabulary contains many fine-grained classes close to the targets; for example, English springer must be distinguished from over one hundred other dog breeds, including Welsh springer spaniel and cocker spaniel, and our exact-set criterion counts any confusion as a failure. The benchmark also clearly separates methods: on the same erased sets, the baselines reach only 4.8–52.0% Recall@5 (Table 3), and the naive weight-difference variant reaches 33.2–64.0% (Table A). The near-perfect accuracy of TRACER therefore reflects the effectiveness of the method rather than an easy benchmark. In addition, Appendix B shows near-perfect identification for artistic styles, a different concept type.

We will clarify these points in the revision, and hope these responses address the reviewer's concerns.

Thanks for the authors’ responses. I have increased the rating from 2 to 3.

We sincerely thank the reviewer for the thoughtful feedback and are glad that our responses have addressed the concerns. We appreciate the increased score and will incorporate the clarifications and additional results into the revision.

Reviewer_mjb5OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies whether the concepts targeted for erasure can be identified from the original and concept-erased diffusion models without knowing the targets or their number. The proposed TRACER constructs erasure footprints from weight differences, matches candidate concept representations to these footprints, and uses progressive subspace coverage and confidence drops to identify multiple targets and estimate their number. The procedure uses weight analysis and limited forward probes without generating images. Experiments cover multiple image and video backbones and concept-erasure methods, reporting high identification accuracy primarily for objects and artistic styles, together with speed advantages over generation-based probing and an adapted membership inference attack.

Critical strengths
  1. The method uses weight changes and limited forward probes to identify erasure targets, avoiding image generation for every candidate. The reported efficiency advantages have practical potential when searching large vocabularies.
  2. The experiments cover multiple erasure methods and both image and video backbones, evaluating target identification and target-count estimation. This supports the method's effectiveness in the controlled settings tested.
Critical weaknesses

1. The practical value and motivation of the task are insufficiently supported

W01: The practical value for an attacker is not sufficiently established. An attacker seeking to generate content depicting a particular concept typically already knows the target and does not necessarily need to identify the complete erased set or its size. They can directly test whether the model generates the desired content and, if it does not, attempt adversarial methods targeting that concept. The proposed setting also requires access to aligned model weights before and after erasure, and is not directly applicable when only the erased model is available or when a commercial service exposes only a black-box interface. Under these restricted access conditions, the paper does not sufficiently establish the additional practical value of complete target identification. Although the experiments demonstrate identification capability, they do not establish whether it improves subsequent attacks, guides more effective concept erasure, or reveals deletion choices with concrete privacy implications in sensitive scenarios.

W02: The motivation needs stronger grounding in prior work and clearer positioning. The motivation for identifying unknown erasure targets is primarily developed through a hypothetical scenario, with insufficient discussion of existing research on deletion-related information leakage. For example, When Machine Unlearning Jeopardizes Privacy, Deletion Inference, Reconstruction, and Compliance, and Learn What You Want to Unlearn have studied inference or reconstruction of deleted information using models before and after unlearning. The authors should connect their work to these studies, clarify how concept-level target identification relates to and differs from existing deletion-inference problems, and better establish the practical need and additional value of the proposed setting rather than relying primarily on scenario-based reasoning.

2. The experiments provide limited support for broader settings

W03: The experiments do not distinguish different causes of missing concept-generation capability. The current experiments primarily test whether the method can recover deliberately selected erasure targets. They do not specifically distinguish concepts that the original model already lacked the ability to generate from concepts that it could previously generate but that were subsequently erased. Controls using models with no concept erasure or only ordinary fine-tuning are also not reported. Consequently, the results primarily support target identification under controlled erasure, while it remains unclear whether the method can reliably distinguish different causes of missing concept-generation capability in practical settings.

References:

  1. When Machine Unlearning Jeopardizes Privacy. CCS 2021.
  2. Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learning. PETS 2022.
  3. Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning. IEEE S&P 2024.
Questions for authors
  1. An attacker who only wants to generate content depicting a particular concept can directly test whether the model can generate it and, if that fails, attempt adversarial methods targeting that concept. For this objective, what additional benefit comes from identifying all erased concepts and their total number? If the paper addresses a different attack objective, please specify the concrete scenario and provide evidence of its practical value.
  2. How is the original model's ability to generate the experimental target concepts verified? With no erasure, ordinary fine-tuning alone, or candidate concepts that were already difficult to generate, can the method avoid falsely identifying erasure targets and return an empty set where appropriate?
4 comments

We thank the reviewer for the detailed and constructive comments, and address each point below.

W01 & Q1: Practical value of identifying the full erased set

We agree that an attacker targeting a single known concept can test it directly. TRACER addresses a different setting: a comprehensive attack on a released unlearned model, where the goal is to uncover and restore everything the provider suppressed. A typical example is red-teaming a released model to assess whether its erasure is reliable. Here the attacker does not know what was erased, and the original model alone cannot reveal which of its many concepts the provider chose to remove. Identifying the complete set provides the following benefits:

  • Making existing attacks applicable. Existing attacks on unlearned diffusion models (e.g., UnlearnDiffAtk, P4D, Ring-A-Bell) all require the target concept as input. Without TRACER, an attacker would have to run them on every candidate in an open vocabulary; TRACER provides the complete target set in seconds, turning an exhaustive search into a targeted attack.
  • Joint restoration. Knowing the full set allows all erased concepts to be restored jointly in a single relearning run, rather than with one attack per concept.
  • Auditing. An auditor must verify that a provider erased exactly what it claimed, and nothing more. One-by-one testing can confirm that a given concept was erased, but cannot establish that no other concept was.
  • Revealing sensitive erasure choices. Following Chen et al. [1], the difference between the two model versions exposes what was removed; in concept erasure, it reveals undisclosed decisions of the provider, such as which artists or copyright holders requested removal.

Table A compares the two pipelines on UCE with K=5K = 5 concepts erased from Imagenette-20 on SD v1.5. Existing attacks are given the true erased concepts, the most favorable setting for them, yet must attack each concept separately. TRACER identifies the erased set without prior knowledge and restores all concepts in a single relearning run, achieving 68.8% ASR at over 40× lower cost than the fastest existing attack.

Table A. Restoring erased concepts from UCE on SD v1.5 (K=5K = 5, Imagenette-20). ASR is measured by an ImageNet-pretrained ResNet-50. Cost is the total GPU time for all five concepts, including identification for TRACER. Joint relearning fine-tunes a LoRA adapter on the unlearned model with images of the identified concepts generated by the original model, following MemoRa (2025).

MethodNeeds known targetsRunsASR (↑)Cost (GPU-h, ↓)
UnlearnDiffAtkYes549.2130.2
P4DYes542.4181.9
Ring-A-BellYes546.822.3
TRACER + joint relearningNo168.80.51

Access assumption.

Access to both the original and unlearned models is a common assumption in this line of work. All three works cited by the reviewer [1–3] assume access to both model versions, and Hu et al. [3] assume white-box access to both sets of parameters. Attacks on unlearned diffusion models adopt similar assumptions; for example, P4D uses the original model to red-team the erased model. This also matches practice: all backbones and unlearning methods we evaluate are open-source, and unlearned checkpoints are typically released as edits of public base models. Black-box settings that expose only an API are beyond the scope of this paper, and we will state this explicitly as a limitation.

References

[1] M. Chen et al. When Machine Unlearning Jeopardizes Privacy. ACM CCS 2021.

[2] J. Gao et al. Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learning. PETS 2022.

[3] H. Hu et al. Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning. IEEE S&P 2024.

W02: Relation to deletion-inference works

We thank the reviewer for these references and will discuss them in the related work. These works study the privacy of training-data deletion in classical models: Chen et al. [1] infer whether a given sample was deleted from classifiers via membership inference on the two model versions; Gao et al. [2] identify or reconstruct a deleted record in regression, 1-NN, and n-gram models; and Hu et al. [3] recover the features and labels of unlearned samples from image classifiers. They share our core observation that the difference between the original and unlearned models leaks what was removed, which further supports our motivation. TRACER differs in three aspects:

  1. Concept erasure rather than data deletion. We target large-scale text-to-image and text-to-video diffusion models, where concept erasure edits the weights to suppress a concept, often without deleting any training sample (e.g., closed-form editing in UCE). There is thus no deleted record to test or reconstruct; the inference target is the erased concept itself.
  2. Open-vocabulary search. The closest setting, label inference in Hu et al. [3], selects the deleted class from the fixed output classes of a classifier. Diffusion models have no such label space, so TRACER searches an open text vocabulary of up to 73,406 entries.
  3. Unknown set size. These works mainly consider a single deletion or a given number of deleted classes, whereas TRACER jointly identifies multiple erased concepts and estimates their number without prior knowledge.

W03 & Q2: Verifying generation ability and avoiding false identification

Generation ability of the erased targets. Our object targets follow Imagenette, the standard object-erasure benchmark used by ESD, UCE, and subsequent unlearning methods. These works verify that the original models generate these classes and that erasure suppresses them, which underlies their reported erasure results. Our evaluation inherits this setting, so the targets are concepts that the original model could generate and that were subsequently erased.

Concepts already difficult to generate. TRACER does not infer erasure from generation failure. It scores each candidate by the alignment between its text embedding and the weight difference, which is independent of whether the original model can generate it. To verify this, we compare five erased concepts with three classes that the original model generates poorly: nail, whiskey jug, and drake (Table B). The erased concepts always occupy the top five ranks and are always selected, whereas the three hard-to-generate classes rank between 149 and 692, around the middle of the 1,000 candidates like ordinary non-targets, and are never selected. Moreover, our exact-set accuracy counts any extra selected concept as a failure (Section 4.2), so the near-perfect accuracy in Tables 1 and 4 further confirms that such concepts are not falsely identified.

Table B. Five erased concepts vs. three hard-to-generate classes on SD v1.5. Original Acc is the ResNet-50 accuracy on images generated by the original model. Rank is the position among the 1,000 ImageNet class names by the matching score of TRACER. Selected is the fraction of checkpoints in which the concept is recovered.

ConceptOriginal Acc (%)MACE rankUCE rankScaPre rankSelected (%)
Erased targets (avg.)84.91–51–51–5100
nail15.23456922720
whiskey jug25.41494245610
drake19.64543172730

No erasure and ordinary fine-tuning. Without erasure, the weight difference is zero, so all candidate scores are zero and TRACER returns an empty set. Ordinary fine-tuning also changes the weights, but these changes concentrate on the concepts it trains, such as the class word in DreamBooth or the style word in a style LoRA. TRACER therefore ranks these concepts highly rather than producing arbitrary false positives; it identifies concepts whose conditioning has been modified, without distinguishing suppression from enhancement. The two cases can be separated by a lightweight check on the few selected candidates: a candidate is kept only if the released model no longer generates it, and an empty set is returned if none remains. Since this check involves only the selected candidates rather than the full vocabulary, the efficiency of TRACER is largely preserved. We will add this extension and a discussion of ordinary fine-tuning to the revision.

We hope these responses address the reviewer's concerns, and we are happy to provide further clarification if needed.

Reviewer_mjb5OpenReview ↗

Official Comment by Reviewer mjb5

The authors adressed my concerns and I have increased my rating.

We sincerely thank the reviewer for the thoughtful feedback and are glad that our responses have addressed the concerns. We appreciate the increased score and will incorporate the clarifications and additional results into the revision.

Reviewer_xBVyOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TRACER to identify which unknown concepts were erased from a diffusion model and estimate how many concepts were erased in total, given the original model weights, the unlearned model weights, and a candidate vocabulary. By directly analyzing weight differences to search for target words one by one, the method avoids generating images or fine-tuning models, achieving high accuracy and large speedups across multiple image and video models and unlearning methods.

Critical strengths

Jointly identifying unknown erased concepts and estimating their number from a large vocabulary is a novel and practical problem setting. Moreover, using weight changes to progressively search for target concepts without generating images matches the goal of fast auditing, and the strong results across different models and unlearning methods show good practical potential. In addition, the method requires no prior knowledge of the specific unlearning algorithm or edited layers, relying instead on label-free self-calibration and confidence drops to automatically determine the stopping point, with each component well validated by ablation studies.

Critical weaknesses

A few details in the description of the main experiments are slightly unclear. Section 4.1 mentions explicit-content evaluation using I2P prompts and generation quality evaluation on MS COCO, but the corresponding results do not appear to be included in the paper. In addition, Sections 4.1 and 4.2 refer to Imagenette-20 and Imagenette-50 respectively when describing the evaluated object concepts, without explaining the relationship between these two sets.

Questions for authors

Could the authors kindly clarify the following two details in their response? First, are there corresponding results available for the I2P explicit-content evaluation and MS COCO generation quality evaluation mentioned in Section 4.1? Second, Section 4.1 and Table 1 use the 20-class Imagenette-20 set, whereas Section 4.2 states that erased concept sets are sampled from Imagenette-50; could you please clarify the definition of Imagenette-50 and the relationship between these two concept sets?

1 comment

Response to Reviewer xBVy

We sincerely thank the reviewer for the positive assessment and careful reading. We address both questions below and will revise the manuscript accordingly.

Q1: Results for I2P and MS COCO

Thank you for pointing this out. These results were omitted from the submission, and we will include them in the Appendix.

I2P (explicit content).

Following prior work, we erase explicit content as the concept nudity, using the same backbones and unlearning methods as in Table 1. This yields 16 unlearned models: ESD, UCE, MACE, FMN, ConceptPrune, Sculpting, SPEED, ScaPre, FIA, AdvUnlearn, STEREO, and Meta-Unlearning on SD v1.5; UCE and EraseFlow on SD 3; and EraseAnything and ScaPre on FLUX.1. Searching the expanded vocabulary of 73,406 entries under the same exact-match criterion, TRACER returns exactly nudity on all 16 models. In every case, nudity ranks first by coverage, and the confidence drops sharply once it is selected.

MS COCO (generation quality).

MS COCO is used to determine the range of erased-set sizes over which each unlearning method erases reliably, and Table 1 evaluates TRACER only within this range. It does not evaluate TRACER itself, which does not generate images.

Even methods that claim to support massive erasure can erase only a limited number of concepts jointly in practice. Before evaluating identification, we therefore test each method across erased-set sizes, measuring both its generation quality on MS COCO (Table A) and whether the targets are effectively erased. A setting is excluded in two cases: (i) erasure becomes ineffective while generation quality is preserved, as for ESD at K≥10K \ge 10 and MACE at K=20K = 20; or (ii) generative capability is damaged, i.e., a CLIP score below 28 or an FID above 20, as for ConceptPrune and FIA at K=20K = 20. In either case, the weight changes no longer reflect selective erasure, so these settings are marked "–" in Table 1. Methods designed for single- or few-concept erasure are evaluated only within their intended setting. We will clarify this role of MS COCO in Section 4.1.

Table A. Generation quality of unlearned checkpoints on SD v1.5, evaluated on MS COCO and reported as FID (↓) / CLIP score (↑).

MethodVenueK=1K=1K=3K=3K=5K=5K=10K=10K=20K=20
ESDICCV 202314.12 / 31.1114.08 / 31.3414.23 / 30.1214.07 / 30.1414.12 / 31.17
MACECVPR 202412.89 / 30.1813.15 / 30.3113.02 / 30.2413.24 / 30.1913.15 / 30.21
UCEWACV 202413.94 / 31.2313.96 / 31.1514.56 / 30.0315.98 / 29.4517.43 / 28.62
ConceptPruneICLR 202414.32 / 30.9214.58 / 30.4116.27 / 29.8318.34 / 28.0222.19 / 24.32
ScaPreICLR 202613.53 / 31.3713.95 / 31.3014.49 / 31.0414.98 / 30.4315.32 / 30.24
FIAICML 202614.05 / 31.2514.24 / 31.1714.56 / 30.7314.92 / 29.9516.57 / 27.42

Q2: Clarification of the erased concept set

We apologize for the confusion. "Imagenette-50" in Section 4.2 is a typo for "Imagenette-20". All object-erasure experiments use the same Imagenette-20 set (Section 4.1 and Appendix A.1): the ten Imagenette classes plus ten non-overlapping ImageNet-1k classes. We will correct this in the revision.

We hope these clarifications address the reviewer's questions, and we are happy to provide further details if needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.