acceptodds
Under review as a conference paper at ICLR 2027

Your T2I model's embeddings know about memorization and enable broad unlearning

Abstract

Text-to-image (T2I) diffusion models are known to memorize training examples, reproducing near-verbatim copies at inference time and raising privacy and copyright concerns. This troubling phenomenon has given rise to a plethora of methods to mitigate memorization, either via prevention measures, specialized inference procedures, or post-hoc unlearning methods. In this work, we make a surprising fundamental observation about memorized examples in text-to-image diffusion models: a simple logistic regression classifier on top of embeddings can predict with very high accuracy which examples are memorized, suggesting the existence of shared representational structure among memorized examples, which is distinct from non-memorized examples. We also show evidence that the relationship between memorization and the structures we identify in representation space is causal. The existence of this representational structure, aside from being scientifically interesting in its own right, is practically important as it opens the door to a new problem setting we refer to as “broad unlearning” of all memorized examples from knowledge of only a subset of them; a realistic setting that prior work has overlooked. Motivated by on our findings, we propose a new method called BLUR that is significantly more effective than prior work at reducing the memorization of both known and unknown (to the unlearning algorithm) memorized examples, while preserving the model's ability to generate high quality and diverse images.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, memorization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RezrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper aims for generalizable unlearning to mitigate training image memorization in diffusion models. Generalizable unlearning requires unlearning a "forget set" results in unlearning a disjoint "held out" set from the same distribution. Concretely, the paper finds that memorized images and non-memorized ones are linearly separable according to their middle embeddings from the model. Then a fine-tuning-based unlearning method is designed to steer embeddings of memorized images towards non-memorized nearest neighbours. Experiment results demonstrate its effectiveness compared to several unlearning baselines.

Critical strengths

The motivation of studying generalizable unlearning for memorization is of large significance and technical challenges. This paper provides an empirically effective method to tackle this problem. The overall idea is meaningful and experimentally verified, which I would judge as the major strength that supports acceptance.

Critical weaknesses
  1. Soundness of the linear classifier. A lot of recent works on security research proposes to use linear separability of latent representations to interprete and mitigate various vulnerabilities in LLMs, diffusion models, etc. However, we should be careful when claiming and leveraging this property in high-dimensional space, especially with limited samples. Acoording to Cover's theorem or VC dimension, it is easy to build a linear classifier in high-dimensional space for a small number of points in general position. The paper does not detail the dimensionality or sample size used in studying this property, which would raise severe concerns on the soundness of the conclusion.
  2. Limited evaluation. As reported in experimental setups, hundreds of samples are used for evaluation. This scale of evaluation may not strongly demonstrate the effectiveness of the proposed method.
Questions for authors
  1. The authors should provide more solid evidence for the linear separability of memorization.
  2. The effectiveness of the proposed unlearning technique should be more comprehensively assessed. Hundreds of images is a trivial scale for a generalizable approach to stand on.
4 comments

We sincerely thank Reviewer Rezr for the careful and thoughtful review. We respond to each point below.

W1, Q1: Soundness of the linear classifier. We thank the reviewer for raising this concern. We agree that high-dimensional representations (1280 dimensions for SD v1.4/v2.0 and 1536 for SD3) can make it easier to linearly separate finite training samples, as suggested by Cover's theorem. However, this does not necessarily imply generalization to held-out unseen examples. Importantly, our reported results in Table 1 are cross-validated accuracies on held-out examples, rather than training accuracies.

Furthermore, our controlled experiment in Table A (and in Section 4.2) provides additional evidence. Before memorization induction, the classifier achieves only 51.48% accuracy, despite operating in the same high-dimensional representation space. After inducing memorization, accuracy increases substantially to 89.45%. This suggests that the observed separability cannot be explained by high dimensionality alone.

Table A: Accuracy of a linear classifier to separate examples in a given set I\mathcal{I} from non-memorized examples in R\mathcal{R}. We apply an intervention to induce memorization on I\mathcal{I}, and then apply BLUR on I\mathcal{I}.

Stage / MethodI\mathcal{I} accuracyI\mathcal{I} SSCD
Pre-mem-induction51.48%0.073
Mem-induced, before unlearning89.45%0.613
After unlearning (BLUR)63.78%0.205

For additional verification, we conduct a random-label control experiment, following the same protocol as in Section 4.1, but with shuffled memorization labels. As shown in Table D, the classifiers achieve near-perfect held-out accuracy with real labels (99.6-100%), whereas accuracy drops to approximately 50% with shuffled labels. Notably, for both settings, the classifier was trained to fit the data reasonably well (training acc close to 100%), yet their held-out accuracies differ dramatically. This further demonstrates that high dimensionality alone does not explain our findings.

Table D: Comparison of separability accuracy between memorized and non-memorized examples, using the memorized dataset with real labels vs shuffled labels.

SettingReal-label testShuffled test
SD1.4 verbatim99.89%49.20%
SD1.4 template100.00%50.03%
SD2 template99.60%50.19%

W2, Q2: Evaluation scale. We thank the reviewer for raising this point. Our original experiments follow the established evaluation protocol in prior studies using the Webster dataset [1-5]. We would like to note that: 1) Memorization, by definition, affects only a small fraction of the training data, which inherently limits the scale of memorization-specific evaluations; 2) Identifying memorized examples in pretrained diffusion models is itself a challenging and active research problem, which presents a practical limitation for memorization-specific evaluations.

Nevertheless, to further address the reviewer's concern, we substantially expand our evaluation to the MemBench (including 3000 memorized prompts identified from LAION), which already goes beyond the commonly adopted Webster benchmark. As shown in Table B-2, BLUR continues to outperform the competing methods, providing further evidence of its effectiveness and generalizability on a broader set of memorization examples.

Table B-2: Broad unlearning results for the MemBench-3k dataset (as M\mathcal{M}). Best and second-best results among mitigation methods are shown in bold and underlined, respectively.

MethodOverall SSCD ↓Overall CLIP ↑Overall CLIP-IQA ↑Overall HPSv3 ↑FID ↓SSCDF\mathrm{SSCD}_{\mathcal{F}} ↓SSCDH\mathrm{SSCD}_{\mathcal{H}} ↓Δgen\Delta_{\mathrm{gen}} ↓
No mitigation0.380 ± 0.0110.310 ± 0.0020.817 ± 0.0056.579 ± 0.10578.5870.197 ± 0.0060.563 ± 0.0110.366
BLUR (ours)0.064 ± 0.0020.288 ± 0.0030.804±0.006‾\underline{0.804 \pm 0.006}5.711 ± 0.13844.0570.051 ± 0.0030.076 ± 0.0040.025
Fine-tune0.202 ± 0.0080.309 ± 0.0020.807 ± 0.0066.308 ± 0.11852.3740.121 ± 0.0050.283 ± 0.0130.162
SISS0.155 ± 0.0050.289 ± 0.0020.742 ± 0.0084.621 ± 0.19659.7150.122 ± 0.0050.188 ± 0.0080.066
NegGrad+0.107±0.005‾\underline{0.107 \pm 0.005}0.294 ± 0.0030.783 ± 0.0075.792 ± 0.15245.665‾\underline{45.665}0.078±0.004‾\underline{0.078 \pm 0.004}0.136±0.008‾\underline{0.136 \pm 0.008}0.058‾\underline{0.058}
RTA0.397 ± 0.0090.284 ± 0.0020.782 ± 0.0055.651 ± 0.01393.1640.315 ± 0.0110.479 ± 0.0130.164
Wen et al.0.255 ± 0.0090.299 ± 0.0020.803 ± 0.0055.861±0.125‾\underline{5.861 \pm 0.125}59.2500.164 ± 0.0080.347 ± 0.0150.183

References:

[1] A reproducible extraction of training images from diffusion models. Webster. 2023.

[2] Understanding and mitigating copying in diffusion models. Somepalli et al. NeurIPS 2023.

[3] Detecting, explaining, and mitigating memorization in diffusion models. Wen et al. ICLR 2024.

[4] Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention. Ren et al. ECCV 2024.

[5] You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models. Zhao et al. ICML 2026.


In summary, we have tried our best to address the reviewer's concerns through additional experiments and clarifications, as outlined below. We hope these efforts will help the reviewer reconsider their assessment of our work, and we would be happy to address any remaining questions.

ConcernResponse
Cover’s theorem, soundness of linear separability (W1, Q1)Clarified that our reported accuracies are cross-validated on held-out examples, and provided controlled artificial memorization experiments and random-label controls to show that high dimensionality alone cannot explain the observed separability.
Evaluation scale (W2, Q2)Expanded our experiments to Webster-500 and MemBench-3k, where BLUR continues to outperform competing methods in memorization reduction.

Dear Reviewer Rezr,

Thank you again for your thoughtful feedback. We believe that we have addressed all of your concerns and we are eager to hear back from you given that the discussion phase is coming to a close soon.

As detailed in our earlier response, we have comprehensively addressed your feedback, including via additional controls to further validate our representational separability findings in high-dimensional space, as well as expanded our evaluation to MemBench-3k. These further strengthen our claims and we will add them to the paper.

Beyond these, the rebuttal also added several results that strengthen the paper:

  • Additional controls separating memorization from distribution effects (responses to Reviewer 7BYF and xfbr, Table A, E) showing that memorization contributes to representational separability beyond distributional differences.
  • A non-memorized control for BLUR’s generalization to held-out memorized examples (response to Reviewer 7BYF, Table C) showing that BLUR generalizes strongly to held-out memorized examples, and this cannot be explained by semantic similarity alone.
  • An evaluation on SD v3, extending our findings to a further architecture (response to Reviewer xfbr, Table D-1, D-2) which gives us further evidence for the generalizability of our findings.

Please let us know if you have any other concerns, we'd love to discuss them if so.

Authors

I thank the authors for their defailed response. I have read the comments from the other reviewers and would not raise any more concerns. I would consider to change my rating based on next-stage discussion with AC and other reviewers.

Reviewer_xfbrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper at hand aims to introduce a general unlearning method for text to image diffusion models, based on U-Nets, by analyzing the internal embedding space. The paper shows that using a linear classifier allows to differentiate between memorized and non-memorized samples in the embedding space. Their BLUR algorithm helps to unlearn the whole distribution around the given forget set. This is done by an empirical test on two unet models (SD1.4 and SD2).

Critical strengths

S1: The paper introduces a broad unlearning technique, allowing the unlearning of memorized training data from the model
S2: The result that the memorized samples are linearly separable in the embedding space is quite surprising to me and opens up new approaches for mitigating memorization in T2I models
S3: The paper is well written and easy to understand

Critical weaknesses

W1: While the memorized samples are only available for SD1.4 and SD2, this is still a very limited number of models which are also only based on the U-Net family. Especially for the derived claims of “memorization is linearly decodable from internal representations of T2I models” is a strong claim given that only 2 models were tested.
W2: The generalization assumption is very strong. There are only 500 memorized samples available for SD1.4 and even less for SD2. So it might be that the embeddings of only these samples are linearly separable, while other memorized samples might be not.

Questions for authors

Q1: Could it be that the samples are only linearly separable because the memorized samples are outliers? Did you test the linear separability using samples from the same domain? So, for example, when having memorized samples of podcast covers, did you test the separability using non-memorized podcast covers?
Q2: Could you please deliver the same evaluation on newer transformer based models?
Q3: Is there any reason to assume the stated claims hold for all T2I models and are not only specific to the two at hand? Q4: Could you also compare against [1] and [2] as another post-hoc method? Because their method also modifies the embedding space by removing neurons and by fine-tuning the model. Q5: Did you try to fine-tune other models to induce memorized samples? While this might be different from memorization during the training process itself, this might still be a good first indication whether the claim is generalizable.

[1] Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models, Hintersdorf et al., NeurIPS 2024 [2] Finding DoRI: Discovery of Retained Images in Diffusion Models, Kowalczuk et al., ICML 2026

6 comments

We sincerely thank Reviewer xfbr for the thoughtful and constructive review. We respond to each point below.

W1, Q2, Q3, Q5: evaluating different model architectures. We thank the reviewer for highlighting the importance of this point. To address this concern, we extend our experiments to SDv3, which uses a DiT-based backbone, rather than the U-Net of SD v1.4/v2.0.

On Q5 (inducing memorization by fine-tuning), this is an informative setting, and it is the one in our Sec. 4.2, so we clarify how it relates to the rest of the paper. We use two kinds of memorization:

  • Natural memorization arises during pretraining. Webster’s (and the added MemBench, in our response to W2 below) examples are of this kind, already memorized by the released models (Sec. 4.1, Table 1, and Table D-1 below for SDv3).
  • Artificial memorization is induced by fine-tuning. We take examples the model has not memorized and fine-tune it on duplicated copies of them to induce memorization. Throughout, “inducing memorization” means this fine-tuning phase. We already study this in Section 4.2, now extending to SDv3 (Table D-2 below).

As the reviewer notes, memorization induced by fine-tuning (i.e. artificial memorization) may differ from memorization that emerges in pretraining (i.e. natural memorization), which is why we study both in Section 4. We now extend both settings to SDv3, as shown in Tables D-1 and D-2.

We observe the same relationship between memorization and representation separability in a non-U-Net architecture, and the slight accuracy drop for SDv3 is consistent with its weaker memorization (lower SSCD). This suggests that our previous finding was not architecture-dependent and is in fact generalizable. This is a very important point that strengthens our contribution and we thank the reviewer for bringing it up.

Table D-1: Linear classification accuracy for distinguishing memorized from non-memorized examples, for memorized examples from Webster’s dataset [1], extending Table 1 of Section 4.1 with SDv3. The mean SSCD of the memorized examples is in parentheses, as a measure of memorization strength.

Memorization typeSD v1.4SD v2.0SD v3.0
Verbatim99.89% (0.581)96.60% (0.185)94.27% (0.183)
Template100.00% (0.671)99.60% (0.545)98.11% (0.208)

Table D-2: Accuracy of a linear classifier to separate examples in a given set I\mathcal{I} from non-memorized examples in R\mathcal{R}. We apply an intervention to induce memorization on I\mathcal{I} for SDv3.

Stage / MethodI\mathcal{I} accuracyI\mathcal{I} SSCD
Pre-mem-induction52.68%0.075
Mem-induced, before unlearning83.58%0.425

On Q3, we emphasize that we observe evidence of memorization-related representation separability in every model evaluated so far, including both U-Net-based models (SD v1.4/v2.0) and the transformer-based SDv3. We thank the reviewer for motivating this extension to a transformer-based architecture, which substantially strengthens the architectural generality of our findings.

W2: Sample-specific results. The reviewer’s concern is that the findings might hold only for the few identified memorized examples. Our artificial memorization experiment (Sec. 4.2, also explained in Q1 below) addresses it directly. By inducing memorization on randomly sampled non-memorized examples, we observe that representational separability increases substantially, and subsequently decreases following unlearning.

We also agree that evaluating a larger and more diverse set of memorized examples is important, so we extend our evaluation to (i) the full Webster’s dataset [1] (following the standard procedure of prior work), and (ii) the MemBench-3k dataset [2], which substantially expands the original evaluation scale. As shown in Tables B-1 and B-2, BLUR continues to achieve the strongest memorization reduction and generalizable unlearning effect among the evaluated methods.

Moreover, we would like to note that memorization, by definition, affects only a small fraction of the training data, which inherently limits the scale of memorization-specific evaluations.

Together, these results strengthen the evidence for generalizable memorization-related structure.

[1] A reproducible extraction of training images from diffusion models. Webster. 2023.

[2] Membench: Memorized image trigger prompt dataset for diffusion models. Hong et al. 2024.

Table B-1: Broad unlearning results for the Webster’s dataset (as M\mathcal{M}) with full 500 memorized examples. Best and second-best results among mitigation methods are shown in bold and underlined, respectively.

MethodOverall SSCD ↓Overall CLIP ↑Overall CLIP-IQA ↑Overall HPSv3 ↑FID ↓SSCDF\mathrm{SSCD}_{\mathcal{F}} ↓SSCDH\mathrm{SSCD}_{\mathcal{H}} ↓Δgen\Delta_{\mathrm{gen}} ↓
No mitigation0.523 ± 0.0280.321 ± 0.0040.819 ± 0.0097.363 ± 0.161128.1100.249 ± 0.0250.796 ± 0.0110.547
BLUR (ours)0.209 ± 0.0170.284 ± 0.0050.802±0.010‾\underline{0.802 \pm 0.010}6.260±0.208‾\underline{6.260 \pm 0.208}75.3150.093 ± 0.0110.325 ± 0.0260.232
Fine-tune0.406 ± 0.0240.316 ± 0.0040.807 ± 0.0107.123 ± 0.171106.9460.195 ± 0.0210.618 ± 0.0240.423
SISS0.277±0.017‾\underline{0.277 \pm 0.017}0.289 ± 0.0040.732 ± 0.0115.756 ± 0.185111.5160.149±0.016‾\underline{0.149 \pm 0.016}0.406 ± 0.0200.257
NegGrad+0.335 ± 0.0200.306 ± 0.0040.691 ± 0.0156.254 ± 0.236120.0160.158 ± 0.0170.511 ± 0.0190.353
RTA0.387 ± 0.0220.308 ± 0.0040.777 ± 0.0116.188 ± 0.20398.8920.209 ± 0.0220.566 ± 0.0230.357
Wen et al.0.282 ± 0.0180.298 ± 0.0040.714 ± 0.0145.202 ± 0.29090.722‾\underline{90.722}0.162 ± 0.0160.402±0.026‾\underline{0.402 \pm 0.026}0.240‾\underline{0.240}

Table B-2: Broad unlearning results for the MemBench-3k dataset (as M\mathcal{M}). Best and second-best results among mitigation methods are shown in bold and underlined, respectively.

MethodOverall SSCD ↓Overall CLIP ↑Overall CLIP-IQA ↑Overall HPSv3 ↑FID ↓SSCDF\mathrm{SSCD}_{\mathcal{F}} ↓SSCDH\mathrm{SSCD}_{\mathcal{H}} ↓Δgen\Delta_{\mathrm{gen}} ↓
No mitigation0.380 ± 0.0110.310 ± 0.0020.817 ± 0.0056.579 ± 0.10578.5870.197 ± 0.0060.563 ± 0.0110.366
BLUR (ours)0.064 ± 0.0020.288 ± 0.0030.804±0.006‾\underline{0.804 \pm 0.006}5.711 ± 0.13844.0570.051 ± 0.0030.076 ± 0.0040.025
Fine-tune0.202 ± 0.0080.309 ± 0.0020.807 ± 0.0066.308 ± 0.11852.3740.121 ± 0.0050.283 ± 0.0130.162
SISS0.155 ± 0.0050.289 ± 0.0020.742 ± 0.0084.621 ± 0.19659.7150.122 ± 0.0050.188 ± 0.0080.066
NegGrad+0.107 ± 0.0050.294 ± 0.0030.783 ± 0.0075.792 ± 0.15245.665‾\underline{45.665}0.078 ± 0.0040.136 ± 0.0080.058
RTA0.397 ± 0.0090.284 ± 0.0020.782 ± 0.0055.651 ± 0.01393.1640.315 ± 0.0110.479 ± 0.0130.164
Wen et al.0.255 ± 0.0090.299 ± 0.0020.803 ± 0.0055.861±0.125‾\underline{5.861 \pm 0.125}59.2500.164 ± 0.0080.347 ± 0.0150.183
NeMo0.103 ± 0.0030.309 ± 0.0020.730 ± 0.0084.620 ± 0.17745.9700.082 ± 0.0040.124 ± 0.0050.042
DoRI0.080±0.002‾\underline{0.080 \pm 0.002}0.286 ± 0.0020.762 ± 0.0074.817 ± 0.17248.5240.061±0.003‾\underline{0.061 \pm 0.003}0.099±0.003‾\underline{0.099 \pm 0.003}0.038‾\underline{0.038}

Q1: Distributional confounding. We thank the reviewer for this suggestion. We agree that memorized examples may be outliers, and this domain difference could contribute to how representations are organized. This is why we introduced the controlled artificial memorization experiment in Section 4.2.

Specifically, we randomly sample initially non-memorized examples I\mathcal{I} from the same distribution of the retain set R\mathcal{R} and induce memorization on I\mathcal{I}. As shown in Table A, without changing the initial sampling distribution, separability increases from approximately 51.48% to 89.45% after memorization induction, and decreases to 63.78% after applying our unlearning method BLUR. These results provide complementary causal evidence that the observed separability is associated with induced memorization, rather than being solely attributable to pre-existing distributional differences.

Table A: Accuracy of a linear classifier to separate examples in a given set I\mathcal{I} from non-memorized examples in R\mathcal{R}. We apply an intervention to induce memorization on I\mathcal{I}, and then apply BLUR on I\mathcal{I}.

Stage / MethodI\mathcal{I} accuracyI\mathcal{I} SSCD
Pre-mem-induction51.48%0.073
Mem-induced, before unlearning89.45%0.613
After unlearning (BLUR)63.78%0.205

Q4: Additional baselines. We thank the reviewer for suggesting these relevant methods. We have additionally evaluated both Finding NeMo and Finding DoRI.

We clarify that NeMo primarily performs neuron deactivation during inference time rather than post-hoc fine-tuning. To adapt it to our broad-unlearning setting, we identify memorization-related neurons using only the forget set F\mathcal F, construct a shared mask using the most frequently identified neurons, and apply it to both F\mathcal F and the held-out set H\mathcal H.

We evaluate these methods under our broad unlearning protocol on the larger MemBench benchmark. As shown in Table B-2, although NeMo and DoRI achieve competitive results, both are outperformed by BLUR and exhibit larger generalization gaps between SSCD_F and SSCD_H than BLUR. Interestingly, as the reviewer pointed out, all three methods intervene in the model's internal representations (albeit through different mechanisms). These findings further support the importance of internal representations in unlearning memorized data, particularly in the context of broad unlearning.


In summary, the table below lists the concerns raised and how we addressed each of them. We hope that our responses and additional experimental results have addressed the reviewer's concerns, and that the reviewer will consider revisiting their assessment of our work. We would be happy to clarify any remaining questions.

ConcernResponse
Only U-Nets (W1, Q2, Q3)Added SD v3 (MMDiT). Same pattern in every model tested.
Fine-tuned memorization (Q5)Clarified natural memorization (arising during pretraining) vs. artificial memorization (induced through fine-tuning). Extended the results to SD v3.
Sample-specific results (W2)Controlled experiments on randomly sampled, initially non-memorized examples. Expanded our evaluation to Webster-500 and MemBench-3k.
Distributional confounding (Q1)Clarified how our artificial memorization experiments control for differences in the initial sampling distribution.
Additional baselines (Q4)Evaluated Finding NeMo and Finding DoRI, with BLUR achieving stronger memorization reduction and smaller generalization gaps.
Reviewer_7BYFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how memorization is encoded in the internal representations of text-to-image (T2I) diffusion models and proposes a representation-based approach for generalizable machine unlearning.

The authors first demonstrate that memorized and non-memorized training examples are highly linearly separable in the U-Net's hidden representation space, with a simple logistic regression classifier achieving over 96% accuracy across Stable Diffusion v1.4 and v2.0. Authors further investigate this relationship through controlled memorization induction and representation steering experiments, providing evidence that changes in memorization are associated with changes in representational separability.

Motivated by these findings, the paper introduces broad unlearning, a setting in which the objective is to reduce memorization across all memorized examples despite having access to only a subset of them. To address this problem, the authors propose Broad Local Unlearning of Representations (BLUR), which fine-tunes the diffusion model by aligning the hidden representations of identified memorized examples with those of their nearest non-memorized neighbors, while using a standard denoising objective to preserve generation utility.

Experiments on Stable Diffusion v1.4 and v2.0, covering both verbatim and template memorization, show that BLUR achieves substantially greater memorization reduction than existing baselines, including on memorized examples excluded from the unlearning procedure. The method also demonstrates a favorable trade-off between memorization reduction and generation quality, as measured by SSCD, CLIP score, and FID.

Critical strengths
  • A realistic and well-formulated problem setting. The paper introduces “broad unlearning” of memorization, removing memorization from all memorized examples when only a subset of them is known. This matches practice, because the full set of memorized training examples is never known in deployment. The evaluation protocol fits the setting well. The F/H split, which reports SSCD_F, SSCD_H and Δgen\Delta_{gen} separately, directly measures whether unlearning generalizes to unseen memorized examples, something prior T2I memorization work does not test.
  • Large empirical gains on held-out memorized examples from a simple method. BLUR is simple, a kNN-based feature alignment loss plus a retain-set denoising loss. Yet BLUR generalizes better than existing unlearning methods. Under the same CLIP constraint, BLUR cuts held-out SSCD from 0.866 to 0.267 on SD v1.4 verbatim memorization and from 0.722 to 0.123 on SD v2.0 template memorization. The gains hold across both memorization types and both models.
Critical weaknesses
  • The probing result may reflect domain shift rather than memorization. Memorized prompts (Webster) are mostly celebrity headlines and product listings, while non-memorized examples come from LAION-Aesthetics 6.5+, a very different distribution. The features also seem to come from t=999 (Fig. 2), where they mostly encode the text prompt. Near-perfect probe accuracy is therefore expected even without any memorization signal. Controls are needed: non-memorized examples from the same source as M, and a probe on CLIP text embeddings alone. Without them, the title claim and the motivation for BLUR are not established.

  • Hyperparameters appear to be selected on the held-out set, and utility evidence is weak. Configurations are chosen by the lowest overall SSCD on M = F ∪\cup H, so H is not truly held out. Results should be re-selected using F only. In addition:

    • FID is computed on only 145–258 images.
    • BLUR sits at the CLIP floor (its CLIP on F is below it).
    • Fig. 3 shows off-prompt generations (e.g., a person for the pillow prompt).
Questions for authors
  • What is the probe accuracy when the non-memorized examples come from the same source as M, e.g., Webster’s non-memorized candidate prompts, or LAION-2B captions matched in style? What does a probe on CLIP text embeddings alone achieve?
  • At which timestep, and from which noised image, are the features h(x, c) extracted, for both the probe and BLUR?
  • Were the hyperparameters for BLUR and the baselines selected using SSCD on H? If so, what are the results when selecting on F only?
  • If BLUR is run with an equal-sized set of non-memorized examples as F, does SSCD on H still drop? This would test whether the generalization really comes from shared memorization structure.
9 comments

We sincerely thank Reviewer 7BYF for the careful reading of our paper and the thoughtful, constructive feedback. We respond to each point below.

W1,Q1: distributional differences vs. memorization. We thank the reviewer for raising this great point. We agree that M\mathcal{M} and R\mathcal{R} can differ in distribution, which may contribute to the observed separability. And this is exactly the concern that our existing experiments in Sec. 4.2 were designed to address where we randomly sample examples that weren’t initially memorized and cause them to become memorized through finetuning, in order to examine what effect memorization has on representations, without the confounding factor of distributional similarity.

Specifically, we randomly sample initially non-memorized examples (set I\mathcal{I}) from the same distribution as the comparison set (remaining non-memorized examples in R\mathcal{R}). Before memorization induction, their representations exhibit weak separability from the comparison set (51.48% accuracy). We then induce memorization through duplicated fine-tuning to increase memorization on set I\mathcal{I} (mean SSCD: 0.073 → 0.613), accompanied by separability rising to 89.45%. After applying BLUR, both separability and memorization decrease again, to 63.78% and 0.205, respectively (Table A). These results provide causal evidence that inducing memorization increases representational separability, which cannot be explained solely by pre-existing distributional differences between the two sets.

Table A: Accuracy of a linear classifier to separate examples in a given set I\mathcal{I} from non-memorized examples in R\mathcal{R}. We apply an intervention to induce memorization on I\mathcal{I}, and then apply BLUR on I\mathcal{I}.

Stage / MethodI\mathcal{I} accuracyI\mathcal{I} SSCD
Pre-mem-induction51.48%0.073
Mem-induced, before unlearning89.45%0.613
After unlearning (BLUR)63.78%0.205

Regarding the suggested distribution-matched comparison in Q1, we clarify that Webster's dataset contains only identified memorized examples, so non-memorized examples cannot be sampled directly from this benchmark. Our existing artificial memorization experiment as described above (also in Section 4.2) instead provides a controlled alternative that accounts for differences in the initial sampling distribution.

On the reviewer’s suggested CLIP-text-embedding control: this is a very interesting experiment that we hadn’t thought about. To examine this, we trained the same probe on CLIP text embeddings alone, see Table E.

Table E: Linear classification accuracy using CLIP text embeddings versus U-Net representations, for (i) Webster’s memorized dataset, and (ii) induced memorization setting.

SettingCLIP text embeddingU-Net embedding
Webster, verbatim memorization)97.08%99.89%
Webster, template memorization)99.56%100.00%
Induced mem. (before induction)50.18%51.48%
Induced mem. (after induction)50.18%89.45%
Induced mem. (after unlearning)50.18%63.78%

For the memorized examples from Webster’s dataset (Sec. 4.1), the CLIP probe reaches 97.08% for verbatim memorization and 99.56% for template memorization, so we agree that the separability in Sec. 4.1 can largely be explained by differences between the caption distributions of M\mathcal{M} and R\mathcal{R} – this is not surprising and in line with prior work that showed that, in text-to-image models, the memorization status of an example is largely driven by its caption [1,2,8]. For the controlled setting (inducing artificial memorization of randomly-selected initially non-memorized examples, via fine-tuning) in Sec.4.2, however, the text-only probe has accuracy of only 50.18% before memorization induction. The captions are fixed and identical throughout, while the U-Net probe goes from 51.48% to 89.45% after memorization induction and back to 63.78% after unlearning. So, while the prompt of an example may influence its initial memorization status (a finding well-known in the literature too), and its initial memorization status indeed affects its “placement” in representation space (and its separability from non-memorized examples), this is a separate finding from our novel observation that there is a causal relationship between an example’s degree of memorization and its linear separability from the set of non-memorized examples.

On the reviewer’s comment on “The features also seem to come from t=999 (Fig. 2), where they mostly encode the text prompt”, we will explain it in detail in Q2.

Q2: How features are extracted. Both BLUR and the probe use timestep t=999 and spatially mean-pooled U-Net mid-block features, conditioning on the corresponding caption c. They slightly differ in the latent input.

  • Probe: it uses an image-independent, fixed noise latent, so that features depend only on the caption across examples. This design is in line with prior findings that text conditioning (rather than their associated images) plays a central role in triggering memorization in T2I diffusion models [1,2,8].
  • BLUR: features are computed on the example’s VAE-encoded image noised to t=999, in order to match the standard training input, so the alignment loss is computed in the same regime as the denoising objective. At t=999, the image’s signal coefficient is small, so these features are also dominated by the caption.

W2, Q3: Hyperparameters selection. We thank the reviewer and agree. We have re-selected hyperparameters for BLUR and all baselines using F\mathcal{F} only (lowest SSCD_F subject to CLIP_F ≥ 90%*ref.CLIP), with H\mathcal{H} used purely for reporting. Our previous findings still hold in the revised Table 2, which shows the outstanding performance of BLUR.

Table 2 (revised): Broad unlearning results for different memorization types and architectures. For each method, we report the configuration with the lowest SSCD on F\mathcal{F} subject to CLIP ≥ 0.276 on F\mathcal{F}. Overall metrics are evaluated on F∪H\mathcal{F} \cup \mathcal{H}, while SSCD_F and SSCD_H are evaluated separately on the set of memorized examples used by the unlearning algorithm as the forget set (F\mathcal{F}) and the heldout set of memorized examples (H\mathcal{H}). Best and second-best results among mitigation methods are shown in bold and underlined, respectively. BLUR achieves by far the largest reduction in SSCD_F while also generalizing far better than previous methods.

(a) SD v1.4 – Verbatim memorization

MethodOverall SSCD ↓Overall CLIP ↑FID ↓SSCDF\mathrm{SSCD}_{\mathcal{F}} ↓SSCDH\mathrm{SSCD}_{\mathcal{H}} ↓Δgen\Delta_{\mathrm{gen}} ↓
No mitigation0.581 ± 0.0520.338 ± 0.008163.9460.292 ± 0.0420.866 ± 0.0220.574
BLUR (ours)0.185 ± 0.0340.296 ± 0.011118.7140.083 ± 0.0190.286 ± 0.0550.203
Fine-tune0.429 ± 0.0460.330 ± 0.009143.5020.217 ± 0.0310.638 ± 0.0530.421
SISS0.325 ± 0.0360.318 ± 0.003151.3760.178 ± 0.0250.469 ± 0.0480.291
NegGrad+0.349 ± 0.0410.320 ± 0.009134.7320.186 ± 0.0270.509 ± 0.0580.323
RTA0.423 ± 0.0470.324 ± 0.009142.9570.210 ± 0.0340.634 ± 0.0550.424
Wen et al.0.299±0.041‾\underline{0.299 \pm 0.041}0.315 ± 0.008128.814‾\underline{128.814}0.166±0.026‾\underline{0.166 \pm 0.026}0.429±0.065‾\underline{0.429 \pm 0.065}0.263‾\underline{0.263}

(b) SD v1.4 – Template memorization

MethodOverall SSCD ↓Overall CLIP ↑FID ↓SSCDF\mathrm{SSCD}_{\mathcal{F}} ↓SSCDH\mathrm{SSCD}_{\mathcal{H}} ↓Δgen\Delta_{\mathrm{gen}} ↓
No mitigation0.671 ± 0.0210.317 ± 0.004223.7660.529 ± 0.0240.813 ± 0.0090.284
BLUR (ours)0.267 ± 0.0200.292 ± 0.005161.4450.214 ± 0.0220.320 ± 0.0260.106
Fine-tune0.513 ± 0.0240.309 ± 0.005194.9600.396 ± 0.0270.630 ± 0.0280.234
SISS0.312±0.012‾\underline{0.312 \pm 0.012}0.278 ± 0.005198.6660.256±0.014‾\underline{0.256 \pm 0.014}0.368±0.017‾\underline{0.368 \pm 0.017}0.112‾\underline{0.112}
NegGrad+0.521 ± 0.0240.307 ± 0.005199.6300.397 ± 0.0270.646 ± 0.0260.249
RTA0.543 ± 0.0240.287 ± 0.006205.7450.440 ± 0.0320.645 ± 0.0240.205
Wen et al.0.356 ± 0.0220.286 ± 0.005183.856‾\underline{183.856}0.303 ± 0.0260.409 ± 0.0350.106

(c) SD v2.0 – Template memorization

MethodOverall SSCD ↓Overall CLIP ↑FID ↓SSCDF\mathrm{SSCD}_{\mathcal{F}} ↓SSCDH\mathrm{SSCD}_{\mathcal{H}} ↓Δgen\Delta_{\mathrm{gen}} ↓
No mitigation0.545 ± 0.0290.307 ± 0.005218.9500.366 ± 0.0360.722 ± 0.0110.356
BLUR (ours)0.133 ± 0.0080.283 ± 0.007158.9660.125 ± 0.0110.140 ± 0.0100.015
Fine-tune0.499 ± 0.0280.311 ± 0.005210.7740.328 ± 0.0350.668 ± 0.0160.340
SISS0.296 ± 0.0140.290 ± 0.005207.2950.213 ± 0.0150.379 ± 0.0110.166
NegGrad+0.266 ± 0.0170.285 ± 0.006180.318‾\underline{180.318}0.166±0.018‾\underline{0.166 \pm 0.018}0.366 ± 0.0150.200
RTA0.381 ± 0.0220.286 ± 0.005199.4330.314 ± 0.0330.447 ± 0.0230.133
Wen et al.0.234±0.017‾\underline{0.234 \pm 0.017}0.275 ± 0.006300.8310.224 ± 0.0260.244±0.021‾\underline{0.244 \pm 0.021}0.020‾\underline{0.020}

W3: FID scale. We thank the reviewer for raising this concern and would like to clarify that FID is computed between 580-1032 generations (4 generated images per memorized prompt) and a reference set of 10k non-memorized images, following the standard practice in prior work [1-4]. We acknowledge that FID has a finite-sample bias, which scales approximately as 1/N [5]. However, within each setting, all methods are evaluated using identical prompts and sample sizes, so this bias already cancels to some extent in comparisons. Moreover, we note that memorization, by definition, affects only a small fraction of the training data, which inherently limits the scale of memorization-specific evaluations.

To further address the concern though, we add results on (i) the full Webster’s dataset [6] including 500 memorized examples identified from LAION, without our filtering procedure, as the standard procedure in prior work, and (ii) MemBench [7], which has 3000 memorized prompts identified from LAION. Table B shows that BLUR continues to outperform the competing methods. We further added two metrics, CLIP-IQA and HPSv3, for generation quality, which we will explain in our response to W4 below.

Table B-1: Broad unlearning results for the Webster’s dataset (as M\mathcal{M}) with full 500 memorized examples. Best and second-best results among mitigation methods are shown in bold and underlined, respectively.

MethodOverall SSCD ↓Overall CLIP ↑Overall CLIP-IQA ↑Overall HPSv3 ↑FID ↓SSCDF\mathrm{SSCD}_{\mathcal{F}} ↓SSCDH\mathrm{SSCD}_{\mathcal{H}} ↓Δgen\Delta_{\mathrm{gen}} ↓
No mitigation0.523 ± 0.0280.321 ± 0.0040.819 ± 0.0097.363 ± 0.161128.1100.249 ± 0.0250.796 ± 0.0110.547
BLUR (ours)0.209 ± 0.0170.284 ± 0.0050.802±0.010‾\underline{0.802 \pm 0.010}6.260±0.208‾\underline{6.260 \pm 0.208}75.3150.093 ± 0.0110.325 ± 0.0260.232
Fine-tune0.406 ± 0.0240.316 ± 0.0040.807 ± 0.0107.123 ± 0.171106.9460.195 ± 0.0210.618 ± 0.0240.423
SISS0.277±0.017‾\underline{0.277 \pm 0.017}0.289 ± 0.0040.732 ± 0.0115.756 ± 0.185111.5160.149±0.016‾\underline{0.149 \pm 0.016}0.406 ± 0.0200.257
NegGrad+0.335 ± 0.0200.306 ± 0.0040.691 ± 0.0156.254 ± 0.236120.0160.158 ± 0.0170.511 ± 0.0190.353
RTA0.387 ± 0.0220.308 ± 0.0040.777 ± 0.0116.188 ± 0.20398.8920.209 ± 0.0220.566 ± 0.0230.357
Wen et al.0.282 ± 0.0180.298 ± 0.0040.714 ± 0.0145.202 ± 0.29090.722‾\underline{90.722}0.162 ± 0.0160.402±0.026‾\underline{0.402 \pm 0.026}0.240‾\underline{0.240}

Table B-2: Broad unlearning results for the MemBench-3k dataset (as M\mathcal{M}). Best and second-best results among mitigation methods are shown in bold and underlined, respectively.

MethodOverall SSCD ↓Overall CLIP ↑Overall CLIP-IQA ↑Overall HPSv3 ↑FID ↓SSCDF\mathrm{SSCD}_{\mathcal{F}} ↓SSCDH\mathrm{SSCD}_{\mathcal{H}} ↓Δgen\Delta_{\mathrm{gen}} ↓
No mitigation0.380 ± 0.0110.310 ± 0.0020.817 ± 0.0056.579 ± 0.10578.5870.197 ± 0.0060.563 ± 0.0110.366
BLUR (ours)0.064 ± 0.0020.288 ± 0.0030.804±0.006‾\underline{0.804 \pm 0.006}5.711 ± 0.13844.0570.051 ± 0.0030.076 ± 0.0040.025
Fine-tune0.202 ± 0.0080.309 ± 0.0020.807 ± 0.0066.308 ± 0.11852.3740.121 ± 0.0050.283 ± 0.0130.162
SISS0.155 ± 0.0050.289 ± 0.0020.742 ± 0.0084.621 ± 0.19659.7150.122 ± 0.0050.188 ± 0.0080.066
NegGrad+0.107±0.005‾\underline{0.107 \pm 0.005}0.294 ± 0.0030.783 ± 0.0075.792 ± 0.15245.665‾\underline{45.665}0.078±0.004‾\underline{0.078 \pm 0.004}0.136±0.008‾\underline{0.136 \pm 0.008}0.058‾\underline{0.058}
RTA0.397 ± 0.0090.284 ± 0.0020.782 ± 0.0055.651 ± 0.01393.1640.315 ± 0.0110.479 ± 0.0130.164
Wen et al.0.255 ± 0.0090.299 ± 0.0020.803 ± 0.0055.861±0.125‾\underline{5.861 \pm 0.125}59.2500.164 ± 0.0080.347 ± 0.0150.183

[1] Understanding and mitigating copying in diffusion models. Somepalli et al. NeurIPS 2023.

[2] Detecting, explaining, and mitigating memorization in diffusion models. ICLR 2024. Wen et al.

[3] Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention. Ren et al. ECCV 2024.

[4] You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models. Zhao et al. ICML 2026.

[5] Effectively unbiased FID and inception score and where to find them. Chong & Forsyth CVPR 2020.

[6] A reproducible extraction of training images from diffusion models. Webster. 2023.

[7] Membench: Memorized image trigger prompt dataset for diffusion models. Hong et al.

[8] Diffusion art or digital forgery? Investigating data replication in diffusion models. Somepalli et al. CVPR 2023.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Contributions and Rebuttal Improvements

Dear Reviewers and Area Chairs,

We sincerely thank the reviewers for their thoughtful feedback and constructive suggestions. We would like to briefly highlight our paper's key contributions and the substantial improvements made during the rebuttal.

Our main contributions:

  • New insights into memorization: Memorization is linearly decodable from T2I embeddings, which is a surprising finding and opens up new approaches to understanding and mitigating memorization [Reviewer xfbr].

  • A realistic and well-motivated problem setting: Broad unlearning of memorization, i.e., removing memorization from all memorized examples when only a subset is known, which prior work does not test. Reviewers highlighted its practical relevance and its evaluation protocol [Reviewers 7BYF, Rezr].

  • A new method for broad unlearning: BLUR, which is simple yet effective [Reviewers 7BYF, xfbr, Rezr], and achieves large gains on held-out memorized examples across both memorization types and models [Reviewers 7BYF, Rezr].

Key improvements during the rebuttal:

  • Isolating the role of memorization: we add new controls that (i) separate memorization-related representational changes from distributional differences, (ii) show that high dimensionality alone does not explain the separability, and (iii) show that BLUR’s generalization is not explained by semantic similarity alone.

  • Broader evaluation: we (i) expand our experiments to Webster-500 and MemBench-3k; (ii) add comparisons with NeMo and DoRI baselines; (iii) extend our findings to SD3’s transformer-based architecture. Our conclusions hold in all of these settings and BLUR continues to outperform competing methods; this is strong evidence in support of the generalizability of our claims and findings.

  • More rigorous protocol: we (i) report results under an alternative hyperparameter selection protocol; (ii) provide additional generation-quality metrics and qualitative examples. BLUR’s advantage still holds under these changes, which again strengthens our claims significantly.

We believe these additions address the reviewers' concerns and substantially reinforce the significance and generalizability of our contributions. We will incorporate them into the revised paper.

Thank you again for your consideration.

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.