- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper introduces a detector for memorization in text-to-image diffusion models that measures the sensitivity of predicted scores to text conditioning. Its primary detection statistic is the Frobenius norm of the condition Jacobian, estimated using Hutchinson’s method, while information from the input-noise Jacobian provides a complementary signal. The approach stems from an analysis of score behavior as the conditioning embedding moves along a straight path from the unconditional embedding to the prompt embedding. By expressing the conditional–unconditional score difference as a line integral, the authors reveal a distinctive pattern associated with memorized prompts: score sensitivity rises sharply as the interpolation approaches the target embedding, coinciding with a sudden shift in generation toward the memorized training image. Evaluations across Stable Diffusion v1.4, Stable Diffusion v2.0, and other diffusion models show that these signals support strong memorization detection, with performance remaining robust when latent resolution is reduced.
Critical strengths
The paper offers an intuitive view of memorization through sharp increases in conditional score sensitivity near memorized text embeddings. The detector substantially improves AUC and TPR at low false-positive rates, with robust results across architectures, noise samples, and latent resolutions.
Critical weaknesses
-
Potential dataset confounding. Memorized and non-memorized prompts come from different sources, potentially introducing linguistic and semantic differences. Without matched negatives, the near-perfect AUC may reflect prompt-distribution differences rather than memorization.
-
Limited methodological novelty and justification. The line-integral formulation, Jacobian norms, and Hutchinson estimation are standard techniques. The paper insufficiently explains why elevated conditioning sensitivity specifically indicates memorization.
-
Unclear evaluation protocol. The separation between fitting and evaluating the logistic-regression projection is insufficiently specified. Although held-out results are provided, random splits within the same prompt collections do not address source-related confounding.
Questions for authors
Overall, despite the impressive reported numerical results, I find the evidence insufficient to establish that the proposed method reliably detects memorization rather than dataset-specific characteristics. The limited methodological novelty and unresolved evaluation concerns substantially weaken the paper's central contribution. I therefore recommend rejection.
8 comments
W2. Empirical justification for elevated conditioning sensitivity as a memorization signal
We provide substantial empirical justification for conditioning sensitivity as a memorization signal by showing that almost always increases sharply around transitions toward memorized images, while varying only the interpolation coefficient for the same prompt and initial noise. This contrasts with the relatively smooth changes in score sensitivity and generated images observed for non-memorized prompts along the interpolation (Figures 1 and 2).
We focus on global memorization, where the entire image is reproduced, and track image similarity using SSCD. We evaluate 544 pairs of prompt and initial noise from 159 globally memorized prompts with accessible training images, keeping the pairs among four initial noises per prompt that reproduce a training image with SSCD ≥ 0.5. For each fixed prompt and initial noise, we locate the transition at the interpolation step with the largest SSCD increase (Figure 2). Across these transitions, mean SSCD rises from 0.159 to 0.738 (+0.580), crossing the 0.5 threshold used to analyze copying in Stable Diffusion by Somepalli et al. [2]. We test whether conditioning sensitivity rises specifically at transitions toward training-image reproduction, identified using SSCD independently of the Jacobian measurements. Using interpolation increments of 0.1, we compare the increase in at and around these transitions with the preceding variation.
The table reports the percentage of these 544 pairs whose increase in conditioning sensitivity is at least 2× or more than 5× the prior variation.
| Sensitivity increase | ≥ 2× prior variation | > 5× prior variation |
|---|---|---|
| At the transition | 83.8% | 76.3% |
| Around the transition | 94.9% | 91.4% |
Prior variation is the mean absolute change over intervals preceding the evaluated interval or region. “Around the transition” averages the changes over the transition and its two adjacent intervals.
The prompt, initial noise, and generation settings remain fixed, and even the text embeddings are nearly identical across the transition. Despite these closely controlled conditions, rises sharply at or around the transition toward reproducing the training image. This links the sensitivity increase to the emergence of memorization within the same prompt, supporting its use as a memorization signal. The pooled Spearman correlation between and original-image SSCD is 0.733, further supporting this association. We have added the detailed experimental settings, selection criteria, and additional results to the revised appendix.
Our contribution lies in a new analysis of memorization in text embedding space and a detection method derived from this analysis that outperforms existing methods, rather than in developing new mathematical techniques. The line-integral formulation, Jacobian norms, and Hutchinson estimation serve as tools for these contributions. The standard nature of these tools does not by itself establish that the resulting insight or detection method lacks novelty.
[1] Ryan Webster. A reproducible extraction of training images from diffusion models. arXiv:2305.08694, 2023.
[2] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? Investigating data replication in diffusion models. CVPR, 2023.
Thank you for the additional experiments. While the rebuttal addresses some of my initial concerns, I remain unconvinced that the proposed method's reliability, specificity, and novelty have been sufficiently established.
- Potential bias in matched-negative construction.
Although the authors introduce semantic and embedding-matched negatives, the candidate exclusion procedure raises concerns. In particular, excluding prompts containing empirically identified memorization-triggering terms may inadvertently remove the most challenging negatives. Moreover, negative candidates are visually screened using only four noise seeds, while evaluation uses twelve. How can the authors ensure that these prompts are genuinely non-memorized across the evaluation settings? Matching average distances alone also does not establish distributional equivalence.
- Insufficiently rigorous evaluation protocol.
The authors acknowledge that the original full-set evaluation fits and evaluates logistic regression on the same prompts, which does not establish generalization. Although the new held-out evaluation is helpful, it remains unclear whether splitting, standardization, and threshold selection are strictly performed at the prompt level without leakage across noise seeds. More importantly, the calibration experiment reports TPR but not FPR on an independent negative test set. A calibration FPR below 1% does not guarantee the same performance on unseen negatives.
- Insufficient evidence for memorization-specific sensitivity.
The interpolation experiment demonstrates an association between conditioning sensitivity and transitions toward training-image reproduction, but does not establish that this behavior is specific to memorization. Would similar sensitivity spikes occur during abrupt generation changes for non-memorized prompts? Additionally, the 544 prompt-noise pairs originate from only 159 prompts, potentially introducing within-prompt dependence into the reported Spearman correlation. A prompt-level statistical analysis and controlled non-memorization comparisons are needed. The authors should also report absolute sensitivity changes, since relative increases can be misleading when prior variation is small.
- Methodological novelty remains unclear.
The authors argue that their contribution lies in a new analysis rather than new mathematical techniques. However, the central novelty is still insufficiently distinguished from existing score-based and gradient-based memorization detectors. In particular, does the conditioning Jacobian provide information beyond simpler alternatives such as score magnitude, conditional-unconditional score differences, or other sensitivity measures? Comparisons under the newly introduced matched-negative protocol would help establish its actual contribution.
The additional experiments are valuable, but they do not fully resolve my concerns. In particular, the negative construction, independent low-FPR evaluation, memorization specificity, and methodological novelty remain insufficiently established. I therefore remain unconvinced that the current evidence supports the paper's central claims.
Our method retains strong detection performance with both semantic and text embedding matched negatives from the same source across multiple evaluation protocols. As in Webster [1], who constructed the benchmark, we use LAION-2B as the source corpus and select matched negatives for the same 500 memorized prompts from a candidate pool of 10 million prompts.
Semantic matching (SM) maximizes cosine similarity between normalized, mean-pooled representations from
all-mpnet-base-v2.Text embedding matching (EM) minimizes mean squared distance between the full SD v1.4 CLIP conditioning embeddings.
We exclude candidates sharing key terms with their paired memorized prompt or containing terms empirically observed to strongly trigger memorization. Each procedure assigns one unique negative per memorized prompt. Every candidate and replacement is visually screened using the same four seeds, regardless of detector score, and candidates exhibiting memorization are replaced. The full matching procedure has been added to the revised appendix and the supplementary material, with excluded prompts and their reported memorizing seeds in the supplementary material.
We follow the evaluation protocol in Table 2, using 12 initial-noise seeds and reporting n=4 results.
To examine prompt-distribution differences, we compare matched-pair distances with average pairwise and nearest-neighbor distances within memorized prompts in both semantic and text embedding spaces. All distances below are measured relative to memorized prompts.
With the same exclusions, semantic matched negatives and text embedding matched negatives are closer on average than the nearest memorized neighbors in their respective spaces. Our detector maintains near-perfect AUC with these closely matched negatives, supporting the interpretation that its performance reflects memorization rather than prompt-distribution differences alone.
We clarify which prompts are used for fitting and evaluation below and show that our method retains strong detection performance across multiple evaluation protocols. Further details have been added to the revised appendix and the supplementary material.
Full-set evaluation. For Table 2, we standardize the joint features and fit logistic regression using all evaluation prompts and their memorization labels. We use its decision scores to compute AUC and TPR@1%FPR.
Held-out evaluation. We use disjoint prompt sets for fitting (20%) and evaluation (80%), with matched negatives and 12 initial-noise seeds (n=4). We select the threshold with calibration FPR ≤ 1%.
Calibration without memorized prompts. Strong detection performance remains possible without memorized calibration examples or a supervised logistic-regression boundary. We fit a Gaussian distribution to the scalar Jc feature or the joint features of 500 non-memorized prompts and set a Mahalanobis-distance threshold allowing at most 1% empirical calibration outliers. We then freeze the threshold and evaluate memorized prompts.
We have added the additional results, detailed experimental settings, and calibration protocols to the revised appendix.