acceptodds
Under review as a conference paper at ICLR 2027

Memorization Detection in Diffusion Models via Text Embedding Interpolation

Abstract

Text-to-image diffusion models generate high-fidelity images from a text prompt but exhibit memorization on certain prompts where the model reproduces a training image. A widely adopted approach to memorization detection quantifies the difference between the conditional and unconditional scores. We express the magnitude of this difference as the norm of the integral of the conditional score's derivative along the linear interpolation from the unconditional embedding to the prompt embedding. For memorized prompts with an anomalously large score difference, the rate of change of the conditional score remains relatively stable over most of the interpolation path and rises sharply within a narrow interval near the prompt embedding. The generated image follows the same trajectory remaining aligned with the prompt semantics across the smooth segment and switching to the memorized training image across the narrow interval. We measure this sharp rise through the condition Jacobian evaluated at the prompt embedding which yields a detection performance that achieves state-of-the-art accuracy. Detection performance remains stable over a range of reduced latent resolutions, allowing the resolution to be selected to reduce memory consumption with minimal performance loss.

Primary area:
generative models
Keywords:
diffusion model, memorization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4mXTOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a detector for memorization in text-to-image diffusion models that measures the sensitivity of predicted scores to text conditioning. Its primary detection statistic is the Frobenius norm of the condition Jacobian, estimated using Hutchinson’s method, while information from the input-noise Jacobian provides a complementary signal. The approach stems from an analysis of score behavior as the conditioning embedding moves along a straight path from the unconditional embedding to the prompt embedding. By expressing the conditional–unconditional score difference as a line integral, the authors reveal a distinctive pattern associated with memorized prompts: score sensitivity rises sharply as the interpolation approaches the target embedding, coinciding with a sudden shift in generation toward the memorized training image. Evaluations across Stable Diffusion v1.4, Stable Diffusion v2.0, and other diffusion models show that these signals support strong memorization detection, with performance remaining robust when latent resolution is reduced.

Critical strengths

The paper offers an intuitive view of memorization through sharp increases in conditional score sensitivity near memorized text embeddings. The detector substantially improves AUC and TPR at low false-positive rates, with robust results across architectures, noise samples, and latent resolutions.

Critical weaknesses
  1. Potential dataset confounding. Memorized and non-memorized prompts come from different sources, potentially introducing linguistic and semantic differences. Without matched negatives, the near-perfect AUC may reflect prompt-distribution differences rather than memorization.

  2. Limited methodological novelty and justification. The line-integral formulation, Jacobian norms, and Hutchinson estimation are standard techniques. The paper insufficiently explains why elevated conditioning sensitivity specifically indicates memorization.

  3. Unclear evaluation protocol. The separation between fitting and evaluating the logistic-regression projection is insufficiently specified. Although held-out results are provided, random splits within the same prompt collections do not address source-related confounding.

Questions for authors

Overall, despite the impressive reported numerical results, I find the evidence insufficient to establish that the proposed method reliably detects memorization rather than dataset-specific characteristics. The limited methodological novelty and unresolved evaluation concerns substantially weaken the paper's central contribution. I therefore recommend rejection.

8 comments

W1, W3. Detection performance on matched negatives and evaluation protocol

Our method retains strong detection performance with both semantic and text embedding matched negatives from the same source across multiple evaluation protocols. As in Webster [1], who constructed the benchmark, we use LAION-2B as the source corpus and select matched negatives for the same 500 memorized prompts from a candidate pool of 10 million prompts.

  • Semantic matching (SM) maximizes cosine similarity between normalized, mean-pooled representations from all-mpnet-base-v2.

  • Text embedding matching (EM) minimizes mean squared distance between the full SD v1.4 CLIP conditioning embeddings.

We exclude candidates sharing key terms with their paired memorized prompt or containing terms empirically observed to strongly trigger memorization. Each procedure assigns one unique negative per memorized prompt. Every candidate and replacement is visually screened using the same four seeds, regardless of detector score, and candidates exhibiting memorization are replaced. The full matching procedure has been added to the revised appendix and the supplementary material, with excluded prompts and their reported memorizing seeds in the supplementary material.

We follow the evaluation protocol in Table 2, using 12 initial-noise seeds and reporting n=4n=4 results.

Matched negativesMetricRenWenHintersdorfJeonAsthanaJcJ_c onlyJoint
SMAUC0.967 ± 0.0030.985 ± 0.0020.935 ± 0.0320.986 ± 0.0100.984 ± 0.0040.996 ± 0.0010.995 ± 0.001
SMTPR@1%FPR0.721 ± 0.0380.859 ± 0.0170.598 ± 0.1680.752 ± 0.2160.834 ± 0.0590.969 ± 0.0060.967 ± 0.006
EMAUC0.971 ± 0.0020.979 ± 0.0030.901 ± 0.0520.985 ± 0.0080.980 ± 0.0030.995 ± 0.0000.995 ± 0.000
EMTPR@1%FPR0.741 ± 0.0140.826 ± 0.0160.363 ± 0.0470.760 ± 0.1570.809 ± 0.0540.970 ± 0.0040.971 ± 0.003

To examine prompt-distribution differences, we compare matched-pair distances with average pairwise and nearest-neighbor distances within memorized prompts in both semantic and text embedding spaces. All distances below are measured relative to memorized prompts.

ComparisonSemantic distance ↓Text embedding RMS distance ↓
All memorized pairs0.7771.266
Nearest memorized pairs (no shared key terms)0.5391.136
SM0.3581.137
EM0.5360.985

With the same exclusions, semantic matched negatives and text embedding matched negatives are closer on average than the nearest memorized neighbors in their respective spaces. Our detector maintains near-perfect AUC with these closely matched negatives, supporting the interpretation that its performance reflects memorization rather than prompt-distribution differences alone.

We clarify which prompts are used for fitting and evaluation below and show that our method retains strong detection performance across multiple evaluation protocols. Further details have been added to the revised appendix and the supplementary material.

  • Full-set evaluation. For Table 2, we standardize the joint features and fit logistic regression using all evaluation prompts and their memorization labels. We use its decision scores to compute AUC and TPR@1%FPR.

  • Held-out evaluation. We use disjoint prompt sets for fitting (20%) and evaluation (80%), with matched negatives and 12 initial-noise seeds (n=4n=4). We select the threshold with calibration FPR ≤ 1%.

    Matched negativesMetricJcJ_c onlyJoint
    SMAUC0.995 ± 0.0010.994 ± 0.001
    SMF1-score0.976 ± 0.0040.975 ± 0.004
    EMAUC0.995 ± 0.0010.994 ± 0.001
    EMF1-score0.974 ± 0.0040.973 ± 0.004
  • Calibration without memorized prompts. Strong detection performance remains possible without memorized calibration examples or a supervised logistic-regression boundary. We fit a Gaussian distribution to the scalar JcJ_c feature or the joint features of 500 non-memorized prompts and set a Mahalanobis-distance threshold allowing at most 1% empirical calibration outliers. We then freeze the threshold and evaluate memorized prompts.

Negative calibration setJcJ_c only (n=4n=4) TPRJoint (n=4n=4) TPR
Original non-memorized prompts0.9940.994
SM0.9720.960
EM0.9660.960

We have added the additional results, detailed experimental settings, and calibration protocols to the revised appendix.

W2. Empirical justification for elevated conditioning sensitivity as a memorization signal

We provide substantial empirical justification for conditioning sensitivity as a memorization signal by showing that ∥Jc∥F\|J_c\|_F almost always increases sharply around transitions toward memorized images, while varying only the interpolation coefficient for the same prompt and initial noise. This contrasts with the relatively smooth changes in score sensitivity and generated images observed for non-memorized prompts along the interpolation (Figures 1 and 2).

We focus on global memorization, where the entire image is reproduced, and track image similarity using SSCD. We evaluate 544 pairs of prompt and initial noise from 159 globally memorized prompts with accessible training images, keeping the pairs among four initial noises per prompt that reproduce a training image with SSCD ≥ 0.5. For each fixed prompt and initial noise, we locate the transition at the interpolation step with the largest SSCD increase (Figure 2). Across these transitions, mean SSCD rises from 0.159 to 0.738 (+0.580), crossing the 0.5 threshold used to analyze copying in Stable Diffusion by Somepalli et al. [2]. We test whether conditioning sensitivity rises specifically at transitions toward training-image reproduction, identified using SSCD independently of the Jacobian measurements. Using interpolation increments of 0.1, we compare the increase in ∥Jc∥F\|J_c\|_F at and around these transitions with the preceding variation.

The table reports the percentage of these 544 pairs whose increase in conditioning sensitivity is at least 2× or more than 5× the prior variation.

Sensitivity increase≥ 2× prior variation> 5× prior variation
At the transition83.8%76.3%
Around the transition94.9%91.4%

Prior variation is the mean absolute change over intervals preceding the evaluated interval or region. “Around the transition” averages the changes over the transition and its two adjacent intervals.

The prompt, initial noise, and generation settings remain fixed, and even the text embeddings are nearly identical across the transition. Despite these closely controlled conditions, ∥Jc∥F\|J_c\|_F rises sharply at or around the transition toward reproducing the training image. This links the sensitivity increase to the emergence of memorization within the same prompt, supporting its use as a memorization signal. The pooled Spearman correlation between ∥Jc∥F\|J_c\|_F and original-image SSCD is 0.733, further supporting this association. We have added the detailed experimental settings, selection criteria, and additional results to the revised appendix.

Our contribution lies in a new analysis of memorization in text embedding space and a detection method derived from this analysis that outperforms existing methods, rather than in developing new mathematical techniques. The line-integral formulation, Jacobian norms, and Hutchinson estimation serve as tools for these contributions. The standard nature of these tools does not by itself establish that the resulting insight or detection method lacks novelty.

[1] Ryan Webster. A reproducible extraction of training images from diffusion models. arXiv:2305.08694, 2023.

[2] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? Investigating data replication in diffusion models. CVPR, 2023.

Thank you for the additional experiments. While the rebuttal addresses some of my initial concerns, I remain unconvinced that the proposed method's reliability, specificity, and novelty have been sufficiently established.

  1. Potential bias in matched-negative construction.

Although the authors introduce semantic and embedding-matched negatives, the candidate exclusion procedure raises concerns. In particular, excluding prompts containing empirically identified memorization-triggering terms may inadvertently remove the most challenging negatives. Moreover, negative candidates are visually screened using only four noise seeds, while evaluation uses twelve. How can the authors ensure that these prompts are genuinely non-memorized across the evaluation settings? Matching average distances alone also does not establish distributional equivalence.

  1. Insufficiently rigorous evaluation protocol.

The authors acknowledge that the original full-set evaluation fits and evaluates logistic regression on the same prompts, which does not establish generalization. Although the new held-out evaluation is helpful, it remains unclear whether splitting, standardization, and threshold selection are strictly performed at the prompt level without leakage across noise seeds. More importantly, the calibration experiment reports TPR but not FPR on an independent negative test set. A calibration FPR below 1% does not guarantee the same performance on unseen negatives.

  1. Insufficient evidence for memorization-specific sensitivity.

The interpolation experiment demonstrates an association between conditioning sensitivity and transitions toward training-image reproduction, but does not establish that this behavior is specific to memorization. Would similar sensitivity spikes occur during abrupt generation changes for non-memorized prompts? Additionally, the 544 prompt-noise pairs originate from only 159 prompts, potentially introducing within-prompt dependence into the reported Spearman correlation. A prompt-level statistical analysis and controlled non-memorization comparisons are needed. The authors should also report absolute sensitivity changes, since relative increases can be misleading when prior variation is small.

  1. Methodological novelty remains unclear.

The authors argue that their contribution lies in a new analysis rather than new mathematical techniques. However, the central novelty is still insufficiently distinguished from existing score-based and gradient-based memorization detectors. In particular, does the conditioning Jacobian provide information beyond simpler alternatives such as score magnitude, conditional-unconditional score differences, or other sensitivity measures? Comparisons under the newly introduced matched-negative protocol would help establish its actual contribution.

The additional experiments are valuable, but they do not fully resolve my concerns. In particular, the negative construction, independent low-FPR evaluation, memorization specificity, and methodological novelty remain insufficiently established. I therefore remain unconvinced that the current evidence supports the paper's central claims.

Reviewer_S7nTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

As motivation, the paper formulates the score-difference memorisation metric as a line integral along the interpolation from the unconditional embedding to the prompt embedding. For memorised prompts, the conditional score changes smoothly along most of the path, then rises sharply in a narrow interval near the prompt. At the same point, the generated image switches to the memorised training image in the image space. Based on this, the authors detect memorisation using the norm of the condition Jacobian at the prompt embedding. Experiments are conducted using Stable Diffusion benchmarks.

Critical strengths

The interpolation formulation is really new and interesting, which also motivates the proposed method.

Many intuitive and informative analyses are provided. Section 3.1 presents a sharp transition along the interpolation space, matched by the image switching to the memorised one in the image space. This is the most interesting part of the paper.

Critical weaknesses
  1. The contribution is detection only. No mitigation or end-to-end evaluation. The value of a better detector would be clearer if it were paired with an existing mitigation method and compared with other detectors paired with the same method. Such experiments are missing.

  2. The analysis only loosely drives the method. The interpolation analysis is the most interesting part of the paper, but the final detector does not use the interpolation directly. The connection between the method and the motivation is loose.

  3. The benchmark is saturated, with nearly 100% results. 100 prompts may also be too few.

  4. Please add a comparison of memory and runtime against other methods, since the method needs backward passes.

Questions for authors

See above

If Weakness 1 can be addressed, I may increase the score.

3 comments

W1. End-to-end mitigation evaluation with our detection method

We pair our detector with an existing mitigation method and compare it with other detectors paired with the same method and settings in an end-to-end evaluation. The pipeline consists of four steps.

  1. Input. Input initial noise and prompts.
  2. Detection. Apply our detector or a baseline detector to the prompt embeddings and initial noises to decide whether mitigation is needed.
  3. Mitigation and generation. If the detector determines that mitigation is needed, apply the batch-wise method of Han et al. [1], including initial noise adjustment and early unconditional sampling. Otherwise, generate images directly without mitigation.
  4. Evaluation. Evaluate how much memorization is mitigated in the generated images.

Because mitigation incurs additional computation and can degrade image quality or prompt alignment [1,2], detectors must identify memorization-inducing inputs while limiting unnecessary mitigation on non-memorized inputs. For SD v1.4, we calibrate each detector using TPR@1%FPR on 3,000 memorized prompts from MemBench [3] and 3,000 non-memorized prompts, using the first four seeds. Of these memorized prompts, we evaluate the 1,735 with available training images using 12 seeds per prompt.

All methods use CFG 7.0, including no mitigation. Following [1,2], we assess mitigation using SSCD between generated and training images and alignment using CLIP similarity to the original prompt.

DetectorSSCD ↓CLIP ↑
No mitigation0.3870.284
Ren0.3110.279
Wen0.1800.267
Asthana0.1800.266
Hintersdorf0.2190.270
Jeon0.1500.263
Ours0.1390.262

Our method achieves the lowest SSCD. It applies mitigation to 20,560 outputs, compared with 5,196–18,436 for the baselines. As mitigation is applied to more memorized outputs, CLIP decreases, consistent with the mitigation–alignment tradeoff discussed in [1,2].

We further examine whether the detectors miss outputs that are highly similar to training images and require mitigation. Following Somepalli et al. [5], we evaluate the 7,483 of 20,820 unmitigated outputs with SSCD > 0.5 against available training images. For this same set of outputs, we report how many each detector misses and the proportion selected for mitigation, denoted as target recall.

DetectorMissed outputs ↓Target recall ↑
Ren4,0970.453
Wen8680.884
Asthana8430.887
Hintersdorf1,8740.750
Jeon1730.977
Ours130.998

At SSCD > 0.3, encompassing 12,006 outputs, our detector’s target recall remains 0.998, compared with 0.357–0.961 for the baselines. Fewer missed high-similarity outputs and lower SSCD after mitigation demonstrate the practical benefit of pairing our detector with an existing mitigation method.

Our interpolation analysis also enables a small improvement to the existing mitigation method. Han et al. [1] use unconditional sampling during the early denoising steps. We replace this unconditional prediction with a single prediction conditioned on the interpolated text embedding at λ=0.7.

Mitigation methodSSCD ↓CLIP ↑
Han et al. [1]0.1390.262
Han et al. [1] + early interpolation (λ=0.7)0.1360.262

This modification provides further mitigation while leaving the CLIP score essentially unchanged.


W2. How the interpolation analysis drives the detector

The interpolation analysis identifies both the sensitivity signal used for detection and the region in which to measure it. We express the conditional–unconditional score difference as a line integral along the linear interpolation path cλ=cϕ+λΔc_λ=c_ϕ+λΔ, where Δ=c−cϕΔ=c-c_ϕ. Along this path, we measure

‖∂λsθ(xt,t,cλ)‖2=‖Jc(xt,t,cλ)Δ‖2.‖∂_λ s_θ(x_t,t,c_λ)‖_2 = ‖J_c(x_t,t,c_λ)Δ‖_2.

Empirically, for memorized prompts, this quantity rises sharply near the prompt embedding, motivating the use of elevated condition sensitivity in this region as a detection signal. Appendix F shows that the sensitivity increase also occurs across other directions, supporting the use of ‖Jc‖F‖J_c‖_F as a memorization detection signal.

To determine where within this region to measure the signal, we compare the joint detector across interpolation positions with n=4. Among the tested positions, λ=1 yields the best detection performance, supporting evaluation at the prompt embedding.

λSD v1.4 AUC ↑SD v1.4 TPR@1%FPR ↑SD v2.0 AUC ↑SD v2.0 TPR@1%FPR ↑
0.70.9220.5980.7100.027
0.80.9690.8440.9910.744
0.90.9950.9680.9970.950
1.00.9990.9920.9980.977

We have clarified this connection in Sec. 3.2 and added the interpolation-position comparison and its evaluation protocol to the revised appendix.

W3. Benchmark saturation and size

The 100 samples refer to initial noise seeds per prompt, not 100 prompts. Sec. 5.1 evaluates 500 memorized and 500 non-memorized prompts on SD v1.4, and 219 memorized and 500 non-memorized prompts on SD v2.0.

Following the evaluation protocol in Sec. 5.1 with 12 initial noise seeds per prompt, we additionally evaluate SD v1.4 on 3,000 memorized prompts from MemBench [3] and 3,000 non-memorized prompts constructed following Wen et al. (2024).

Methodn=1 AUCn=1 TPR@1%FPRn=4 AUCn=4 TPR@1%FPR
Ren0.863 ± 0.0220.312 ± 0.0750.873 ± 0.0200.297 ± 0.046
Wen0.910 ± 0.0180.611 ± 0.0600.963 ± 0.0050.763 ± 0.031
Hintersdorf0.777 ± 0.0710.450 ± 0.0980.859 ± 0.0390.569 ± 0.123
Jeon0.936 ± 0.0150.697 ± 0.0590.984 ± 0.0010.898 ± 0.005
Asthana0.926 ± 0.0160.629 ± 0.0590.968 ± 0.0060.774 ± 0.033
Ours (JcJ_c-only)0.996 ± 0.0030.962 ± 0.0280.999 ± 0.0000.992 ± 0.003
Ours (Joint)0.996 ± 0.0020.970 ± 0.0180.998 ± 0.0010.990 ± 0.005

We further evaluate the same prompt sets without memorized examples for calibration. For each method, we fit a Gaussian to its scores or features using only the 3,000 non-memorized prompts and set a squared Mahalanobis distance threshold that classifies at most 1% of these calibration prompts as memorized. We keep the distribution and threshold fixed when evaluating the 3,000 memorized prompts.

MethodTPR, n=1TPR, n=4
Wen0.5530.748
Ren0.3360.322
Hintersdorf0.4270.558
Jeon0.6430.904
Asthana0.5780.751
Ours (JcJ_c only)0.9800.988
Ours (joint)0.9790.990

Our near-perfect performance is not merely a consequence of a small prompt set. On the expanded benchmark, both our joint detector and scalar JcJ_c-only feature maintain substantially higher detection rates than all evaluated baselines, even without memorized calibration examples.

We have provided the prompt sets in the supplementary material and documented their construction and the experimental settings for both evaluations in the revised appendix.


W4. Comparison of memory and runtime

Our method improves detection performance without substantially increasing memory or runtime. Reduced latent resolution lowers both costs while retaining strong performance. The reported costs include our method’s backward passes.

The table combines newly measured costs with the n=4 detection results in Tables 2–3 of the paper. Time is averaged per prompt for a batch of four prompts, while memory is the peak for the entire batch.

All methods use the default latent resolution (64×64 for SD1.4 and 96×96 for SD2), except Ours (reduced), which uses 32×32 and 16×16, respectively.

MethodSD1.4 AUC ↑SD1.4 TPR@1% FPR ↑SD1.4 Wall clock (s/prompt) ↓SD1.4 Peak allocated (GiB) ↓SD2 AUC ↑SD2 TPR@1% FPR ↑SD2 Wall clock (s/prompt) ↓SD2 Peak allocated (GiB) ↓
Ren0.9030.2480.0641.8770.8700.0000.1372.205
Wen0.9860.8890.1172.1320.9760.8280.2572.784
Hintersdorf0.9340.5230.1192.1320.9850.8730.2612.783
Jeon0.9940.9652.32222.5640.9750.812OOM—
Asthana0.9850.8960.2392.1320.9770.8480.5172.785
Ours (joint)1.0000.9950.6272.5811.0000.9951.3463.393
Ours (reduced)0.9960.9820.2631.8760.9960.9690.2681.734

We have added these results and the detailed measurement settings to the revised appendix.

[1] Han et al. Adjusting Initial Noise to Mitigate Memorization in Text-to-Image Diffusion Models. arXiv:2510.08625, 2025.

[2] Jeon et al. Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes. ICML, 2025.

[3] Hong et al. MemBench: Memorized Image Trigger Prompt Dataset for Diffusion Models. TMLR, 2025.

[4] Webster. A reproducible extraction of training images from diffusion models. arXiv:2305.08694, 2023.

[5] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? Investigating data replication in diffusion models. CVPR, 2023.

Thanks, I have increased my score. Please prepare a comprehensive revision.

Reviewer_L8gZOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a memorization detection method for text-to-image diffusion models using the sensitivity of the score to the text embedding. They express Wen’s metric, the norm of the difference between conditional and unconditional scores, as a line integral from the unconditional to the prompt embedding. Their main finding is that, for memorized prompts, the score changes sharply only near the prompt embedding. The detector therefore uses the Frobenius norm of the Jacobian of the score with respect to the prompt embedding at the first denoising step. On the Webster prompts for SD v1.4 and v2.0, the results are very strong, particularly at low false-positive rates. Further experiments follow on the FLUX.1 model.

Post-rebuttal:

Thank you for your detailed answers. I am satisfied with the authors' answers. I also read through the other reviews and responses. Overall, the rebuttal does a very good job. I will keep my positive rating.

Critical strengths
  • The paper is well-written and easy to follow. One can easily understand the method and main ideas.
  • The paper presents the novel idea of interpreting the score difference as a line-integral to investigate where changes in the conditional score occur along the interpolation path. It is creative and insightful compared to the prior works.
  • The experiments are broad, thorough and the results are significantly better compared to the previous work.
  • The experiment on transformer-based FLUX models is a huge plus (although preliminary). It adds clear value to the paper.
  • The paper also introduces the reduced latent resolution experiment, which is another useful methodological contribution of this paper. The resolution ablations are particularly useful because they reveal that performance is not uniformly preserved across model families or operating resolutions.
Critical weaknesses
  • The joint-feature results (Table 2) use a logistic-regression decision boundary to project the two-dimensional feature onto a scalar. If it was fitted on the given prompt set. The approach requires some memorized prompt examples to work, which limits its applicability in cases where a memorized prompt set is not available.
  • The paper reports neither runtime nor memory for its method, although it makes the claim a favorable trade-off between memory consumption and performance. Reporting wall-clock time and peak memory per prompt, against previous approaches, would support the claims.
  • The paper presents the inequality (Eq. 6) but it is unclear if this would affect the localized increase of the score difference near the target embedding. A constant small increase over a broad interval can also make the integral large.
  • The submission does not provide any code or reproducibility statement, which limits the reproducibility of its strong results.
Questions for authors
  • How can one find the learned parameters of the regression model if there are no memorized prompts available?
  • What is the wall-clock time for the proposed detection method? And how does it compare with previous approaches?
2 comments

W1. Limited applicability without memorized examples for calibration

When a memorized prompt set is not available, our method achieves higher detection rates than the evaluated baselines using only non-memorized prompts for calibration.

We compute exactly the same Jacobian scores and change only how they are classified. For classification, we use a one-class Gaussian (Mahalanobis-distance) detector calibrated using only non-memorized prompts.

Using the prompts from Table 2 of the paper, we fit a Gaussian to each method’s scores on the 500 non-memorized prompts and set a squared-Mahalanobis-distance threshold to classify at most 1% as outliers. We then evaluate on the memorized prompts while keeping the fitted distribution and threshold fixed. No memorized examples are used for calibration.

MethodSD v1.4, n=1 TPRSD v1.4, n=4 TPRSD v2.0, n=1 TPRSD v2.0, n=4 TPR
Wen0.4620.9400.7990.776
Ren0.7580.7460.0090.009
Hintersdorf0.4680.7260.4200.685
Jeon0.6700.9780.8170.808
Asthana0.4760.9520.7990.781
Ours (JcJ_c only)0.9880.9940.9540.954
Joint0.9880.9940.9730.977

Our JcJ_c-only variant, which uses a single scalar score like the baselines, also achieves substantially higher detection rates than all evaluated baselines under the same calibration procedure.

These results show that our features support accurate detection even without memorized calibration examples. We have added this experiment and protocol to the appendix of the revised manuscript.


W2. Wall-clock time and peak memory

Our method improves detection performance with an acceptable increase in memory consumption and computation time, and reducing latent resolution lowers both costs while retaining strong detection performance. The reported costs include the backward passes required by our method.

The table combines newly measured costs with the n=4 detection results in Tables 2–3 of the paper. Time is averaged per prompt for a batch of four prompts, while memory is the peak for the entire batch.

All methods use the default latent resolution (64×64 for SD1.4 and 96×96 for SD2), except Ours (reduced), which uses 32×32 and 16×16, respectively.

MethodSD1.4 AUC ↑SD1.4 TPR@1% FPR ↑SD1.4 Wall clock (s/prompt) ↓SD1.4 Peak allocated (GiB) ↓SD2 AUC ↑SD2 TPR@1% FPR ↑SD2 Wall clock (s/prompt) ↓SD2 Peak allocated (GiB) ↓
Ren0.9030.2480.0641.8770.8700.0000.1372.205
Wen0.9860.8890.1172.1320.9760.8280.2572.784
Hintersdorf0.9340.5230.1192.1320.9850.8730.2612.783
Jeon0.9940.9652.32222.5640.9750.812OOM—
Asthana0.9850.8960.2392.1320.9770.8480.5172.785
Ours (Joint)1.0000.9950.6272.5811.0000.9951.3463.393
Ours (reduced)0.9960.9820.2631.8760.9960.9690.2681.734

We have added these results and the detailed measurement settings to the appendix of the revised manuscript.


W3. Eq. (6) and the localized increase near the target embedding

The localized increase near the target embedding is observed empirically in Figure 1, rather than inferred from Eq. (6). The mathematical analysis identifies the sensitivity signal to investigate, the experiment identifies its behavior along the path, and the detector measures the resulting signal.

  1. The mathematical analysis narrows the question. In Sec. 3.1, we express the change from the unconditional score to the conditional score as a line integral along the embedding interpolation path. Applying the triangle inequality to this representation gives the following bound on the magnitude of that change,

    dt≤∫01rλ dλ.d_t≤∫_0^1r_λ\,dλ.

    As already stated in the paper, a large dₜ requires “rλr_λ to be large along at least part of the interpolation path.” This requirement does not specify where the increase occurs or distinguish a broadly distributed response from a localized one.

  2. We therefore conducted the experiment in Figure 1 to determine whether the score changes through “A constant small increase over a broad interval.” The results empirically reveal a “localized increase in the rate of score change near the target embedding”.

  3. Eq. (6) then connects this observed directional sensitivity to the condition Jacobian,

    ≤\|J_c(c_λ)\|_F\|Δ\|_2.$$

We have refined the wording in Secs. 3.1–3.2 of the revised manuscript to make this existing distinction between the mathematical requirement and the empirical finding clearer.


W4. Code availability and reproducibility

We have added our code, prompt sets, and execution instructions as Supplementary Material.

Thank you for your detailed answers. I am satisfied with the authors' answers. I also read through the other reviews and responses. Overall, the rebuttal does a very good job. I will keep my positive rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.