acceptodds
Under review as a conference paper at ICLR 2027

From Modes to Memories: Characterizing the Scale-Space Dynamics of Diffusion Models

Abstract

Diffusion models are typically viewed as stochastic processes that transform noise into data. We take a complementary perspective: a diffusion model defines a family of deterministic dynamical systems indexed by noise scale. At each fixed scale , we treat the denoiser as a self-map and study its dynamics. For an exact denoiser, fixed points correspond to critical points of the smoothed data density, while attractors correspond to its modes; as increases, sample-level modes merge into progressively coarser ones. This suggests a geometric view of memorization: examples that receive excess probability mass due to duplication or overfitting, as well as outliers, should remain distinguishable under stronger smoothing than ordinary examples. We quantify this persistence by the critical scale , the largest noise scale at which an example is retained by the fixed-scale dynamics. In conditional models, the same construction extends naturally to image-caption pairs. Experiments in controlled settings and on large-scale models show that tracks memorization arising from duplication, overfitting, and outliers, and identifies both memorized and partially memorized examples in Stable Diffusion. Moreover, yields interpretable measures of the image spatial distribution and caption dependence of memorization.

Primary area:
generative models
Keywords:
Diffusion models, attractors, memorization, self map, dynamics

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_sPEMOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The to repeatedly apply diffusion model denoising on the same noise level to find attractors at each level of noise, hypothesizing that the most prominent modes, the modes of memorized samples, would keep have theor attractors intact for hghger levels of noising (the critical sigma_c): The largest noise level at which an example remains retained under these dynamics. They experiment with memorization which they induced for cifar-10 training, and known memorized prompts in SD.

Critical strengths

The fixed-scale dynamical interpretation provides a useful connection between mean-shift dynamics and memorization. The controlled CIFAR-10 experiments cover all main mechanisms of memorizatoin: duplication, rarity, and overfitting.

Critical weaknesses

My main concern is that the novelty relative to prior work on when memorization emerges during denoising should be positioned more carefully. Several works have already shown that memorized images become identifiable or effectively determined unusually early in generation. Wen et al. detect memorized prompts from signals available at the first denoising step [1]. Humayun et al. show that memorized images resolve faster along diffusion trajectories [2]. Jain et al. identify an early attraction-basin regime in which classifier-free guidance commits trajectories toward memorized images [3]. Kim et al. similarly report that memorized training-image information is overestimated during early denoising, causing rapid trajectory collapse toward memorized examples [4]

These results are conceptually close to the observation that memorized examples persist to larger noise scales. I therefore see the main novelty less as showing that memorization is visible “earlier” or at higher noise, and more as the fixed-σ dynamical-system construction, its mean-shift interpretation, and the resulting persistence statistic σc. A direct comparison between σc and prior notions of memorization emergence/transition timestep would substantially clarify what additional information the proposed method provides.

References

[1] Wen et al., “Detecting, Explaining, and Mitigating Memorization in Diffusion Models” ICLR 2024.

[2] Humayun et al., “What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models” ICLR 2025.

[3] Jain et al., “Classifier-Free Guidance Inside the Attraction Basin May Cause Memorization” CVPR 2025.

[4] Kim et al., “How Diffusion Models Memorize” arXiv, 2025.

Questions for authors

Kindly see weaknesses.

4 comments
AuthorsOpenReview ↗

Rebuttal (1 of 2)

We thank the reviewer for their feedback and suggestions. In the following we address the reviewers comments: we uploaded a revised version of the manuscript where all edits are highlighted in red. We remain available during the discussion period for further clarifications.

Relation to prior work on early memorization (Wen et al., Humayun et al., Jain et al., Kim et al.)

We thank the reviewer for their comment. We agree that the fact that memorized images show up early in generation is known in the literature (Wen et al., Humayun et al., Jain et al., Kim et al.) and we do not claim it as our contribution. Our contribution lies in proposing the dynamical system framework for diffusion models, its connection to mean shift theory and to memorization through the critical scale. This framework yields a theoretically grounded, interpretable measure that can localize memorized content and distinguish the roles of the image, the caption, and their interaction. To show what the critical scale σc\sigma_c adds, we proceed in three steps:

  • (a) We contrast it methodologically with sampling-based signals.
  • (b) We show experimentally cases that critical scale detects and sampling-based methods miss.
  • (c) We compare it with each cited method.

All details and new experiments are now fully reported in the new App. H.

(a) What σc\sigma_c measures that an emergence time does not. The measures in [1-4] are properties of a sampling trajectory starting from noise. That is a question of reachability of sampling trajectories. σc\sigma_c is a property of a candidate image: the orbit starts at xx, and one asks up to which noise level xx survives as a fixed point. That is a question of persistence of candidate input. The two are related but measure different things. A memorized image can persist as a strong attractor even when its basin is reached too rarely to be observed with a finite sampling budget. The theory shows that they depend on the data differently. σc\sigma_c grows linearly with an image's isolation and only logarithmically with its mass (Thm. 5). The probability of sampling the image grows with its mass.

(b) Memorization that persistence detects and reachability misses.

To test what critical scale can predict differently than prior measures conducted a new experiment on Stable Diffusion v1.4.

  • Setup. We take the 395 memorized prompts of [1] whose training image is available and draw 32 samples per prompt. This estimates how often sampling reproduces the training image. For a set of prompts, the training image appears in only 1-3 of the 32 samples ($\le$10%): sampling reaches these memorized images only occasionally.
  • Result. σc\sigma_c is computed at the training image and its caption, without sampling. It separates the set of rarely memorized prompts from random LAION controls with AUC 0.99. The sampling-based signals of Jain et al. [3] and Kim et al. [4] reach 0.57 and 0.59 on the same prompts (Table R1, Fig. R1).

Table R1. AUC of memorized prompts vs. controls, by probability of sampling the training image. ± is the half-width of the 95% bootstrap interval. Humayun et al.: local scaling of the decoder at the current latent xtx_t, mean AUC over 5, 15 and 50 of 50 DDIM steps, scored on the 100 controls on which it was computed.

sampling probabilitypromptsJain [3]Kim [4]Humayun [2]σc\sigma_c
$\le$10%350.57 ± 0.090.59 ± 0.090.46 ± 0.090.99 ± 0.01
10-50%510.61 ± 0.070.67 ± 0.080.63 ± 0.080.98 ± 0.01
>50%720.70 ± 0.070.97 ± 0.030.92 ± 0.030.98 ± 0.02

Additional evidence already in the paper.

  • Rarely sampled images on CIFAR-10. σc\sigma_c flags 22-88% of planted training images that never appear in 10510^5 samples (Sec. 3.2).
  • Overfitting (CIFAR-10). At N=2000N = 2000 training images, only 4 of 5⋅1045\cdot10^4 samples are copies, yet σc\sigma_c separates training from held-out images with AUC 0.90 (App. E.2).

[1] Wen et al., ICLR 2024. [2] Humayun et al., ICLR 2025. [3] Jain et al., CVPR 2025. [4] Kim et al., arXiv 2025.

AuthorsOpenReview ↗

Rebuttal (2 of 2)

(c) Comparison with each method.

  • Wen et al. [1] score the prompt alone. We extensively compare in the paper (Tab. 1) showing are compared in, where they cannot detect image-caption pairing, and on cost (App. G.5); they also cannot localize memorized content or detect unconditional memorization (Secs. 3, 4.2, 4.3). New (App. G.4): on 298k LAION images (SD 2), how much the caption helps inpaint an image tracks local gap better than [1] (ρ=0.64\rho = 0.64 vs. 0.440.44).
  • Humayun et al. [2] show group-level curves of decoder geometry along sampling, without a per-prompt score. We turned their local scaling into one (Table R1): It tracks how often the image is sampled (ρ=0.68\rho = 0.68-0.780.78), so it can detect frequently sampled memorization but misses rare memorization.
  • Jain et al. [3] locate where guidance commits a trajectory to a memorized image. As a detector their transition point stays weak at every sampling probability (Table R1). Moreover, σc\sigma_c finds the basin without guidance (App. B.5): guidance only changes how often sampling enters it (copies drop from 66% to 32% at w=1w = 1).
  • Kim et al. [4] measure early trajectory collapse. This detects frequently sampled images but not rare ones (Table R1). It also fires on template prompts that rarely reproduce their image, which σc\sigma_c instead separates from verbatim copies (AUC 0.83 vs. 0.59).

Methodological comparison summary Table.

Wen et al. [1]Humayun et al. [2]Jain et al. [3]Kim et al. [4]σc\sigma_c (ours)
Detects caption-conditioned memorization✓✓✓✓✓
Detects memorization in unconditional models✗not tested✗✗✓
Detects memorized images that sampling rarely or never producesprompt only✗ (Tab. R1)✗ (Tab. R1)✗ (Tab. R1)✓
Detects whether an image belongs to its caption (pairing)✗✗✗✗✓
Localizes memorized content spatially✗✗✗✗✓
Defined without guidance✓✗✗✗✓
Needs no sampling from noise✓✗✗✗✓
Needs no access feature encoder or decoder✓✗✓✓✓
Has an analytical model of the signal✗✗✗✓✓

[1] Wen et al., ICLR 2024. [2] Humayun et al., ICLR 2025. [3] Jain et al., CVPR 2025. [4] Kim et al., arXiv 2025.

Thank you to the authors for their response - I would raise my score accordingly

We thank the reviewer for their follow-up, and for raising their score. Thank you again for the constructive feedback and suggestions that helped us improve the paper.

Reviewer_soaKOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a scale-space perspective on diffusion models by treating the diffusion models as a deterministic dynamics systems indexed by noise scale. At each time scale, memorization can be detected by iteratively feed the sample to the diffusion model. The memorized samples correspond the critical points of this iterative mapping below a critical scale, hence can be used for detecting memorization. Th authors provide extensive experiments to support their claims.

Critical strengths
  1. The conceptual framework for interpreting diffusion models as dynamic systems are novel and interesting. The mean-shift perspective provides an intuitive understanding on the different behaviors of memorized samples and non-memorized samples.

  2. The experiments regarding how duplication, overfitting and outliers influence the critical scales are useful, further supporting the theoretical claims.

Critical weaknesses
  1. The theories are derived under the empirical optimal denoiser. While providing insights, it is not clear how the theoretical results change when considering other types of denoisers. For example, can similar conclusions hold if analyzing the score function of other distributions?

  2. The proposed critical scale depends on the escape radius (r) and the iteration budget (K). Although the authors provide some ablation studies, it is unclear how these hyperparameters should be selected for different models and datasets without access to memorization labels.

Questions for authors
  1. The theoretical analysis focuses on the empirical optimal denoiser. Could the authors elaborate on whether similar conclusions hold for more general denoisers, particularly those approximating the score functions of other distributions? What aspects of the theoretical results rely specifically on the empirical data distribution?

  2. How should the escape radius (r) and iteration budget (K) be selected in practice, especially when memorization labels are unavailable? Are there general guidelines for choosing these hyperparameters across different models and datasets?

2 comments

We thank the reviewer for their feedback and suggestions. Below, we address the reviewer's comments. We uploaded a revised version of the manuscript where all edits are highlighted in red. We remain available during the discussion period for further clarifications.

On denoisers other than the empirical optimum

We thank the reviewer for this question. Below we first clarify which parts of the theory hold for a general data distribution and which are specific to the empirical one. In short, the dynamical picture holds for any distribution. Only the location of the attractors and the value of σc\sigma_c depend on the empirical distribution. We then describe a new preliminary experiment in Appendix D.3 that uses a patch-wise exact denoiser to aim to bridge the empirical denoiser and trained networks.

What holds for any distribution. For any data distribution pp, Tweedie's formula gives Mσ(x)=x+σ2∇log⁡pσ(x)M_\sigma(x) = x + \sigma^2 \nabla \log p_\sigma(x), with pσ=p∗N(0,σ2I)p_\sigma = p * \mathcal{N}(0,\sigma^2 I). The proofs of Proposition 1 only use that Mσ(x)M_\sigma(x) is a posterior mean. So for every pp, fixed points are critical points of pσp_\sigma, attractors are its modes, and orbits increase pσp_\sigma and cannot cycle. The definition of σc\sigma_c as the scale at which the mode holding xx disappears is therefore unchanged.

What is specific to the empirical distribution. With atoms at the training points, every training image is its own attractor at small σ\sigma, and the two-atom law gives the value of σc\sigma_c. A smooth density has no per-image attractors: its attractors are the modes of pσp_\sigma. This is what we mean by memorization. A model memorizes xx to the extent that it behaves like the empirical denoiser at xx.

The transition between the two cases is explicit. Replace each training point by a Gaussian of width ss. The blurred density is then the empirical one blurred at σ2+s2\sqrt{\sigma^2+s^2}, so the attractors are those of the empirical map at that scale, and

σc(s)(x)=σcemp(x)2−s2.\sigma_c^{(s)}(x) = \sqrt{\sigma_c^{\mathrm{emp}}(x)^2 - s^2}.

Memorization therefore fades continuously as ss grows, and it vanishes once s≥σcemp(x)s \ge \sigma_c^{\mathrm{emp}}(x).

Trained networks: a patch-wise denoiser (new, Appendix D.3). A trained network does not correspond to the posterior mean of any known distribution. To address this we consider the analytic model of the patch-wise exact denoiser (Niedoba et al., 2024; Kamb & Ganguli, 2024). It applies the empirical denoiser to each patch separately, so it can build new images out of training patches. We fit its patch size to each trained CIFAR model and computed σc\sigma_c with it, on the overfitting sweep and on the duplication model. Unlike the empirical map, the patch-wise map also gives held-out images a finite σc\sigma_c, and it still ranks training images above held-out ones.

  • Generalizing networks (N≥2000N \ge 2000). They follow the patch-wise map at a roughly constant ratio, σcnet≈0.2 σcloc\sigma_c^{\mathrm{net}} \approx 0.2\,\sigma_c^{\mathrm{loc}} on training images, whereas their critical scales are 1–4%1–4\% of the empirical denoiser’s critical scales.

  • Memorizing networks. As NN decreases, the network moves from the patch-wise map towards the empirical one. At N=500N=500 it lies above the patch-wise map for 86% of training images and below the empirical map for all of them. At N=100N=100 the fitted patch already covers the whole image, so the two maps coincide.

  • Duplication. Images seen once stay close to the patch-wise map. The patch-wise map, however, over-reacts to two copies and does not reproduce the network's saturation in the number of copies.

Across all our CIFAR experiments, no probe exceeds the empirical ceiling (0/1047).

Although promising, these results are preliminary. Averaging overlapping patch posteriors does not give the mean shift of a single smoothed density. The guarantees above (e.g. fixed points as critical points, no cycles) therefore do not directly apply to it. Validating the patch-wise model on conditional and larger-scale models is left for future work.

Choice of KK and rr without memorization labels

We have added Sec. G.3 to the appendix, proposing a simple heuristic for selecting KK and rr without requiring memorization labels. Specifically, we fix KK based on the computational budget and select rr such that the median image in a reference set of non-memorized images escapes at a small noise scale σ0\sigma_0 within KK iterations. Obtaining such a reference set is not restrictive, as one can use held-out images or simply swap the captions of available images. Our additional experiments (Fig. 24) show that, for small values of σ0\sigma_0 (as expected for non-memorized reference images), the estimated radii are close to those selected using labeled images. Moreover, detection performance is robust to the choice of σ0\sigma_0. We refer the reviewer to Sec. G.3 and Fig. 24 for further details.

We thank the reviewer again for their feedback and suggestions. We hope to have addressed all their questions in our rebuttal and we remain available for any further questions during the remainder of the discussion period.

Reviewer_YnQMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the analysis of diffusion model as family of deterministic maps indexed by noise level: fix σ\sigma and iterate the denoiser on its own output, xk+1=Dσ(xk)x_{k+1} = D_\sigma(x_k). For the exact posterior-mean denoiser this iteration is Gaussian mean shift on the smoothed density pσp_\sigma, so its fixed points are critical points of pσp_\sigma and its attractors are modes; sweeping σ\sigma produces a scale space in which per-example attractors merge into coarser ones. Memorization is then defined as persistence in this scale space: the critical scale σc(x)\sigma_c(x) is the largest σ\sigma at which the orbit started at x stays within r∣x∣r|x| for K steps. Basically, the denoiser outputs a very similar result for a given data at σ\sigma over iterations.

The authors prove, for the exact denoiser, that adding mass to an example can only raise σc\sigma_c (Theorem. 4), and that in a two-atom model with mass θ\theta and neighbour distance dd, linear in distance and only logarithmic in mass (Theorem. 5), so σc\sigma_c should flag both duplicates and outliers.

For conditional models they define the caption gap Δlog⁡σc=log⁡σc(x;c)−log⁡σc(x;null)\Delta \log \sigma_c = \log \sigma_c(x;c) - \log \sigma_c(x; null) and a per-location variant. Controlled CIFAR-10 experiments with duplication, planted outliers and overfitting show σc\sigma_c responds to each intervention and detects retained examples that sampling never reproduces. On Stable Diffusion v1.4, local ∣Δlog⁡σc∣|\Delta \log \sigma_c| separates matching-verbatim (MV) and template-verbatim (TV) examples from controls, the signed gap detects caption swaps, local gaps align with memorized regions, and strongly negative gaps under wrong captions point to images retained by the unconditional branch.

Critical strengths
  1. The authors provide a clean, well-grounded framework with a theory that is actually tested. The fixed-scale dynamical-system view gives a geometric reading of several memorization mechanisms at once. Also, the paper shows in Figure 9 and Appendix D, that the exact empirical denoiser obeys the predicted scaling without fitting (log--log slope 1.03 in distance; 32 copies raise σc\emp\sigma_c^{\emp} by only 1.26 X), that the trained network orders images the same way (ρ=0.78\rho = 0.78), and that κ≤1\kappa \le 1 holds on all 1047 images tested. This is more careful than most memorization papers.

  2. Controlled experiments that isolate mechanisms and report the honest comparison with sampling. The duplication, rarity and overfitting interventions each come with a reference or paired model, and the central empirical finding is stated plainly: σc\sigma_c depends on distance as well as mass, so it detects retained examples that sampling misses (88.2% of never-sampled colour-MNIST plants; AUC 0.897 at N = 2000 where only 4 of 50,000 samples are copies), while explicitly not detecting membership when the model generalizes. The caption gap and local gap add forms of interpretation that prompt-only scores cannot provide.

Critical weaknesses
  1. The headline detection advantage on Stable Diffusion is marginal and the evaluation is thin. On the primary task (memorized vs. control) the proposed best score does not beat the prompt-level baseline of Wen et al., which is far cheaper; the gains are on caption-swap and MV-vs-TV tasks whose practical importance is argued but not demonstrated. The evaluation set is small (70 MV, 24 TV, 94 controls), no confidence intervals are given, and the primary hyperparameters (K, r) appear to have been selected on the same examples that produce Table 1. The same happened in the rarity experiment, where the pre-specified r failed and another r was chosen from a sweep over (K,r)(K, r) on the evaluation images. Figure 16 shows this matters: global σc\sigma_c for MV vs. control ranges from ≈0.5\approx 0.5 at r=10−2r = 10^{-2} to ≈0.9\approx 0.9 near r=0.25r = 0.25. The localization claim (section 4.2) is qualitative only, and existing image-level and localization detectors (Jiang et al.; Chen et al/) are cited but not compared against. Compute cost per image is never reported.

  2. The quantity measured on trained networks is not the quantity the theory describes, and the main text blurs this. For the exact denoiser σc\sigma_c is a saddle-node bifurcation scale, independent of K and r. For trained networks the paper shows that non-memorized images are transients whose σc\sigma_c falls roughly as 1/K1/K — by factors of 165165 and 193193 between K=50 and K=500 — so σc\sigma_c is in practice an escape-time statistic at a chosen budget, and the separation between groups partly reflects that budget. The main text nevertheless presents σc\sigma_c as “the scale up to which a sample remains an attractor” and claims it “measures the width of that basin,” although the orbit always starts at x and never off it, so basin width is never probed except qualitatively (Figure 7, second row; the inpainting of section 4.3).

Questions for authors
  1. Were K and r for Table 1 and for the rarity experiment selected on the same images used to report AUCs? If so, please either report AUCs under a held-out selection or state that the numbers are optimistic, and add bootstrap confidence intervals.

  2. What is the cost of σc\sigma_c, Δlog⁡σc\Delta \log \sigma_c and the local gap per image, in U-Net evaluations (K iterations X bisection steps X two caption conditions, or a 24-level grid X K for the local gap), compared with the n = 32 forward passes of Wen et al.?

  3. The duplication experiment trains m for 2, 8, 25, 50, 100, 200, 400, 800, but Figure 4 and the text report only m >= 8 versus m = 1. What happens at m = 2?

  4. At N = 2000, σc\sigma_c separates training from held-out images with AUC 0.897 while essentially nothing is copied. Under which definition does retention-without-generation count as memorization rather than membership, and why does it matter (privacy, copyright, or purely as a diagnostic of the learned map)?

  5. All Stable Diffusion runs use guidance w = 1, and Lemma B.15 shows that for w > 1 the guided map climbs the tilted density pσ1−wpσ(⋅∣c)wp_\sigma^{1-w} p_\sigma(\cdot \mid c)^w, whose critical points are not those of the conditional model. Since memorized images are reproduced in practice at w=7.5w = 7.5, does σc\sigma_c measured at w=1w=1 predict which pairs are reproduced under guidance?

5 comments
AuthorsOpenReview ↗

Rebuttal (1 of 3)

We thank the reviewer for their feedback and suggestions. In the following we address the reviewers comments: we uploaded a revised version of the manuscript where all edits are highlighted in red. We remain available during the discussion period for further clarifications.

Additional experiments on Stable diffusion V2

We acknowledge that Wen et al. method achieves top performance on memorized-vs-control detection at a low computational cost, although we note that the benchmark on that specific task is saturated. To further investigate the potential benefits of beyond the relatively small Webster benchmark, we have added larger scale experiments on Stable Diffusion 2, detailed in Appendix G.4.

So far, we have processed 298,024 LAION images, computing both our local ∣Δlog⁡σc∣|\Delta \log \sigma_c| (k=1,r=0.1k=1,r=0.1) and Wen et al. with n=4n=4 and first-step. We select five groups of 149 images: the highest- and lowest-scoring under each measure, plus a random reference group. We test whether these scores predict image reconstruction, even for images rarely generated during standard sampling, using paired inpainting with and without the original caption. We find that the caption-induced reduction in reconstruction error correlates more strongly with our score (Spearman 0.64) than with Wen et al.'s (0.44) (Fig. 26, left).

We select the 149 images with the highest local gap and divide each into 16 non-overlapping latent patches. Each patch is inpainted while keeping the surrounding regions fixed, once with the original caption and once with empty conditioning. Across the resulting 2,384 patches, higher patch-wise local gaps are associated with lower inpainting reconstruction error (spearman -0.49 with original caption vs -0.29 with empty caption)(Fig. 26, right). This suggests that our local measure can help identify image regions where caption conditioning facilitates more accurate reconstruction.

These preliminary results suggest a practical benefit of our image-dependent measure: it provides information about the reconstruction of specific images and spatial regions that is not directly available from a prompt-level score. However, a conclusive evaluation would require more images, additional inpainting samples per image, and further analysis of how reliably reconstruction reflects memorization. We therefore regard these findings as initial evidence of our method's potential for larger-scale applications, leaving a more comprehensive evaluation to future work.

Hyperparameter tuning KK and rr

We thank the reviewer for pointing this out. We ran held out selection for kk and rr for both Stable diffusion (Table 1) and CIFAR rarity (Sec 3.2) experiments as detailed below. For both experiments chosen KK and rr are close to the one in the paper and performance values are comparable and conclusions unchanged.

  • Stable diffusion (Table 1) - Appendix G.2

    We use five-fold cross-validation (four folds for choosing KK, rr and one for evaluation) repeated over 20 random splits. The results (Table 3, Fig. 23) show that the selected parameters are stable and that performance remains comparable to the original results. In particular, the local ∣Δlog⁡σc∣|\Delta \log \sigma_c| achieves mean AUCs of 0.9870.987 and 0.9780.978 for MV and TV vs controls ( 0.9970.997 and 0.9930.993 in Table 1), while its MV-versus-TV AUC increases from 0.9350.935 to 0.9560.956. Similarly, the signed Δlog⁡σc\Delta \log \sigma_c improves detection of MV and TV vs caption-swapps. We also apply the same procedure to the baseline of Wen et al. We will change Table 1 accordingly.

  • CIFAR (rarity) - Appendix E.4 We chose KK and rr on half of the images and tested on the other half, over 20 random splits. The selected k,rk,r fall in the same range we used (0.25–0.5) and never near the failed value of 0.02 (Fig. 17). The conclusion is unchanged (Fig. 18): σc\sigma_c still flags memorized images that sampling never produces, and the order MNIST > SVHN > CIFAR-10 > CIFAR-100 holds in every split.

AuthorsOpenReview ↗

Rebuttal (2 of 3)

Localization quantitative results and comparison.

We ran quantitative experiments and compared our local gap map ∣Δlog⁡σcu∣=∣log⁡σcu(c)−log⁡σcu(∅)∣|\Delta \log \sigma_c^u| = |\log \sigma_c^u(c) - \log \sigma_c^u(\varnothing)| with local detector from Bright Ending (BE) from Chen et al.,2025 in Appendix G.9. The baseline BE computes end-token cross-attention at the last denoising step. We used SD v1.4 generations from BE's own prompt bank and from the Webster template prompts. Both maps are scored on the 64×6464 \times 64 latent grid with per-image AUROC against two ground truths (full results in Table 5):

(1) Webster's hand-made template masks mean AUC: BE 0.8420.842, ours 0.9000.900 (+0.058+0.058 95% CI [+0.035,+0.082][+0.035, +0.082]).

(2) Measured copy maps (the latent positions where a partially memorized generation actually reproduces its training image) mean AUC: BE 0.5560.556, which is near chance; ours 0.7770.777 (Δ=+0.221\Delta = +0.221, 95% CI [+0.173,+0.275][+0.173, +0.275]).

BE does detect memorization at the image level: its attention is about 2×2\times brighter on copies. Spatially, however, attention follows the captioned object, which is precisely the varying, non-memorized region in template-style memorization, thus highlighting the wrong region (Figure 33). Our gap measures where the caption changes the stability of the denoising dynamics, highlighting the reproduced background and layout without requiring training data or attention internals.

Computational complexity and reduced cost experiment.

We count network evaluations (one forward pass of the UNet on one latent) per image–caption pair. We have added Appendix G.5 with the full analysis.

  • Caption gap Δlog⁡σc\Delta\log\sigma_c: at most 2K(n+2)2K(n+2), for bisection with nn steps and at most KK iterations per retention test, run under the caption and under the empty caption.

  • Local gap: 2LK2LK on a grid of LL scales.

  • Wen et al.: 2nT2nT, for nn seeds and the first TT sampling steps.

With the original settings of Table 1, this gives 435 evaluations on average for the caption gap (K=16K=16, n=14n=14; escaping tests stop early), 768 for the local gap (L=24L=24, K=16K=16), 64 for Wen et al. (n=32n=32, T=1T=1) and 640 for our image variant (n=32n=32, T=10T=10).

However we remark that efficiency was not an original goal of the paper. To address this, we ran a new experiment on the benchmark of Table 1 in Appendix G.5. We recompute every score at budgets from 2 to 768 evaluations per pair, varying KK, nn and LL for our scores and nn and TT for the baselines. The mean AUC over the five tasks of Table 1 is shown below (± half-width of the 95% bootstrap CI):

scoreTable 1 config: evalsmean AUCcheapest config: evalsmean AUC
Δlog⁡σc\Delta\log\sigma_c (ours)4350.87 ± 0.0520.90 ± 0.04
local ∣Δlog⁡σc∣\lvert\Delta\log\sigma_c\rvert (ours)7680.85 ± 0.0380.84 ± 0.03
Wen et al.640.74 ± 0.0320.73 ± 0.05
our image variant of Wen et al.6400.89 ± 0.0420.83 ± 0.04

With K=1K=1, our scores need only a few evaluations and lose almost nothing. The caption gap reaches its best mean AUC at 2 evaluations per pair, the cost of the cheapest baseline configuration. It stays the best score for telling a caption's own image from a swapped one: 0.96 ± 0.03 at 2 evaluations, against 0.79 ± 0.09 for our image variant of Wen et al. at 640. The local gap matches its Table 1 performance with 8 evaluations instead of 768. We added the complexity analysis and the AUC-versus-budget curves to the appendix (Table 4, Figure 27,28, Appendix G.4).

CIFAR duplication experiment clarification

In Appendix E.1 Figure 14 we now show the duplication results for every multiplicity. Images with m=2m=2 lie between m=1m=1 and m=8m=8. For small iteration budgets they are initially retained: their median σc\sigma_c is above that of m=1m=1 images (0.67 vs 0.55 at K=10K=10, 0.45 vs 0.30 at K=20K=20). But they never reach a fixed point. Their σc\sigma_c then falls as 1/K1/K, like a transient, and from K≈50K\approx50 on it is the same as for m=1m=1. For m≥8m\ge8, σc\sigma_c stays constant in KK. Two copies are therefore not enough to make an image a fixed point of the denoising map.

AuthorsOpenReview ↗

Rebuttal (3 of 3)

Relationship between theory and practice

We thank the reviewer and we agree with the distinction. On a trained network, σc\sigma_c is a retention scale at budget KK. It connects to the theory's saddle-node scale only for images that are true fixed points of the network's map. The main text overstated this. Below we list the changes and the new evidence, now reported in a new Appendix D.4 (Figs 10–11-12).

Edits to the manuscript and clarifications

  1. Introduction. "the scale up to which a sample remains an attractor" → the scale up to which a sample is retained by the network dynamics"

  2. Sec. 2.3. "σc\sigma_c measures the width of that basin" → "σc\sigma_c is the largest noise scale at which the example keeps its basin"

  3. Appendix D. The budget factors: 165 and 193 refer to K=1−>500K=1 -> 500. Over K=50−>500K=50->500 the drop is 6.7.

  4. New Appendix D.4 measures the basin directly.

The budget dependence is confined to non-memorized images. On the full-CIFAR duplication model, the median σc\sigma_c of memorized images (m=800m=800) is identical at K=50K=50 and K=500K=500. Non-memorized and held-out images instead fall roughly as 1/K1/K. For them σc\sigma_c is an escape time rather than a fixed point, and we now report it as an upper bound. The network also never goes beyond the theory: under the same test, the exact empirical denoiser has a larger critical scale for all 264 training probes, and the two are rank-correlated for memorized images (Spearman +0.77).

Basin width, measured from starts away from xx. We perturb xx, iterate the map from the perturbed point, and check whether the orbit returns to xx.

  1. CIFAR-10. Memorized images return from Gaussian noise up to 1.2∥x∥1.2\lVert x\rVert and from occlusion of 14% of the image (all images) or 25% (~85%). This holds at every noise scale up to σc\sigma_c, and the basin closes around σc\sigma_c. Non-memorized images never return. Their orbits fall into a shared flat attractor that is not a training image, so there is no basin around xx.

  2. Stable Diffusion v1.4. With its own caption, a memorized image repairs occlusion: the median fraction removed is 0.75–0.89 at 1% of the area, 0.60–0.77 at 4% and 0.50–0.60 at 9%. With an empty caption, or for control images, nothing is repaired. The basin therefore belongs to the caption–image pair.

How σc\sigma_c relates to basin width. For the exact denoiser, a training image's basin shrinks as σ\sigma grows and closes at the saddle-node. σc\sigma_c marks where the basin vanishes. Trained networks show the same picture for memorized images: the basin is roughly constant in width below σc\sigma_c and closes near it. We now report the two quantities separately. σc\sigma_c says up to which scale a memorized image keeps a basin, and in App. D.4 shows how wide that basin is.

CIFAR overfitting (N=2000N=2000) and clarification on critical scale

In the overfitting experiment DDPMs trained on CIFAR subsets of increasingly smaller sizes NN, keeping the model capacity fixed. At N≤500N \le 500 samples copy training images, therefore membership and memorization tasks overlap. At N≥2000N \ge 2000 they do not anymore, yet σc\sigma_c separates training from held-out images (AUC 0.897). At N=2000N = 2000* training images stay stationary up to slightly larger σ\sigma, which gives the statistical separation to held out images, but they carry negligible mass in order to be sampled.

In general cases σc\sigma_c measures retention means xx is a stationary point of MσM_\sigma. This notion is stronger than membership, which needs only a statistical difference, and weaker than extraction, which relies on sampling. For privacy settings the critical scale matters because σc\sigma_c can flag memorized images that sampling rarely produces, e.g. planted outliers below the sampling floor. For copyright: σc\sigma_c detects memorized caption–image pairs in Stable Diffusion without sampling, therefore can be used to inspect target datasets directly. In general σc\sigma_c remains as a diagnostic measure to interpret what the network has learned about the data.

Does σc\sigma_c at w=1w = 1 predict reproduction under guidance?

Yes. The ground-truth labels are already defined under guidance: MV/TV are filtered by whether the caption reproduces the image at cfg 7.5, and Wen-500 labels use cfg 7.5 with SSCD. So Table 1 measures how well w=1w = 1 scores predict reproduction at w=7.5w = 7.5. On Wen-500, the graded correlation between the w=1w = 1 caption gap and SSCD of the guided generation is ρ=+0.62\rho = +0.62.

We deliberately don't measure σc\sigma_c at w=7.5w = 7.5: in accordance to Lemma B.15, guidance scales ϵc−ϵ∅\epsilon_c - \epsilon_{\varnothing}, which is largest exactly on memorized pairs, so the guided map takes its largest steps there. w=1w = 1 is the scale at which σc\sigma_c describes a fixed point of the conditional model. We'll clarify this in the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.