acceptodds
Under review as a conference paper at ICLR 2027

Spectral Signatures of Memorization Basins in Diffusion Trajectories

Abstract

Diffusion models are known to reproduce individual examples from their training data as a form of memorization with implications for generalization, privacy, and downstream use. Recent work demonstrated that memorized training samples reside inside contiguous regions of the data/latent space known as basins of memorization; once a denoising trajectory enters a basin, under finite perturbations it irreversibly reproduces a memorized example. Prior empirical work on such basins in trained models has primarily focused on classifier-free guidance based detection and mitigation strategies in text-to-image models, leaving basin entry in trained diffusion trajectories and the formation of such basins largely uncharacterized. In this paper, we present a geometric characterization of memorization basins for conditional, unconditional, and rectified-flow-based diffusion models. We observe a characteristic signature in the local spectrum of the function geometry that distinguishes basin entry: the emergence of a dominant singular mode followed by a sharp spectral recovery, both markedly stronger along memorized trajectories. This signature appears at two timescales: along denoising trajectories during inference, and across training. Crucially, these geometric signatures are actionable: regularizing them can suppress memorization during training, while during inference they can help identify and redirect trajectories falling into memorization basins. The resulting geometry-informed interventions remain effective under low-dimensional approximations, and geometry-informed steering extends to large-scale text-to-image diffusion models. Our results show that local spectral geometry provides a common basis for characterizing memorization and for suppressing, detecting, and steering away from it across training and generation.

Primary area:
generative models
Keywords:
diffusion models, generative modeling, memorization, Jacobian geometry, spectral analysis, memorization mitigation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_JJqKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Despite significant attention in recent years, detecting and mitigating memorization in diffusion models remain unresolved. This paper considers the local spectral profiles of the denoiser along generation trajectories, and observes that objects such as the spectral gap and entropy effective rank exhibit interesting behaviors that appear to characterize the trajectories leading memorized samples. Based on these observations, the authors propose and develope three mechanisms: one to regularize training to suppress memorization, one to detect memorized generations during inference, and the third one is a steering strategy to redirect memorized trajectories during inference to reduce memorization.

Critical strengths

(S1) Studying memorization and generalization from a trajectory perspective is less explored in the literature and is a natural and meaningful direction.

(S2) The observation that the spectral gap shows a sharper transition across time within a memorized trajectory (than a non-memorized trajectory) is new and interesting. (That said, studying local geometry or inspecting generation trajectory has appeared in the literature.)

(S3) The paper provides careful experiments. The three proposed mechanisms, especially the GeoSteer-DM, appear to be effective.

Critical weaknesses

(W1) The paper empirically observes that the spectral gap exhibits an interesting sharp transition for memorized trajectories. It hints that this transition might be associated with the entry of the memorization basin. However, no (ideally theoretical) justification is really provided for why such a transition should appear in general. Given that the main trajectory-level experiments are only carried out only on CelebA32 with about 4K images, it might be that the observed spectral-gap behavior captures only one aspect of the underlying phenomenon. It is not clear whether this is the fundamental object governing memorization, and also it would be important to provide more insights on why memorized trajectories should have such behavior. (The spectral recovery observation and analysis is less surprising.) Without further justification or insights, this part of the paper feels somewhat incremental.

As a more minor remark: the paper begins with memorization basin, and gives a formal definition in Section 3.1. The memorization basin itself is a very interesting object. Unfortunately, the later observations and studies are only tangentially connected to the memorization basin. In my view, the paper hasn't provided no strong evidence that the spectral gap transition indeed characterizes the entry of the memorization basin. The experiments in later sections also do not really use or leverage this concept. So there seems to be a gap between the motivating concept (of memorizatoin basin) and later studies.

(W2) While the experimental results are promising, some design choices are not fully aligned with the earlier observations/ narratives. A few examples are:

  • Spectral gap is the quantity that shows the sharp transition. But GeoReg-DM uses the first singular value for reguliarization. The reasoning for this is not clear, and ideally the paper should also report resultings using a spectral-gap based regularization.

  • Similarly, spectral gap transition is highlighted in early part of the paper. However, the proposed JER for memorization detection was based on effective rank. At the very least, I feel that the paper should report results using spectral gap for memorization detection empirically even if it is not as good as the current JER.

  • How is the range of timesteps over which the steering is applied chosen? (I saw some choices are given in the appendix.) It will be interesting to probe where GeoSteer actually gets its most benefit. For example, the paper describes the dominant intermediate-noise mode. Perhaps it will be interesting to test whether most of the steering gain comes from that regime.

  • The stable diffusion setup is only used for the GeoSteer experiments. I understand that for unconditional diffusion memorization may be less common when input training set is not too small. Nevertheless, it would be interesting to study the behavior of such spectral objects for the conditioned case, as well as to test the effectiveness of GeoReg-DM and JER there.

Minor comment: -- The paper [R1] also studied memorization from a trajectory perspective, and it showed that the middle range is the danger zone where memorization forms. I wonder whether there are any deeper connections of the two studies.

[R1] Dodson et al, "Two Calm Ends and the Wild Middle: A geometric picture of memorization in diffusion models", ICML26.

Very minor comment: -- The PDF file I downloaded from OpenReview does not render properly on my Windows machines (I tried multiple laptops). In particularly, many plots / figures are not showing. Eventually I found that it works on my iPad (although not all plots show proper labeling). I am not sure whether this is an issue only happened to me. Nevertheless, the review above is independent of this small presentation issue.

Questions for authors

see Weakness

Reviewer_GTuMOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper “Spectral signatures of memorization basins in diffusion trajectories” isolates an interesting phenomenon in the ODE sampling dynamics of diffusion models, consisting in the emergence of a dominant direction with which memorizing trajectories align at intermediate diffusion times. The behaviour of diffusive trajectories shows that basins of attraction of memorized points are particularly narrowed along large variance directions, consistently with the study of the anisotropy of memorization by Achilli et al. (2024).

Critical strengths

Despite the effect of different local variance directions on the geometric aspects of memorization in diffusion models had already been studied in the past (e.g. Achilli et al., 2024; Merger & Goldt, 2026) this work is the first one that performs a spectral analysis of the Jacobian along the diffusion trajectory.

Critical weaknesses

All figures in the main text, and some figures in the appendices are not readable, which means that the main text lacked supervision by the Authors. Other figures in the appendices show a very bad quality (e.g. fig. 7), which corroborates the previous point.
Some crucial modeling choices are not extensively commented: see for instance the local linear score assumption.

Questions for authors

I will now present my comments and questions in form of Major and Minor Issues, with the goal of improving the quality of the presented work. I ask the Authors to address my points extensively.

Major issues:

Q1: The SVD Jt=UtΣtVtJ_t = U_t \Sigma_t V_t should be true only when the data manifold is linear. Authors should specify that Σt\Sigma_t is a local covariance matrix. What is the “locality” criterion to include the points across which computing the covariance matrix (and its SVD) ?

Q2: Can the Authors determine whether trajectories, at the time when the spectral gap is maximum, have already approached the local manifold (i.e. there are already vanishing singular values on the tails of the spectrum) ? It would be interesting to confront the observation with the paper by Shen et al. (2026), and similar ones from He’s group analyzing the manifold estimation with respect to generalization and memorization.

Q3: Biroli et al. (2024) also study the emergence of data modes as an ergodicity breaking event along diffusive trajectories. In principle, at a smaller scale, one can generalize this picture to memorization, where now modes are centered around training points, instead of classes. The emergence of multistability in the diffusion potential along certain directions should be accompanied by the emergence of negative singular values of the Hessian of the energy i.e. the negative Jacobian of the score on the center of mass of the data. I ask the Authors to confront their spectral observation with Biroli’s analysis.

Q4: Does GEOSTEER-DM relies on the fact that in practice not all trajectories are memorized by diffusion models? Imagine that we are in perfect memorization regime, then what would the model generate at t=0t = 0?

Q5: Did the Authors measure the effectiveness of GEOSTEER-DM in diffusion time? I would expect it to be effective only in correspondence of the ergodicity breaking in the diffusion potential, i.e. near t=t∗t = t^*. At this point one could tune the guidance strength δn\delta_n as a function of the spectral gap itself at step nn: this approach is computationally costful, but maybe there are other cheaper quantities to compute, for instance the conditional entropy (see Stancevic & Ambrogioni, 2026).

Minor issues:

Q6: Most of the Equations are not numbered. I suggest numbering them all for simplicity of reference.

Q7: Spectral gap Δ12\Delta_{12} in line 187 is not defined.

Q8: In Section 3 it should clarified more explicitly that the analysis makes use of deterministic samplers, e.g. DDIM, instead of SDE samplers.

Reviewer_C2QsOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies memorization through the Jacobian spectrum of diffusion and rectified-flow models. It reports a transient concentration around a dominant singular mode followed by spectral recovery, with a stronger effect along memorized trajectories. Similar spectral structure develops during training. The paper connects these observations to a formal definition of memorization basins and proposes three applications: spectral regularization during training (GeoReg), local memorization detection using Jacobian effective rank (JER), and sampling-time steering along dominant singular directions (GeoSteer). The main evaluation covers synthetic distributions, CelebA, and Stable Diffusion 1.4, examining memorization, population coverage, detection accuracy, and generation quality.

Critical strengths
  1. The temporal analysis provides an interesting perspective on memorization geometry. Section 3 examines how the Jacobian spectrum evolves along sampling trajectories and across training, extending analyses based on local geometry at a single state. The emergence of a dominant mode and subsequent recovery provide a concrete phenomenon to investigate. Using this geometry to motivate regularization, detection, and steering gives the analysis practical relevance beyond a descriptive observation.

  2. The controlled experiments provide useful evidence that spectral interventions can reduce memorization. Table 1 evaluates precision and recall against independent samples from known synthetic populations, helping distinguish reduced training-set copying from reduced distributional coverage. Section 5.1 also reports substantial reductions in memorization under GeoSteer: from 51.6% to 7.6% for DDIM and from 60.2% to 8.0% for rectified flow. Low-dimensional approximations retain much of this effect. These results support the usefulness of the intervention, although its advantage over the closest alternatives remains insufficiently established.

Critical weaknesses
  1. The submitted manuscript appears unfinished, particularly in its central figures. Figures 1–4 contain missing or unreadable axis labels, legends, and annotations. These figures support nearly every major empirical claim: the spectral transition during sampling, its development during training, the effectiveness of steering, and the memorization–quality trade-off in Stable Diffusion. The surrounding text and captions do not adequately compensate for the missing information, since the reader cannot reliably identify the plotted quantities, conditions, and methods. This prevents independent assessment of the central evidence and substantially limits the evaluability of the submission.

  2. The contribution over closely related approaches is insufficiently established. Jain et al. (CVPR 2025) study attraction basins and develop guidance interventions based on their dynamics, making their work directly relevant to both the conceptual motivation and mitigation experiments. Yet it is neither discussed nor compared experimentally. SAIL, the sharpness-based mitigation method of Jeon et al. (ICML 2025), is another relevant missing mitigation baseline. Moreover, Figure 4 compares existing methods before and after adding GeoSteer at particular settings. The reduction in reference similarity is accompanied by reduced CLIP alignment, leaving unclear whether GeoSteer offers a better trade-off than adjusting the intervention strength of existing methods. Comparisons across operating points, including matched text alignment and computational cost, are needed to establish the practical value of the proposed spectral intervention.

[1] Jain, A., Kobayashi, Y., Shibuya, T., Takida, Y., Memon, N., Togelius, J., & Mitsufuji, Y. (2025, June). Classifier-free guidance inside the attraction basin may cause memorization. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 12871-12879). IEEE.

[2] Jeon, D., Kim, D. & No, A.. (2025). Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes. Proceedings of the 42nd International Conference on Machine Learning

Questions for authors
  1. Can you provide complete, readable versions of Figures 1–4, including all axes, legends, and annotations? These figures are necessary to assess the main empirical claims.

  2. How does the proposed basin interpretation differ from Jain et al.’s attraction-basin analysis? For mitigation, are comparisons with their guidance methods and SAIL available at matched CLIP alignment, with runtime reported? Such comparisons would materially affect the assessment of GeoSteer’s added value.

  3. What evidence establishes the correspondence between the empirical spectral transition and the basin entry defined in Section 3.1? In particular, how reliably does the transition identify commitment to memorization, given that a similar spectral pattern occurs in non-memorized trajectories?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.