acceptodds
Under review as a conference paper at ICLR 2027

Beyond Sample Copying: Structural Memorization in Diffusion Models

Abstract

Diffusion models generalize well in practice. Paradoxically, an optimal diffusion model fully memorizes the training data and therefore fails to generalize, raising the question of what induces generalization in a real diffusion model. We show that diffusion models progressively overfit the denoising training objective, creating a generalization gap between validation and training performance at intermediate noise levels. In a fully analytic 2D toy model with a controlled denoising error, we trace this gap to the interaction between model error and the density of the data distribution's support. The optimal denoising flow field localizes sharply around individual training points, whereas model error suppresses exact recall of training points, yielding a smooth, generalizing flow field. Finally, we examine how training-time overfitting manifests along inference trajectories. We find that predictions made from intermediate trajectory states occupy a distinct feature-space regime from predictions made from noised training and validation images. As training progresses and model size increases, these predictions develop greater relative affinity to the training data, despite the absolute similarity to the validation data not decreasing. Together, these findings show that the denoising objective and inference trajectories express structural memorization differently.

Primary area:
generative models
Keywords:
Deep Learning, Generative Models, Diffusion Models, Generalization, Memorization, Image Synthesis, Toy Model, Flow

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Z7fYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work studies momorization and generalization in diffusion models. The authors introduce the noise dependent generalization gap, a metric that subtracts the training from validation reconstruction error. The authors then consider a 2D, fully analytic toy model, and qualitatively reproduce the phenomena of image diffusion models. The authors then continue to study "training affinity", how close the denoiser outputs are to training elements in the dataset. Finally, the authors consider what happens in class conditional diffusion models.

Critical strengths
  • I appreciate the effort to dive deep into the "generalization mystery" of diffusion models, which has been discussed by ample papers, but no satisfying solution has been found yet.
  • I like that the authors use a combination of empirical results, as well as an analytic model to build intuition
Critical weaknesses
  • Overall I was hoping for stronger conclusions, or perhaps suggestions on how to improve diffusion models based on existing results. It would be great if the authors could address this in the rebuttal.
  • The amount of content in the paper seems limited for a full conference submission.
Questions for authors
  • Why do the plots go negative (Fig 2a/b, Fig 4b)? It implies that in the high and sometimes low noise regime, the validation loss is lower than the training loss, why? I would expect them to converge on the same value, because at the endpoints the predictions are trivial (either dataset mean, or closest training sample)
  • Can the authors define "traj" samples, how are they different from the train and val ones? Do the authors mean these are reverse process samples (traj) vs forward process samples (train/val)?
  • At what sigma is figure 6 taken, and why is the train affinity of the val and train sets constants with training images seen? I would expect it to change since it the metric includes the predictor y_theta for train/val too.
1 comment
AuthorsOpenReview ↗

Reply to reviewer Z7fY

We thank the reviewer for the positive assessment and the helpful questions.

W1: Stronger conclusions and implications. We agree that concrete improvements regarding generalization would have been great. The goal of our paper was to shed light on the problem of memorization/generalization in diffusion models from what we see as an underexplored angle: Structural memorization. As you mentioned, this is a notoriously difficult problem. We see the value of our contributions

  1. in the insight that the significant levels of overfitting we measure during training do not transfer to generated samples, within the resolution of our test,
  2. in deepening the understanding of how the training generalization gap arises (toy model), and
  3. in showing that strong conditioning and guidance, which are both very successful for diffusion models, have non-trivial interactions with memorization behavior, which is an underexplored problem.

W2: Amount of content. The revision adds null and positive controls, a calibrated memorization test across five EDM2 sizes and CIFAR-10, and a train/val decomposition of the toy model.

Q1: Why does GG go negative? The endpoint values are not model effects: You can see in Fig.13 and Fig. 15 that these tails are identical across all checkpoints and model sizes (e.g., ImageNet-64: −0.010-0.010 at σ=80\sigma=80). At high σ\sigma, the prediction is the (class) mean, so the error measures how spread out each image set is around that mean. At low σ\sigma, a real model passes the input through, so the error is again determined by the images, not the model. Because the two splits differ slightly (the ImageNet validation set is known to be shifted), GG converges to small split-dependent constants of either sign rather than to zero. For ImageNet-512, this happens to be negative at both tails, and for ImageNet-64 it's negative at high σ\sigma and positive at low σ\sigma. The model-dependent signal is the growth of GG over training at intermediate σ\sigma.

With the toy model, the symmetries in the data and how they interact with the noise level can lead to a negative gap. Both tails eventually converge to 0, as the splits are i.i.d. here.

Q2: Definition of "traj". Yes. Trajectory queries are the intermediate states x(t)\mathbf{x}(t) of the reverse process, i.e., the states the EDM sampler visits while denoising pure noise into a sample, fed to the denoiser at their own σ(t)\sigma(t). Training and validation queries are real images, forward-noised to the same σ\sigma. We will make this more explicit in Sec. 4.

Q3: σ\sigma in Fig. 6, and why are the train/val curves flat? σ=0\sigma = 0. We will clarify that in the revised PDF. There, the model is not called: trajectory queries are the final samples, and training and validation queries are clean images. Their affinity depends only on the images and reference sets, so it is constant across checkpoints by construction. This observation, together with Reviewer kxpp's, led us to the calibrated test described in our reply to kxpp (will be posted soon, if it is not yet).

Reviewer_kxppOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how memorization shows up in diffusion models beyond verbatim copying of training samples. The authors introduce “structural memorization” for a systematic difference in how a model treats training versus unseen data, and traces it at three levels: the training objective, an analytic toy model, and inference trajectories.

First, the authors measure the one-step denoising error on noised training and validation images at matched noise levels, and summarize the difference as a relative generalization gap, G(σ)=(Lval(σ)−Ltrain(σ))/Ltrain(σ)G(\sigma) = (L_{val}(\sigma) - L_{train}(\sigma))/ L_{train}(\sigma)). Using released EDM2 checkpoints on ImageNet-64/512 and EDM models on CIFAR-10/100, they report that G(σ)G(\sigma) peaks at intermediate noise levels, grows roughly linearly with training, increases with model size, and decreases with dataset size. For large models, validation loss at intermediate \sgima\sgima eventually rises while training loss continues to fall.

Second, to explain the shape of this gap, they compute it in a 2D toy model with data on a circle, following Adaloglou et al., where model error is simulated by smoothing the data distribution with Gaussian noise of variance δ2\delta^2. In closed form, the gap vanishes at high noise, where all predictions collapse to the data mean, and at low noise or high model error, where the error is dominated by the input noise. It is largest at intermediate noise, where the optimal flow field localizes around individual training points. The toy model predicts larger gaps for sparser data, finer conditioning, and moderate guidance weights. The authors test these predictions on real models, finding that finer pseudo-class conditioning on CIFAR-10 enlarges the gap, and that CFG and autoguidance enlarge it at moderate weights but shrink it at high weights.

Third, because intermediate sampling states have no matched clean target, the authors compare model predictions from three query types (noised training images, noised validation images, and intermediate trajectory states) against training and validation reference sets, using DINOv2 kNN cosine similarity. They define a train-affinity score BkB_k as the difference in similarity to the two reference sets. They report that trajectory-based predictions are less similar to both reference sets than predictions from noised real images, and that their train-affinity is positive and grows with training and model size, while their similarity to validation data does not decrease. An appendix experiment shows that restricting the denoiser’s receptive field shrinks the gap, which the authors read as evidence that long-range pixel dependencies are preferentially memorized.

Critical strengths
  1. A simple, well-executed, σ\sigma-resolved measurement applied uniformly across two model families, many sizes, four datasets, and many checkpoints. The resulting picture is clean and consistent (peak at intermediate σ, near-linear growth with images seen, eventual validation-loss increase even for SOTA ImageNet models), and the compute cost is tiny compared with FD-based metrics (Table 1).

  2. The error-prone denoiser (Eqs. 8–10) gives a transparent, closed-form account of why the gap vanishes at both ends of the noise range and is largest in between, and the toy model’s predictions about N, conditioning granularity, and guidance are actually tested against real models, with the failure case (high guidance weights) reported honestly.

Critical weaknesses
  1. The relative train-affinity BkB_k does not isolate memorization, because the null hypothesis Bk=0B_k = 0 for non-memorizing behavior is never established and the paper’s own data suggest it is false. In Fig. 6(b), noised training queries sit at B≈+0.010B \approx +0.010 and noised validation queries at B≈−0.007B \approx −0.007, and both are flat across all checkpoints. At low σ\sigma the one-step prediction is essentially the input image, so these offsets are properties of the images and reference sets, not of the model. A positive B for train queries and a negative B for val queries is exactly the signature of two reference sets that are not exchangeable, which is plausible for ImageNet, whose validation set is known to differ in distribution from the training set, and which would also arise from unequal reference set sizes or per-class counts (50 val images per class versus up to 1280 train images, if full sets are used). Under such a shift, a model that simply learns ptrainp_{train} better will generate samples whose B approaches the train-query value, which is precisely what the trajectory curve does (rising toward +0.010). The reported trend is therefore equally consistent with improved generalization to the training distribution as with memorization. The clean control is a same-distribution held-out set (train on a split, use the unseen split as DvalD_{val} reference), or at minimum two disjoint halves of DtrainD_train as the two reference sets to measure the baseline. Without this, the headline claim about inference trajectories is not supported.

  2. Modest novelty and an overclaimed mechanism. A train/validation loss gap is the most basic overfitting measure; the concentration of memorization at intermediate σ\sigma is already established (Dodson et al. and the statistical-physics line of work the paper cites); and the toy model, Eq. 8, and the affine-mixing interpretation are taken from Adaloglou et. The genuinely new mechanistic content is thin, and the claim that model error “causes” the gap is backwards relative to the paper’s own mathematics (the optimal denoiser has the largest gap; error only shapes it). The toy model’s isotropic Gaussian δ\delta is also an assumed, not learned, form of error, and the paper’s own Appendix D shows that real network error is structured (locality), which the toy cannot capture. The paper reads as a careful observational study whose explanatory layer is weaker than the framing suggests.

Questions for authors
  1. What are the sizes and per-class counts of DtrainD_train and DvalD_val used as reference sets in Eq. 11? Were they matched? If not, please recompute BkB_k with matched sizes.

  2. Please report BkB_k for noised validation queries at a very early checkpoint, and BkB_k computed with two disjoint halves of DtrainD_{train} as the two reference sets. If the former is non-zero or the latter is near zero while train-vs-val is not, the ImageNet train/val shift is driving the offsets.

  3. Can you run the same-distribution control (train on half of CIFAR-10 or a subset of ImageNet-64, use the held-out half as DvalD_{val} reference)? CIFAR runs are cheap per Table 1. Does Bk(traj)B_k(traj) remain positive and growing?

  4. Which σ\sigma is used in Fig. 6? The L2 gap peaks at σ=1 2\sigma = 1~2, but Fig. 5(c) shows the trajectory affinity is near zero there and only grows at low σ. How do you reconcile the claim that the two measures are “consistent” when they live at different noise levels?

  5. Does the toy model reproduce the rise in LvalL_{val} (classical overfitting) as δ\delta decreases, or only the rise in G? Please show Ltrain(σ)L_{train}(\sigma) and Lval(σ)L_{val}(\sigma) separately for the toy.

2 comments
AuthorsOpenReview ↗

Reply to reviwer kxpp (1/2)

We thank the reviewer for the careful, constructive review. A revised version of the manuscript will be uploaded before the end of the discussion period; however, we address the review's concerns here.

W1: BkB_k does not isolate memorization. We agree, and ran the suggested controls.

  • Null: With two disjoint halves of the same split as references, B20B_{20} vanishes for all query types (∣B20∣≤0.002|B_{20}| \le 0.002, ImageNet-64 and CIFAR-10). The metric itself is unbiased.
  • Split shift: The ImageNet-64 offsets of clean training (+0.010+0.010) and validation (−0.007-0.007) images are 6–10× the null. As you suspected, they reflect a real difference between the splits (FD$_\text{DINOv2}$ train vs. val: 20.6, vs. a train-vs-train floor of 13.5).

To remedy this issue, we made two adjustments to the metric and analysed the results.

  • Calibrated test: The right baseline for a non-memorizing model is not B=0B=0 but the affinity of new images from the training distribution, i.e., training images outside the reference set (denote that BgenB_\text{gen} for 'generalizer'). As the baseline for memorization, we measure the level of a model that copies random training images (BmemB_\text{mem}). The calibrated score becomes: Mk=(Bk(traj)−Bgen)/(Bmem−Bgen)M_k = (B_k(\text{traj}) - B_\text{gen}) / (B_\text{mem} - B_\text{gen}), with 0 = generalizer and 1 = memorizer.
  • Separating fidelity and memorization: Because BgenB_\text{gen} is computed from real images, it corresponds to high-fidelity images. If generated samples have lower fidelity, their calibrated score is lower. Therefore, MkM_k mixes fidelity and memorization. We account for this by computing BgenB_\text{gen} from training images degraded to the samples' fidelity: we noise them and denoise them in one step (with the model of Bk(traj)B_k(\text{traj})), choosing the noise level so that their similarity to DvalD_\text{val} (which cannot contain copies) equals that of the samples.
  • Result: These two corrections show two things: (i) our metric detects no structural memorization, i.e., no systematic bias toward the training set, in the generated samples, and (ii) the rise in BkB_k for longer training times is fully explained by the samples’ rising fidelity, as opposed to an increase in memorization. The samples of all five EDM2 sizes and of CIFAR-10 sit at the generalizer level (∣M1∣≤0.12|M_1| \le 0.12, all 95% CIs include 0) for all checkpoints (except the very first CIFAR-10 checkpoint). This holds while GG on CIFAR-10 grows from 0.01 to 0.23. Your reading is therefore correct: the rising trajectory affinity reflects learning the training distribution, not memorization. We revised the claim of Sec. 4 accordingly: within the resolution of the test, overfitting on the denoising objective does not translate into memorization in the generated samples.

Validating the new metric:

  • Since the degrader was trained on the images it degrades, its own overfitting could in principle raise BgenB_\text{gen}. CIFAR-10 argues against a large effect: despite the largest gap (G=0.23G=0.23), degraded training images have a lower affinity than clean ones (−0.0001-0.0001 vs. +0.0008+0.0008), not a higher one.
  • Does the metric detect memorization? A positive control (replacing a known fraction ff of samples by copies) raises M1M_1 by ff. It reliably detects 5% copies on CIFAR-10 and 10–20% on ImageNet-64.
  • The metric has useful resolution only for k=1k=1. Diffuse proximity to many training images is not resolved: larger kk point in the same direction, but with wide confidence intervals.

W2: Novelty and mechanism.

  • We agree that "model error causes the gap" is backwards. We will correct the wording, including the Sec. 3 title.
  • We agree that isotropic δ\delta is an assumed form of error, while real model error is structured. As all models are, the toy model is wrong, however, we still believe this form of error to be useful to the analysis. We will make this limitation more explicit though.
  • The toy model and Eq. 8 build on Adaloglou et al., and the concentration of memorization at intermediate σ\sigma is known. Our contribution is (i) a σ\sigma-resolved measurement showing classical overfitting (rising validation loss) in released SOTA EDM2 checkpoints, across sizes, datasets, and training time, at a fraction of the cost of FD; (ii) toy-model predictions (data density, conditioning granularity, guidance) tested on real models; and (iii) the calibrated test above, which shows that this overfitting does not carry over to the generated samples.
AuthorsOpenReview ↗

Reply to reviewer kxpp (2/2)

Q1: Reference-set sizes. They are matched: 50 images per class in both sets on ImageNet-64 (50k each), 1000 per class on CIFAR-10 (10k each). Queries: 10 per class. The half-split controls use 25 and 500 per class, with candidate counts equalised after self-exclusion.

Q2: Validation queries at an early checkpoint; disjoint halves. Fig. 6 is at σ=0\sigma=0, where the model is not called (see Q4), so the validation-query value (−0.007-0.007) is the same at every checkpoint, including the earliest. With two disjoint halves of DvalD_\text{val}, ∣B20∣≤0.002|B_{20}| \le 0.002, while train vs. val is not. As you predicted, the ImageNet split shift drives the offsets.

Q3: Same-distribution control. The CIFAR-10 test set is an i.i.d. held-out split of the same distribution, which our controls confirm: new validation images land at the generalizer level. CIFAR-10 therefore serves as this control without retraining. As above, it shows no train affinity beyond the generalizer level in the samples, despite a large and growing objective-level gap.

Q4: Which σ\sigma in Fig. 6? σ=0\sigma = 0: trajectory queries are the final samples, and training and validation queries are clean images, which is why those two curves are flat. We agree that "consistent" was not justified. After calibration, the two measures disagree: the objective-level gap grows, while the train affinity of the samples does not exceed the generalizer level. We will remove the claim.

Q5: LtrainL_\text{train} and LvalL_\text{val} in the toy model. At the peak σ\sigma, as δ\delta decreases toward the optimal denoiser, LtrainL_\text{train} falls and LvalL_\text{val} rises monotonically. The toy model thus reproduces classical overfitting, not only the rise in GG. We will add this panel to the appendix.

Ethics flag. The Author Contributions section is removed in the revision.

Reviewer_YDFDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The Author Contributions section (page 10, lines 535–539) reveals the authors' names, which violates ICLR's double-blind policy. I am thus obliged to recommend rejection. I have not assessed the scientific content of this submission, so the scores do not reflect an evaluation of this paper.

Critical strengths

N/A

Critical weaknesses

Double-blind policy violation

Questions for authors

N/A

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised manuscript and summary of changes

Dear reviewers and AC,

Thank you all for your time and your thoughtful reviews. Your feedback substantially improved the paper.

We have uploaded a revised version. All changes are highlighted in color. The main ones are:

  • Section 4, rewritten around a calibrated memorization test. The relative train-affinity is now placed between a perfect generalizer (training images degraded to the samples' fidelity) and a perfect memorizer (copies of training images). Across all EDM2 sizes on ImageNet-64 and EDM on CIFAR-10, the samples stay at the generalizer level throughout training, even where the generalization gap grows. We revised the claims of the abstract, introduction, and conclusion accordingly.
  • Validation of the metric (Appendix B.4): null controls, a same-distribution control, a positive control with injected copies, and robustness to the choice of degrader and of kk.
  • Toy model: we credit Adaloglou et al. more clearly, name isotropic model error as a limitation, and add training and validation errors separately (Appendix E).
  • Reproducibility: an improved Appendix B with the full experimental setup.

We replied to the individual reviews on Friday. We would be glad to hear whether the replies and the revision address your concerns, and we welcome any further questions or suggestions that could help improve the paper.

Best regards, The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.