acceptodds
Under review as a conference paper at ICLR 2027

Local Coverage Governs Memorization in Diffusion Models

Abstract

Memorization in diffusion models is often treated as a global property of the model or dataset. In practice, however, a single diffusion model can simultaneously generate both memorized and novel samples. Which training samples are most likely to be memorized? In this work, we show that memorization is governed by local data coverage. Leveraging the connection between diffusion models and kernel density estimation (KDE), we derive a theoretical criterion that predicts whether a point is memorized based on the density of training data in its neighborhood and the size of the training dataset. In the high-dimensional limit, this leads to a sharp, local transition: regions of low coverage are dominated by isolated training samples, which are memorized, while dense regions support interpolation and generalization. We validate these predictions empirically, showing that memorization increases with local sparsity and that diffusion models exhibit a coexistence of memorized and novel samples within the same model. Extending this framework to multi-class settings, we further show that classes with higher intra-class diversity (and thus lower local coverage) are more strongly memorized. Our results provide a local view of memorization in diffusion models, explaining when and where memorization occurs in terms of data geometry.

Primary area:
learning theory
Keywords:
Memorization, Diffusion Models, Kernel density estimation, Theory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_R6hfOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why memorization varies across regions within the same diffusion model. It develops a point-dependent analysis of kernel density estimation in the high-dimensional regime (P=\exp(\alpha N)). For hard-sphere kernels, it derives a transition governed by local probability mass, and a leave-one-out argument connects low coverage to self-kernel dominance at training points. Experiments with pixel-space U-Nets on CIFAR-10, grayscale CelebA, and a MNIST/CIFAR-10 mixture report increasing nearest-neighbor-ratio memorization with local sparsity. Additional experiments examine class differences, training dynamics, and model capacity. The paper interprets these observations through a finite effective kernel width.

Critical strengths

1.The paper identifies local effective sample size, rather than dataset size alone, as a relevant quantity for understanding which examples are vulnerable to memorization. The hard-sphere analysis makes this intuition explicit through the competition between sample complexity and local coverage. This provides an interpretable framework for discussing the coexistence of copying and novel generation.

2.The experiments examine dataset size, local sparsity, class composition, training duration, and model capacity. The held-out-data dominance baseline is a valuable control for geometric correlations. The training-dynamics observations are particularly interesting and suggest a connection between data isolation and the onset of memorization.

Critical weaknesses

1.The main qualitative conclusion overlaps substantially with prior work. Zhou et al. [1] discuss single-sample dominance, interpolation among nearby samples, and memorization of isolated samples. Chen [2] studies how score smoothing produces interpolation. Zhang et al. [3], already cited in the submission, connect locally sparse data to memorization and locally abundant data to learning shared statistics. These works already provide much of the intuition behind the claim that sparse regions favor memorization while dense regions support interpolation. The local coverage criterion is a useful addition, but the paper should explain more clearly what new predictions it provides beyond these results. In particular, the explicit threshold is derived for hard-sphere KDE, while its connection to Gaussian kernels and trained diffusion models remains largely qualitative. Section 4.1 reports correlations with nearest-neighbor sparsity without directly testing the predicted threshold, and the effective kernel width proposed in Section 4.3 is not directly estimated. This makes the additional predictive value of the theory difficult to assess.

2.The memorization measures are affected by the same distances used to describe sparsity. Sparsity is measured using distances between neighboring training samples, while both local dominance and binary memorization depend on the ratio (d_1/d_2). Even if the distance to the nearest training sample, (d_1), stays fixed, a larger (d_2) increases dominance and makes (d_1/d_2<\tau) easier to satisfy. Thus, isolated samples may receive higher memorization scores without being reproduced more accurately in absolute distance. The authors acknowledge the possible correlation and provide a useful test-data baseline. However, this comparison does not fully separate differences in copying accuracy from differences in training-sample spacing. Moreover, dominance and binary memorization are computed from the same ratio, so their agreement does not provide independent confirmation. Reporting absolute nearest-neighbor distances and an additional copying measure would help establish whether the observed trend reflects more accurate reproduction of isolated training samples.

[1] Smoothing the Score Function to Enhance Generalization in Diffusion Models

[2] On the Interpolation Effect of Score Smoothing in Diffusion Models

[3] Generalization of Diffusion Models Arises with a Balanced Representation Space

Questions for authors

1.What predictions does the local coverage criterion provide beyond the related work above? Can these predictions be tested in a setting where local coverage is known?

2.How does the hard-sphere result extend to Gaussian KDE and the outputs of trained diffusion models? What evidence supports describing training and capacity effects through an effective kernel width?

3.Does the relationship between sparsity and memorization remain when copying is measured using absolute distance or perceptual similarity?

5 comments

We thank the reviewer for their detailed comments and helpful suggestions.

Critical point 1: Relation to prior work and novelty

Thank you for suggesting these relevant works. Our main theoretical contribution is a general characterization of local KDE statistics for arbitrary kernels and data distributions (Theorem 1). This framework describes how individual samples locally compete with the combined contribution of nearby training points as sample size increases. We then test empirically how local geometry predicts differences in memorization risk across training samples. We agree that the qualitative sparsity-memorization connection also appears in Zhang et al. and other cited works. Our contribution makes the joint dependence on local geometry and sample complexity in KDE quantitative; neither Zhou et al. nor Chen provides the local KDE characterization of Theorem 1.

Chen studies how score smoothing promotes interpolation, with the analytical results focusing on uniformly spaced samples on a one-dimensional manifold. Zhou et al. analyze Gaussian-component dominance and develop smoothing methods to reduce memorization. We therefore view these works as complementary: they substantiate the relevance of understanding KDE, while our analysis advances that understanding locally, and now discuss these references in our manuscript. In particular, building on Zhou et al.'s suggestion of geometry-dependent smoothing, we also discuss using our local coverage criterion to guide smoothing as a future direction.

Memorization measure

Thank you for highlighting the possible confound between our memorization metric and local sparsity. We have added experiments using SSCD features (Pizzi et al., 2022), which compare images in the feature space of a pretrained copy-detection model rather than pixel space. We report both raw cosine similarities, which do not involve the second-neighbor distance, and local dominance. Both measures support the hypothesis on CIFAR-10 and CelebA (Section C.5). Together with the held-out-data baselines, these checks provide evidence that the relationship is not simply an artifact of the shared pixel-distance statistics. We did not include raw pixel distances to the nearest neighbor on their own, as these have been found not to be good predictors of memorization on CIFAR-10 before (see e.g. Carlini et al. 2023, Section 5).

Effective kernel width

We would first like to clarify that within our KDE description, a finite effective kernel width is a modeling assumption needed to describe novel generation: as the width approaches zero, the KDE collapses onto the training data (Section 4.3)

However, we noticed several trends in the empirical results in in section 4.3 and section C.4 of the appendix that relate to this effective description.

First, alignment between the trained denoiser and the corresponding KDE estimator increases during training, with agreement strongest at high noise levels and improving subsequently at lower noise levels (Figures 3c and 8). Thus, as training progresses, the model approximates KDE estimators with progressively smaller kernel widths. We evaluate this alignment on both noised training and held-out data.

Second, memorization initially occurs predominantly in sparse regions and later extends to denser regions (Figures 3b and 7). This is consistent with a decreasing effective kernel width, which moves the threshold for single-sample dominance toward regions of higher coverage.

Varying model capacity produces corresponding trends (Figures 8–9): smaller U-Nets retain strong alignment with KDE at high noise levels but show larger discrepancies at low noise levels, while larger models exhibit stronger memorization. Together, these observations support interpreting increased training time and capacity as allowing the model to approximate KDE at progressively smaller effective kernel widths.

Further questions, addressed together

Can these predictions be tested in a setting where local coverage is known? How does the hard-sphere result extend to Gaussian KDE and the outputs of trained diffusion models?

Our main theoretical result, Theorem 1, applies to general kernel functions, including Gaussian kernels, under the stated convergence assumptions. Obtaining a corresponding explicit threshold for Gaussian kernels requires evaluating the kernel statistic appearing in Theorem 1 for that case.

We have also added experiments measuring memorization directly against local coverage (Section C.6). We observe the expected decrease in memorization as coverage increases. Here, coverage is estimated from the available data rather than known analytically. These results provide a more direct empirical check of the coverage dependence predicted by the hard-sphere analysis in trained diffusion models.

We hope these clarifications and additional experiments address your concerns. If so, we would be grateful if you would consider raising your score.

Thank you for the response and the additional experiments. The rebuttal clarifies the mathematical generality of Theorem 1 and the proposed local coverage criterion. However, I'm not convinced that these results provide sufficiently distinct, testable predictions beyond the established relationship between local sparsity, sample dominance, and memorization. In particular, the explicit transition threshold is derived for hard-sphere kernels, while the additional experiments mainly demonstrate qualitative correlations rather than quantitatively validate the predicted transition in trained diffusion models.

Although I think the paper is well-developed and the additional experiments are useful, I still find the incremental novelty and additional insights beyond prior work somewhat limited. Therefore, I will maintain my original score.

Thank you for your follow-up. We are glad that our response clarified the local coverage criterion, and we appreciate your positive assessment of the paper’s development and the additional experiments. We understand that your remaining reservation concerns the significance of the theoretical advance beyond the established qualitative picture and its testable predictions.

We appreciate your acknowledgment of the mathematical generality of Theorem 1. As our paper is primarily a theoretical contribution, we believe that its novelty should be assessed through the mathematical results it establishes relative to prior work. Theorem 1 provides a general local characterization of KDE statistics for arbitrary kernels and data distributions, including their dependence on sample complexity. Its scope extends substantially beyond the specific settings analyzed in the cited works: separated clusters and Gaussian mixtures in Zhang et al., and uniformly spaced training samples on a one-dimensional subspace in Chen (2026). We ask that the paper be judged primarily on the novelty and significance of its contribution to theoretical understanding.

Connecting the theory to trained diffusion models involves several approximations: describing the learned models by an effective KDE, using hard-sphere kernels to obtain an explicit threshold, and taking the high-dimensional limit (N\to\infty) with (P=e^{\alpha N}). The hard-sphere kernel serves as an idealization illustrating the general result, while Gaussian kernels are covered by the framework of Theorem 1. Quantitatively comparing the explicit threshold’s numerical location with trained models would additionally require accounting for these approximations.

Despite these approximations, our experiments support the relevance of the theoretical results. We recognize that qualitative links between sparsity and memorization have been observed previously. However, our experiments also examine a more specific prediction of Theorem 1 than the sparsity-memorization correlation alone: as dataset size increases, memorization recedes first in well-covered regions and persists longest in poorly covered regions.

Reviewer_iSwpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The key message this paper advocates is that memorization in diffusion model is governed by the local coverage of training samples. To theoretically justify this, the authors consider the behavior of KDE as the ambient dimension N tends to infinity. (KDE can be viewed as a proxy of the denoiser at a fixed noise level.) The authors show that there is an interesting phase transition from memorization to generalization locally, depending on certain local statistical coverages. Hence the same model may exhibit both memorization and generalization in different regions of the data space, depending on local data coverage. The paper also provides experiments (on CIFAR-10 and CelebA) to show the relationship between local sparsity in memorization.

Critical strengths

++ I find the local view of memorization natural and potentially useful. It is interesting to study memorization from this perspective.

++ The sharp local phase transition (although established for KDE) is interesting. I like the theoretical analysis and appreciate the general theorem and also the one for the special case of hard-sphere, which helps further clarify the intuitions behind the result.

Critical weaknesses

-- The practical utility of the theoretical results unfortunately is not clear. How can these be used in practice, for example, to develop better local memorization detection or mitigation methods? These are not explored in the paper. That memorization depends on local information is also not entirely new (see the minor comment at the end).

-- The theoretical results have several important limitations: (a) the analysis is asymptotic and assumes that the dimension N tends to infinity; (b) the number of samples need to grow exponentially with N; and (c) the results are established for KDE, and there is a gap between KDE and a learned denoiser.

(The authors do discuss some of these in Section 5, and mention studying the implicit (or explicit) lower-dimensional metric spaces that diffusion models might operate in as a future direction. )

Regarding the relationship between KDE and a learned denoiser, while KDE is certainly a good proxty (as the empirical optimial denoiser at a fixed noise scale is determinated by a KDE), I think the relationship is more nuanced than assuming that it will always be a good approximation. This for example depends on noise level and data distribution: [Ref-1] below observes that practical denoisers are effectively trained primarily within certain Gaussian spherical shells around sampled points, and intuitively the denoiser may instead extrapolate outside such well-supervised regions.

-- The experimental section is not very strong. It shows that local sparsity can correlate with memorization (which is not too surprising); but it is unclear how this might lead to practical guidance or useful strategies. Also, the memorization rates decreases as as the number of samples increase; while practical diffusion models are often trained on very large image datasets. It would be helpful to discuss the relevance of the observed effect in such regimes.

I am also not sure I follow the logic in Section 4.2: higher intra-class variability does not necessarily lead to lower local coverage. A class could have high variability but consist of several locally dense clusters. Furthermore, the discussion above Eqn (18) suggests that locally μin,R\mu_{in, R} is dominated by the locally densest class. So even if a sample is has low local coverage within its own class, it might still have high local coverage due to nearby samples from other dense classes. So it may not memorize this way.

Minor comments:

  • The paper emphasizes that "our results suggest that memorization in diffusion models is not a global binary property". While I like the local analysis, I think the dependency of memorization on local sample coverage is not entirely new. In fact, this also depends on the noise level which will affect the kernel width for KDE. (For example, if the cardinality of the training set is fixed, then as the noise level changes, the number of samples locally contributing will change, and for sufficiently small noise level local sparsity will be low.) The reference [Ref-1] and [Ref-2] both study the interaction of noise level with the geometry and coverage of training samples.

  • In caption of Figure 1, there is an incomplete sentence around line 083/084.

[Ref-1]: "Selective underfitting in diffusion models", Song et al., https://selective-underfitting.github.io/

[Ref-2]: "Two Calm Ends and the Wild Middle: A Geometric Picture of Memorization in Diffusion Models". Dodson et al., https://arxiv.org/abs/2602.17846

Questions for authors

See the "Weakness" above. The key is to provide examples and experiments to demonstrate the practical utilities.

1 comment

We thank the reviewer for their detailed comments and helpful suggestions.

Practical Utility

The practical utility of our results is the following: we have shown, that memorization risk has a meaningful connection to local coverage. Importantly, memorization vulnerability of training data can therefore be evaluated prior to training any diffusion model at all. The implications are especially relevant in the context of highly diverse data, where samples of high sparsity are of particular interest, e.g. in medical imaging. We have also shown empirically that data from sparse regions are more vulnerable to memorization earlier in training, thus we expect these insights to also be relevant in the context of very large datasets on which training encompasses few epochs.

We now additionally discuss two avenues towards practical interventions that arise from our work in section 5. We especially thank the reviewer for suggesting Ref-1, which inspired one of these avenues. We hope that these suggestions further illustrate the practical utility of our contribution.

Finally, we maintain that the main contribution of our paper is a theoretical one. As such, our main goal is to gain an improved understanding of diffusion model behavior through the connection between data geometry and memorization. Many highly influential theoretical works on diffusion models of the past years, including the award-winning contributions by Kadkhodaie et al. at ICLR 2024 and Bonnaire et al. at NeurIPS 2025, do not contain explicit practical interventions.

Novelty and prior work

Our work characterizes when a single training sample dominates the collective contribution of neighboring samples, jointly as a function of local geometry and sample complexity (Theorem 1), and empirically evaluates the resulting differences in memorization risk between training samples. Neither of these references provides the local characterization of KDE established in Theorem 1, nor empirically evaluates how local geometry predicts individual training samples’ vulnerability. These papers make use of KDE from complementary perspectives:

Song et al. investigate where learned scores approximate the empirical score and where they extrapolate. Our analysis could further distinguish supervised regions dominated by individual samples from those governed by collective contributions.

Dodson et al. study at which noise levels single-sample dominance is most conducive to memorization. Their Gaussian-shell coverage is different from our pointwise local coverage: it measures how much of the noisy population distribution falls within training supervision regions.

We thank the reviewer for suggesting these references, which reinforce the relevance of a more precise characterization of KDE. We discuss these works and connections in the revised manuscript and have clarified the positioning of our contribution in the introduction, including the sentence mentioned in the reviewer's minor comments.

Applicability of the KDE hypothesis

KDE is a powerful hypothesis on diffusion models, ubiquitous across theoretical and empirical studies. Our empirical results, both those originally included in the submission and those added in response to the other reviewers comments demonstrate its usefulness for predicting memorization risk for single samples, which is the purpose of our study. Beyond this, we provide several ablations that address this point in Section 4.3 and the appendix C, including new experiments that include a more direct test of our hypothesis.

Intra-class variability

We agree with this point, thank you for pointing it out. In the setting where a class consists of few highly concentrated clusters, we do not expect higher levels of memorization. In the context of these results, the relevant distinction is higher average sparsity within a class (as we observe e.g. in the car and truck classes in the CIFAR-10 data). On the level of individual samples, the prediction remains the same: samples in high sparsity regions are more prone to memorization. We have clarified in the abstract and the corresponding text that the decisive quantity is higher average sparsity within a class, which must not necessarily be induced by higher intra-class variability.

Assumptions and scope

While the assumptions (the high-dimensional limit and exponential sample-size scaling) do limit the regime of our theoretical guarantees, within this regime, Theorem 1 accommodates general kernels and data densities, substantially broadening the scope of existing KDE analyses. Moreover, our finite-dimensional experiments support the relevance of the resulting predictions to trained diffusion models.

We hope these clarifications and our additional discussion of practical utility address your concerns. If so, we would be grateful if you would consider raising your score.

Reviewer_VnogOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors study the effect of local data sparsity in diffusion model memorization.

The authors find that the empirical denoiser in diffusion models behaves as a KDE estimation with Gaussian kernels, where local sparisty/coverage matters for the estimations. From this connection, the authors analyze the log density from the KDE. By approximation, the authors prove that it either converges to a sample-agnostic limit or to −∞-\infty (zero Prob.) asymptotically. The authors show that the estimated density has a thresholding form, and when the data is locally sparse around a sample, the density is only determined by that sample itself and therefore memorization, and otherwise generalization.

Empirically, the authors measure local sparsity of a sample with the average squared distance to nearest training neighbors. They plot the sparsity levels of data from different classes in CIFAR10, and find that the samples with high local sparsity tend to be more memorized, while the classes with higher average sparsity are also generally more memorized. They finally discuss the effect of diffusion training in their KDE analysis, and argue that a better-trained diffusion model has a smaller effective kernel width and approximates the empirical denoiser better.

Critical strengths
  1. The authors manage to formalize and study an important factor in diffusion memorization and generalization, which is the local data geometry.
  2. The theory is rigorous and understandable, and the authors conduct compatible experiments for them.
Critical weaknesses
  1. The authors employ a different definition of diffusion memorization and generalization than previous papers. Here the authors seem to study memorization and generalization in the estimated density, while normally previous paper study the sampled data distribution. Also, the analysis here is asymptotic and requires approximations.
  2. The experiments, though self-contained, are relatively small-scale. Also, Fig.2(e) does not seem to show a perfect alignment or sharp transition as the theory indicates.
Questions for authors
  1. How well does NN2NN1\frac{NN_2}{NN_1} or the average squared distance to nearest training neighbors measure local sparsity? How does it align with your theory or with other metrics?
  2. Can the KDE analysis be applied to the case of data duplication, which is also a cause for memorization?
4 comments

We thank the reviewer for their positive evaluation of our work. We have added an experiment addressing the reviewer’s second critical point and clarifications that address the remaining points raised in the review.

Critical Point 1: Definition of Memorization

We do define memorization as reproduction of generated samples in the generated samples, consistent with the literature. Our experiments also measure generated samples, not the estimated density. However, we don't distinguish between estimated density and sampled density, as we approximate them to be equal. Given the agreement between the experiments and the predictions by the theory, we believe that this approximation is reasonable in the context of our results. Thank you raising this subtle point. We have added a clarifying paragraph in Section 2.

Critical Point 2.1: The scale of the experiments

In total, in the manuscript, we report the results of over twenty separately trained diffusion models of different architectures and on different datasets, we have extensively validated our results on over one hundred different models (to validate our hypothesis on different data complexities), each of a size between 2,132,13 million and 33,1833,18 million parameters. Moreover, for each diffusion model, we have generated at least 40,00040,000 samples. Together, experiments have required several tens of thousands of GPU hours. In response to some of the reviewer comments, we have further evaluated several ablations. We believe that the scale of the experiments is adequate for a theoretical work, especially since across all datasets, we observe consistent trends with the hypothesis.

Before we answer the second question in this critical point, we briefly address one of the reviewer's questions as they are related.

Q1: How well does NN2NN1\frac{NN_2}{NN_1} or the average squared distance to nearest training neighbors measure local sparsity?

We distinguish three quantities: local coverage is the probability mass within a neighborhood, as defined in our theory; local sparsity is measured empirically by the average squared distance to nearby training examples, with larger distances indicating lower coverage at the corresponding neighborhood scale. This is a proxy rather than a direct estimate of coverage. The ratio NN2/NN1\mathrm{NN}_2/\mathrm{NN}_1 measures memorization, not sparsity: larger ratios indicate that a generated sample is much closer to one training example than to its competitors. In response to another reviewer’s comments, we have also added an alternative memorization measure.

Our theory predicts that memorization should depend on local coverage ln⁡pin,R(x)\ln p_{\text{in},R}(x), that measures the probability that another sample is within radius RR of xx, intuitively, how closely packed samples are around xx.

In practice, measuring ln⁡pin,R(x)\ln p_{\text{in},R}(x) is possible, but the results are very sensitive to the radius RR, which we assume to be finite (see section 4.3). We therefore use local sparsity as a proxy to determine how densely/how sparsely packed samples are locally. We have added a figure that compares ln⁡pin,R(x)\ln p_{\text{in},R}(x) over a range of radii and shows that the two quantities have the expected inverse relationship: local coverage increases as local sparsity decreases, and vice versa.

Critical Point 2.2: Gradual vs abrupt increase in memorization

Figures 1 and two use local sparsity as a proxy for (inverse) local coverage. If one instead uses local coverage, which is the relevant order parameter in the theory, one can recover a more abrupt behavior as expected. We have added the corresponding figures in the appendix (Figures 14 and 15) and further discuss this point in the main text. We would also like to note, that we expect a sharp transition only in the infinite-dimensional limit, for finite-dimensional experiments, fluctuations can "smear out" the transition. Our main motivation for using local sparsity instead of local coverage is that local sparsity is a little less sensitive to parameter choices such as the choice of the order k of the nearest neighbor and the radius.

Q2: Can the KDE analysis be applied to the case of data duplication, which is also a cause for memorization?

Yes, and we thank the reviewer for this interesting question. For example, one can make a modified version of the leave-one out argument where the self-kernel is doubled, which would correspond to conditioning onto a twice-duplicated sample. It would then be interesting to see that there could be situations in which the self-kernel dominates even in the case where there are many nearby training data. We have not yet added this analysis to the manuscript, but we expect to do so by the end of the rebuttal period.

We hope these clarifications and additional experiments address your concerns. If so, we would be grateful if you would consider raising your score.

Thank you again for your helpful question about data duplication, we have added an analysis of this setting in appendix B.5. There, we explain how duplication increases an individual example contribution relative to its neighbors and may thereby dominate the KDE estimator locally, leading to memorization. We have also showed how to extend theorem 1 to account for a random number of repetitions per training example. We hope that this addition answers your question.

I thank the authors for their dedicated rebuttal, and I also appreciate the localized perspective on diffusion memorization and generalization that the authors proposed, as well as their effort in enriching the paper (e.g. the data duplication case).

I now understand the metrics more clearly, where the authors use local sparsity as a less hyperparameter-sensitive metric instead of local coverage. However, both metrics are unnormalized, and I am a bit concerned about their transferability and applicability to real-world large models.

Therefore, I lean towards keeping my original score, which is already positive.

Thank you for your thoughtful follow-up and positive assessment of our contribution and additions.

We agree that transferring quantitative predictions across datasets and models requires some care, but it can be managed: our theoretical criterion accounts explicitly for dataset size as the transition is governed by whether the expected number of neighboring samples, Ppin,R(x)P p_{\mathrm{in},R}(x), is small or large. Local coverage pin,R(x)p_{\mathrm{in},R}(x) is a probability mass and thus normalized. Choosing the radius RR is indeed something that must be done for each setting individually. In our work we first tried a set of exponentially spaced radii and then chose those that yielded a large range of values for the given training set (if RR is too small or too large,pin,R(x)p_{\mathrm{in},R}(x) can collapse to zero or one, respectively). Alternatively, one can use local sparsity, which is more robust to hyperparameter selection, though it could be affected by data duplication. In the case of heavy data duplication we would recommend the use of local coverage (which is usually computed over a wider range of close neighbors and thus more robust). We hope this clarifies how the measures can be applied beyond the particular settings studied.

We appreciate your recognition of the localized perspective and the additional analysis.

Comments on the paper

2 comments
Area_Chair_TYwVOpenReview ↗

Reviewer-author discussion kickoff

Dear Reviewers,

Thank you for your review. The authors have now provided their rebuttal. We would appreciate it if you could take some time to read their responses and engage in the discussion, particularly on whether your concerns have been adequately addressed.

Thank you for your time and contributions to the review process!

AuthorsOpenReview ↗

Summary of the discussion

We would like to briefly summarize the discussion and rebuttal process below.

We thank all reviewers for their constructive engagement, which helped clarify the scope, novelty, and empirical support of our contribution. The reviewers appreciated the mathematical generality and understandability of our results, and deemed the local theoretical perspective natural and the sharp transition in memorization it predicts interesting. Our rebuttal addressed several concerns about the metrics we used to determine memorization and local sparsity through additional experiments that the reviewers acknowledged as useful and strengthening the paper. We are also grateful to the reviewer for helping us clarify the the novelty and scope of our paper.

In response to the reviews, we provided additional experiments addressing questions about the memorization metrics, the relationship between local sparsity and local coverage, in addition to the existing experiments on the effects of dataset size, training time, and model capacity. Our empirical investigation covers CIFAR-10, CelebA, and an MNIST/CIFAR-10 mixture across three U-Net sizes. Furthermore, we provided an additional analysis that shows how the theorem 1 can be extended to account for data duplication. We also clarified the distinction from prior work: Theorem 1 characterizes local KDE statistics for arbitrary kernels and data distributions, while the hard-sphere specialization yields an explicit local criterion for single-kernel dominance and its dependence on dataset size. The connection to trained diffusion models is investigated empirically.

We were particularly pleased that reviewer Rh6f deemed that our paper " is well-developed and the theoretical analysis has merit", reviewer iSWP said that they "like the theoretical analysis and appreciate the general theorem" and reviewer Vnog appreciated "the localized perspective on diffusion memorization and generalization that the authors proposed, as well as their effort in enriching the paper".

The remaining reservations primarily concern the standard of empirical evidence needed to establish practical utility. The reported experiment encompass over twenty different diffusion models, and showed consistent trends across three training data sets such as CIFAR-10, greyscale CelebA, and an MNIST/CIFAR-10 mixture and across three different sizes of trained U-Nets. We acknowledge that further experiments on larger-scale models and datasets beyond the settings studied here would be valuable. We nevertheless believe that the theoretical generality, local predictive perspective, and empirical evidence constitute a substantive contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.