acceptodds
Under review as a conference paper at ICLR 2027

Diagnosing and Mitigating Memorisation in Generative Models with Metric Space Magnitude

Abstract

Memorisation in deep generative models is often reduced to the question of whether a similarity threshold is crossed. Current successful approaches typically analyse sample-wise comparisons between generated outputs and individual training examples. In this work, we propose a framework based on metric-space magnitude, an invariant that measures the effective number of distinct samples across multiple scales, providing a notion of diversity. For example, in an image embedding space, similar images contribute less effective diversity than distinct ones. By comparing the effective diversity of the training and generated sets and their combination, we shift from pointwise copy detection to a set-level description of how strongly and in what form the generated distribution is absorbed into the training set. We apply our framework to image, 3D, and neural-network parameter generation, where it provides a richer quantitative account of how memorisation is expressed in these settings, for example, whether it is broadly distributed over the training set, concentrated in a small part of it, or accompanied by self-collapse. This broader view remains consistent with the main findings of previous analyses. Finally, we use our approach to construct post-training interventions for image and face generation, substantially reducing memorisation while avoiding undesirable geometric side effects.

Primary area:
generative models
Keywords:
Metric-space Magnitude, Deep Generative Models, Memorisation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xJvEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper uses multiscale metric-space magnitude to measure training-set absorption, coverage deficit, and internal redundancy relative to a healthy counterfactual population. It evaluates these diagnostics on image, 3D-shape, and neural-network generation, then develops counterfactual-calibrated post-training interventions for image copying and identity memorisation.

Critical strengths
  1. The framework distinguishes several geometrically different failures that a copy count alone cannot describe. Controlled experiments with planted copies, displaced modes, and appended near-duplicates provide useful evidence for the intended interpretations.
  2. Evaluating neural-network generators in parameter, functional, and representation spaces reveals meaningful distinctions. HR’s strong parameter-space absorption is substantially weaker in the other spaces. This demonstrates the value of examining memorisation beyond a single representation.
  3. The mitigation experiments use held-out prompts or identities, fresh generation seeds, retention-only controls, and evaluation representations distinct from those used for optimisation.
Critical weaknesses
  1. The image diagnostics compare MEMBENCH-triggered generations against COCO-caption generations without establishing that these populations are semantically matched. Consequently, differences in subject matter or prompt diversity could affect absorption and redundancy; matched non-copying controls and counterfactual-sensitivity experiments would strengthen the memorisation interpretation.
  2. The claimed recovery of the 3D guidance effect needs further evidence. Appendix C reports that the positive absorption change disappears in every leave-one-category-out evaluation and depends on cross-category generated–generated affinities. Validation with known copying levels under varying category mixtures is needed to establish whether this reflects memorisation rather than sensitivity to population composition.
Questions for authors
  1. Since 150 MEMBENCH prompts correspond to only 63 unique reference images, do the 24 training prompts and 12 held-out prompts share any target images? Please clarify whether Table 9 demonstrates transfer to unseen memorised images or to unseen prompts potentially targeting familiar images.
  2. How are magnitude computations handled at t=0, where the similarity matrix is singular, and when exact duplicates occur in the controlled experiments?
  3. Are the reference photographs in the face experiment verified training examples, or merely photographs of the named individuals?
6 comments
AuthorsOpenReview ↗

Response to W1. MemBench generations are compared with COCO-caption generations without semantic matching.

We use COCO-caption generations as the counterfactual for three reasons.

  • Absorption. COCO captions are the ordinary-prompt set of MemBench itself. Their generations share nothing with the memorised images, so the magnitude they share with the reference set is as close to zero as possible.

  • Redundancy. We take the self-redundancy of COCO generations as that of a healthy set of prompts.

  • Fixed counterfactual. We base our memorisation reading on comparisons that keep the counterfactual fixed. For example, in Table 1 we measure the original and the post-trained model against the same counterfactual. We read the coverage deficit in the same relative way.

We do not use MemBench prompts without detected copies as a counterfactual set because MemBench construction is based on the prompts' susceptibility to memorisation. We therefore consider COCO captions as a reference of healthy generations.

AuthorsOpenReview ↗

Response to W2. The claimed recovery of the 3D guidance effect.

We no longer claim that absorption reproduces the moderate-guidance peak, and we removed the leave-one-category-out analysis. We revised two points (Appendix C.2, Table 9).

  • Distance. The Light-Field Distance is not of negative type. We now compute the magnitude measures on a Euclidean embedding of this distance.

  • Training reference. The submitted analysis used a subsample of each category’s training set, which omitted several of the shapes the prompts were taken from. We now use every training shape of the category that we could extract, and we keep the generations whose source shape is in it: the same 767 at every guidance scale.

We find the following in the revised analysis.

  • Z_U now correctly reproduces the peak reported by Pu et al. (2026): it is strongest at w = 3.

  • Absorption, coverage deficit and redundancy decrease monotonically with guidance. The generated set is most absorbed and most redundant at w = 1, and least at w = 10.

  • We give the per-category results in Table 10.

In Figure 11 we illustrate why our metrics trend differs from Pu et al.(2026). Z_U reduces every generated shape to the distance to its single nearest training shape. Our measures respond to how many training shapes surround a generation. At w = 1 the generations are similar to the training set in a way the nearest-neighbour distance does not show. The nearest training shape is no closer than at w = 10, but each generation is close to many training shapes at once (Figure 11). Z_U therefore reads w = 1 as non-memorising, while the set-level measures register this proximity as the highest absorption and redundancy of the sweep. Pu et al. do not treat w = 1 as a good setting either. They report that generation without guidance gives low memorisation but poor generation quality.

AuthorsOpenReview ↗

Response to Q1. Do the 24 treated and 12 held-out prompts share target images?

Yes, partly. The 36 prompts correspond to 21 training images.

  • Eight of the 12 held-out prompts share a training image with a training prompt.
  • Four held-out prompts do not. They correspond to three images that have no training prompt targets.

We therefore show transfer to unseen prompts and, on the four image-disjoint prompts, to unseen memorised images. Held-out copies fall from 34 to 12, 8 and 8 in the three runs. In each run, at least 12 of the removed copies come from two of the four image-disjoint prompts.

Reviewer_QbmMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

In the paper, the authors propose a framework built on the magnitude of metric spaces, a quantity that captures a notion of diversity. The resulting fingerprint distinguishes three complementary geometric effects: training-set absorption (μ\mu), coverage deficit (κ\kappa), and internal redundancy (ρ\rho). All three are derived from a single construction that applies across different metric representations. The framework is evaluated on image diffusion, text-to-3D generation, and neural-network generation, and the authors report results consistent with the main findings of prior work. Finally, they use the approach to design post-training interventions for image and face generation that reduce memorization without introducing undesirable geometric side effects.

Critical strengths
  • Using metric-space magnitude to detect memorization in generative models is a novel and interesting idea worth exploring. Although the experimental setup is limited, the results are promising.
Critical weaknesses
  • The experiments focus largely on settings chosen to showcase the proposed method, rather than on a broad benchmark. An extensive comparison with existing memorization metrics (e.g., CTC_T score, AuthPct, or Generalization Gap FLD), in the spirit of the large-scale study by Stein et al. (NeurIPS 2023), is missing. In addition, the source code is not provided, which limits the reproducibility of the results.

  • The theoretical part of the paper, which presents the mathematics behind metric-space magnitude, is somewhat difficult to follow in detail. The authors rely mainly on intuitive descriptions rather than formal statements and proofs (e.g., in the appendix) to support their claims.

Questions for authors
  • Could the authors provide a broader experimental comparison, on standard benchmarks, showing the advantages of their three-effect fingerprint over existing metrics?

  • Could the authors strengthen the theoretical presentation by adding precise formal definitions and statements, with proofs (e.g., in the appendix), for the properties of the proposed quantities (μ\mu, κ\kappa, ρ\rho)?

5 comments
AuthorsOpenReview ↗

Response to W1 / Q1. Comparison with existing memorisation metrics, in the spirit of Stein et al. (2023), on standard benchmarks.

We added this comparison in Appendix B.4 (Table 6). We reproduce the six synthetic conditions of Stein et al. (2023, Appendix D.6.3): fresh sampling, mode shrinkage, training replay and three levels of underfitting. In Table 6, we report AuthPct, C_T, modified C_T, the FLD gap and the FLD overfit percentage next to our three measures.

We draw three conclusions from the results.

  • Original C_T and AuthPct confound memorisation with other departures, as Stein et al. found. C_T is negative under shrinkage (−2.50), although no training point is copied. AuthPct assigns 39.7% authenticity to fresh samples.

  • Modified C_T and the FLD gap flag replay, but each returns a single value. In Table 5 (Appendix B.3), we concentrate the same 48 copy-derived outputs on four examples. Modified C_T then goes from −3.64 to −0.16, and the FLD gap goes from −235.9 to −20.6.

  • Our measures report each effect separately. With the reference-substitution control, we separate replay from shrinkage. For replay, we measure an absorption of 1.000 against the training reference and of 0.026 against an independent reference. For shrinkage, we measure 0.373 and 0.391.

This fills the gap noted by Stein et al. (NeurIPS 2023): existing scores do not separate memorisation from underfitting or mode shrinkage. We made this explicit in the text, in the new Figure 1 and in the revised Figure 2. In the new Appendix A, we state which changes move which measure (Proposition 3) and what each value means (Proposition 2).

Regarding setting choices: We take the three diagnostic settings from published memorisation studies (MemBench; Pu et al., 2026; Zeng et al., 2026). We did not build them for the method. In these settings the nearest-neighbour statistic already shows the limitation on real data. In the 3D sweep, Z_U scores w = 1 and w = 10 alike while our measures separate them (Table 9). For P-diff, Z_U is +13.6 in all three spaces, while the coverage deficit ranges from 0.717 to 0.928 (Table 11).

AuthorsOpenReview ↗

Response to W2 / Q2. Formal definitions, statements and proofs.

We revised Section 3 and added Appendix A. In Appendix A, we state the assumptions, cite the magnitude results we use, and prove four propositions.

  • Proposition 1 (bounds). The shared magnitude lies between 0 and the smaller of the two magnitudes. Both shared fractions therefore lie in [0, 1].

  • Proposition 2 (meaning of the values). Each measure equals one minus the ratio of a generated integral to a counterfactual integral. Each measure is at most 1 and is zero when G = P. A corollary gives absorption 1 when every generation lies on a training point.

  • Proposition 3 (invariances). The measures are invariant under a common isometry. Moving G rigidly preserves redundancy. Repeating the whole sample q times preserves absorption and coverage and changes redundancy to 1 − (1 − ρ)/q.

  • Proposition 4 and Section A.5 (numerics). The measures are continuous on a fixed grid. In Section A.5, we give the conventions for exact duplicates and for t = 0.

I would like to thank the authors for their rebuttal. Regarding the theoretical concerns, Appendix A now provides satisfactory details. However, on the experimental side, the additional experiments conducted by the authors, though valuable, rely only on synthetic data and are far from the large-scale study proposed by Stein et al. (2023), which involves several generative models trained on a few real-world datasets (CIFAR-10, ImageNet, LSUN Bedroom, and FFHQ). In my opinion, the lack of such an extensive experimental comparison against existing metrics significantly weakens the contribution. Therefore, I will maintain my score.

Reviewer_QeDuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a magnitude-based framework for analyzing three different geometrical aspects of memorization: training-set absorption, reference-coverage deficit, and internal redundancy. The framework compares generated samples with reference data and a separate comparison set at multiple distance scales. The authors also develop post-training interventions for image and identity memorization, using diagnostics to guide training and adding constraints to address undesirable side effects.

Critical strengths
  • The paper applies the diagnostics across different generative settings and shows consistency in the given experiments. The results are consistent with previous findings while distinguishing reference overlap, limited coverage, and internal redundancy in the evaluated cases.

  • The controlled experiments separately vary planted copies, reference coverage, and internal redundancy. Repeated trials and uncertainty estimates support the intended verification of the proposed diagnostics.

  • The post-training interventions reduce copying on held-out prompts and identity similarity on held-out identities. This suggests that the learned corrections transfer beyond the conditions used for post-training.

Critical weaknesses
  • The added constraints help address the observed side effects, but their effects could be examined further. In particular, it would help to clarify whether they reduce memorization on their own and whether the final combination introduces other forms of degradation on the modified prompts.

  • The additional value of the proposed diagnostics could be evaluated more directly through comparisons with existing copying, coverage, and diversity metrics used jointly.

Questions for authors
  • Could the authors clarify how the three diagnostic values would reflect patch-level memorization or memorization that occurs only for certain rare prompts? I am unsure whether such cases would produce a clear signal, although I understand that the diagnostics may not capture every form of memorization.

  • Could the authors clarify how patch-level memorization or copying limited to rare prompts would be reflected in μ\mu, κ\kappa, and ρ\rho? I understand that the diagnostics may not capture every form of memorization.

  • Could the authors further evaluate the quality and coverage of images generated from the memorizing prompts after post-training? Mitigating memorization is beneficial only when generalization is not degraded simultaneously. Generalization metrics such as FID with a suitable reference set would complement the copy-reduction results.

  • In the image diagnostics, PP consists of generations from the COCO set. Would the conclusions persist if PP were instead generated from non-memorizing prompts with content similar to the MemBench prompts?

8 comments
AuthorsOpenReview ↗

Response to W1: Do the added constraints reduce memorisation on their own, and does the final combination degrade the modified prompts?

Memorisation is mainly reduced by the repulsion term in its normal and magnitude versions. In the "Magnitude self + cross" row of Table 2, we use the magnitude constraints. With this variant alone, we lower identity similarity by 0.409 on training identities and by 0.258 on held-out identities. We also lower absorption from 0.232 to 0.095. The role of the plausibility term is to act as a ceiling that limits drift, and it does not act on the identity, while the role of the cross term is to avoid the newly generated identities collapsing between them, which is the degradation we found was added by the repulsion term.

Does the final combination degrade the modified prompts? No. On the treated identities, the final model stays below the original model in coverage deficit and redundancy, and it keeps the face detection rate on the name prompts. We measured this in two ways.

  • Geometry (Table 2). We add the magnitude terms because distance repulsion increases the coverage deficit and the redundancy. With the final model, we bring them to 0.007 and 0.053. Both are below the values of the original model (0.016 and 0.134). We show the corresponding images in Figure 5.

  • Generation quality on the name prompts (new Table 16). We detect no face in 3.1% of the generations for the original model and in 2.6% for the final model. On perturbed names, we measure the same CLIP similarity to the prompt (0.263 and 0.259). On real names, we measure a move from 0.333 to 0.274, which is the perturbed-name level. This is the intended effect: the model no longer produces the named person.

We report ordinary prompts in Table 15. We also report, through our own framework, the post-training results where we observe that the generations are less repetitive (redundancy falls in all three encoders) and the coverage deficit is unchanged (Table 1); Figure 4 shows the image results.

AuthorsOpenReview ↗

Response to W1 and Q3. Quality and coverage on the memorising prompts after post-training.

We evaluate quality after post-training with measures that do not need a real reference set.

  • Name prompts: face detection and prompt adherence (Table 16).

  • Ordinary prompts: diversity and prompt adherence (Tables 13 and 15).

  • We also see that the coverage deficit stays at its original value in all three encoders (Table 1), and Figure 4 shows the images.

FID needs a large set of real images that match the evaluated prompts. For the memorising prompts and the memorised names, we only have the memorised training images from MemBench and one reference photograph per person from Namesakes.

AuthorsOpenReview ↗

Response to W2. Comparison with existing copying, coverage and diversity metrics used jointly.

Metrics used jointly give conflicting readings when copying and repetition effects overlap. Our three measures report each effect on its own axis. We show this with two new experiments in Appendix B (the former Appendix A).

  • Each score alone (Appendix B.2, Table 4): For Vendi, coverage, recall and Z_U, we build a case in which the score does not move or moves in the wrong direction. Our measures register the change in every case.

  • All scores together (Appendix B.3, Table 5): We read Z_U, Vendi, recall, coverage, modified C_T and the FLD gap on sets where copying and repetition overlap. We keep 48 copy-derived outputs and move them from 48 training points onto 4. Z_U does not change (−5.88). Modified C_T goes from −3.64 to −0.16, and the FLD gap goes from −235.9 to −20.6, although the number of copy-derived outputs is the same.

  • Our measures separate the effects: absorption falls from 0.392 to 0.120, redundancy rises from −0.003 to 0.339, and the coverage deficit rises from −0.218 to 0.141.

We derive the three measures from one construction, so a reading on one can be interpreted against the other two. This fills a gap noted by Stein et al. (NeurIPS 2023): existing scores do not separate memorisation from underfitting or mode shrinkage. We made this explicit in the text, in the new Figure 1 and in the revised Figure 2. In the new Appendix A, we state which changes move which measure (Proposition 3) and what each value means (Proposition 2).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank the reviewers for their careful reading. We summarise here the changes in the revised version. All table, figure and appendix numbers in our responses refer to the revision.

Corrections:

  • 3D analysis (Section 4.1 and Appendix C.2). Thanks to the comment of Reviewer xJvE, we found two problems in the submitted analysis. The Light-Field Distance is not of negative type, so magnitude is not well defined on it, and the training reference was a subsample of each category. We fixed this by using a Euclidean embedding of the distance and used the correct training reference. We withdraw the claim that absorption reproduces the moderate-guidance peak, and we removed the submitted Appendix C. In the revised analysis, Z_U now correctly reproduces the peak reported by Pu et al. (2026), while our three measures decrease with guidance and are highest at w = 1, a setting that Pu et al. also report as poor (Tables 9 and 10, Figures 11 and 12). While the specific analysis of the 3d diagnostic changed, the conclusions of the paper remain the same.

  • Table 1. We recomputed this table after correcting a floating-point error in the self-distances in the distance matrices. The DINOv2 and CLIP rows changed; the SSCD row did not. The analysis and conclusions remain the same.

Additions:

  • Formal statements (Appendix A). Following Reviewer QbmM request. We state the assumptions and prove bounds, the meaning of the values, invariances and continuity (Propositions 1–4). Section 3 is revised accordingly for clarity purposes.

  • Comparison with existing scores (Appendix B.2–B.4, Tables 4–6). Following the reviewers' QbmM and QeDu suggestions, we compare with Vendi, coverage, recall, Z_U, C_T, modified C_T, AuthPct and the FLD gap, including the six synthetic conditions of Stein et al. (2023).

  • Generation quality on name prompts (Table 16). We report the face detection rate and prompt adherence after post-training to add completeness to the reviewer QeDu request.

Presentation:

  • Figure 1 is new; Figure 2 is revised to add more clarity to our magnitude-based approach and emphasise our metric’s contribution. For clarity, Figure 3 now shows Z_U beside our measures. Table 2 reports ArcFace for training and held-out identities; the AdaFace results moved to Table 14 to improve readability. Table format modifications were done to improve readability and interpretation.

  • Text. We revised the abstract, the introduction and the related work to state the contributions over other metrics more directly. We also clarified the description of the MemBench data and the reading of Tables 7 and 8 in Appendix C.1. We also unified our metrics name to improve readability.

  • Numbering. Appendix A is new; the submitted Appendices A and B are now B and C. The submitted Figures 1–5 are now Figures 2–6. Tables 4–6, 10, 14 and 16 are new; the submitted Tables 4, 5, 6, 7, 9, 10 and 11 are now Tables 7, 8, 9, 11, 12, 13 and 15.

None of these modifications changes any result or analysis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.