- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
In this work, the authors criticize existing tests for memorization in generative image models by proposing two new null hypotheses: one an exchangeability argument for swapping labels of train and held-out evaluation images at the model level, and one at the image-level based on the intersection of clouds around the train and held-out sets with the generated images.
Critical strengths
The authors take a principled approach to designing good null hypotheses for memorization tests in generative models. They introduce tests both for models and individual images, which are two complementary problems in the area of memorization detection in generative models.
Critical weaknesses
The writing and clarity is quite poor, to the point of hindering my ability to understand and appreciate the author's work. This is often vague feedback so I'll try to make concrete points. My personal perspective is that, if your goal is to get practitioners to adopt your rigorous approach to memorization testing, then you should opt for clarity over technical complexity.
- The work is mainly written in passive voice to the point where I can't tell what the authors did and what prior work did. The experiments section briefly goes into active voice which is a better example of good clarity in writing.
- The authors use a lot of pronouns instead of nouns which sometimes make me confused on what they are referencing, especially combined with the passive voice.
- You introduce results and methods without justifying or explaining them, instead of walking through prior methods/results and what you do differently. Walk the reader through your thought process!
- Maybe related to this, you mention (baseline) methods and tests in passing without explicitly introducing them. This is especially confusing in intro where we don't even know what your approach is yet.
For example on Line 343: "For memorized image xi with its ni trigger-prompt generations (median ni = 21), the statistic is the median SSCD similarity (Pizzi et al., 2022) of the generations to xi. Embedding space is the right place for it, since it sees layout-level copies, and the wrong place for the topological statistic, since SSCD distances concentrate."
- You just introduced SSCD similarity in-line, and then talk about embedding spaces with no context. At least say "SSCD similarity, a metric based on similarity in model embedding space"
Examples of confusing phrases:
- "Where topology enters is narrower, and we say so."
- "At the operating scale its specificity matches the count’s."
- "Read against its normal reference, the data-copying statistic flags 10 of 24 generators that reproduce nothing;"
- What is the normal reference?
- What is the data copying statistic?
- Isn't reproducing nothing (in a memorization benchmark) good? So the data copying statistic is flagging 10/24 generators, should it be flagging all of them or none of them? I have no clue what your contribution is here
- "the nearest-neighbour fraction rejects on 7, the data-copying statistic on 5, and a count restricted to the near-duplicate scale on none."
- You introduce these methods without any context or explanation. Which is yours? which are baselines? do we want to reject on all or none?
- "The median of 21 values resampled from the pool has spread (standard deviation) 0.016"
- Why are you resampling 21 values from the pool, or what procedure is resulting in 21 values? Earlier in the paragraph you mentioned 47 discoveries not 21.
To show an example of more clear, active voice, consider contribution 1:
- "The exact null for memorization, and what the existing one gets wrong." ->
- We point out challenges with existing tests for memorization and propose an exact null for memorization.
- "Ten copies among 500 samples, injected into a model that copies nothing, are caught by the scale-restricted statistics in every replicate; the nearest-neighbour fraction needs fifty." ->
- Using the scale-restricted statistics we are able to find 10 data copies inserted in a set of 500 unmemorized generations across multiple trials, whereas the nearest-neighbor fraction needs 50 (needs 50 what?)
Line 212: Where is Prop 3?
Notation: you define you test statistic as but then in the image-level null section write
I believe that the method underlying this work is solid, but it is difficult for me to evaluate due to the presentation. I am willing to increase my score but I fear that the presentation issues would require changes on a scale beyond the normal revisions of a discussion cycle.
Questions for authors
- If is looking at balls around points, how is it different than the naive threshold tests you were criticizing earlier? Isn't the radius an arbitrary hyperparameter again here?
- On your CIFAR-10 trained models, what is your ground truth for memorization? How do you know your statistic is finding the true memorized generations?
- Why does statistical power change with the number of replicated samples? I understand it being dependent on the number of held-out samples but aren't you testing each replicated sample independently, or are you testing the entire set jointly?
Response to Reviewer 6Ktu
Thank you for the concrete examples; they are the most useful part of the reviews for us, and the revision will follow them (active voice throughout, every baseline introduced before it is used, no results before their justification). To your questions:
How is T_small at r₀ different from the threshold tests we criticise? The threshold rules we criticise compare a similarity to a fixed cutoff with no null, so the cutoff sets the error rate. Under the swap null, r₀ sets what is measured (how close a sample must be to count as a copy), not the error rate: the permutation test is exact at any r₀, and the choice is reported. The choice does matter for what is detected, and we will show the sensitivity (four models rerun at 0.6× to 2× the default r₀; at 2× one marginal model rejects).
Ground truth on CIFAR-10. Pixel distance: a sample within a quarter of the training-image spacing of a training image is a copy, and a generator with at least a fifth of its samples copies is a reproducer. Reviewer 2VGb points out that this makes the count's specificity partly circular, and we agree; the non-circular evidence is on Stable Diffusion, where the ground truth is MemBench's reverse-image-search verification.
Why does power depend on the number of copies? The model-level test is one statistic for the whole sample of 500 generations, tested jointly against the permutation null; it is not a per-sample test. More copies among the 500 move that one statistic further from its null, which is why power rises with the number injected. The image-level audit is the per-image test, and there the question is how many generations per image are needed (two, on this benchmark).
Line 212 / Prop. 3 / notation. Proposition 3 is in Appendix A and will move to the main text; S(G, T, H) is the model-level statistic and S(G_i, x_i) the image-level one, and the revision will use different letters.