- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The papers' argument is that current definitions of memorization for outputs of image generative models focus on exact/near-exact copies of training samples. While doing this the methods miss a more group-level kind of reproduction, that is copying an artists style. The question is when generating images in the style of a specific artist, when can it be classified as generalization and when as memorization? The main contribution of the paper is a new definition of memorization for style called (τ, λ)-memorization. A style is counted as memorized if the style is unusual in both captions and visuals, and the model can reproduce it. To test this, the authors group LAION samples by shared caption words finding 3400 groups, 160 of which have artist name in caption. They prompt Stable Diffusion models with short prompts like the artists name and label a style as memorized if reverse image search names the correct artist. From their outputs SD1.4 memorized about a third of the artists and SD2.1 a little more. They further show that after only keeping unusual styles (based on their metric) a simple style-similarity score detects memorized artists much better than existing definition.
Critical strengths
- The paper responds to an important and relevant problem. With the widespread usage of generative models, style mimicry is a major copyright issue. The argument that sample-level definitions fall short of correctly expressing that a model reproduces a style even if the exact individual pieces are not generated, is a convincing angle.
- Using atypicality rather than frequency in the training data to separate expected from unexpected reproduction is a new approach.
- Employing reverse image search to identify the artist is a practical proxy to see if the generated images could be classified under an artist or compete with their work.
Critical weaknesses
- In baseline comparison the paper evaluates their method on a subset of evaluation set which is already pruned to cases that are less confusing or vague. The baseline is however tested on their own messier set.
- The definition that is compared against does not seem to be fair. The adapted Carlini et al's definition so a style counts as memorized if the generated images are as close to artists paintings as those paintings are to each other. This is a tough bar to meet, so beating it seems easy. Maybe some simple comparisons like using CLIP could be another baseline.
- Some prompts name the painting itself like "claude monet woman parasol madame son 1875". This is built to fail.
- Style measure presented of how close the outputs are or how unusual a style is, needs to be validated and verified. Not sure if the last point is a weakness or a confusion from me, but 5. Writing an artist, especially a very well known artist, in the prompt will get that artists style in the output. That is what is expected and probably desired, so how this is considered memorization needs to be verified or validated better.
Questions for authors
- Better comparison on fair data
- Prompt with only artist name and then with something very unexpected (from the artists general style, like maybe if they do realistic portraits, you can write a dragon or something) and see how it works.
- How is entropy in the caption calculated?
- Better validate the custom style score
3 comments
Entropy calculation (Question 3)
We thank the reviewer for pointing out this ambiguity. The proposed caption atypicality score (Eq. 1) uses an entropy-based notion of token importance, where entropy is estimated through the effective rank of the token embedding matrix. Specifically, the importance of token is computed as the decrease in effective rank when that token is removed from the caption: . As a result, tokens that contribute more to the caption’s semantic information cause a larger decrease in effective rank when they are removed, receiving higher importance. We will clarify in the revised version that Eq. 1 uses an effective-rank-based estimate of entropy.
Metrics validation (Weakness 4 and Question 4)
Style atypicality measures how distinctive and well-defined an artistic style is relative to other styles, based on its distance with remaining groups’ styles and weighted by a style-complexity score, which captures the existence of a single and meaningful artistic style in each group. These metrics rely on the Gram-matrix representation, which is proposed by several works on style representation/transfer (Gatys et al., 2016; Li et al., 2017; PyTorch, 2025; TensorFlow, 2025) and experiments reveal that it is enough to distinguish between different styles. However, to improve this distinction on similar styles (coming from different artists), we performed rank-based dimensionality reduction, as explained in Section A.1. In addition, experiments revealed that the score that computes style complexity, which is intended to distinguish well-defined styles such as artistic styles and unlike random pictures, is based on the same representation and tends to agree with artistic styles, meaning that groups containing artistic styles tend to have higher style complexity, as illustrated in Figure 11 (Section A.7) where memorized and non-memorized artistic styles (red and green points) are mostly in the rightmost part of the plot. So, the choice of the Gram-matrix style representation is supported by prior work and results after applying the improvement (rank-based dimensionality reduction) reveal that this representation sufficiently captures human perception of artistic styles.
Style memorization vs generalization
We agree that reproducing characteristics of an artistic style when prompted with the artist name is expected, especially for a well-known artist. However, our goal is not to identify every successful artist-name-conditioned generation as memorization, but instead to identify cases where a general-purpose model retains artist-specific characteristics to an unusual extent such that those generations constitute a threat for the artist’s work, which is undesirable regardless of whether the artist is well known and may conflict with data owners’ consent. Our notion of memorization is aligned with the view that memorization and generalization are distinct but complementary phenomena, supported by prior work [1-7], where memorization corresponds to the retention and consistent reproduction of training-specific information, while generalization fills in the remaining visual details, enabling the generation of distinct images that are not exact replicas of the training examples but share training-specific information (i.e. artistic style). In contrast with prior work, we address the interplay between memorization and generalization by decoupling the extraction accuracy (style loss) from the concept of memorizable style (style and caption atypicality). This allows us to detect memorization beyond exact extraction, particularly when the model generates synthetic images that exhibit training-specific information traceable to training data (with artist re-identification) while being pixel-wise distinct from training images reflecting the model's generalization capability.
To distinguish this undesired phenomenon from normal text-conditioned generation, our definition incorporates several additional requirements related to 1) reproduction quality; 2) repetitive behaviour; 3) insufficient prompt guidance; 4) undesired model use. First, style memorization differs from normal text-conditioned generation due to the fact that synthetic samples exhibit artist-specific characteristics remarkably similar to original works to the point of allowing visual search engines, with access to training and non-training artworks from several artists, to re-identify the original author, which underscores the threat that this capacity can have. Second, style memorization differs from normal text-conditioned generation because it is consistent across many generations, revealing the failure of the model’s generalization capacity to produce distinct images that still follow the prompt, for example artist-name-only prompts, which led to 72 memorized cases (18%), could result in stylistically similar images as well as images of persons sharing the same genre, nationality or cultural origin. We address this requirement by clustering synthetic samples according to style similarity and analyzing only sufficiently large clusters with at least 25 samples (10%). In practice, however, clusters corresponding to memorized artists were substantially larger than this minimum threshold, typically containing around 200 samples. Third, style memorization differs from normal text-conditioned generation because the eliciting prompt does not include enough information to guide the model into reproducing artist-specific characteristics, which we address by prompting the model with small prompts, with an average of 8.5 tokens on memorized cases. We further confirm that the model is able to reproduce artistic styles without sufficient information to guide reproduction with artist-only-prompts, although we suggest for future research to improve the strategy to rule out the hypothesis of prompt-guided extraction. Fourth, while several tools have been developed for style mimicry, style memorization differs from expected text-conditioned generation because it persists in models whose training objective was not style mimicry, which we address by selecting Stable Diffusion - a general-purpose model not trained for style mimicry. This explanation will be improved in the paper, particularly in Section 3.
[1] Somepalli et al., Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models, CVPR 2023
[2] Schwarzschild et al., Re-thinking LLM Memorization Through the Lens of Adversarial Compression, NeurIPS 2024
[3] Di et al., Demystifying Foreground-Background Memorization in Diffusion Models, 2025
[4] Morris et al., How Much Do Language Models Memorize?, 2025
[5] Huang et al., Neuron-Level Differentiation of Memorization and Generalization in Large Language Models, EMNLP 2025
[6] Wang et al., Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data, ICLR 2025
[7] Huang et al., Demystifying Verbatim Memorization in Large Language Models, EMNLP 2024
Summary
We thank the reviewer for the thoughtful feedback. Our response clarifies the baseline comparison and highlights the main limitation of the baseline, thereby addressing Weakness 1, 2 and Question 1. Question 2 and Weakness 3 regarding concerns about prompt guided extraction, we analysed the influence of different prompts in the extraction of the same artist, which we will expand and summarize the main insights in the beginning of Section 4, as well as add an illustration to Figure 1 of artists that were re-identified even when the generated content clearly differs from known artworks. Concerns regarding metrics validation, raised in Question 4 and Weakness 4, are already addressed in the appendix A.1 and A.7, but we will add a summary about the experimental validation of the proposed metrics to A.1. Our response to Question 3 clarifies the calculation of entropy in the caption, particularly its effective-rank-based estimate, which we will clarify in the introduction of Caption Atypicality in Section 3. To address Weakness 5, where the reviewer raised concerns about the extent to which reproducing artistic styles is considered memorization, we will improve the explanation of the distinction between style memorization and generalization in Section 3, explaining that our concept of memorization is aligned with the view of memorization and generalization as distinct but complementary phenomena, which is supported by prior work, and the criteria used to distinguish style memorization from normal model behaviour.
Baseline comparison (Weakness 1, 2 and Question 1)
We agree that the adapted (k,l,epsilon)-eidetic definition imposes a strict extraction criterion, which is a direct consequence of preserving the original interpretation, since memorizable styles and the memorization status itself are both defined by epsilon, which controls the extraction accuracy. As a result, the definition cannot simultaneously allow some degree of generalization around artist-specific characteristics (by relaxing epsilon) while restricting memorizable styles to groups of samples with the same artistic style (by tightening epsilon). While alternative metrics, such as CLIP similarity, could be explored, the original formulation of (k,l,epsilon)-eidetic memorization already accommodates different distance metrics through the parameter l. Consequently, we expect the main challenge to remain the dependence on epsilon as both an extraction criterion and a definition of what constitutes a memorizable style. Additionally, we want to clarify that the detection strategy and the baseline comparison are evaluated on the same subset.
Prompt guided extraction (Weakness 3 and Question 2)
We thank the reviewer for this observation. While some prompts contain content related to specific artworks, results of Section 4.3, where we use artist-name-only prompts, show that style memorization is not dependent on content specific to known artworks, as approximately 18% of a much larger set of artistic styles are consistently reproduced and re-identified by visual search even when the prompt only includes the artist name, without requiring a painting, and without any content-related cues. Focusing on the 20 artists common on both experiments (prompts with and without content-related words), 9 remain memorized when using artist-name-only prompts, showing that artist re-identification can succeed without content-related cues, whereas 5 artistic styles are memorized only when content words are included, suggesting that certain content descriptions can help elicit memorized styles. However, for 4 artists, adding content-related words steers the generation process away from the artist's specific style. Results from this subset of overlapping artists suggest that content-related words are not the main factor driving style reproduction, as artistic styles can be reproduced even when the prompt contains only the artist's name, as illustrated in Figure 1. Regarding cases where the generated content clearly differs from known artworks, these can still be re-identified by visual search, such as the style of L.S. Lowry which is extracted with the prompt “ls lowry going painting match steps wick” resulting in images similar in style but visually very distinct. We will expand experiments of overlapping artists to further strengthen the hypothesis that content-related words are not the main factor driving memorization and summarize the main insights in the beginning of Section 4. We will also add to Figure 1 cases where the generated content clearly differs from known artworks that still allow visual search to re-identify the artist.