acceptodds
Under review as a conference paper at ICLR 2027

Style Memorization in Text-to-Image Generative Models

Abstract

While the intended use of generative models is to create novel content, they can generate expressive content on a large scale that closely resembles training data, particularly memorized styles of specific artists, competing with human creators in existing markets. Past research overlooks this type of subtle memorization by focusing on exact/near-exact copies and by failing to account for the inherit difficulty of reproducing atypical semantic properties, like artistic styles. To distinguish expected from unexpected/undesired capabilities, this work proposes (, )-memorization that defines an extractable artistic style as memorized if it is sufficiently atypical at the image-level () and caption-level (), as these outlier-properties are inherently difficult to reproduce. Experiments show that the popular general-purpose model Stable Diffusion memorized 34% of artistic styles, producing fake artworks based on non-overly descriptive prompts that enable artist re-identification with visual search, highlighting potential market impact. With adequate thresholds and to define caption and style atypicality, style memorization is detected with over 20% TPR at 1% FPR, unlike prior work that only achieves TPR as high as FPR. This work shows that atypical artistic styles are more likely to be memorized, particularly underrepresented ones, exposing an underestimation of memorization risks of current frequency-based indicators, thus informing data owners of risks of style memorization and the need for appropriate mitigation strategies.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Generative Models, Memorization, Computer Vision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_nGGMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The papers' argument is that current definitions of memorization for outputs of image generative models focus on exact/near-exact copies of training samples. While doing this the methods miss a more group-level kind of reproduction, that is copying an artists style. The question is when generating images in the style of a specific artist, when can it be classified as generalization and when as memorization? The main contribution of the paper is a new definition of memorization for style called (τ, λ)-memorization. A style is counted as memorized if the style is unusual in both captions and visuals, and the model can reproduce it. To test this, the authors group LAION samples by shared caption words finding 3400 groups, 160 of which have artist name in caption. They prompt Stable Diffusion models with short prompts like the artists name and label a style as memorized if reverse image search names the correct artist. From their outputs SD1.4 memorized about a third of the artists and SD2.1 a little more. They further show that after only keeping unusual styles (based on their metric) a simple style-similarity score detects memorized artists much better than existing definition.

Critical strengths
  1. The paper responds to an important and relevant problem. With the widespread usage of generative models, style mimicry is a major copyright issue. The argument that sample-level definitions fall short of correctly expressing that a model reproduces a style even if the exact individual pieces are not generated, is a convincing angle.
  2. Using atypicality rather than frequency in the training data to separate expected from unexpected reproduction is a new approach.
  3. Employing reverse image search to identify the artist is a practical proxy to see if the generated images could be classified under an artist or compete with their work.
Critical weaknesses
  1. In baseline comparison the paper evaluates their method on a subset of evaluation set which is already pruned to cases that are less confusing or vague. The baseline is however tested on their own messier set.
  2. The definition that is compared against does not seem to be fair. The adapted Carlini et al's definition so a style counts as memorized if the generated images are as close to artists paintings as those paintings are to each other. This is a tough bar to meet, so beating it seems easy. Maybe some simple comparisons like using CLIP could be another baseline.
  3. Some prompts name the painting itself like "claude monet woman parasol madame son 1875". This is built to fail.
  4. Style measure presented of how close the outputs are or how unusual a style is, needs to be validated and verified. Not sure if the last point is a weakness or a confusion from me, but 5. Writing an artist, especially a very well known artist, in the prompt will get that artists style in the output. That is what is expected and probably desired, so how this is considered memorization needs to be verified or validated better.
Questions for authors
  1. Better comparison on fair data
  2. Prompt with only artist name and then with something very unexpected (from the artists general style, like maybe if they do realistic portraits, you can write a dragon or something) and see how it works.
  3. How is entropy in the caption calculated?
  4. Better validate the custom style score
3 comments

Summary

We thank the reviewer for the thoughtful feedback. Our response clarifies the baseline comparison and highlights the main limitation of the baseline, thereby addressing Weakness 1, 2 and Question 1. Question 2 and Weakness 3 regarding concerns about prompt guided extraction, we analysed the influence of different prompts in the extraction of the same artist, which we will expand and summarize the main insights in the beginning of Section 4, as well as add an illustration to Figure 1 of artists that were re-identified even when the generated content clearly differs from known artworks. Concerns regarding metrics validation, raised in Question 4 and Weakness 4, are already addressed in the appendix A.1 and A.7, but we will add a summary about the experimental validation of the proposed metrics to A.1. Our response to Question 3 clarifies the calculation of entropy in the caption, particularly its effective-rank-based estimate, which we will clarify in the introduction of Caption Atypicality in Section 3. To address Weakness 5, where the reviewer raised concerns about the extent to which reproducing artistic styles is considered memorization, we will improve the explanation of the distinction between style memorization and generalization in Section 3, explaining that our concept of memorization is aligned with the view of memorization and generalization as distinct but complementary phenomena, which is supported by prior work, and the criteria used to distinguish style memorization from normal model behaviour.

Baseline comparison (Weakness 1, 2 and Question 1)

We agree that the adapted (k,l,epsilon)-eidetic definition imposes a strict extraction criterion, which is a direct consequence of preserving the original interpretation, since memorizable styles and the memorization status itself are both defined by epsilon, which controls the extraction accuracy. As a result, the definition cannot simultaneously allow some degree of generalization around artist-specific characteristics (by relaxing epsilon) while restricting memorizable styles to groups of samples with the same artistic style (by tightening epsilon). While alternative metrics, such as CLIP similarity, could be explored, the original formulation of (k,l,epsilon)-eidetic memorization already accommodates different distance metrics through the parameter l. Consequently, we expect the main challenge to remain the dependence on epsilon as both an extraction criterion and a definition of what constitutes a memorizable style. Additionally, we want to clarify that the detection strategy and the baseline comparison are evaluated on the same subset.

Prompt guided extraction (Weakness 3 and Question 2)

We thank the reviewer for this observation. While some prompts contain content related to specific artworks, results of Section 4.3, where we use artist-name-only prompts, show that style memorization is not dependent on content specific to known artworks, as approximately 18% of a much larger set of artistic styles are consistently reproduced and re-identified by visual search even when the prompt only includes the artist name, without requiring a painting, and without any content-related cues. Focusing on the 20 artists common on both experiments (prompts with and without content-related words), 9 remain memorized when using artist-name-only prompts, showing that artist re-identification can succeed without content-related cues, whereas 5 artistic styles are memorized only when content words are included, suggesting that certain content descriptions can help elicit memorized styles. However, for 4 artists, adding content-related words steers the generation process away from the artist's specific style. Results from this subset of overlapping artists suggest that content-related words are not the main factor driving style reproduction, as artistic styles can be reproduced even when the prompt contains only the artist's name, as illustrated in Figure 1. Regarding cases where the generated content clearly differs from known artworks, these can still be re-identified by visual search, such as the style of L.S. Lowry which is extracted with the prompt “ls lowry going painting match steps wick” resulting in images similar in style but visually very distinct. We will expand experiments of overlapping artists to further strengthen the hypothesis that content-related words are not the main factor driving memorization and summarize the main insights in the beginning of Section 4. We will also add to Figure 1 cases where the generated content clearly differs from known artworks that still allow visual search to re-identify the artist.

Entropy calculation (Question 3)

We thank the reviewer for pointing out this ambiguity. The proposed caption atypicality score (Eq. 1) uses an entropy-based notion of token importance, where entropy is estimated through the effective rank of the token embedding matrix. Specifically, the importance of token tt is computed as the decrease in effective rank when that token is removed from the caption: it=entropy(T)−entropy(T\t)i_t = \text{entropy}(T) - \text{entropy}(T_{\backslash t}). As a result, tokens that contribute more to the caption’s semantic information cause a larger decrease in effective rank when they are removed, receiving higher importance. We will clarify in the revised version that Eq. 1 uses an effective-rank-based estimate of entropy.

Metrics validation (Weakness 4 and Question 4)

Style atypicality measures how distinctive and well-defined an artistic style is relative to other styles, based on its distance with remaining groups’ styles and weighted by a style-complexity score, which captures the existence of a single and meaningful artistic style in each group. These metrics rely on the Gram-matrix representation, which is proposed by several works on style representation/transfer (Gatys et al., 2016; Li et al., 2017; PyTorch, 2025; TensorFlow, 2025) and experiments reveal that it is enough to distinguish between different styles. However, to improve this distinction on similar styles (coming from different artists), we performed rank-based dimensionality reduction, as explained in Section A.1. In addition, experiments revealed that the score that computes style complexity, which is intended to distinguish well-defined styles such as artistic styles and unlike random pictures, is based on the same representation and tends to agree with artistic styles, meaning that groups containing artistic styles tend to have higher style complexity, as illustrated in Figure 11 (Section A.7) where memorized and non-memorized artistic styles (red and green points) are mostly in the rightmost part of the plot. So, the choice of the Gram-matrix style representation is supported by prior work and results after applying the improvement (rank-based dimensionality reduction) reveal that this representation sufficiently captures human perception of artistic styles.

Style memorization vs generalization

We agree that reproducing characteristics of an artistic style when prompted with the artist name is expected, especially for a well-known artist. However, our goal is not to identify every successful artist-name-conditioned generation as memorization, but instead to identify cases where a general-purpose model retains artist-specific characteristics to an unusual extent such that those generations constitute a threat for the artist’s work, which is undesirable regardless of whether the artist is well known and may conflict with data owners’ consent. Our notion of memorization is aligned with the view that memorization and generalization are distinct but complementary phenomena, supported by prior work [1-7], where memorization corresponds to the retention and consistent reproduction of training-specific information, while generalization fills in the remaining visual details, enabling the generation of distinct images that are not exact replicas of the training examples but share training-specific information (i.e. artistic style). In contrast with prior work, we address the interplay between memorization and generalization by decoupling the extraction accuracy (style loss) from the concept of memorizable style (style and caption atypicality). This allows us to detect memorization beyond exact extraction, particularly when the model generates synthetic images that exhibit training-specific information traceable to training data (with artist re-identification) while being pixel-wise distinct from training images reflecting the model's generalization capability.

To distinguish this undesired phenomenon from normal text-conditioned generation, our definition incorporates several additional requirements related to 1) reproduction quality; 2) repetitive behaviour; 3) insufficient prompt guidance; 4) undesired model use. First, style memorization differs from normal text-conditioned generation due to the fact that synthetic samples exhibit artist-specific characteristics remarkably similar to original works to the point of allowing visual search engines, with access to training and non-training artworks from several artists, to re-identify the original author, which underscores the threat that this capacity can have. Second, style memorization differs from normal text-conditioned generation because it is consistent across many generations, revealing the failure of the model’s generalization capacity to produce distinct images that still follow the prompt, for example artist-name-only prompts, which led to 72 memorized cases (18%), could result in stylistically similar images as well as images of persons sharing the same genre, nationality or cultural origin. We address this requirement by clustering synthetic samples according to style similarity and analyzing only sufficiently large clusters with at least 25 samples (10%). In practice, however, clusters corresponding to memorized artists were substantially larger than this minimum threshold, typically containing around 200 samples. Third, style memorization differs from normal text-conditioned generation because the eliciting prompt does not include enough information to guide the model into reproducing artist-specific characteristics, which we address by prompting the model with small prompts, with an average of 8.5 tokens on memorized cases. We further confirm that the model is able to reproduce artistic styles without sufficient information to guide reproduction with artist-only-prompts, although we suggest for future research to improve the strategy to rule out the hypothesis of prompt-guided extraction. Fourth, while several tools have been developed for style mimicry, style memorization differs from expected text-conditioned generation because it persists in models whose training objective was not style mimicry, which we address by selecting Stable Diffusion - a general-purpose model not trained for style mimicry. This explanation will be improved in the paper, particularly in Section 3.

[1] Somepalli et al., Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models, CVPR 2023

[2] Schwarzschild et al., Re-thinking LLM Memorization Through the Lens of Adversarial Compression, NeurIPS 2024

[3] Di et al., Demystifying Foreground-Background Memorization in Diffusion Models, 2025

[4] Morris et al., How Much Do Language Models Memorize?, 2025

[5] Huang et al., Neuron-Level Differentiation of Memorization and Generalization in Large Language Models, EMNLP 2025

[6] Wang et al., Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data, ICLR 2025

[7] Huang et al., Demystifying Verbatim Memorization in Large Language Models, EMNLP 2024

Reviewer_ES4gOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a new definition of style memorization based on caption atypicality and style atypicality. It tests this mainly on Stable Diffusion 1.4 and 2.1 and uses visual search to decide if generated images can be linked back to an artist.

Critical strengths

The paper looks at a type of memorization that prior work mostly misses. The experiments are fairly extensive and checking SD1 and SD2 gives some evidence the effect is not only one model.

Critical weaknesses

The biggest issue is the definition itself. Artist re-identification by visual search does not clearly show memorization versus normal learning of a recognizable style. Also some thresholds are selected to maximize ROC AUC on the same setup which makes the reported detection numbers less convincing.

Questions for authors

Can you validate the memorized labels with an independent human study or another ground truth not based on visual search? Also what happens if tau and lambda are fixed on one dataset or model and then used directly on another?

3 comments

Summary

We thank the reviewer for the thoughtful feedback. To address concerns about the extent to which reproducing artistic styles is considered memorization, raised in Weakness 1, we will improve the explanation of the distinction between style memorization and generalization in Section 3, explaining that our concept of memorization is aligned with the view of memorization and generalization as distinct but complementary phenomena, which is supported by prior work, and the criteria used to distinguish style memorization from normal model behaviour. To address concerns about less convincing results, raised in Weakness 2, we will clarify that the optimized threshold selection strategy is used to estimate an upper bound in the paragraph Threshold-selection strategies in the beginning of Section 4. Furthermore, our response to this concern explains that we further evaluate the robustness of the proposed detection strategy to more realistic threshold selection strategies. This new result comes from addressing concerns from another reviewer, which we will add to a new subsection “Robustness Analysis” in Section 4 with the goal of evaluating the robustness of the proposed detection strategy to other threshold-selection strategies. Regarding the need for human validated labels, our response to Question 1 explains the reason behind the choice of automated labels and introduces a summary of results computed with labels from human annotators, an experiment that we will expand and add results to the appendix while summarizing the main insights in a new subsection “Robustness Analysis” in Section 4 with the goal of evaluating the robustness of the proposed detection strategy to other labelling procedures. Our response to Question 2, regarding model and dataset transferability, explains how the experimental phase with 2 models can support model transferability to a certain extent, even though experiments with other models are left for future work, and explains how a new and more robust threshold selection process, estimating thresholds with non-overlapping data, can support dataset transferability.

Human labels (Question 1)

We thank the reviewer for this suggestion. In addition to the automated labels determined by visual search, we conducted a human assessment of style memorization in which authors used the annotation html file available in our repository, which presents the top-5 most stylistically similar synthetic images per candidate style, and independently classified each case with respect to style memorization. Annotations are aggregated over all authors such that a style is labeled as memorized only when all authors independently agree, making these human labels somewhat conservative. Thus, instead of 48 memorized artistic styles (34%) human annotators agreed that 45 artistic styles were memorized, in which 69% of these (31) were also classified as memorized by visual search. As discussed in Section A.5, these human labels assign a positive label to 14 additional styles compared with automated labels which correspond to generated images that are visually close to training images from the target artist, but are also similar to works from other artists, causing the automated visual-search pipeline to assign a negative label. Even though the automated pipeline can yield a more conservative estimate of the number of memorized styles for this reason, the overall conclusion remains unchanged, as detection with human labels leads to a similar trend, close to 21-25% TPR at 1% FPR, which suggests that detection based on the proposed definition is not sensitive to the specific labeling. We reported automated labels because the automated visual search pipeline is a scalable, practical and reproducible proxy for assessing whether generated images could be attributed to the target artist or compete with their work in real-world settings. Nevertheless, we will expand the number of human annotators, add this human-annotation procedure as an alternative labelling, report these results and highlight their agreement with results in the main document.

Dataset and model transferability (Question 2)

Regarding model transferability, our experiments on Stable Diffusion v1 (SD1) and Stable Diffusion v2 (SD2) showed that, when the dataset is fixed, using style and caption atypicality thresholds computed with SD1 yields very similar detection results on SD2. This suggests that style and caption atypicality do not strongly depend on the underlying model used to compute elements of those metrics (gram matrix and text embedding). However, since SD1 and SD2 share similar diffusion architectures, further validation with more diverse generative models would be valuable, which we will propose as future work. Regarding dataset transferability, we evaluated the robustness of threshold selection through 5-fold cross-validation procedure adapted to threshold selection, where thresholds were estimated on 20% of the dataset and then applied to the remaining 80%, which resulted in a mean TPR at 1% FPR of 20.7% with a standard deviation of 2.5%, suggesting that the proposed detection strategy is robust to non-optimal thresholds, particularly when estimated with a small number of non-overlapping artistic styles.

Less convincing results (Weakness 2)

We thank the reviewer for highlighting this point. We agree that selecting thresholds to maximize ROC AUC on the same evaluation set, in the optimized threshold selection strategy, can lead to optimistic estimates. However, we also evaluate style-guided and caption-guided threshold-selection strategies, where thresholds are determined from style and caption atypicality statistics and the desired number of artistic styles. For this reason, we view the optimized strategy as an upper bound of style memorization detection rather than as a realistic strategy for deployment. Nevertheless, results do not change significantly with this optimized strategy, particularly comparing with the caption-guided as shown in Figure 2 (b) and (c). To further address the concern about less convincing results, we performed 5-fold cross-validation procedure adapted to threshold selection, partitioning the target dataset into 5 folds, estimating thresholds on 20% of the dataset (1 fold), and evaluating detection performance on the remaining 80% using only thresholds derived from the validation fold, which resulted in a mean TPR at 1% FPR of 20.7% with a standard deviation of 2.5% across folds, which exhibits a similar trend as reported in the main document (approximately 21-25% TPR at 1% FPR). This result shows that detection performance is not highly sensitive to the particular dataset used to estimate style and caption atypicality thresholds, nor to the threshold selection strategy. We will add this result to the main document to further support the robustness of the proposed approach.

Style memorization vs generalization (Weakness 1)

We agree that artist re-identification by visual search alone does not clearly distinguish memorization from normal learning of a recognizable style. However, our goal is not to identify every successful artist re-identification as memorization, but instead to identify cases where the model retains artist-specific characteristics to an unusual extent such that it consistently reproduces those characteristics failing to generate sufficiently diverse images based on non-overly descriptive prompts. Our notion of memorization is aligned with the view that memorization and generalization are distinct but complementary phenomena, supported by prior work [1-7], where memorization corresponds to the retention and consistent reproduction of training-specific information, while generalization fills in the remaining visual details, enabling the generation of distinct images that are not exact replicas of the training examples but share training-specific information (i.e. artistic style). In contrast with prior work, we address the interplay between memorization and generalization by decoupling the extraction accuracy (style loss) from the concept of memorizable style (style and caption atypicality). This allows us to detect memorization beyond exact extraction, particularly when the model generates synthetic images that exhibit training-specific information traceable to training data (with artist re-identification) while being pixel-wise distinct from training images reflecting the model's generalization capability.

To distinguish this undesired phenomenon from normal text-conditioned generation, our definition incorporates several additional requirements related to 1) reproduction quality; 2) repetitive behaviour; 3) insufficient prompt guidance; 4) undesired model use. First, style memorization differs from normal text-conditioned generation due to the fact that synthetic samples exhibit artist-specific characteristics remarkably similar to original works to the point of allowing visual search engines, with access to training and non-training artworks from several artists, to re-identify the original author, which underscores the threat that this capacity can have. Second, style memorization differs from normal text-conditioned generation because it is consistent across many generations, revealing the failure of the model’s generalization capacity to produce distinct images that still follow the prompt, for example artist-name-only prompts, which led to 72 memorized cases (18%), could result in stylistically similar images as well as images of persons sharing the same genre, nationality or cultural origin. We address this requirement by clustering synthetic samples according to style similarity and analyzing only sufficiently large clusters with at least 25 samples (10%). In practice, however, clusters corresponding to memorized artists were substantially larger than this minimum threshold, typically containing around 200 samples. Third, style memorization differs from normal text-conditioned generation because the eliciting prompt does not include enough information to guide the model into reproducing artist-specific characteristics, which we address by prompting the model with small prompts, with an average of 8.5 tokens on memorized cases. We further confirm that the model is able to reproduce artistic styles without sufficient information to guide reproduction with artist-only-prompts, although we suggest for future research to improve the strategy to rule out the hypothesis of prompt-guided extraction. Fourth, while several tools have been developed for style mimicry, style memorization differs from expected text-conditioned generation because it persists in models whose training objective was not style mimicry, potentially conflicting with data owners consent, which we address by selecting Stable Diffusion to perform experiments - a general-purpose model not trained for style mimicry. This explanation will be improved in the paper, particularly in Section 3.

[1] Somepalli et al., Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models, CVPR 2023

[2] Schwarzschild et al., Re-thinking LLM Memorization Through the Lens of Adversarial Compression, NeurIPS 2024

[3] Di et al., Demystifying Foreground-Background Memorization in Diffusion Models, 2025

[4] Morris et al., How Much Do Language Models Memorize?, 2025

[5] Huang et al., Neuron-Level Differentiation of Memorization and Generalization in Large Language Models, EMNLP 2025

[6] Wang et al., Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data, ICLR 2025

[7] Huang et al., Demystifying Verbatim Memorization in Large Language Models, EMNLP 2024

Reviewer_9KrTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates style-level memorization in text-to-image generative models, arguing that existing definitions of memorization, which primarily focus on exact or near-exact reproduction of individual training images, fail to capture more subtle forms of memorization, particularly the reproduction of distinctive artistic styles.

The authors propose (τ,λ)(\tau,\lambda)-memorization, a new definition that considers an artistic style memorized if it can be extracted from a generative model and is sufficiently atypical at both the caption level (τ)(\tau) and image/style level (λ)(\lambda). The central intuition is that reproducing common visual characteristics may reflect normal generalization, whereas consistently reproducing an unusual, artist-specific style from a relatively simple prompt may indicate memorization.

To operationalize this definition, the authors introduce two metrics:

  • Caption Atypicality: Measures how unusual and informative the textual tokens associated with an artistic style are.
  • Style Atypicality: Measures how distinctive an artistic style is relative to other styles, using image representations based on Gram matrices. The authors evaluate their approach on Stable Diffusion 1.4 and 2.1 using artistic styles extracted from their training datasets. They identify potentially memorized styles by generating synthetic images and using an automated visual search pipeline to determine whether the original artist can be re-identified from these images.

The experiments report that Stable Diffusion 1.4 memorizes approximately 34% of the evaluated artistic styles, while the proposed definition enables detection with a true positive rate (TPR) exceeding 20% at a 1% false positive rate (FPR), substantially outperforming an adapted version of conventional sample-level memorization detection. Additional analyses suggest that caption and style atypicality are associated with memorization risk, particularly for artists less frequently represented in the training data.

Critical strengths
  • This work picks a real, underexplored problem and frames it clearly. Most memorization work looks at exact or near-exact copies of single samples. This paper moves to a group-level, partial kind, artistic style. That’s the kind of copying at the center of current artist lawsuits and the U.S. Copyright Office reports. Table 1 sorts prior definitions by granularity, type of extraction, and the principle used to separate expected from unexpected behavior, which places the gap clearly. Using atypicality (separately in captions and in images) as that principle is a new choice. It fits the known finding that outliers get memorized more, and it splits “what is memorizable” (τ,λ)(\tau, \lambda) from “was it extracted” (ϵ)(\epsilon). That split lets extraction be loose without making the definition loose. Here, loose extraction means the style can be recognized while the content differs.
Critical weaknesses
  1. The evaluation is not reliable enough.

    • The labels come from Google Vision + Gemini and were never checked by humans. The authors themselves found 14 styles they judged reproduced that the pipeline missed, against only 48 positives.
    • Prompts often contain content words, so a match may come from what the image shows rather than its style.
    • τ\tau and λ\lambda are tuned on the test labels. There is no held-out split and no confidence intervals.
    • With about 150 groups, 1% FPR comes down to roughly one false positive, so the 20–50$\times$ TPR/FPR ratios are fragile.
  2. The paper doesn't separate memorization from generalization.

    • Producing an artist's style from their name may simply be normal text-conditioned generation.
    • There are no counterfactual controls, such as retraining without the artist's works or testing artists absent from the training data.
    • The "atypicality" cutoff is tuned on the outcome, which makes the argument risk being circular: distinctive styles are both easier to reproduce and easier for visual search to match.
Questions for authors
  1. Can you check the visual-search labels against human annotators on a subset and report agreement? Given that 14 styles you judged reproduced were labeled non-memorized, how do the results change if those are counted as positives?
  2. Many prompts include content words (e.g., "woman wheelbarrow"). Does re-identification still succeed with artist-name-only prompts, or when the generated content clearly differs from known artworks?
  3. τ\tau and λ\lambda are tuned on the evaluation labels. Can you report results with thresholds chosen on a held-out split (or via cross-validation), with confidence intervals for TPR at 1% FPR?
  4. What evidence separates style memorization from expected text-conditioned generalization? For instance: a fine-tuning or ablation experiment that removes an artist's works, or results on artists absent from (or very rare in) LAION.
  5. Section A.5 reports 160 groups with an artistic style, while the figures use 143, and 48 memorized is described as $\approx$34%. Which number is correct, and how were groups excluded?
  6. In the adapted (k,ℓ,ϵ)(k, \ell, \epsilon)-eidetic baseline, kk is effectively not varied. Would a stronger or more natural adaptation (e.g., per-sample extraction with a relaxed ϵ\epsilon) close the gap with (τ,λ)(\tau, \lambda)-memorization?
4 comments

Summary

We thank the reviewer for the thoughtful feedback. Our response to Question 1 and Weakness 1.1, explains the reason behind the choice of automated labels and introduces a summary of results computed with labels from human annotators, an experiment that we will expand and add results to the appendix while summarizing the main insights in a new subsection “Robustness Analysis” in Section 4 with the goal of evaluating the robustness of the proposed detection strategy to other labelling procedures. Weakness 2.3 is about the same topic, which we addressed by performing a risk factor analysis based on human labels, which we will summarize the main insights in Section 4.3 (Risk Factors) and present results in Section A.10. In our response to Question 2 and Weakness 1.2 regarding concerns about prompt guided extraction, we analysed the influence of different prompts in the extraction of the same artist, which we will expand and summarize the main insights in the beginning of Section 4. To answer Question 3 and Weakness 1.3, we evaluated detection with a more robust threshold estimation, which we will add to a new subsection “Robustness Analysis” in Section 4 with the goal of evaluating the robustness of the proposed detection strategy to other threshold-selection strategies. To address concerns about the extent to which reproducing artistic styles is considered memorization, raised in Question 4 and Weakness 2.1, we will improve the explanation of the distinction between style memorization and generalization in Section 3, explaining that our concept of memorization is aligned with the view of memorization and generalization as distinct but complementary phenomena, which is supported by prior work, and the criteria used to distinguish style memorization from normal model behaviour. Our response to Question 5 clarifies the number of memorized artistic styles and we will correct this inconsistency in Section A.5 in the revised version. Regarding Weakness 1.4, our response clarifies the dimension of the target dataset used to compute the reported results, which we will clarify in the Experimental Setup Overview paragraph, in the beginning of Section 4, and in Figures 2 and 3. Our response to Question 6 explains the reason why one of the past definitions’ parameters is not varied in the main document, as explained in the appendix A.12 and why we believe that the suggested per-sample adaptation of the baseline would have the same limitation. Regarding the concern about the lack of counterfactual controls, useful to guarantee that the prompt does not contain enough information to guide style mimicry by itself, raised in Weakness 2.2, we explained how we currently ensure insufficient prompt guidance and leave the development of a stronger approach for future research.

Human labels (Question 1 and Weakness 1.1)

We thank the reviewer for this suggestion. In addition to the automated labels (determined by visual search) used in the paper, we conducted a human assessment of style memorization using the annotation html file available in our repository, which presents the top-5 most stylistically similar generated images for each candidate style. The repository also includes the resulting aggregated annotations, where a style is labeled as memorized only when all authors independently agree, making these human labels somewhat conservative. These human labels assign a positive label to 14 additional styles compared with automated labels, thus instead of 48 memorized artistic styles (34%) human annotators agreed that 45 artistic styles were memorized, in which 69% of these (31) were also classified as memorized by visual search. As discussed in Section A.5, these 14 cases correspond to generated images that are visually close to training images from the target artist, but are also similar to works from other artists, causing the automated visual-search pipeline to assign a negative label. Recomputing detection using these human labels does not significantly change results, as it leads to a similar trend of close to 21-25% TPR at 1% FPR. While the automated pipeline can yield a more conservative estimate of the number of memorized styles, by returning similar artists including non-training ones, the overall conclusions remain unchanged, indicating that results from using the proposed definition for detection are not sensitive to the specific labeling. We mainly reported automated labels because the automated visual search pipeline is a scalable, practical and reproducible proxy for assessing whether generated images could be attributed to the target artist or compete with their work in real-world settings. Nevertheless, we will expand the number of human annotators, add this human-annotation procedure as an alternative labelling, report these results in the appendix, and highlight their agreement with results in the main document obtained with automated labels.

Risk factor analsys with human labels (Weakness 2.3)

We agree that distinctive styles could be easier for visual search engines to identify and trace back to the original artist, however the memorization label is attributed based on the analysis of synthetic samples and not real artworks, which highlights that generative models retain artist-specific information and reproduce it with remarkable quality allowing visual search engines to recognize the distinctive factor of original artworks and return the correct artist. Nevertheless, we acknowledge that the automated visual search pipeline can have limitations, particularly by providing a more conservative estimation of the number of memorized styles. For that reason, we will add results obtained from human labelling, as mentioned in answer to Question 1, that present similar insights regarding detection performance and risk factors, despite identifying 14 additional memorized styles that the pipeline missed by returning other artists with similar styles. Particularly regarding the analysis of risk factors, style and caption atypicality achieve odds ratio above 1 in the low support regime, meaning that both metrics are a risk factor particularly for artists less frequently represented in the training dataset. Insights from this risk factor analysis for human labels will be summarized in Section 4.4 and detailed in Section A.10.

Prompt guided extraction (Question 2 and Weakness 1.2)

We thank the reviewer for this question. Our results in Section 4.3 already evaluate artist-name-only prompts and show that approximately 18% of a much larger set of artists are memorized according to the automated visual search pipeline, showing that the model consistently reproduces recognizable stylistic characteristics even when prompted only with the artist's name, without requesting a painting, and without any content description. Focusing on the 20 artists common on both experiments, 9 remain memorized when using artist-name-only prompts, showing that artist re-identification can succeed without content-related cues, whereas 5 artists are memorized only when content words are included, suggesting that certain content descriptions can help elicit memorized styles. However, for 4 artists, adding content-related words steers the generation process away from the artist's specific style. Results from this subset of overlapping artists suggest that content-related words are not the main factor driving style reproduction, as artistic styles can often be reproduced even when the prompt contains only the artist's name. We will expand experiments of overlapping artists to further strengthen the hypothesis that content-related words are not the main factor driving memorization and summarize the main insights in the beginning of Section 4.

Robust threshold estimation (Question 3 and Weakness 1.3)

We thank the reviewer for this suggestion. First, we would like to clarify that thresholds are not selected using memorization labels, they are estimated from the distribution of style and caption atypicality values across artists, regardless of whether those artists are labeled as memorized or not. To further evaluate the robustness of the proposed detection strategy, we performed 5-fold cross-validation procedure adapted to threshold selection, partitioning the target dataset into 5 folds, estimating thresholds on 20% of the dataset (1 fold), and evaluating detection performance on the remaining 80% using only thresholds derived from the validation fold, which resulted in a mean TPR at 1% FPR of 20.7% with a standard deviation of 2.5% across folds. The cross-validated results exhibit the same trend as those reported in the main document, approximately 21-25% TPR at 1% FPR, falling within the confidence interval. Although threshold selection is an important part of the detection strategy, as it distinguishes styles that are easy and expected to be reproduced (objects in a monochromatic background) from those that are harder to reproduce (artistic styles), this confidence interval shows that detection performance is not highly sensitive to the particular dataset used to estimate style and caption atypicality thresholds, as it is stable when thresholds are estimated on held-out data. We will add this result to the main document to further support the robustness of the proposed approach.

Target dataset dimension (Weakness 1.4)

Regarding weakness 1.4, we want to clarify that the reported TPR at 1% FPR is not computed over only the 143 artistic-style groups, the negative class also includes more than 3200 groups without an artistic style (i.e. groups without artworks, see Section A.5), in addition to the 95 groups with an artistic style labeled as non-memorized, resulting in a negative class with over 3300 examples. For each reported TPR/FPR value along the x-axis in Figure 2, these 3300 negative examples are filtered according to the corresponding threshold of style and caption atypicality. Moving to the right along the x-axis corresponds to progressively relaxing thresholds, meaning that these negative examples, particularly groups without artistic styles, are no longer filtered out, hence the reported trend of 21-25% TPR at 1% FPR corresponds to 10-12 memorized styles detected (21-25% x 48 positive examples) and approximately 33 false positives. Analogously, 14-16 memorized styles detected and 165 false positives (30-33% TPR at 5% FPR) and 20-22 memorized styles detected and 330 false positives (42-46% TPR at 10% FPR). We will clarify in the Experimental Setup Overview paragraph, in the beginning of Section 4, and in the x-axis of Fig. 2 and Fig. 3, that the reported TPR at 1% FPR is not computed over only the 143 artistic-style groups but instead over the whole target dataset.

Clarification number of memorized styles (Question 5)

We thank the reviewer for pointing this out. The correct numbers are 143 groups with an artistic style, of which 48 are labeled as memorized (approximately 34%). The value of 160 artistic-style groups reported in Section A.5 is an outdated number that was inadvertently left in the appendix during manuscript revisions. We removed 17 groups from the set of groups containing an artistic style because none of their captions included the artist name, despite containing images from websites of arwork prints, thereby removing noisy groups (e.g. with several art-like prints not associated with a real artist). We apologize for this inconsistency and will correct Section A.5 in the revised version.

Prior work comparison (Question 6)

We thank the reviewer for this question. In the adaptation of k-eidetic memorization, k exhibits limited variability because groups are restricted to at most 50 samples (Section A.12), and detection performance is mainly driven by epsilon, which determines both the target dataset composition and the extraction accuracy. Regarding the suggestion of a more natural adaption, we acknowledge that a per-sample extraction with a relaxed epsilon could also detect memorized artistic styles, however the main limitation remains - samples without an artistic style are not filtered out and therefore a relaxed epsilon will predict them as memorized, increasing FPR. In addition, considering a style to be memorized based on a single memorized artwork could introduce a different type of false positive, namely when a single artwork is heavily duplicated and therefore memorized in contrast with the remaining works. In this case, the model cannot meaningfully compete with the artist, since it only generates images similar to a well-known and publicly available artwork, whose authorship is already known.

Lack of counterfactual controls (Weakness 2.2)

Regarding the concern about the lack of counterfactual controls, such as retraining without the artist's works or testing artists absent from the training data, we agree that it could further strengthen the hypothesis that the prompt does not contain enough information to guide style mimicry by itself, since any successful reproduction of non-training artistic styles would be attributed to the prompt. We addressed the requirement of insufficient prompt guidance by constructing non-overly descriptive and small eliciting prompts, with an average of 8.5 tokens, based on captions of artworks, making prompt-guided reproduction unlikely. However, we believe that failing to extract non-training artistic styles does not guarantee that the eliciting prompt does not guide reproduction. We leave the development of stronger methodologies for ensuring insufficient prompt guidance as future work. This concern is particularly relevant for memorization in LLMs, which requires special attention to the eliciting prompt to prevent prompt-guided extraction, since prompts and outputs share the same modality and carefully crafted prompts can directly elicit memorized content. In contrast, prompts with an average of 8.5 tokens provide less information than the generated image, making prompt-guided reproduction less likely in text-to-image models.

Style memorization vs generalization (Question 4 and Weakness 2.1)

We agree that reproducing characteristics of an artistic style when prompted with the artist name is expected model behavior. However, our goal is not to identify every successful artist-name-conditioned generation as memorization, but instead to identify cases where the model retains artist-specific characteristics to an unusual extent such that those generations constitute a threat for the artist’s work. Our notion of memorization is aligned with the view that memorization and generalization are distinct but complementary phenomena, supported by prior work [1-7], where memorization corresponds to the retention and consistent reproduction of training-specific information, while generalization fills in the remaining visual details, enabling the generation of distinct images that are not exact replicas of the training examples but share training-specific information (i.e. artistic style). In contrast with prior work, we address the interplay between memorization and generalization by decoupling the extraction accuracy (style loss) from the concept of memorizable style (style and caption atypicality). This allows us to detect memorization beyond exact extraction, particularly when the model generates synthetic images that exhibit training-specific information traceable to training data (with artist re-identification) while being pixel-wise distinct from training images reflecting the model's generalization capability.

To distinguish this undesired phenomenon from normal text-conditioned generation, our definition incorporates several additional requirements related to 1) reproduction quality; 2) repetitive behaviour; 3) insufficient prompt guidance; 4) undesired model use. First, style memorization differs from normal text-conditioned generation due to the fact that synthetic samples exhibit artist-specific characteristics remarkably similar to original works to the point of allowing visual search engines, with access to training and non-training artworks from several artists, to re-identify the original author, which underscores the threat that this capacity can have. Second, style memorization differs from normal text-conditioned generation because it is consistent across many generations, revealing the failure of the model’s generalization capacity to produce distinct images that still follow the prompt, for example artist-name-only prompts, which led to 72 memorized cases (18%), could result in stylistically similar images as well as images of persons sharing the same genre, nationality or cultural origin. We address this requirement by clustering synthetic samples according to style similarity and analyzing only sufficiently large clusters with at least 25 samples (10%). In practice, however, clusters corresponding to memorized artists were substantially larger than this minimum threshold, typically containing around 200 samples. Third, style memorization differs from normal text-conditioned generation because the eliciting prompt does not include enough information to guide the model into reproducing artist-specific characteristics, which we address by prompting the model with small prompts, with an average of 8.5 tokens on memorized cases. We further confirm that the model is able to reproduce artistic styles without sufficient information to guide reproduction with artist-only-prompts, although we suggest for future research to improve the strategy to rule out the hypothesis of prompt-guided extraction. Fourth, while several tools have been developed for style mimicry, style memorization differs from expected text-conditioned generation because it persists in models whose training objective was not style mimicry, potentially conflicting with data owners consent. We address this requirement by selecting Stable Diffusion to perform experiments - a general-purpose model not trained for style mimicry. This explanation will be improved in the paper. Furthermore, experiments with non-training artists or with artists with very few training images (<5) based on artist-only-prompts will be added to the paper to further strengthen the fact that artist re-identification makes synthetic samples traceable to training data.

[1] Somepalli et al., Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models, CVPR 2023

[2] Schwarzschild et al., Re-thinking LLM Memorization Through the Lens of Adversarial Compression, NeurIPS 2024

[3] Di et al., Demystifying Foreground-Background Memorization in Diffusion Models, 2025

[4] Morris et al., How Much Do Language Models Memorize?, 2025

[5] Huang et al., Neuron-Level Differentiation of Memorization and Generalization in Large Language Models, EMNLP 2025

[6] Wang et al., Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data, ICLR 2025

[7] Huang et al., Demystifying Verbatim Memorization in Large Language Models, EMNLP 2024

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.