- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper focuses on optimizing image data "non-trainable" for image generative models.
The problem significance is to prevent protected content, such as celebrity faces or IPs, from training and reproducing by the image generative model, meanwhile preserving the image utility to humans.
The paper poses this problem as "data distillation in a reverse direction of training", but reviewer would summarizes it as adversarial examples training in the image generative model context, and will keep using term "adversarial example" for the remaining of the review.
The paper shows the derivation of their adversarial objectives (image cloaking technique), which is a first-order approximation (1-step update), over the auto-regressive image generative model paradigm.
In the experiment, the paper demonstrated that a surrogate auto-regressive model yields adversarial examples can prevent both auto-regressive model and diffusion-based model from fine-tuning over them, and re-generate the target content.
The major claims are 1) novelty of the introduced optimization algorithm; 2) computational efficiency; 3) transfer across different model architectures
Critical strengths
-
The optimization problem (Eqn. 6), and the derivation of the objective mathematically make sense, and it is concise. The derivation shows how a model parameter updates for unlearning, , can connect to a data sample optimization problem, where its form minimizes the inner product of two gradients computed over original data () and the cloaked data (), mathematically enforcing two updating directions (gradients) geometrically orthogonal to each other.
-
The proposed method demonstrated cross-architecture transfer. Experiment shows that cloaked example yield over auto-regressive image generative models (emu3) can prevent diffusion model (SDXL) fine-tuning over them. This black-box transferability is valuable.
Critical weaknesses
Method
- The proposed method is hard-coded with auto-regressive text-to-image generative models. The paper discusses and implements of Eqn. 5, only with auto-regressive score function form, which reads this is a method over-engineered for the auto-regressive image gen. Conceptually, diffusion-based, flow-matching-based image generation all model a score function with different forms , can the proposed method works with those generative paradigms? The paper should includes or discusses that.
Experiment
-
Experiment lacks of model diversity. All experiments in this paper are run on two models, emu3 (auto-regressive model) and SDXL (diffusion model). While paper demonstrated that non-trainable examples optimized over emu3 transfer to prevent fine-tuning for SDXL, it is worth including results on other models beyond just emu3 and SDXL.
-
The paper claims "non-trainable", while supported evidence is only from fine-tuning tasks. The whole paper is articulated with claim "the proposed method yield data examples that image generative models cannot trained with and produce relevant examples". But the actual notion of training discussed in the paper is fine-tuning the pretrained image generative models. Training and Fine-tuning are different objectives and non-fine-tunable cannot be used to claim non-trainable. Reviewer understands that conducing controlled experiments by training generative models over the yield examples from scratch is computationally expensive, but still, current experiment results cannot support that the yield non-trainable examples {} with their original captions {} can prevent a model trained from scratch from generating the cloaked data.
-
Image utility preservation is not demonstrated or discussed in the paper. The paper claimed proposed method yield examples that are non-trainable to image generative model, while preserving image utility to humans. However, such a preservation is not demonstrated beyond visually imperceptibility to human (e.g., an optimized image visually look similar to its original version).
-
The evaluation is incomplete that paper currently missing evaluation for preserved generation quality on non-protected concepts. More specifically, all reported metrics are focusing on "how bad is the generation on the protected data, after model is fine-tuned on the cloaked data", but after fine-tuning, do models still function well for general data generation? The paper provides no answer to that. (Motivation-wise, the delivered data-cloaking method does not make sense: if I have to prevent model generate X's faces, I have to optimize cloaked samples of X's faces, then ask the model providers to fine-tune model over them --- how do you make sure models are not damaged after fine-tuning over your data)
Questions for authors
-
After fine-tuning over the cloaked examples, does the fine-tuned model still preserve generation quality over the non-protected concepts?
-
Does the proposed formulation (Eqn. 6) work with diffusion models as surrogate models? If not, any particular reason why it won't
9 comments
Response to Reviewer aiwi 2/3
3. Effectiveness Against Training From Scratch
The current experiments evaluate fine-tuning rather than training from scratch. Can the proposed method prevent models trained from scratch from learning protected data?
Thank you for raising this important question. We have conducted additional training-from-scratch experiments using the 41,600-frame Frozen dataset.
As the reviewer noted, full-scale pretraining would be prohibitively expensive, so we conducted a smaller-scale experiment. Specifically, we randomly initialized the weights of Emu3 and independently trained the model on clean and CoReDistill-cloaked Frozen frames for 3 epochs. Throughout training, we measured the loss on two datasets:
- Clean training set: The original, uncloaked versions of the frames used for training.
- Clean held-out evaluation set: Additional Frozen frames not included in the training dataset.
We found that Emu3 trained from scratch on CoReDistill-cloaked images exhibits substantially higher loss on both the clean training set and the held-out evaluation set, compared with the model trained on clean images.
The final-checkpoint results are shown below:
| Training data | Clean training loss ↓ | Clean evaluation loss ↓ |
|---|---|---|
| Clean | 6.4755 | 6.4970 |
| CoReDistill-cloaked | 6.7169 | 6.7314 |
For the complete training loss trajectories, please refer to Section 3.4 (page 9-10, lines 481–509) of the revised manuscript.
These results indicate that CoReDistill makes the original protected content significantly harder to learn not only during fine-tuning, but also during training from scratch.
4. Preservation of Image Utility
Image utility preservation is not quantitatively demonstrated beyond visual similarity.
We have additionally evaluated the visual fidelity of CoReDistill-cloaked images using PSNR, SSIM [7], LPIPS [8], and ArcFace embedding cosine similarity [9].
The evaluation covers all 50 CelebA-HQ identities, with four images per identity (200 images in total).
| Metric | Result |
|---|---|
| PSNR ↑ | 27.86 dB |
| SSIM ↑ | 0.7968 |
| LPIPS ↓ | 0.1255 |
| ArcFace Cosine Similarity ↑ | 0.8932 |
These results show that CoReDistill-cloaked facial images retain substantial structural similarity (SSIM), relatively small perceptual differences (LPIPS), and high facial identity similarity (ArcFace), while introducing only moderate pixel-level distortion (PSNR).
5. Generation Quality on Non-Protected Concepts
After fine-tuning on cloaked images, does the model still preserve generation quality on non-protected concepts? How can the method be useful if training on protected data also damages the model's general capabilities?
Evaluating generation quality on non-protected concepts is beyond the scope of this work, as CoReDistill is designed to protect data independently of AI developers' cooperation. We do not expect creators to ask AI developers to fine-tune their models on cloaked data.
This motivation is shared by existing image-cloaking research [10, 12–14], they all primarily focus on how bad is the generation quality of the protected concepts, rather than how well the generation quality of unrelated concepts is preserved.
Glaze [10], particularly, has attracted significant attention from creators and reportedly millions of downloads [11]. Such adoption demonstrates the emergent demand for proactive data protection, while the high computational cost of iterative optimization remains a barrier to large-scale deployment.
CoReDistill thus builds on this established research direction by offering a more efficient and scalable solution.
Preserving general model capabilities while respecting creators' rights is a research question primarily from the perspective of AI developers, involving a broader consideration of both parties' interests. In contrast, our work focuses exclusively on the perspective of data owners, and therefore primarily evaluates how effectively CoReDistill prevents unauthorized learning and reproduction of their protected content.
We hope this explanation clarifies why our evaluation prioritizes protection of the target content rather than generation quality on non-protected concepts.
Response to Reviewer aiwi 3/3
We sincerely thank the reviewer again for the constructive feedback, which has helped us strengthen both our experimental evaluation and the presentation of our work.
The central motivation of CoReDistill is to enable creators and data owners to protect their content from unauthorized AI training without requiring negotiation or cooperation with AI developers.
To this end, our work addresses two key challenges: cross-model transferability, protecting against unknown downstream models, and computational efficiency, shifting data cloaking from optimization to generation, making large-scale protection more practical.
We hope our additional cross-model and training-from-scratch experiments, together with the clarification of our motivation, help address the reviewer's concerns.
We believe CoReDistill can bring new insights to the research community while helping creators better protect their work. In light of these additional results and clarifications, we would be grateful if the reviewer would reconsider their assessment.
We would be happy to address any further questions or comments.
Sincerely,
The Authors
[1] “No Artists, No Art”: The Importance of Artists Rights to the genAI Ecosystem. Graphic Artists Guild. February, 2026. https://graphicartistsguild.org/no-artists-no-art/
[2] Statement on the Use of AI. Joan Mitchell Foundation. October, 2026. https://www.joanmitchellfoundation.org/journal/statement-on-the-use-of-ai
[3] Our Joint Statement Calling for Transparency, Fairness and Respect for Creators’ Rights in the Age of AI. Design and Artists Copyright Society (DACS) and Partner Organizations. October 16, 2025. https://www.dacs.org.uk/news-events/joint-statement-on-creators-rights-in-age-of-ai
[4] Protect Artists' Rights: Stop Unlicensed AI Training. Artist Management Association (AMA), via MoveOn. July 2025. https://sign.moveon.org/petitions/protect-artists-rights-stop-unlicensed-ai-training
[5] Require Meta to Give Artists a Real Opt-Out From AI Scraping. Brianna Herder, Change.org Petition. November, 2026. https://www.change.org/p/require-meta-to-give-artists-a-real-opt-out-from-ai-scraping
[6] CODA Issues Written Request to OpenAI Regarding Sora 2. Content Overseas Distribution Association (CODA), Japan. October, 2025. https://coda-cj.jp/en/news/817/
[7] Image quality assessment: from error visibility to structural similarity. Wang et al. IEEE Transactions on Image Processing, 2004
[8] The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. Zhang et al. CVPR, 2018.
[9] ArcFace: Additive Angular Margin Loss for Deep Face Recognition. Deng et al. CVPR, 2019.
[10] Shan et al. Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models. USENIX Security 2023
[11] Tech to protect images against AI scrapers can be beaten, researchers show. The Register. July, 2025. https://www.theregister.com/software/2025/07/11/tech-to-protect-images-against-ai-scrapers-can-be-beaten/1136183
[12] Ven Le et al. Anti-DreamBooth: Protecting users from personalized text-to-image synthesis. ICCV 2023.
[13] Liang et al. Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples. ICML 2023.
[14] Salman et al. Raising the cost of malicious AI-powered image editing. ICML 2023.
Response to Reviewer aiwi 1/3
Dear Reviewer aiwi,
Thank you for your thoughtful feedback. We have conducted additional experiments and revised the manuscript to address your concerns.
Before responding point by point, we would like to clarify the motivation behind our work.
CoReDistill aims to help creators and data owners protect their content from unauthorized AI training, without requiring negotiation or cooperation with individual AI developers.
Many creators and data owners have publicly demanded an end to unauthorized AI training, calling for opt-out rights, consent, and compensation [1–6]. Yet litigation and negotiation are costly and time-consuming, especially for individual creators who lack the resources to deal with AI companies worldwide, let alone individual developers who fine-tune models on their work without permission.
As a result, creators have limited practical means to prevent unauthorized training, even after explicitly expressing their objections.
This is precisely what motivates CoReDistill: providing a technical solution that allows data owners to protect their content independently of AI developers.
We hope this clarifies the motivation of our work. Our point-by-point responses follow.
1. Applicability Beyond Autoregressive Models
The reverse-distillation objective in Equation 6 is applicable to any differentiable training loss, including diffusion noise-prediction and flow-matching velocity-regression losses: δ∗=argmax∣δ∣p≤ϵ∇θL(θ,x+δ)⊤(−∇θL(θ,x)).
However, directly optimizing this objective requires expensive mixed second-order derivatives, making it difficult to scale to real-world datasets.
This motivates our central contribution: realizing reverse distillation through generation rather than iterative optimization.
While directly optimizing the objective for diffusion or flow-matching models incurs this computational cost, contrastive decoding with autoregressive models provides an efficient, one-shot generation process aligned with the reverse-distillation objective.
Therefore, our methodological reasoning is:
Reverse distillation for data protection → Avoid expensive iterative optimization → Construct samples through generation → Contrastive decoding using autoregressive models.
Autoregression is therefore a deliberate choice enabling efficient generation-based cloaking, not an overengineered design.
2. Model Diversity and Cross-Model Generalization
We agree that broader model evaluation is important. In response, we have expanded our experiments on all 50 CelebA-HQ identities to two additional downstream models: FLUX.1-dev (flow matching, DiT) and Stable Diffusion 1.4 (diffusion, U-Net).
(1) FLUX.1-dev
We expanded FLUX.1-dev evaluation from one identity (Sec 3.2) to all 50, comparing every baseline.
sksISM ↓dslrISM ↓FLUX.1-dev generates detectable faces under all conditions (FDFR = 0). CoReDistill achieves the lowest ISM under both prompts, outperforming all baselines and demonstrating cross-model protection.
(2) Stable Diffusion 1.4
We additionally evaluated CoReDistill on SD1.4 across all 50 ids.
sksFDFR ↑sksISM ↓dslrFDFR ↑dslrISM ↓We found that SD1.4, being an older model with less capable image generation than SDXL and FLUX.1-dev, is particularly sensitive to fine-tuning configurations. Despite extensive hyperparameter tuning during the rebuttal period, we failed to find a configuration that consistently produced high-quality generations, even when fine-tuning on clean images. Consequently, the FDFR results are less consistent than those on newer, more capable models.
Nevertheless, CoReDistill reduces ISM under both prompts, indicating weaker reproduction of the protected identities.
Together with the original results, these experiments demonstrate consistent reductions in identity similarity across three downstream models. The protection effects are particularly clear on the more capable models, SDXL and FLUX.1-dev.
We also recognize the value of further cross-model evaluation. We will continue cross-model evaluations during rebuttal and report new results here; otherwise, we commit to expanding them in the camera-ready version if accepted.