- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes data laundering, an attack in which a licensee fine-tunes a generator on a licensed dataset, regenerates a synthetic version of it, and trains deployed models on the regenerated data. Evaluated against nine state-of-the-art audits, laundered models score at or near the level of models trained without the licensed data, while language models still retain 82%–85% of the capability gain from the original dataset.
Critical strengths
The motivation of this paper is sound: it aims to address the problem of data laundering, which is increasingly important in an era of rapidly advancing large models such as LLMs, where the community has reached a consensus that high-quality data matters more and more.
Critical weaknesses
Weakness 1: The experiments are far from sufficient. For images, a licensee could regenerate data through many different operations, such as rotation, style transfer, resizing, or color transformation. I believe that these transformations, which vary in both granularity and modality, pose substantially different challenges and difficulties for data-licensing audits. However, I did not find sufficiently fine-grained experiments that analyze these fundamental factors one by one.
Weakness 2: The situation becomes even less convincing for text. Language can be rewritten at the word level, or augmented at the semantic level without changing the original meaning, both of which can be regarded as important instantiations of regeneration. Therefore, I do not believe that the current text experiments are sufficient to demonstrate the effectiveness of the proposed attack.
Weakness 3: The field of fingerprinting is also advancing rapidly, with a growing body of work on natural fingerprints. I believe that if such natural fingerprints can be well preserved in the regenerated dataset, then the significance of auditing is substantially weakened, since one could simply rely on natural fingerprints as the basis for attribution.
Questions for authors
See Weaknesses
2 comments
Thank you for the additional experiments. However, in my understanding, two concerns still remain: the image study evaluates mostly isolated transformations rather than composed or adaptive regeneration pipelines, and several text rewrites substantially reduce capability, making it difficult to disentangle audit robustness from utility loss. Therefore, I maintain my original rating.
Response to Reviewer i82p
We thank the reviewer for the insightful comments. We are committed to making all necessary changes and improvements. We performed the three experiments the reviewer suggested and added the results to the revised paper (in blue).
Fine-grained analysis of image transforms (W1)
We agree with the reviewer that the paper tested its membership audit (CLiD scores with CDI's t-test, Sec. 4.1) only on standard data augmentation (crops, flips and color jitter, App. E.4). However, new tests on 17 settings of rotation, resizing, color change and style transfer (Gatys et al., 2016) show that the audit still detects all 102 models trained on transformed images (p < 0.001), including those at the strongest of these settings (30° rotation, resizing to 25%, grayscale, the highest style weight). The average AUC per transform is between 0.89 and 0.99, against 0.42 for the innocent models in Table 3. This is because a transform edits each licensed image in place, so the model trains on all the licensed images. Regeneration instead trains the model only on a dataset D′ sampled from a generator fine-tuned on D (Sec. 3.2).
We added the results for each setting to Sec. 4.3 and App. E.7 (Table 14).
Word-level and semantic text rewrites (W2)
We agree with the reviewer that the paper compared laundering with only one semantic rewrite (Table 2) and no word-level rewrite. However, new tests on 8 more rewrites show that, unlike laundering, each one is either detected or loses much of the capability.
Word swaps replace 10% to 70% of the content words at 4 rates with a masked language model (RoBERTa-base). Round-trip translation uses OPUS-MT, and the other rewrites use the paper's rewriter, Qwen2.5-7B-Instruct.
We added these results to Table 2 and App. E.8 (Table 15).
Natural fingerprints (W3)
We agree with the reviewer that natural fingerprints can survive regeneration. On images, an auditor who has D and reproduces our regeneration pipeline flags laundered models more often than innocent ones (78% vs 47% at the standard threshold, 47% vs 8% at the strict one). However, at a base rate of 1 in 100, about 95% of flagged models are false positives at the strict threshold, and 98% at the standard one, against 40% to 60% for the membership audit on direct training (Fig. 3). An audit of released models can face such a low base rate. On text, the fingerprint does not separate laundered models from innocent models trained with the same pipeline (22% of each flagged at the strict threshold).
Similar to the dataset classifier of Mansour & Heckel (2025), the fingerprint audit trains a classifier to distinguish the outputs of reference models trained on D and on public data. It flags a suspect model whose outputs score above all models trained on the same artist's other public works that the auditor never saw (standard threshold), or above all other innocent models (strict threshold). Each rate combines 36 measurements from 6 suspect models (2 artists, 3 seeds), each evaluated by classifiers trained on 3 seeds of the auditor's reference models, with 2 feature sets (CLIP embeddings, and the Fourier spectrum of each image's median-filter residual).
We added this audit and its results to Sec. 5 and App. G, with a sentence on how future audits could use natural fingerprints.
Please let us know whether these results address your three concerns.