- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper is on data laundering, which a model is trained on rewritten versions of documents so that membership-inference audits run on the originals fail. The paper argues that prior evaluations cannot tell apart three cases: the detector was weak to begin with, the rewriting really did hide the data, or the model did not learn from the rewrites. The paper proposes LaunderBench, which trains three matched models from the same Pythia-1.4B checkpoint: one on original documents mixed with filler text, one on rewrites mixed with filler, and one on filler only. By auditing all three models on the original documents, the researchers evaluate detection success through AUROC and true positive rates alongside utility measured by perplexity reduction. The experiments find: (1) Models trained on rewrites still learn from the source documents, but less than models trained on the originals, and the gap grows with more training. (2) When the model has seen the data only a little, detectors that use the model alone barely work even without rewriting; only the detector that compares against a reference model works. (3) Which rewriting method looks hardest to detect depends on the metric: the ranking by AUROC can differ from the ranking when almost no false positives are allowed.
Critical strengths
- The paper points out an interesting question that a low detection score after rewriting could simply mean the detector was weak from the start. The three-model design (originals, rewrites, filler only) separates these cases cleanly.
- The main experiments on Pythia-1.4B are broad: four corpora, nine rewriting methods, two seeds, several training lengths, and several source ratios. The detectors are also grouped by what access they need (target model only, extra prefix text, or a reference model), so methods with different requirements are not compared as if they were equal. This makes the results more convincing than a study with one rewriting method, one detector, or one training setting.
- The benchmark is easy to extend, since new detectors can be tested on the released checkpoints without retraining.
Critical weaknesses
-
The utility metric and the detectors rely on the same signal, the model's likelihood on the original documents. Utility is the perplexity reduction on the originals relative to the filler-only model, and the Reference detector thresholds almost the same quantity per document. As a result, the rewriting methods rank in the same order on both detection and utility in Table 1, and the paper does not test whether a rewrite actually hides the data or just makes the model learn less. A model that learned the facts without the exact wording would be scored as low utility, even though that is the most concerning form of laundering. The utility criterion (gain > 0) is also too weak: every setting passes it, including SAGE-R, where most factual entities are masked out. A utility measure that does not depend on the verbatim text, such as question answering on the facts in the source documents, and a control experiment in which an unlaundered model is trained with less exposure so that it reaches the same gain as the laundered model, which would show whether rewriting hides the data or the model has simply learned less.
-
The paper claims coverage across model scales and architectures, but the experiments on the larger Pythia models and GPT-Neo (Appendix F) do not follow the paper's own protocol. They train only the laundered model, with no unlaundered or filler-only counterpart, use the pretrained checkpoint as the reference, and measure perplexity on the rewritten training texts instead of the originals. Without these controls, none of the three criteria the paper proposes can be checked in these settings, so it is unclear whether the detection numbers reflect real evasion or whether the models learned anything about the original sources. In practice, the three main findings are supported only on Pythia-1.4B. It would be better to rerun a subset of these experiments with the full three-model setup.
-
The paper would be stronger if it also evaluated laundering-aware audits [1] and dataset-level inference [2], not only document-level audits on the verbatim originals.
-
The training setup is quite different from how LLMs are really trained, so I am not sure how much the numbers mean. The filler data is only 1,000 documents from the Pile training split, which Pythia has probably already seen in pretraining, and each one is repeated about 40 times. The paper says the filler-only model gets worse on the originals because of domain drift, but I think it is more likely just overfitting to this small set. This matters because utility is measured against this baseline, so the gains can look bigger than they are. In fact, in 62 of 72 cases the rewrite-trained model gets worse on the originals in absolute terms as training goes on. Also, each source document is seen 8–12 times, while in real pretraining a document is usually seen about once, so the setting is closer to fine-tuning with heavy memorization. I suggest using a larger filler set without repetition, adding the pretrained checkpoint as a baseline, and testing the case where each source is seen only once.
-
The description of the surrogate-selected rewriting is inconsistent. Section 3.1 says five candidate rewrites are generated for each document and the one with the lowest surrogate detection score is kept. Appendix A.1 instead says five prompt variants are tested on the first 20 documents of each corpus, and the best prompt is then used to rewrite all remaining documents. These are different procedures, and the appendix version seems to be the one actually used (Table 3 also calls it "surrogate-selected prompt"). If so, the adversary is much weaker than the main text suggests, since nothing is selected per document.
[1] Combating Data Laundering in LLM Training. Li et al., arXiv 2026. [2] LLM Dataset Inference: Did You Train on My Dataset? Maini et al., arXiv 2024.
Questions for authors
- If an unlaundered model is trained with less exposure so that its gain matches that of a laundered model, is the laundered model still harder to detect?
- Do the three main findings still hold on Pythia-2.8B, Pythia-6.9B, or GPT-Neo when the unlaundered and filler-only models are also trained?
- Does the filler-only model's perplexity on the originals still rise when the filler is a larger set of documents that were not in pretraining and are not repeated?
- Which surrogate-selection procedure was actually used: the per-document candidate selection in Section 3.1 or the per-corpus prompt selection in Appendix A.1?
13 comments
Responses to Reviewer Ki7m (Part 1/5)
W1 & Q1: "The utility metric and the detectors rely on the same signals ... If an unlaundered model is trained with less exposure so that its gain matches that of a laundered model, is the laundered model still harder to detect?"
Thanks for the constructive question. Indeed, both and all verbatim-query detectors score target likelihoods on the original documents. Thus, this shared signal means that lower detectability and lower verbatim learning move together (fixed-quota comparisons in Table 1), leaving genuine evasion confounded with simply learning less.
To separate the two (answer to Q1), we conducted 3 controlled comparisons at 313 optimization updates by adjusting source-token quotas within the fixed 313-update budget, so that models match on source perplexity reduction over the pretrained checkpoint . We found that rewritten models that reach the original-text learning gain are no harder to detect on average under threshold-free Reference AUROC (), whereas lower detectability appears only at the strict zero-false-positive threshold (, where most bootstrap confidence intervals span zero).
Gain-matched exposure controls at 313 updates. Because cycling a small filler pool degrades at later steps, we fix the budget at 313 updates and match source perplexity reductions over () across three setups: (i) lowering original-text quotas from 5% to 2.25% and 3.0% against 5%-quota rewrites under repeated filler across all four corpora; (ii) using 40,064 non-repeating Pile filler documents on Wikipedia and arXiv to compare single-pass originals (each source seen once) against single-pass rewrites and against 10% and 20% quota rewrites (which also alter filler allocation); and (iii) comparing those 10% and 20% non-repeating-filler rewrites against our earlier 313-update multi-pass 5% original-text baseline under the same non-repeating filler pool. Positive gains below indicate lower source perplexity than , and positive indicates higher detectability on the rewritten model (by Reference detector):
| Original Baseline (313 Updates) | Rewrites Matching Original Gain | Ref. | Strict and Seed Notes |
|---|---|---|---|
| 2.25% Originals (repeated filler, 4 corpora) | Lexical & Syntactic only (23/72 runs) | +0.006 (mean) | Lower point estimates in most matched runs; 17/23 95% CIs span 0 |
| 3.0% Originals (repeated filler, 4 corpora) | Syntactic only (4/72 runs) | +0.013 (mean) | Lower in half of matched runs; 2/4 95% CIs span 0 |
| Single-pass Originals (non-repeating filler) | 10% & 20% Wiki SAGE (all 4 runs; no arXiv or SAGE-R) | +0.009 to +0.059 | Reference AUROC Originals in every matched run |
| 5% Multi-pass Originals (non-repeating filler) | 20% Wiki SAGE, seed 137 only (+0.03 pp gain) | -0.023 (seed 137) | Seed 137 95% CI ; seed 42 misses gain (-0.03 pp), CI spans 0 |
What this means for evaluating laundering. For rewritten runs that fall below the original-text gain threshold (including all 5%-quota SAGE and SAGE-R runs), lower detectability remains tied to lower verbatim learning. Against original-text likelihood audits, laundering therefore is considered a lossy trade-off rather than lossless hiding: aggressive rewrites suppress detectability by forfeiting part of the verbatim sequence gain (lost gain), which is why LaunderBench pairs detection with lost gain against unlaundered training. Tracking both metrics still remains necessary because shifts in the upper tail of non-members decouple Reference trajectories and strict rankings from , while our further responses below (Parts 2/5 and 4/5) remove the verbatim-text coupling on both sides via unseen paraphrases and multi-paraphrase audit queries.
Responses to Reviewer Ki7m (Part 2/5)
W1: "A model that learned the facts without the exact wording would be scored as low utility ... The utility criterion (gain > 0) is also too weak: every setting passes it, including SAGE-R, where most factual entities are masked out."
We agree that original-text perplexity measures distributional sequence modeling rather than abstract factual recall, and we will replace "source knowledge" with source sequence-modeling utility in the revision to reflect this more precisely.
To examine the exact scenario you raised, i.e., training texts that preserve facts while replacing surface wording, we contrast SAGE [1] (which retains factual anchors while suppressing lexical overlap) directly against SAGE-R (which masks out factual entities) across our unseen-paraphrase and a set of newly constructed fact cloze probes. We conducted the supplementary studies as follows:
| Probe and Scope | Quantity Measured | Empirical Outcome |
|---|---|---|
| Whole unseen paraphrases (Wikipedia, 18 repeated-filler cells) | Source-over-held-out on syntactic rewriting and both SAGE generators | 95% CIs in all 18/18 trajectory cells (lexical and SAGE-R not run) |
| Factual spans, summed (Wikipedia, non-repeating filler) | Source-over-held-out conditional log-likelihood gain summed per entity/date/number span vs. | 95% CIs on lexical (from 313 up.) and SAGE (both seeds at 1,252 up., plus seed 137 at 626 up.); spans 0 on SAGE-R at all steps |
| Content words, per-token (Wikipedia & arXiv, non-repeating) | Source-over-held-out conditional log-likelihood gain per content-word token vs. (nats/token) | Unlaundered at all steps; at 1,252 up., on lexical (both corpora) and Wiki SAGE-R (seed 42), spanning 0 at 313–626 up. and on SAGE |
| Factual minus matched non-factual (Wikipedia, 1,252 up.) | Within-paraphrase per-token gain difference against non-factual content words with similar loss under | 95% CIs span 0 in every condition and seed (e.g., per-token factual-span gain is 0.057–0.063 on SAGE, with seed 42 lower CI at -0.002, vs. 0.127–0.139 on Originals) |
| Four-way factual cloze QA (Wikipedia, 313 up., 5% quota) | Top-1 conditional log-likelihood accuracy vs. and | Failed positive control: unlaundered originals achieved 33.3%, matching (33.3%) vs. (32.5%) |
Fact-preserving rewrites (SAGE) and the QA extraction boundary. On the detectability side, fact-preserving paraphrasing does not become invisible. At 1,252 non-repeating-filler updates, Reference still detects SAGE with 0.988 AUROC on Wikipedia and 0.947 on arXiv (gain-matched comparisons are present in Part 1/5, paraphrase-query audits in Part 4/5). On the utility side, SAGE transfers positive likelihood gains to unseen paraphrases at the document and summed factual-span levels, yet its per-token factual-span gain is roughly half of Original training and the within-document factual minus matched non-factual difference spans zero across all runs. Moreover, as [2] suggest, verbatim pretraining memorization does not guarantee extractable QA knowledge without explicit QA fine-tuning, whereas paraphrasing may facilitate downstream QA extraction (consistent with [1] reporting preserved LLM-judged QA utility under 3-epoch LoRA fine-tuning on LLaMA-3.2-3B).
To probe factual recall directly on our checkpoints, we built a four-way cloze test from the Wikipedia sources (non-verbatim factual prompts with 1 true answer and 3 distractors) and scored each candidate completion by its conditional log-likelihood under the 313-update models. However, even unlaundered verbatim training achieved only 33.3% top-1 accuracy, matching the unexposed filler-only model (33.3% vs. 32.5% on ). Because this probe failed its positive control even on verbatim training, we consider downstream factual QA an open boundary and leave it as an important future direction.
Response Overview
Thanks for your constructive review! We ran new controlled experiments to address each of your concerns and will incorporate all updates into the revision. Below is a quick overview before the detailed responses:
Please find the full details and tables below. We hope these additions resolve your concerns.