- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether near-duplicate training records cause membership-inference auditors to attribute an exact record to training when only related records were included. A four-world framework independently varies exact-record inclusion and family presence, distinguishing ordinary membership inference, family-presence inference, and exact inclusion given a family. The experiments combine natural text families with controlled image/text interventions and show substantial false attribution for an auditor calibrated only on family-absent non-members. They also find that a family-aware auditor can retain exact-record information in the language-model settings.
Critical strengths
-
The distinction between exact-record inclusion and exposure to related content is important for interpreting provenance audits. The composite-null formulation makes clear why calibration on family-absent non-members alone does not control errors when related records may be present.
-
The interventions provide useful evidence beyond observational score correlations. Exact membership is varied while family presence is controlled, with discovery/calibration separation, no-sibling controls, alternative scores, and several family constructions. The explicit family-aware comparison is especially valuable because it distinguishes false attribution from disappearance of exact-record information.
Critical weaknesses
-
The strongest provenance and mechanism claims need narrower scope. The contribution statement describes exact provenance as empirically unidentifiable. Section 6.2 does evaluate composite and oracle-GLR tests, reporting text AUCs of 0.69 and 0.74 and sharply reduced power at a low false-positive rate. This supports a practical limitation of the evaluated audit scores. Section 5.4 appropriately disclaims an information-theoretic guarantee; the contribution statement should maintain that limited scope rather than imply unidentifiability for every model-only audit. The family-conditioned results in Table 3 and Appendix H address a different estimand and do not, by themselves, resolve the unknown-family composite hypothesis. Likewise, the matched AG-News intervention in Section 5.3 supports an effect of changing the training task in that setting: the backbone, records, families and exposure are held fixed. The baseline memorization difference may be a consequence of that change, rather than an external confound. What remains less established is the broader substitution-versus-addition mechanism across memorization, exposure and objective-relative family-fidelity regimes. Appendix G's exposure controls and Appendix I's regime caveats are helpful, but the matched cross-objective contrast itself is not traced across those regimes. Corresponding matched controls or more limited main-text mechanism wording would address this concern.
-
The experiment-to-estimate mapping is insufficiently explicit for the main empirical conclusions. Appendix H states that Table 3 uses a weaker-memorization Pythia run, while other reported results use stronger full-corpus or deduplicated runs. The several AUCs, advantages, per-target thresholds, and pooled estimators cannot safely be read as interchangeable measurements. Appendix F helps, but the main results need a compact provenance table linking every headline figure/table to its checkpoint/run, target set, family intervention, calibration population, estimator, and uncertainty. The text experiments are controlled fine-tuning studies, so their conclusions should also be scoped to that regime rather than extrapolated quantitatively to arbitrary web-scale pretraining audits.
Overall assessment: I currently lean against acceptance, narrowly. The core calibration failure is important and supported by controlled experiments, including composite-null and family-aware analyses. My reservations concern the scope of the unidentifiability and general mechanism claims and the traceability of the quantitative evidence. A clearer scope and run-provenance table could move my recommendation toward acceptance.
Questions for authors
- Please scope the unidentifiability claim to the evaluated audit channels, distinguishing limited low-FPR composite-test power from family-conditioned recoverability and from a general impossibility claim.
- Please provide an experiment-provenance table covering Table 1, Table 3, Table 8, and the deduplication results, distinguishing AUC from advantage and per-target from pooled estimates, and making the run/checkpoint differences explicit alongside the estimator map in Appendix F.
- For the matched AG-News intervention, how stable is the objective effect across exposure levels or lexical-family constructions? Appendix G's text-only controls and Appendix I's regime caveats are useful; please clarify whether the cross-objective contrast itself persists in those regimes, or limit the broader mechanism claim.
- For the headline 1% per-target false-attribution result, please make explicit the threshold estimator and calibration/evaluation split sizes within the disjoint-shadow protocol, and provide the crossed-bootstrap interval. These details would help assess low-FPR precision.
4 comments
Q2 — "please provide an experiment-provenance table covering Table 1, Table 3, Table 8, and the deduplication results"
Done. We re-derived every number in Tables 1, 3, 8, and 9, Figure 2, Section 4, and the appendices from stored per-shadow scores with the paper's analyzers, and everything reproduces. The revision will add the full table (8 runs, each with model, exposure count, background size, shadow counts, and the exact results it feeds) plus an estimator tag on every reported value (per-target LiRA or pooled, AUC or advantage). The audit also surfaced items we correct.
- Table 1 "Pythia/repeat" is a second regime (4 exact exposures) rather than a replicate of the headline run (1 exposure). Relabeled.
- The repeat, Min-K20, and Min-K++ rows marked "(P)" are in fact per-target values (98/97/82%). The true pooled values are 0.994, 0.968, and 0.932. Corrected, and Min-K%++ was additionally re-run natively in the headline configuration on 96 new shadows: false attribution 0.882 [0.836, 0.921] at 1% against a held-out control of 0.016–0.019.
- Table 1 "semantic 10%" compared against a family-absent world with a different exposure. The matched comparison gives pooled 0.175 and per-target 0.089. In the headline configuration with full 32+32-shadow re-runs it is 0.331 [0.278, 0.410] per-target at 1%, independently reproduced at 0.349 on fresh shadows.
- Table 1 Qwen "37%" is a pooled estimate on the surviving band. Per-target over all targets it is 0.955 [0.852, 0.977] at 5%.
- Table 1 neighborhood "58%" came from a 6+6-shadow run. Re-run at 32+32 shadows it is 0.632 [0.524, 0.727] per-target at 1%.
- The "dedup sharpens the bands" sentence compared runs with different exposure. It is replaced by the executed target-unaware full-corpus dedup results, with false attribution 0.992 and 0.923 at τ = 0.8 and 0.5 and the accompanying cost table.
- The WikiText "28–85% across training environments" range is an exposure difference (1 against 4 exact insertions), not an environment effect. Restated.
- The Appendix F text exact-given-family "0.50" is a pooled-AUC advantage. The per-target advantage is 0.873 [0.847, 0.899]. Both are now labeled.
- Exact-record LR+ values are point ratios TPR/FPR, and only the family LR+ are bootstrap medians. The family-presence per-target thresholds run above nominal on CC-News (point estimates 0.079 and 0.106 at nominal 1% and 5%, bootstrap medians 0.110 and 0.137). With one global threshold the verdict is properly calibrated (TPR 1.000, FPR 0.012, LR+ 83). All restated.
- The shadow-free dataset-inference test's control was inflated by a bystander validation population. The reference-calibrated repair brings the control to 0.001–0.007 with false attribution still at 1.000, and the powerless WikiText cells are dropped.
None of these corrections changes a conclusion. The rows that were weakest become stronger evidence of the confound when re-run at scale. We agree the paper's estimate-to-run mapping was not readable without this table, and we thank you for prompting the audit.
Q3 — "how stable is the objective effect across exposure levels or lexical-family constructions? … please clarify whether the cross-objective contrast itself persists in those regimes, or limit the broader mechanism claim"
Both: the contrast persists in every tested cell, and we limit the mechanism claim, because its magnitude is graded exactly as you suspected. The first grid crosses 4 family constructions, 2 objectives, and 3 exposures (24 cells with 32 paired seeds each, where one cell reuses the paper's identical configuration and a batch-size control matches it, pooled exact-given-family AUC 0.532 against the paper's 0.529). A second grid adds fidelity from J ≈ 0.09 to verbatim copies and family mass from k = 1 to 8 (672 additional fine-tunes). Per-target exact-given-family AUC:
| family | cls 1/5/25 ep | LM 1/5/25 ep |
|---|---|---|
| semantic (J .01) | 0.54 / 0.53 / 0.54 | 0.99 / 1.00 / 1.00 |
| word dropout (J .52) | 0.52 / 0.51 / 0.52 | 0.88 / 0.78 / 0.81 |
| substitution (J .62) | 0.52 / 0.51 / 0.51 | 0.86 / 0.73 / 0.71 |
| substitution (J .81) | 0.52 / 0.51 / 0.51 | 0.83 / 0.66 / 0.59 |
| verbatim copies (J 1.0), 5 ep | 0.51 [0.47, 0.55] | 0.60 [0.56, 0.63] |
In all 12 matched cells at 5 epochs the LM's confidence interval lies above the classifier's, so the qualitative contrast is stable. For verbatim copies the margin shrinks to about 0.005, and the LM's R_exact falls from 0.85 (semantic) to 0.03 (copies) along fidelity and from 0.82 (k = 1) to 0.10 (k = 8) along mass. The revision will therefore replace "substitution versus addition" with a quantitative statement: the autoregressive residual is proportional to the part of the target the family fails to cover. The coverage analysis supports this directly. The cell-level Spearman correlation between the uncovered-12-gram fraction and the exact-given-family AUC is +0.95 for the LM at both 1 and 5 epochs, while the classifier stays flat at 0.50–0.54 across all coverage levels. At target level the correlation is +0.59 [0.48, 0.68] at 5 epochs and +0.68 [0.60, 0.75] at 25, robust to controlling for standard-MI AUC and length, and on the headline CC-News run it is +0.28 [0.14, 0.41]. Coverage is not the whole story, since fully covered families still retain AUC 0.60–0.86 depending on exposure and fewer siblings retain more at equal coverage, and the revision will say so.
Regarding your observation that the baseline memorization difference may itself be a consequence of the objective change: we agree, and the grid handles it by normalizing within each cell. R_exact divides each cell's exact-given-family separation by the same cell's standard-MI separation, so the reported substitution is relative to each objective's own memorization level, at matched records, families, and exposure.
Q4 — "please make explicit the threshold estimator and calibration/evaluation split sizes within the disjoint-shadow protocol, and provide the crossed-bootstrap interval"
Full specification, which we will add to the appendix.
- Calibration: 32 family-absent seeds (0–31). Per-target IN and OUT Gaussians are fitted on about 16 scores each, and one global threshold is set at the (1 − α) quantile of the 3,187 pooled calibration non-member log-likelihood ratios.
- Evaluation: 32 disjoint family-present seeds (32–63), with 3,219 family-present non-member scores.
- Crossed bootstrap (B = 1,000) over calibration seeds (refitting every Gaussian and the threshold), targets, and evaluation seeds: false attribution 0.997 [0.976, 1.000] at α = 1%, with the identical point value at 5% and 10%. The realized clean FPR is 0.016 [0.009, 0.030], TPR on clean members is 0.996, and the bystander control is 0.016 [0.003, 0.043].
- Estimator robustness: global-variance LiRA 0.997, offline one-sided LiRA 0.997, median/MAD z-score 0.996, and an assumption-free rule ("score exceeds every calibration score of the target") 0.997. Only a pooled raw-score threshold is lower (0.600 [0.241, 0.825]), and the revision will report per-target and pooled estimates side by side throughout. A split sweep from 16/48 to 48/16 calibration and evaluation seeds moves the estimate by at most 0.002, and the realized controls were re-confirmed on fresh held-out family-absent shadows at 0.016–0.018.
- Why the rate is insensitive to α: the median family-present non-member log-likelihood ratio is 274.6, against 89.5 for a true clean member, and 99.2% of family-present non-member scores exceed the largest calibration non-member score. Indeed, for 96.5% of targets a family-present non-member scores above a clean member (per-target H01-versus-H10 AUC 0.948): 2–8 trained siblings move the target's loss more than one exposure of the target itself. The failure is a distribution-level shift, not a marginal threshold effect.
Finally, we adopt your scoping request on regime. The revision will state explicitly that the text results are controlled fine-tuning studies (pythia-2.8b full fine-tuning, GPT-2, and one Qwen3.5-2B LoRA replication) and that the quantitative rates should not be extrapolated to arbitrary web-scale pretraining audits. The transferable claim is the composite-null failure mode itself.
Thank you for the careful review. We executed all four of your requests, and we believe the two conditions you named for moving toward acceptance (scoped claims and run-level traceability) are now met.
Q1 — "scope the unidentifiability claim to the evaluated audit channels, distinguishing limited low-FPR composite-test power from family-conditioned recoverability and from a general impossibility claim"
Agreed. The revised contribution statement will read:
The three quantities you asked us to distinguish are now measured separately and in composition. Composite and low-FPR power: the end-to-end gated protocol and its error decomposition. Family-conditioned recoverability: the oracle-reference auditor, with per-target exact-given-family AUC 0.82–0.94 by channel. Non-impossibility: stated explicitly, with the matched-configuration paraphrase result (false attribution 0.331 at 1%, exact record recoverable at per-target AUC 0.999) replacing the mismatched-exposure 10% figure, and a semantic family shown to be detectable at AUC 0.998 by a held-out paraphrase probe, so the family-report recommendation carries no hidden blind spot. The mechanism claim is correspondingly narrowed (your Q3). The revision will also include a claims table mapping each changed claim to its evidence.