acceptodds
Under review as a conference paper at ICLR 2027

Near-Duplicate Families Break Exact-Record Membership Inference

Abstract

Membership inference (MI) asks whether a specific record appeared in a model's training set and is increasingly used as evidence for data provenance and copyright auditing. These applications require determining whether the exact queried record was used for training, rather than merely whether the model was exposed to similar content. Making this distinction is challenging because web-scale datasets naturally contain near-duplicates, including syndicated articles, mirrored pages, and lightly modified images. We show that this creates a fundamental confound for standard MI. A clean-reference audit typically calibrates membership against a null in which neither the queried record nor its near-duplicate family is present. In deployment, however, the queried record may be absent while a non-identical family member was used for training. We introduce a four-world audit that independently varies exact-record inclusion and family presence to separate these cases. Natural near-duplicate families cause severe false attribution. On CC-News, a clean-reference LiRA auditor labels 99.7% of family-present exact non-members as members at 1% false-positive rate. This failure persists across alternative scores, model architectures, and executed deduplication and retraining. Controlled interventions further reveal that the effect depends on the learning objective. In classification, faithful families largely substitute for the exact record, reducing exact-given-family inference to near chance. In autoregressive language modeling, the exact sequence retains a detectable residual, while family presence still confounds clean-reference decisions. These results show that positive model-only membership evidence may establish family-level exposure without establishing exact-record provenance.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Membership inference, Training-data provenance, Content-family auditing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oWZvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether near-duplicate training records cause membership-inference auditors to attribute an exact record to training when only related records were included. A four-world framework independently varies exact-record inclusion and family presence, distinguishing ordinary membership inference, family-presence inference, and exact inclusion given a family. The experiments combine natural text families with controlled image/text interventions and show substantial false attribution for an auditor calibrated only on family-absent non-members. They also find that a family-aware auditor can retain exact-record information in the language-model settings.

Critical strengths
  1. The distinction between exact-record inclusion and exposure to related content is important for interpreting provenance audits. The composite-null formulation makes clear why calibration on family-absent non-members alone does not control errors when related records may be present.

  2. The interventions provide useful evidence beyond observational score correlations. Exact membership is varied while family presence is controlled, with discovery/calibration separation, no-sibling controls, alternative scores, and several family constructions. The explicit family-aware comparison is especially valuable because it distinguishes false attribution from disappearance of exact-record information.

Critical weaknesses
  1. The strongest provenance and mechanism claims need narrower scope. The contribution statement describes exact provenance as empirically unidentifiable. Section 6.2 does evaluate composite and oracle-GLR tests, reporting text AUCs of 0.69 and 0.74 and sharply reduced power at a low false-positive rate. This supports a practical limitation of the evaluated audit scores. Section 5.4 appropriately disclaims an information-theoretic guarantee; the contribution statement should maintain that limited scope rather than imply unidentifiability for every model-only audit. The family-conditioned results in Table 3 and Appendix H address a different estimand and do not, by themselves, resolve the unknown-family composite hypothesis. Likewise, the matched AG-News intervention in Section 5.3 supports an effect of changing the training task in that setting: the backbone, records, families and exposure are held fixed. The baseline memorization difference may be a consequence of that change, rather than an external confound. What remains less established is the broader substitution-versus-addition mechanism across memorization, exposure and objective-relative family-fidelity regimes. Appendix G's exposure controls and Appendix I's regime caveats are helpful, but the matched cross-objective contrast itself is not traced across those regimes. Corresponding matched controls or more limited main-text mechanism wording would address this concern.

  2. The experiment-to-estimate mapping is insufficiently explicit for the main empirical conclusions. Appendix H states that Table 3 uses a weaker-memorization Pythia run, while other reported results use stronger full-corpus or deduplicated runs. The several AUCs, advantages, per-target thresholds, and pooled estimators cannot safely be read as interchangeable measurements. Appendix F helps, but the main results need a compact provenance table linking every headline figure/table to its checkpoint/run, target set, family intervention, calibration population, estimator, and uncertainty. The text experiments are controlled fine-tuning studies, so their conclusions should also be scoped to that regime rather than extrapolated quantitatively to arbitrary web-scale pretraining audits.

Overall assessment: I currently lean against acceptance, narrowly. The core calibration failure is important and supported by controlled experiments, including composite-null and family-aware analyses. My reservations concern the scope of the unidentifiability and general mechanism claims and the traceability of the quantitative evidence. A clearer scope and run-provenance table could move my recommendation toward acceptance.

Questions for authors
  1. Please scope the unidentifiability claim to the evaluated audit channels, distinguishing limited low-FPR composite-test power from family-conditioned recoverability and from a general impossibility claim.
  2. Please provide an experiment-provenance table covering Table 1, Table 3, Table 8, and the deduplication results, distinguishing AUC from advantage and per-target from pooled estimates, and making the run/checkpoint differences explicit alongside the estimator map in Appendix F.
  3. For the matched AG-News intervention, how stable is the objective effect across exposure levels or lexical-family constructions? Appendix G's text-only controls and Appendix I's regime caveats are useful; please clarify whether the cross-objective contrast itself persists in those regimes, or limit the broader mechanism claim.
  4. For the headline 1% per-target false-attribution result, please make explicit the threshold estimator and calibration/evaluation split sizes within the disjoint-shadow protocol, and provide the crossed-bootstrap interval. These details would help assess low-FPR precision.
4 comments

Thank you for the careful review. We executed all four of your requests, and we believe the two conditions you named for moving toward acceptance (scoped claims and run-level traceability) are now met.

Q1 — "scope the unidentifiability claim to the evaluated audit channels, distinguishing limited low-FPR composite-test power from family-conditioned recoverability and from a general impossibility claim"

Agreed. The revised contribution statement will read:

"Across the evaluated black-box channels (loss, Min-K%, Min-K%++, zlib, reference-model, neighborhood, whole-block, and semantic-probe scores), a clean-reference exact-record verdict is invalid whenever lexical near-duplicates may be present. Realized false attribution is 0.63–1.00 at nominal 1%, and 0.882 even for the most robust tested score. This is a claim about the evaluated audit channels, not an impossibility theorem. With the true family in hand, a gated family-conditioned auditor recovers the exact-record verdict, at false attribution 0.068 with power 0.917 on clean members and 0.644 on family-present members. Every reference built without the true family fails, with one true sibling at 0.972, half the siblings at 0.848, retrieval from a matching crawl at 0.434, and synthesized surrogates at 0.103 with power 0.180, because the verdict requires matching the family's duplicate count in training, which a model-only auditor cannot observe. In the high-fidelity, high-exposure, and classification regimes the exact-record separation itself additionally degrades."

The three quantities you asked us to distinguish are now measured separately and in composition. Composite and low-FPR power: the end-to-end gated protocol and its error decomposition. Family-conditioned recoverability: the oracle-reference auditor, with per-target exact-given-family AUC 0.82–0.94 by channel. Non-impossibility: stated explicitly, with the matched-configuration paraphrase result (false attribution 0.331 at 1%, exact record recoverable at per-target AUC 0.999) replacing the mismatched-exposure 10% figure, and a semantic family shown to be detectable at AUC 0.998 by a held-out paraphrase probe, so the family-report recommendation carries no hidden blind spot. The mechanism claim is correspondingly narrowed (your Q3). The revision will also include a claims table mapping each changed claim to its evidence.

Q2 — "please provide an experiment-provenance table covering Table 1, Table 3, Table 8, and the deduplication results"

Done. We re-derived every number in Tables 1, 3, 8, and 9, Figure 2, Section 4, and the appendices from stored per-shadow scores with the paper's analyzers, and everything reproduces. The revision will add the full table (8 runs, each with model, exposure count, background size, shadow counts, and the exact results it feeds) plus an estimator tag on every reported value (per-target LiRA or pooled, AUC or advantage). The audit also surfaced items we correct.

  • Table 1 "Pythia/repeat" is a second regime (4 exact exposures) rather than a replicate of the headline run (1 exposure). Relabeled.
  • The repeat, Min-K20, and Min-K++ rows marked "(P)" are in fact per-target values (98/97/82%). The true pooled values are 0.994, 0.968, and 0.932. Corrected, and Min-K%++ was additionally re-run natively in the headline configuration on 96 new shadows: false attribution 0.882 [0.836, 0.921] at 1% against a held-out control of 0.016–0.019.
  • Table 1 "semantic 10%" compared against a family-absent world with a different exposure. The matched comparison gives pooled 0.175 and per-target 0.089. In the headline configuration with full 32+32-shadow re-runs it is 0.331 [0.278, 0.410] per-target at 1%, independently reproduced at 0.349 on fresh shadows.
  • Table 1 Qwen "37%" is a pooled estimate on the surviving band. Per-target over all targets it is 0.955 [0.852, 0.977] at 5%.
  • Table 1 neighborhood "58%" came from a 6+6-shadow run. Re-run at 32+32 shadows it is 0.632 [0.524, 0.727] per-target at 1%.
  • The "dedup sharpens the bands" sentence compared runs with different exposure. It is replaced by the executed target-unaware full-corpus dedup results, with false attribution 0.992 and 0.923 at τ = 0.8 and 0.5 and the accompanying cost table.
  • The WikiText "28–85% across training environments" range is an exposure difference (1 against 4 exact insertions), not an environment effect. Restated.
  • The Appendix F text exact-given-family "0.50" is a pooled-AUC advantage. The per-target advantage is 0.873 [0.847, 0.899]. Both are now labeled.
  • Exact-record LR+ values are point ratios TPR/FPR, and only the family LR+ are bootstrap medians. The family-presence per-target thresholds run above nominal on CC-News (point estimates 0.079 and 0.106 at nominal 1% and 5%, bootstrap medians 0.110 and 0.137). With one global threshold the verdict is properly calibrated (TPR 1.000, FPR 0.012, LR+ 83). All restated.
  • The shadow-free dataset-inference test's control was inflated by a bystander validation population. The reference-calibrated repair brings the control to 0.001–0.007 with false attribution still at 1.000, and the powerless WikiText cells are dropped.

None of these corrections changes a conclusion. The rows that were weakest become stronger evidence of the confound when re-run at scale. We agree the paper's estimate-to-run mapping was not readable without this table, and we thank you for prompting the audit.

Q3 — "how stable is the objective effect across exposure levels or lexical-family constructions? … please clarify whether the cross-objective contrast itself persists in those regimes, or limit the broader mechanism claim"

Both: the contrast persists in every tested cell, and we limit the mechanism claim, because its magnitude is graded exactly as you suspected. The first grid crosses 4 family constructions, 2 objectives, and 3 exposures (24 cells with 32 paired seeds each, where one cell reuses the paper's identical configuration and a batch-size control matches it, pooled exact-given-family AUC 0.532 against the paper's 0.529). A second grid adds fidelity from J ≈ 0.09 to verbatim copies and family mass from k = 1 to 8 (672 additional fine-tunes). Per-target exact-given-family AUC:

familycls 1/5/25 epLM 1/5/25 ep
semantic (J .01)0.54 / 0.53 / 0.540.99 / 1.00 / 1.00
word dropout (J .52)0.52 / 0.51 / 0.520.88 / 0.78 / 0.81
substitution (J .62)0.52 / 0.51 / 0.510.86 / 0.73 / 0.71
substitution (J .81)0.52 / 0.51 / 0.510.83 / 0.66 / 0.59
verbatim copies (J 1.0), 5 ep0.51 [0.47, 0.55]0.60 [0.56, 0.63]

In all 12 matched cells at 5 epochs the LM's confidence interval lies above the classifier's, so the qualitative contrast is stable. For verbatim copies the margin shrinks to about 0.005, and the LM's R_exact falls from 0.85 (semantic) to 0.03 (copies) along fidelity and from 0.82 (k = 1) to 0.10 (k = 8) along mass. The revision will therefore replace "substitution versus addition" with a quantitative statement: the autoregressive residual is proportional to the part of the target the family fails to cover. The coverage analysis supports this directly. The cell-level Spearman correlation between the uncovered-12-gram fraction and the exact-given-family AUC is +0.95 for the LM at both 1 and 5 epochs, while the classifier stays flat at 0.50–0.54 across all coverage levels. At target level the correlation is +0.59 [0.48, 0.68] at 5 epochs and +0.68 [0.60, 0.75] at 25, robust to controlling for standard-MI AUC and length, and on the headline CC-News run it is +0.28 [0.14, 0.41]. Coverage is not the whole story, since fully covered families still retain AUC 0.60–0.86 depending on exposure and fewer siblings retain more at equal coverage, and the revision will say so.

Regarding your observation that the baseline memorization difference may itself be a consequence of the objective change: we agree, and the grid handles it by normalizing within each cell. R_exact divides each cell's exact-given-family separation by the same cell's standard-MI separation, so the reported substitution is relative to each objective's own memorization level, at matched records, families, and exposure.

Q4 — "please make explicit the threshold estimator and calibration/evaluation split sizes within the disjoint-shadow protocol, and provide the crossed-bootstrap interval"

Full specification, which we will add to the appendix.

  • Calibration: 32 family-absent seeds (0–31). Per-target IN and OUT Gaussians are fitted on about 16 scores each, and one global threshold is set at the (1 − α) quantile of the 3,187 pooled calibration non-member log-likelihood ratios.
  • Evaluation: 32 disjoint family-present seeds (32–63), with 3,219 family-present non-member scores.
  • Crossed bootstrap (B = 1,000) over calibration seeds (refitting every Gaussian and the threshold), targets, and evaluation seeds: false attribution 0.997 [0.976, 1.000] at α = 1%, with the identical point value at 5% and 10%. The realized clean FPR is 0.016 [0.009, 0.030], TPR on clean members is 0.996, and the bystander control is 0.016 [0.003, 0.043].
  • Estimator robustness: global-variance LiRA 0.997, offline one-sided LiRA 0.997, median/MAD z-score 0.996, and an assumption-free rule ("score exceeds every calibration score of the target") 0.997. Only a pooled raw-score threshold is lower (0.600 [0.241, 0.825]), and the revision will report per-target and pooled estimates side by side throughout. A split sweep from 16/48 to 48/16 calibration and evaluation seeds moves the estimate by at most 0.002, and the realized controls were re-confirmed on fresh held-out family-absent shadows at 0.016–0.018.
  • Why the rate is insensitive to α: the median family-present non-member log-likelihood ratio is 274.6, against 89.5 for a true clean member, and 99.2% of family-present non-member scores exceed the largest calibration non-member score. Indeed, for 96.5% of targets a family-present non-member scores above a clean member (per-target H01-versus-H10 AUC 0.948): 2–8 trained siblings move the target's loss more than one exposure of the target itself. The failure is a distribution-level shift, not a marginal threshold effect.

Finally, we adopt your scoping request on regime. The revision will state explicitly that the text results are controlled fine-tuning studies (pythia-2.8b full fine-tuning, GPT-2, and one Qwen3.5-2B LoRA replication) and that the quantitative rates should not be extrapolated to arbitrary web-scale pretraining audits. The transferable claim is the composite-null failure mode itself.

Reviewer_vh7kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work provides an investigation of the impact of near-duplicates on MIA scores. The authors claim that we should restate the problem by distinguishing between exact-record MI and near-duplicates-aware (family-aware) MI. They show, that if training set contains near-duplicates to the sample x, it inflates the result of MIA for such a sample, leading to false attribution. They propose to distinguish four world audit for only exact membership, family only membership, none of the two, and both of them at the same time. The final takeaway I read from what authors propose is that MIA methods based on black-box model audit are not only weak in real world, but additionally have confounded family vs exact membership and thus are not reliable.

Critical strengths
  1. The presentation is clean, the idea of the authors is easy to grasp and understand,
  2. The idea is shown in experiments - how near-duplicates change MIA scores etc. The investigation of what is represented by current MI attacks results when similar examples are in the dataset are interesting.
Critical weaknesses

W1. Big picture: the motivation for the proposed framework is weak, or not shown good enough. It is not clear, why near-duplicates should be banned as (partial) evidence of training on very similar data. In practice the question in training data attribution is not as strongly connected to exact-single sample attribution as the authors show it in this work.

Single sample evidence are weak for real-world models (thus dataset inference is more broadly used for stronger signal), if they show something it should rather be a strong signal to further check, rather than be split and diffused with explanations that near-duplicates are everywhere, so we cannot tell anything. If there seems to be very strong evidence on x's close family in training (samples x' etc super close to x) we already have what we can obtain at best as evidence from current SOTA MIA signal, no matter if family-or-exact.

I would like Authors comment on that point with practical motivation for their framework.

W2. If I am not mistaken - there is no direct definition of what authors define as 'family', which would be a major weakness, since now, the work is not "mapping the exact boundaries" as claimed in 481 line. Without this definition it has only near-duplicate and not concrete meaning.

W3 I cannot understand the reason behind the deduplication experiment - since authors define what the family is each time, there can always be deduplication experiment that would not show deduplication from the family samples...

Questions for authors

See weaknesses W1-W3. Additionally:

Q1. Could you share more details about MIA benchmarks you used, how they were cleaned to ensure we measure membership and not distribution drift?

Q2. How families are created for the vision corpora exactly? I have not found it in referenced appendix F in line 235.

7 comments

Thank you for the careful review.

W1 — "It is not clear why near-duplicates should be banned as (partial) evidence … dataset inference is more broadly used for stronger signal."

We do not ban family evidence. We validate it and ask that it be reported as what it is. With a global threshold, the family-presence verdict on CC-News reaches TPR 1.000 at a realized FPR of 0.012 (LR+ 83), and the gated two-stage auditor composes it with the exact-record question in one protocol. Family evidence is excellent evidence of family presence. The paper's claim is only that silently converting it into an exact-record verdict produces a stated 1% FPR whose realized value is 99.7%.

Three new experiments give the practical motivation you asked for.

  • Family evidence implicates the wrong party. We mapped all 200 natural CC-News families back to their publishers. In 84% of families, every trained sibling comes from a different publisher than the target, the median family spans 3 publishers, and 83.3% of the headline false attributions fall on targets whose own publisher's copy was never trained (in 85 families a sibling even predates the target). In a copyright or provenance dispute the question is precisely whose record was used, and under syndication strong evidence on a record's close family routinely points at the wrong plaintiff.
  • Dataset inference inherits and amplifies the confound. We ran dataset-level tests in which all m queried records are absent but their families are present: a calibrated dataset-LiRA, and a shadow-free test in the style of Maini et al. 2024, which we repaired after finding that its bystander validation set inflated the control. Reference calibration against the public base model brings that control from 0.274 to 0.001 at m = 25, and false attribution stays at 1.000. Even semantic paraphrase families go from 0.05 at m = 1 to 1.00 at m = 25. The only variant that removes the false attribution needs validation records whose families were trained exactly like the suspects', which is the dataset-level analogue of the oracle and unavailable in practice. Aggregation strengthens the signal, including the confounded part of it.
  • A strong single hit as an investigative lead: we agree, and the retrieval experiment shows its limit. Even an investigator who then retrieves the near-duplicates from a matching crawl and calibrates on them still falsely attributes 43% of family-present non-members, because the crawl reveals the family's content but not its count in the training set.

W2 — "there is no direct definition of what authors define as 'family'"

We agree and will add a definition box to Section 3, where families first appear. The definition used throughout: F(x) is the connected component containing x in the graph joining two records whenever their token 5-gram Jaccard similarity is at least τ = 0.5 (single linkage). Natural text families are mined with MinHash (128 permutations) and LSH (32 bands × 4 rows) over 256-token blocks. Constructed families state their edit process and the resulting Jaccard. Three experiments show the conclusions do not hinge on this choice.

  • Threshold sensitivity. We recomputed exact (MinHash-free) 5-gram Jaccard over all 40,000 corpus blocks and rebuilt families at τ from 0.3 to 0.9. Lowering τ from 0.5 to 0.3 adds one target family and leaves the median family size unchanged. Mining precision at τ = 0.5 is high: 97.2% of stored siblings verify in the exact component, and 1 of 200 families fails verification.
  • Band and size splits. False attribution is 0.97–1.00 in every similarity band, including the least similar one (J in [0.49, 0.6)), and for every family size.
  • Below the mining threshold. Constructed families at J ≈ 0.09–0.39, below any threshold we would mine at, already produce false attribution 0.96–1.00 on the LM at 5 epochs, even with a single sibling. The phenomenon is continuous in similarity. The threshold only determines which instances we can find, not whether the effect exists.

We will also reword the boundary claim the review points to: what the data support is a graded fidelity dependence, and the revision will claim exactly that.

W3 — "there can always be a deduplication experiment that would not show deduplication from the family samples"

You are right that a dedup pass defined relative to our own family definition would be circular, so we replaced it with fully target-unaware experiments that never see the family definition or the audit targets.

  • Survival under standard pipelines. NearDup (5-gram J ≥ 0.8), FineWeb-style (≥ 0.75), SlimPajama-style (13-gram ≥ 0.8), ExactSubstr-50, and an aggressive 5-gram ≥ 0.5 rule, applied over 50 random corpus orders. Every rule leaves at least one sibling in 100% of the 200 natural families, while the same rules delete x itself in 33–72% of the cases where x is in the corpus.
  • The whole dial, executed, with its cost. We ran target-unaware component dedup over the full training corpus at τ = 0.8 and at the aggressive τ = 0.5, retrained, and audited with unchanged clean calibration:
executed passfalse attribution (per-target, 1%)corpus removedincluded x deletedvalidation perplexity
none0.9970012.40
full corpus, τ = 0.80.992 [0.974, 0.999]4.3%43%12.32
full corpus, τ = 0.50.923 [0.894, 0.952]8.8%72%12.28

Dedup is cheap here, and perplexity even improves slightly because the pass removes duplicates rather than distinct text. It still does not fix the verdict. When x is absent, its family is a component with no x in it, so dedup keeps one sibling as the representative, and one sibling suffices. Worse, aggressive dedup creates a second kind of false attribution. At τ = 0.5 the pass deletes 72% of the x the owner believes were included, and the auditor still declares 90% of those deleted x to be members, because a sibling survived in x's place.

The reason behind the experiment is therefore: no setting of the dial removes the problem, because dedup decides which copy of a story survives, not whether the story is in the model, and an exact-record verdict cannot tell the survivor from x. We will rewrite the paper's deduplication discussion around this executed trade-off.

Q1 — "how they were cleaned to ensure we measure membership and not distribution drift"

We use no external MIA benchmark. Every result comes from our own four-world shadow constructions on CC-News, WikiText-103, AG-News, and CIFAR-10/100, which removes the usual cleaning question at its root: there is no cross-population comparison anywhere in the protocol. All four worlds use the same target records, background, and seeds, and only the two inclusion bits change, drawn as independent Bernoulli(0.5) per target and seed, so the member and non-member populations are exchangeable by construction. Three checks confirm this empirically.

  • Model-free attacks are blind. Length, zlib, surprisal, and label features give AUC 0.49–0.51 on every contrast in every setting.
  • The family-absent world is verified family-free. An exact inverted 5-gram index of the training background against the 200 targets shows no J ≥ 0.8 relative of any target, J ≥ 0.5 relatives (max 0.523) for only 2 of 200 targets, and a median closest-background similarity of J = 0.008.
  • Bystander controls. 100 no-family targets pass through the identical selection and calibration pipeline and stay at the nominal rate (0.016 at α = 1%, CI [0.003, 0.043]). Retraining a seed reproduces the stored scores at correlation 0.994.

Q2 — "How families are created for the vision corpora exactly? I have not found it in referenced appendix F in line 235."

We apologize for the broken pointer. We will fix it and add the full recipe, which we have verified by re-execution: all 21 CPU-buildable family files reproduce byte-identically from the released builders. The submission's vision families are constructed from the target itself. The "near-duplicates" are random-flip plus 4-pixel reflect-pad-crop views of x, with raw SSIM 0.16–0.19 but, after rigid alignment, RMSE 0 and SSIM 1.000. The other constructions measure aligned SSIM 0.973 (reinforcement), 0.875, 0.589, and 0.207 (interpolations), 0.231 (density control), and 0.139 (random same-label).

Because constructed families are a limitation, we additionally mined natural vision families: descriptor-cosine candidate pairs at cosine ≥ 0.90, alignment-tolerant SSIM verification, and single-linkage components at SSIM ≥ 0.80, with all near-matches removed from the base training pool. This yields 649 CIFAR-10 families (label agreement 1.000) and 820 CIFAR-100 families (0.962). With 192 paired shadow seeds, the result is now unambiguous.

  • On the vulnerable targets that the paper's vision tables address, natural families cause substantial false attribution: 0.260 [0.215, 0.312] at nominal 1% (control 0.013) on CIFAR-10 without augmentation, with target selection, calibration, and evaluation on disjoint shadow thirds, and 0.479 on CIFAR-100. Population averages are lower (0.05–0.15 at 1%) mostly because a ResNet barely memorizes most targets at all.
  • Vision and text differ in magnitude, and the revision will say so. A family-present non-member in vision scores about half as member-like as a true member (ratio 0.47–0.65), against about 1.0 in the text headline.
  • Matched crop/flip constructions are not a uniform proxy for natural families. They overstate them under training augmentation and understate them by more than half without it, so the revision will report natural families directly and state vision claims per construction and training recipe.
Reviewer_ad1ROpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Full disclosure: I reviewed this paper and dictated my review to a speech-to-text model. The same has been cleaned up, expanded and presented below.

This paper studies how near-duplicate examples in training data confound exact-record membership inference. The authors introduce a four-world experimental framework that independently varies whether the exact queried example is present in training and whether members of its near-duplicate family are present. In particular, they identify an important alternative null hypothesis in which the exact example is absent but a near-duplicate family member is present.

Through experiments on natural near-duplicate families in CC-News and WikiText-103, the authors demonstrate that conventional membership inference attacks, calibrated on a clean null distribution, can produce extremely high false-positive rates when related examples appear in training. They further investigate this phenomenon across different models, attack scores, and deduplication settings.

The paper also studies how near-duplicates affect exact-record membership signals under different learning objectives. In classification models, near-duplicates can largely substitute for the exact example, making its additional membership signal nearly undetectable. In autoregressive language models, however, the exact example retains an additional detectable signal even when its near-duplicate family is present

Critical strengths

1. A useful and concrete characterization of a membership inference failure mode.

There is already a substantial literature questioning the reliability of membership inference attacks, particularly their tendency to capture distribution-level differences rather than evidence of individual-example membership. More recent work, including Zhang et al., has also challenged the validity of using membership inference for exact training-data provenance.

Within this context, I find the paper's four-world framing informative. In particular, the distinction between an example being absent with and without its near-duplicate family present provides a clean way to study an important source of miscalibration. While I view this as one instantiation of a broader class of known problems with membership inference, the authors do a good job of isolating and empirically demonstrating the mechanism. The natural near-duplicate experiments make the concern particularly concrete.

2. Interesting differences between classification and autoregressive language modeling.

I found the analysis of how near-duplicates affect different learning objectives particularly interesting. In classification settings, near-duplicates can almost completely substitute for the exact member, whereas autoregressive language models retain an additional signal attributable to the exact example.

The controlled comparison using the same backbone under classification and autoregressive objectives is particularly useful for understanding this distinction. This goes beyond simply demonstrating another failure of membership inference and provides insight into how different training objectives affect memorization and membership detectability.

Critical weaknesses

1. The conceptual novelty is somewhat limited relative to the existing literature.

The broader observation that membership inference attacks can capture distributional properties rather than exact-example membership is well established. There has also been substantial discussion of the difficulties of calibrating membership inference for provenance claims, including the recent critique by Zhang et al.

I therefore see the primary contribution of this paper as a careful consolidation and concrete empirical demonstration of one such failure mode, rather than a fundamentally new challenge to membership inference. This is still a meaningful contribution, but I would encourage the authors to position it more clearly relative to the existing literature and distinguish what is newly established from what was previously understood.

2. Some of the framing overstates the extent to which near-duplicates break exact-record membership inference.

The abstract and introduction sometimes suggest that near-duplicate families fundamentally break exact-record membership inference. However, the experimental results support a more nuanced conclusion.

In particular, for autoregressive language models, there remains a detectable difference between training on a near-duplicate family and training on that family together with the exact example. This suggests that the central difficulty is not necessarily the absence of an exact-record membership signal, but rather the ability to calibrate an inference attack against the appropriate null distribution.

If an auditor can construct a suitable family-conditioned reference distribution, exact-record membership inference may still be possible. Indeed, some of the paper's own experiments demonstrate this. I would encourage the authors to distinguish more carefully between failure due to miscalibration and genuine loss of exact-record information.

Questions for authors

How should we interpret the continued detectability of exact-record membership in autoregressive language models? The paper reports a substantial residual signal even when near-duplicate family members are present. Does this suggest that appropriately calibrated membership inference attacks can still succeed in these settings? If so, is the central limitation primarily the practical difficulty of constructing the relevant reference distribution, rather than a fundamental failure of exact-record membership inference?

2 comments

Thank you for the careful review.

W1 — "position it more clearly relative to the existing literature"

We accept this and will add an explicit positioning paragraph to the revision. Two new experiments also show that the delta from prior critiques is empirical, not only conceptual.

  • The effect is not a distribution-level signal. The "MIA captures distributional differences" literature predicts that member-like scores track population features. We ran model-free attacks (length, zlib ratio, unigram surprisal, label) on the same four-world shadows. Every AUC is 0.49–0.51 with confidence intervals covering 0.5 in all settings. The same records appear in all four worlds and only the inclusion bits change, so the confound is record-level and survives even when every distributional critique is answered.
  • Relative to Zhang et al. Their obstacle is that the production null cannot be sampled without retraining the deployed model, which is a calibration problem. Our failure persists after granting unlimited shadow retraining. The null is composite in a nuisance variable the auditor cannot enumerate, and the new experiments identify that variable precisely as the family's duplicate count in training. The obstacle is identification, not only sampling.
  • The stakes are concrete. 83.3% of headline false attributions implicate a publisher whose own copy was never trained, a provenance error that prior critiques do not surface.

What was previously understood: membership scores respond to related content, and duplication affects memorization. What is newly established: the family-present null occurs naturally at scale, defeats clean calibration at 0.997 false attribution while bystander controls stay at 0.016, survives executed target-unaware deduplication at every setting of the dial, splits into three measurable regimes by objective, fidelity, and exposure, is mechanistically explained by n-gram coverage, and defeats even a correctly designed two-stage repair unless the auditor knows the training set's duplicate structure.

W2 — "distinguish more carefully between failure due to miscalibration and genuine loss of exact-record information"

We agree, and we can now draw this line quantitatively. The new 45-cell grid over objective, fidelity, mass, and exposure yields exactly the distinction you request:

regimeexact-given-family AUCclean-reference false attribution
LM, low family fidelity or mass0.98–1.000.54–1.00
LM, high fidelity × mass × exposure0.60–0.87 (R_exact 0.03–0.19)1.00
classifier, every tested family0.50–0.540.04–0.17

In the headline text regime the failure is indeed miscalibration: the exact-record signal is present (per-target exact-given-family AUC 0.94 across scores) and only the verdict is wrong. This is not the whole story. With enough faithful exposure, autoregressive models also genuinely lose exact-record separation, and classifiers lose it in every tested condition. We also completed the score-channel list: Min-K%++ is the most robust tested score and still declares 88% of family-present non-members to be members at 1% (control 0.019), while itself losing exact-given-family signal (per-target AUC 0.818 against 0.937 for loss). Its robustness comes from discarding signal, not from isolating the exact record. We will rescope the abstract claim accordingly (exact wording in our response to Reviewer oWZv).

Q1 — "Does this suggest that appropriately calibrated attacks can still succeed? Is the central limitation primarily the practical difficulty of constructing the relevant reference distribution?"

Short answer: yes, with the true family. We built the repaired auditor your question describes and measured where it breaks.

The repaired auditor composes the two tests an auditor can already calibrate: a family-presence gate, calibrated against both family-absent worlds, that routes each query to a clean or a family-conditioned exact-record verdict. With the true family in the reference, this works end to end. False attribution falls from 0.997 to 0.068, clean-member power stays at 0.917, and the family-absent FPR stays at 0.016. Your conjecture is therefore right: the exact-record information is there, and a correctly calibrated attack recovers it.

We then degrade the auditor's family knowledge realistically, holding the gate fixed (α = 1%, false attribution measured on family-present non-members):

family referencefalse attributionTPR on H11
true family (oracle)0.0680.644
half of the true siblings0.8480.971
one true sibling0.9720.992
retrieval from a matching crawl0.4340.610
synthesized surrogate (J ≈ 0.89)0.1030.180

The retrieval case is decisive for your question. The retrieved blocks are near-copies of the true siblings (median Jaccard 0.82 to the nearest one, 96% above 0.5), so the auditor found the right content, yet false attribution stays at 0.434. Splitting by size shows why. When retrieval happens to match the true family's size, false attribution drops to 0.227, against 0.058 for the oracle on the same targets. With too few relatives it fails (0.735). With too many it over-corrects and loses power (0.148 at TPR 0.193). The reference must reproduce the family's training exposure, not only its content. The limitation is "practical" in your sense, but the missing ingredient is the training set's duplicate count, the very fact the audit exists to establish. The revision will keep the unidentifiability conclusion for model-only audits while reporting the gated design as the recommended protocol with this stated limit.

Outside the headline regime the answer changes character. At high fidelity, mass, and exposure, R_exact falls to 0.03–0.19, and no reference distribution can recover separation that training removed. The coverage analysis shows that the residual, where it exists, lives in the part of the sequence the family does not reproduce (target-level correlation up to +0.68 for the LM and about 0 for the classifier). The revision will state both halves of this answer explicitly.

Comments on the paper

1 comment

We thank the reviewers for their concrete and actionable questions. After the submission deadline we ran a substantial second round of experiments and re-derived every headline number from its stored per-shadow scores. The audit surfaced several places where a number was reported under the wrong label or estimator. We correct each one in the individual responses, none changes a conclusion, and the affected results become stronger when re-run at scale. The new experiments confirm the central confound and sharpen the claims.

The individual responses build on five shared results.

  1. We built the repaired audit protocol and located exactly where it breaks. The two-stage auditor first tests whether a near-duplicate family was trained and then judges exact membership against a family-conditioned reference. When the reference contains the true siblings, it cuts false attribution from 0.997 to 0.068 and keeps a true-positive rate of 0.917 on clean members, so the exact-record information is recoverable in principle. Every reference an auditor could actually build fails. Calibrating on one true sibling leaves false attribution at 0.972, on half of the siblings at 0.848, and on near-duplicates retrieved from a matching crawl at 0.434. The reference must match the family's duplicate count in the training set, which is the very fact the audit exists to establish.

  2. Three regimes replace the paper's substitution-versus-addition dichotomy. We crossed the learning objective with family fidelity, family size, and training exposure on a fixed GPT-2 backbone. Language models with low-fidelity or small families show pure miscalibration: the exact record stays detectable given its family (AUC 0.98–1.00) while clean-reference false attribution reaches 0.54–1.00. At high fidelity, mass, and exposure they additionally lose genuine exact-record information, with the surviving signal falling to 3–19% of its no-family level. Classifiers lose the signal under every tested family (exact-given-family AUC 0.50–0.54). What survives is carried by the part of the target the family fails to reproduce, with coverage and detectability correlated at +0.95.

  3. Semantic paraphrase families also confound the verdict, and we provide the detection channel that repairs the recommendation. Eight paraphrase siblings per target cause false attribution of 0.331 at a nominal 1% rate. The paper's 10% figure came from a comparison at mismatched exposure and is corrected. A probe built from the auditor's own held-out paraphrases, calibrated against the public base model, detects such families at AUC 0.998, so the recommendation to report family-level evidence stands once this detection channel is part of the protocol.

  4. The vision claims now rest on natural families, and we state the effect at its measured size rather than extrapolating from text. We mined and verified natural near-duplicate image families in CIFAR-10 and CIFAR-100 and reran the four-world audit with 192 paired shadow seeds. On the vulnerable targets the paper's vision tables address, they produce false attribution of 0.260 at a nominal 1% rate on CIFAR-10 and 0.479 on CIFAR-100. In vision a family-present non-member scores about half as member-like as a true member, whereas in text the two are indistinguishable.

  5. Deduplication was executed across its whole dial, together with its cost. A target-unaware pass over the full corpus at the standard threshold leaves false attribution at 0.992, and the aggressive threshold still leaves it at 0.923 while costing no validation perplexity. The aggressive pass also deletes 72% of the records their owners believe were included, and the auditor still declares 90% of those deleted records to be members, because a surviving sibling takes their place.

The revision will add a provenance table linking every headline figure to its run, estimator, and uncertainty, a formal definition of the near-duplicate family with its threshold sensitivity analysis, and claims rescoped to the evaluated audit channels rather than an impossibility statement. Detailed answers to every question follow in the individual responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.