acceptodds
Under review as a conference paper at ICLR 2027

Localizing Verbatim Memorized Spans in Language Model Outputs

Abstract

Large language models (LLMs) can reproduce training data verbatim, raising copyright and privacy concerns. To detect such reproduction, existing methods typically frame the problem as membership inference, classifying entire sequences as members or non-members. However, a long generated sequence may contain only a short memorized passage, for example, roughly 70 verbatim tokens within an up to 512-token output. Sequence-level aggregation can therefore dilute the memorization signal. While token-level scoring methods exist, they commonly flag tokens independently. Our key insight is that verbatim memorization should be treated as a structured span-localization problem rather than a sequence-level or independent-token detection problem. Unlike membership inference, which asks whether a candidate text was used in training, our task asks where verbatim matches occur within generated text. We make three contributions. First, we develop a benchmark for predicting the boundaries of verbatim spans and demonstrate the benefit of contiguity-aware extraction: on Hubble models, applying it to InfoRMIA improves mean intersection over union (IoU) from 0.326 to 0.683. Second, we introduce Token Span Localization (TSL), which converts token-level scores into contiguous span predictions using two scoring functions, Final Loss (FL) and Sample Loss Difference (SLD). Third, we provide a theoretical analysis showing that these scores offer complementary evidence: FL captures how likely the final model is to reproduce a complete span, while SLD measures how much its likelihood has increased since a reference checkpoint. On GPT-2 XL, TSL achieves 0.851 mean IoU (with SLD) and 0.984 detection AUC (with FL), a gain of 0.122 over the strongest sequence-level baseline (0.862). Across eight Hubble configurations, TSL obtains 0.986-0.993 AUC, an average gain of 0.207 over InfoRMIA (0.672-0.850), compared with 0.601–0.796 for the strongest conventional sequence-level baseline in each configuration. TSL also achieves the highest detection AUC among the evaluated methods on 12 of 14 open-weight models tested against an external corpus. Together, these results establish TSL as a more accurate and fine-grained approach to auditing verbatim memorization in LLM outputs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
memorization, large language models, verbatim memorization, privacy auditing, training data extraction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UgQfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Token Span Localization (TSL), which identifies verbatim-memorized spans in language model outputs by thresholding token-level scores (Final Loss or Sample Loss Difference), merging contiguous tokens, and selecting the longest valid span. Experiments across GPT-2 XL, eight Hubble configurations, and 14 open-weight models show substantial improvements in detection AUC and localization IoU over existing baselines.

Critical strengths
  1. Fine-grained memorization localization: The paper formulates verbatim memorization detection as a span-localization problem rather than sequence-level classification, enabling the identification of specific reproduced passages within long outputs.

  2. Promising empirical results: TSL substantially improves localization when applied to existing token-level scores, increasing mean IoU from 0.326 to 0.683 for InfoRMIA on Hubble models and achieving a detection AUC of 0.984 on GPT-2 XL.

Critical weaknesses
  1. Mismatch between hyperparameter selection and practical validity: The minimum span length (L_{\min}) is selected to maximize sequence-level detection AUC, without ensuring that the detected spans provide meaningful evidence of memorization. Specifically, short, common phrases may be flagged even when their overlap is not sufficiently distinctive to support a memorization claim [1]. The paper does not adequately examine whether the AUC-optimal (L_{\min}) avoids such false positives.

  2. Insufficient evaluation of practical reliability: Given the low prevalence of exact matches in several settings, high ROC-AUC does not guarantee high precision at low false-positive rates.

[1] 'Language Models May Verbatim Complete Text They Were Not Explicitly Trained On.' ICML 2025.

Questions for authors
  1. As noted in Weakness 1, my main concern is: Why are (L_{\min}) and the token-score threshold optimized for sequence-level AUC rather than localization quality? Given the issue raised in Weakness 1, can the authors compare different minimum-length constraints and report practically meaningful metrics, such as true-positive rates at low false-positive rates?

  2. How does selecting only the longest span affect performance when multiple disjoint memorized spans occur in one output?

2 comments
AuthorsOpenReview ↗

Response to Reviewer UgQf

We agree that sequence AUC neither optimizes boundaries nor guarantees precision at low prevalence. We now tune for each goal separately and report low-FPR operating points against the baselines.

Q1: Tuning objective. The original objective was detection-oriented (§5.1). We now select ζ\zeta and Lmin⁡L_{\min} on validation (50k) for either AUC or mean IoU, and freeze them for test (100k). Missing predictions count as IoU 0.

SettingScoreAUC-tuned: IoU / AUCIoU-tuned: IoU / AUC
Hubble (mean of 8)FL0.762 / 0.9900.795 / 0.984
Hubble (mean of 8)SLD0.728 / 0.9840.761 / 0.970
GPT-2 XLFL0.826 / 0.9880.859 / 0.986
GPT-2 XLSLD0.851 / 0.9820.858 / 0.977

The AUC-tuned column reports newly validation-selected controls. We recomputed both objectives using the same parameter grid and evaluation protocol to provide a matched comparison of AUC and IoU tuning.

Q1: Minimum length Lmin⁡L_{\min}. For each Lmin⁡L_{\min} and FPR target, we selected the token threshold and sequence cutoff on validation, then fixed both for test. The table gives the FL mean over the four Hubble 100B configurations. Cells are test recall / achieved test FPR (%).

Lmin⁡L_{\min}1% target0.1% target
1070.84 / 0.96924.84 / 0.098
2071.19 / 0.97425.52 / 0.099
3082.00 / 1.00431.87 / 0.098
4091.24 / 0.99440.36 / 0.089
5085.22 / 0.99051.11 / 0.101

Recall is highest with long minimum spans (40–50 tokens), so performance does not depend on flagging short phrases. Longer is not uniformly better: Lmin⁡=40L_{\min}=40 beats 50 at the 1% target.

Q1: Low-FPR reliability. We evaluate recall at test-ROC FPR caps of 1% and 0.1%. For TSL, the token threshold ζ\zeta and minimum predicted run length Lmin⁡L_{\min} are jointly selected on validation, separately for each cap, and then frozen. For all methods, the final sequence decision cutoff is swept on test. The Hubble results are equal-weight means across all eight configurations, covering both 100B and 500B training settings. GPT-2 XL results use unconditional generations. Each cell reports recall (%) at 1% / 0.1% FPR.

MethodHubble (mean of 8)GPT-2 XL
PPL13.6 / 3.614.4 / 0.6
Zlib10.7 / 4.438.4 / 19.2
Min-K% (k=20)30.7 / 21.311.1 / 0.6
InfoRMIA mean38.8 / 20.027.8 / 0.5
TSL-FL93.9 / 63.184.7 / 40.6
TSL-SLD89.7 / 39.253.0 / 2.5

TSL-FL achieves the highest recall among the compared methods in all four comparisons, exceeding twice the strongest baseline recall at the 1% cap. TSL-SLD ranks second in three comparisons.

To assess operational calibration separately, we also select the final decision cutoff on validation and freeze it for test. Across the same eight Hubble configurations, TSL-FL achieves 93.7% mean recall at 0.9715% achieved test FPR, with 53.8% precision, and 63.5% recall at 0.0944% FPR, with 88.7% precision. These results show the practical trade-off: reducing false positives increases precision but misses more genuine matches.

Q2: Multiple disjoint spans. At γ=50\gamma=50, 133 of the 1,533 positive GPT-2 outputs (8.7%) contain several disjoint matched regions, 287 regions in total. We used one-to-one region matching at IoU ≥ 0.5 against all-region references, with extraction settings fixed:

ScoreRegion recall: longest / all runsRegion precision: longest / all runs
FL45.3% / 96.5%97.7% / 87.7%
SLD42.9% / 96.2%92.5% / 86.5%

With one span per output, recall on this subset can be at most 133/287 = 46.3%. Returning all runs recovers most of the remaining regions, at a precision cost. Detection AUC is unchanged when the sequence score is kept.

W1: Common phrases. An overlap shorter than γ=50\gamma=50 does not by itself make an output positive. In the displayed four-configuration FL ablation, the highest mean recall occurs at Lmin⁡=40L_{\min}=40 for the 1% target and Lmin⁡=50L_{\min}=50 for the 0.1% target. Commonness is nonetheless real at the passage level. Our content audit finds that 73.3% of all 1,533 GPT-2 hits and 90.94% of the 1,600 sampled Hubble hits are boilerplate. These are correct matches under the benchmark definition, but, consistent with Liu et al. (ICML 2025), overlap does not establish distinctive or record-caused memorization.

W2: Practical reliability at low prevalence. The validation-frozen results under Q1 explicitly report achieved FPR and precision at the observed test prevalence. They quantify the recall–precision trade-off rather than treating ROC-AUC as a reliability guarantee. Precision will change with match prevalence.

AuthorsOpenReview ↗

Thank you and follow-up

Dear Reviewer UgQf,

Thank you for your constructive comments and the time you dedicated to reviewing our work. We hope our rebuttal has adequately addressed your concerns. We would be happy to provide any further clarification if needed. Should our response have resolved your concerns, we would appreciate it if you would consider updating your evaluation.

Best regards,

The Authors

Reviewer_7dB2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Token Span Localization (TSL), a method for identifying the boundaries of contiguous verbatim-matched spans within language-model generations. Rather than assigning a single membership or memorization score to an entire sequence, TSL computes token-level scores using either Final Loss (FL) or Sample Loss Difference (SLD), thresholds these scores, groups adjacent positive tokens into contiguous runs, removes runs below a minimum length, and returns the longest remaining span. Ground-truth spans are constructed by querying a reference-corpus index with sliding 50-token windows and selecting the longest matching interval. The method is evaluated using a controlled GPT-2 XL setup, the Hubble benchmark, and fourteen open-weight language models.

Critical strengths
  • The paper evaluates the method in a controlled training setting, on multiple Hubble configurations, and across fourteen open-weight models. The corpus indexing and large-scale generation effort are substantial.

  • The paper generally distinguishes verbatim overlap, output reproduction, training membership, and record-level causal memorization. It also acknowledges that overlap with an external corpus does not establish that the corresponding text was used for training.

Critical weaknesses
  • TSL is essentially token-score thresholding followed by connected-component extraction, minimum-length filtering, and longest-run selection. The target span is also defined as the longest contiguous exact match, making the method’s inductive bias unusually well matched to the benchmark.

  • The theoretical claims are stronger than their empirical support. The FL and SLD results largely formalize standard autoregressive probability identities. The exact interpretation assumes ancestral sampling from the scored distribution, while the experiments use top-k and top-p decoding. The causal PN/PNS analysis additionally depends on matched-control assumptions that are not instantiated or validated experimentally.

Questions for authors
  • The ground truth is the longest contiguous exact match, while the method predicts the longest contiguous run of low-loss tokens. Could the authors evaluate on independently annotated or synthetically inserted spans whose positions, lengths, and multiplicities are not defined by the same longest-match rule? In the current form, it is currently unclear how the benchmark avoid structurally favoring the proposed method.

  • Are the theoretical reproduction bounds valid for the decoder used in the experiments? The experiments use top-k and top-p sampling, while FL and SLD appear to be computed from the unmodified model distribution. Could the authors recompute scores after truncation and renormalization and compare those results with raw-model scoring?

  • What exactly is being detected in the external-corpus setting? Since membership in the actual training corpus is unknown, could TSL be detecting low-loss or stylistically typical DCLM passages rather than training-derived reproduction? Can the authors evaluate matched-domain corpora known to be excluded from training, control for passage frequency, and stratify results by corpus source?

  • How much of the localization improvement is attributable to the token score rather than the longest-run extraction procedure? Table 3 suggests that applying the same extractor to InfoRMIA closes much of the gap with FL and SLD. Could the authors evaluate every token-aware score using the same set of aggregation methods, such as longest run, moving-average thresholding, maximum-window search, and dynamic programming? This would clarify whether the main contribution is the scoring function or the structured post-processing.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 7dB2 (1/2)

We agree that the shared longest-run geometry of target and predictor was the key open question. We tested it in three ways: a synthetic insertion experiment with independent boundaries, four extractors under identical tuning, and all-region references.

Q1: Synthetic insertion. We inserted corpus-verified training passages into GPT-2 XL generations of 512 tokens, replacing equal-length intervals at uniformly random valid positions. The experiment uses 1,000 validation and 3,334 test examples, with test lengths approximately balanced across 20–40, 41–80, 81–160, 161–210 and 211–260 tokens. The planted intervals define ground truth independently of the corpus matcher and detector. Each extractor selects its own parameters on the same synthetic validation set, containing 20–160-token insertions, and freezes them across all test lengths. Scoring uses ordinary FL and SLD.

ExtractorFL IoU, 20–260 tokensSLD IoU, 20–260 tokens
TSL (longest run)0.72640.7318
Moving average, width ≥30.72010.7234
Max-window0.36230.3630
DP0.75280.7493

DP achieves the highest overall IoU, while TSL and moving average have similar point estimates. Performance depends strongly on passage length:

Inserted lengthTest examplesTSL-FL IoUTSL-SLD IoU
20–406670.22710.2182
41–806670.47730.5018
81–1606660.96390.9704
161–2106670.97800.9808
211–2606670.98580.9879

These results demonstrate accurate recovery of long passages with independently planted boundaries, while identifying short-span recovery as a limitation. TSL has the highest point-estimate IoU on the 161–260-token subset (1,334 test examples), with FL/SLD IoU of 0.9819/0.9843, whereas DP performs best overall. The frozen max-window extractor selects an 80-token window, limiting its coverage of longer targets.

No sequence-level acceptance cutoff is applied; these results measure boundary recovery rather than detection FPR. Donors are previously reproduced, corpus-verified passages.

Q2: Decoder-aware rescoring. This comparison uses fresh paired forward passes; the separate extractor comparison uses archived token scores. We ran paired GPT-2 XL forward passes on 50,048 validation and 100,096 test outputs. Each pass computes raw FL and FL after the actual top-k=40 then top-p=0.96 filtering, renormalized at temperature 1. Each score is tuned independently on validation:

Tuning objectiveRaw FL: AUC / IoURenormalized FL: AUC / IoU
AUC0.9891 / 0.8580.9891 / 0.854
IoU0.9885 / 0.8660.9836 / 0.877

Under independent AUC tuning, GPT-2 detection AUC is essentially unchanged. Under IoU tuning, renormalization increases IoU by 0.0108 and lowers AUC by 0.0050.

Hubble: all eight configurations. We completed paired raw and decoder-aware FL scoring on 50k validation and the full designated 100k test set for each configuration. Each independently tuned variant selects its own token threshold and minimum run length on validation. Cells below show equal-weight mean test AUC / IoU across all eight configurations:

Parameter selectionRaw FL: AUC / IoURenormalized FL: AUC / IoU
Each independently AUC-tuned0.9917 / 0.77230.9925 / 0.7953
Each independently IoU-tuned0.9841 / 0.81490.9890 / 0.8192

With independent AUC tuning, decoder-aware FL increases mean IoU from 0.7723 to 0.7953, with AUC 0.9917 to 0.9925. With independent IoU tuning, its mean IoU gain is smaller (0.8149 to 0.8192). The localization benefit therefore depends on the tuning objective and is not uniform across configurations.

Theory. Summary of the updated proof (full proof in the revised paper):

Let rr be the distribution after temperature and repetition processing, S(h)S(h) the retained set, and Z(h)=∑v∈S(h)r(v∣h)Z(h)=\sum_{v\in S(h)} r(v\mid h). The sampling distribution is

q(v∣h)=r(v∣h) 1{v∈S(h)}Z(h).q(v\mid h)=\frac{r(v\mid h)\,\mathbf{1}\{v\in S(h)\}}{Z(h)}.

For a span yy in a fixed context xx, with every token retained by the decoder,

q(y∣x)=∏jr(yj∣hj)Z(hj)=r(y∣x)∏jZ(hj).q(y\mid x)=\prod_j \frac{r(y_j\mid h_j)}{Z(h_j)}=\frac{r(y\mid x)}{\prod_j Z(h_j)}.

Consequently, defining FL as summed token loss,

FLdec=FLr−C,C=−∑jlog⁡Z(hj)≥0.\mathrm{FL} _ {\mathrm{dec}}=\mathrm{FL} _ {r}-C,\qquad C=-\sum _ j \log Z(h _ j) \ge 0.

With temperature-1 truncation and no other logit changes, r=pr=p, so

q(y∣x)≥p(y∣x)=exp⁡(−FLraw),FLraw=−∑jlog⁡p(yj∣hj).q(y\mid x) \ge p(y\mid x)=\exp(-\mathrm{FL} _ {\mathrm{raw}}),\qquad \mathrm{FL} _ {\mathrm{raw}}=-\sum _ j \log p(y _ j\mid h _ j).

With a repetition penalty, r≠pr\neq p in general, so this raw-model bound does not carry over.

AuthorsOpenReview ↗

Response to Reviewer 7dB2 (2/2)

Q3: External reference. In the open-weight setting, a DCLM-100B match establishes overlap with that corpus, while a non-match does not rule out other sources. Low loss can also reflect commonness or domain typicality. Our claims concern overlap localization, not training membership. These experiments do not establish specificity against matched-domain corpora known to be excluded from training.

Q4: Alternative extractors. All extractors use the same raw scores and confidence definition, and each is tuned on validation for IoU:

ExtractorGPT-2 FL / SLD IoUHubble (mean of 8) FL / SLD IoU
TSL0.859 / 0.8580.795 / 0.761
Moving average0.860 / 0.8590.771 / 0.752
Max-window0.736 / 0.7190.712 / 0.680
DP0.863 / 0.8500.739 / 0.713

TSL achieves the highest mean Hubble IoU, while alternative extractors can perform better on GPT-2 or detection AUC. For example, Hubble FL AUC is 0.993 for DP versus 0.984 for TSL under IoU-selected settings. These matched comparisons show that extractor choice affects both localization and detection.

Q4: Score versus extraction. The score also matters. With the extractor and IoU tuning held fixed, FL beats InfoRMIA on all five Pythia models (410M: 0.573 vs 0.537; 1.4B: 0.547 vs 0.507; 2.8B: 0.427 vs 0.391; 6.9B: 0.599 vs 0.454; 12B: 0.509 vs 0.412). Our contribution is therefore the benchmark together with the finding that contiguity matters. FL/SLD are effective score choices within it, and the theory explains how to interpret them.

W1: Alignment between predictor and benchmark. The shared longest-span structure motivates the additional evaluations in Q1 and Q4. Synthetic insertion specifies target boundaries independently of the longest-match rule, while the matched extractor comparison tests alternatives to longest-run selection. DP performs best overall on synthetic insertions, whereas TSL achieves the highest mean Hubble IoU.

W2: Theoretical interpretation and empirical support. The probability identities clarify how FL and SLD relate to reproduction likelihood; our empirical contribution rests on the localization benchmark and extraction results. Q2 extends the interpretation to the renormalized decoder and compares raw with decoder-aware scoring, explicitly distinguishing the raw model from the distribution after logit processing.

Causal assumptions. Let Q1Q_1 and Q0Q_0 be the reproduction probabilities under the same decoder when the record is included or omitted. For Q1>0Q_1>0:

PN≥[1−Q0Q1]+,PNS≥[Q1−Q0]+.\mathrm{PN} \ge \left[1-\frac{Q _ 0}{Q _ 1}\right] _ {+},\qquad \mathrm{PNS} \ge \left[Q _ 1-Q _ 0\right] _ {+}.

Calibrating these bounds requires the matched-control exchangeability/domination assumptions stated in §4.2. Our experiments do not validate those assumptions. The revision presents this as conditional theory, not as an empirically validated certificate of causal memorization.

AuthorsOpenReview ↗

Follow-up on rebuttal

Dear Reviewer 7dB2,

We sincerely thank you for your careful reading of our paper and your valuable feedback. We hope our response has clarified the points you raised. If you have any remaining questions, please let us know, and we will gladly address them. If you feel your concerns have been resolved, we would be grateful if you would consider revisiting your score.

Best regards,

The Authors

Reviewer_wqzUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper reframes detection of verbatim memorization in language-model outputs as a span-localization problem. It introduces TSL, or SLD, groups qualifying adjacent tokens into runs, and returns the longest run that meets a minimum-length criterion.

Critical strengths
  1. TSL requires per-token scores and a linear scan to form contiguous runs; it does not require corpus lookup during prediction.

  2. Experiments span a controlled model, models with open training data and checkpoints, and open-weight models.

Critical weaknesses

Limited scope of memorization evaluation. The benchmark adopts a narrow operational definition of memorization, relying on exact token matches with a minimum length of 50 tokens to establish ground truth. Furthermore, the evaluation is restricted to unconditional generation, explicitly excluding paraphrastic and structural forms of memorization. These limitations raise concerns about whether the benchmark adequately captures the broader phenomenon of memorization in LLMs.

Distinguishing memorization from harmful information leakage. It is unclear whether reproducing snippets of training data should inherently be considered harmful. For instance, if a user asks a model to quote a passage from a well-known poet, reproducing that passage would not necessarily constitute undesirable behavior. The key concern should be whether memorization leads to the unauthorized disclosure of sensitive or protected content. Therefore, I suggest that the experiments focus more specifically on detecting memorization involving copyrighted or private information, rather than treating the reproduction of general knowledge or publicly available content as inherently problematic.

Evaluation on more recent models. The evaluation would benefit from including more recent language models, as GPT-2 is relatively outdated and may not adequately represent the capabilities and memorization behaviors of modern LLMs. Expanding the experiments to contemporary model families would strengthen the relevance and generalizability of the findings.

Questions for authors

How sensitive is localization to the exact-match definition? Do results persist for shorter copied spans, different minimum match lengths, or approximate matching that includes paraphrases?

Does the method generalize beyond unconditional generation? Since real-world leakage may follow a prompt, it would be useful to evaluate prompted outputs and distinguish copied continuations from text made likely by the prompt itself.

3 comments
AuthorsOpenReview ↗

Response to Reviewer wqzU (1/2)

Our contribution is a benchmark for localizing verbatim corpus matches, together with evidence that contiguity-aware extraction substantially improves localization from token-level scores. We added a match-length sweep, a prompted-generation experiment, a content audit and corrected Gemma-2 results.

Q1: Sensitivity to the match definition. On unconditional GPT-2 XL generations, we varied the minimum match length γ∈{10,15,20,25,30,35}\gamma\in\{10,15,20,25,30,35\}. This experiment uses 49,987 validation and 50,109 test outputs. For each γ\gamma, we selected the token threshold ζ\zeta and minimum predicted run length Lmin⁡L_{\min} on validation for either IoU or AUC, then froze them for test. The table reports test IoU; the positive count is shared by FL and SLD.

γ\gammaTest positives / 50,109FL: IoU-tunedFL: AUC-tunedSLD: IoU-tunedSLD: AUC-tuned
1046,5960.11120.07140.14000.1391
159,4120.36720.34800.35350.3463
202,7990.68490.66320.67750.6692
252,0660.76360.71220.77510.7662
301,6970.79630.73940.80850.8062
351,5060.81690.80430.82850.8248

Localization is substantially weaker at γ=10\gamma=10–15, while IoU-tuned FL/SLD reaches 0.685/0.678 at γ=20\gamma=20 and 0.817/0.829 at γ=35\gamma=35. Changing γ\gamma also changes the positive cohort, so this is sensitivity to the match definition, not a controlled effect of passage length alone.

Synthetic insertion experiment. We ran a separate synthetic insertion experiment. We inserted verified training passages of random length (20–160 tokens) at random positions in GPT-2 XL generations, so the ground-truth span is exactly the inserted interval. TSL recovers 81–160-token spans at IoU 0.96–0.97, 41–80-token spans at 0.48–0.50, and 20–40-token spans at only 0.22–0.23. We further evaluated 1,334 test insertions of 161–260 tokens, keeping the original validation set and settings fixed. Mean TSL FL/SLD IoU is 0.9780/0.9808 for 161–210 tokens and 0.9858/0.9879 for 211–260 tokens; longer-span recovery is strong, while the shorter-span limitation remains.

Approximate matching. Paraphrastic or structural reproduction would require semantic alignment annotations rather than exact-match boundaries. We do not claim to detect it.

Q2: Prompted generation. We generated GPT-2 XL continuations (up to 512 new tokens) from random 50-token OpenWebText windows: 50,000 for validation and 100,000 for test. The prompts come from the training corpus, so this is a discoverable-extraction-style setting, distinct from the unconditional task in §3. It tests whether localization still works when a training passage supplies the context. Prompt tokens are excluded from scoring and matching. Each continuation is searched against the full training corpus. At γ=50\gamma=50, 2,634 test continuations (2.6%) contain a verified match:

ScoreAUC-tuned: IoU / AUCIoU-tuned: IoU / AUC
FL0.728 / 0.9680.748 / 0.952
SLD0.758 / 0.9460.758 / 0.932

Localization carries over to prompted outputs. Excluding the prompt prevents supplied tokens from being counted as reproduction. However, because the prompts are training text, this design cannot separate training-caused copying from a continuation made likely by the prompt. We state this limitation explicitly in the revision.

AuthorsOpenReview ↗

Response to Reviewer wqzU (2/2)

W1: Scope. Our benchmark targets verbatim reproduction, for which exact corpus matches provide independently verifiable token boundaries and a reproducible localization objective. We have expanded the evaluation beyond the original 50-token, unconditional setting: Question 1 reports an unconditional minimum-match-length sweep over γ∈{10,15,20,25,30,35}\gamma\in\{10,15,20,25,30,35\}, and Question 2 evaluates training-prefix prompted generation at γ=50\gamma=50. The synthetic-insertion study additionally tests recovery of 20–260-token passages at independently chosen positions, using planted boundaries as ground truth. These experiments establish where localization remains effective and where it becomes difficult, particularly for short spans. Exact overlap can include common language, so it should not by itself be interpreted as record-specific training causation or harmful disclosure. Near-verbatim and paraphrastic reproduction are distinct evaluation targets requiring appropriate alignment annotations and validation of the localization method.

W2: Harm versus reproduction. To address this, we measured what is actually reproduced. We categorized the longest verified passage in every GPT-2 hit (1,533) and in 200 random hits from each of the eight Hubble configurations (1,600). These samples were drawn independently of detector predictions. Categorization was LLM-assisted (GPT-6.1 Sol), using fixed template-family rules.

CategoryGPT-2 (1,533)Hubble (1,600)
Boilerplate (ads, forms, navigation, footers)73.26%90.94%
News/related-content widgets23.16%1.81%
Educational content0.00%4.13%
Other substantive content1.96%1.25%
Repetitive artifacts1.57%1.44%
Software code0.00%0.00%
Passage unavailable0.07%0.44%

Beyond boilerplate, TSL localizes substantive, publisher-authored content. News headlines and related-article text make up 23.2% of GPT-2 hits. Together with educational and other substantive passages, they account for 25.1% of GPT-2 hits and 7.2% of Hubble hits. This is content that can raise copyright concerns, and it is reproduced without any prompt requesting it. Deciding whether a passage is harmful requires first knowing where the reproduced text is. TSL provides that localization, so that an auditor can review the flagged span for copyright or privacy rather than the entire output.

W3: Contemporary models. In addition to GPT-2 XL, the paper evaluates eight Hubble configurations and fourteen open-weight models, including Llama-3.1/3.2, Gemma-2 and Nemotron-H. Gemma-2 was the weakest family, with AUC-tuned IoU of 0.072 (2B) and 0.058 (9B). We traced this to our scoring pipeline: Gemma inputs were scored without the beginning-of-sequence token and with inconsistent left-padding handling. After the fix:

ModelAUC-tuned IoUIoU-tuned IoU
Gemma-2-2B0.2330.382
Gemma-2-9B0.2050.360
Falcon-RW-1B (reference)0.7150.855

For open-weight models, DCLM-100B is an external reference. A match establishes overlap with that corpus, not training membership.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer wqzU,

Thank you again for your time and thoughtful review. We hope our response has addressed your concerns. Please let us know if you have any further questions; we would be happy to clarify. If our response has resolved your concerns, we would appreciate it if you would consider updating your assessment.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for the feedback. The reviews raise complementary questions about benchmark geometry, practical reliability and scope. We added experiments addressing each.

Contribution. Our contribution is a benchmark for localizing verbatim corpus matches in generated text, together with the finding that exploiting contiguity substantially improves localization from existing token-level scores.

New evidence, in brief.

  1. Synthetic insertion. We inserted verified training passages at random positions in GPT-2 XL generations, with planted intervals as ground truth. We evaluated 3,334 test cases: 2,000 with 20–160-token insertions and 1,334 additional cases with 161–260-token insertions, keeping validation data and settings unchanged. Dynamic Programming (DP) has the highest IoU on the original cohort (FL/SLD 0.6053/0.5957), while TSL has the highest point estimates on the longer extension (0.9819/0.9843). Short inserted spans remain difficult.
  2. Four extractors, same scores and tuning. TSL, moving average, max-window and DP: TSL has the highest mean Hubble IoU (FL 0.795), whereas DP achieves slightly higher GPT-2 FL IoU, while TSL achieves higher SLD IoU. DP also gives higher detection AUC. The finding that holds across all of them is that contiguity matters.
  3. The score also matters. With the extractor and tuning fixed, FL beats InfoRMIA on all five Pythia models by 0.036–0.145 IoU.
  4. Decoder-aware rescoring. With independent AUC tuning, GPT-2 FL AUC is essentially unchanged (0.9891 vs 0.9891). For the eight Hubble configurations, with independent IoU tuning, decoder-aware FL increases mean test IoU from 0.8149 to 0.8192 and AUC from 0.9841 to 0.9890.
  5. What is reproduced. A content audit shows that most verified hits are boilerplate. The verified matches also include substantive news headlines, related-article text and educational content. News/related-content, educational and other substantive or mixed content together make up 25.1% of all 1,533 GPT-2 hits and 7.2% of the 1,600 sampled Hubble hits.
  6. Gemma-2. The near-zero Gemma IoU came from a scoring-pipeline error (missing BOS token and inconsistent left padding). Correcting BOS and left-padding handling increases AUC-tuned IoU from 0.072 to 0.233 for Gemma-2-2B and from 0.058 to 0.205 for Gemma-2-9B. On the corrected scores, IoU tuning yields 0.382 and 0.360, respectively.
  7. Operating points. We tuned for IoU and for fixed FPR targets, and report recall, achieved FPR and precision at 1% and 0.1% FPR alongside the baselines.

Design choices. Exact matching provides reproducible boundaries for evaluating verbatim reproduction. Our principal benchmark uses unconditional generations and a minimum match length of γ=50\gamma=50; the additional experiments examine shorter matches, prompted continuations and multiple regions. A verified match establishes overlap with the reference corpus.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.