- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Token Span Localization (TSL), which identifies verbatim-memorized spans in language model outputs by thresholding token-level scores (Final Loss or Sample Loss Difference), merging contiguous tokens, and selecting the longest valid span. Experiments across GPT-2 XL, eight Hubble configurations, and 14 open-weight models show substantial improvements in detection AUC and localization IoU over existing baselines.
Critical strengths
-
Fine-grained memorization localization: The paper formulates verbatim memorization detection as a span-localization problem rather than sequence-level classification, enabling the identification of specific reproduced passages within long outputs.
-
Promising empirical results: TSL substantially improves localization when applied to existing token-level scores, increasing mean IoU from 0.326 to 0.683 for InfoRMIA on Hubble models and achieving a detection AUC of 0.984 on GPT-2 XL.
Critical weaknesses
-
Mismatch between hyperparameter selection and practical validity: The minimum span length (L_{\min}) is selected to maximize sequence-level detection AUC, without ensuring that the detected spans provide meaningful evidence of memorization. Specifically, short, common phrases may be flagged even when their overlap is not sufficiently distinctive to support a memorization claim [1]. The paper does not adequately examine whether the AUC-optimal (L_{\min}) avoids such false positives.
-
Insufficient evaluation of practical reliability: Given the low prevalence of exact matches in several settings, high ROC-AUC does not guarantee high precision at low false-positive rates.
[1] 'Language Models May Verbatim Complete Text They Were Not Explicitly Trained On.' ICML 2025.
Questions for authors
-
As noted in Weakness 1, my main concern is: Why are (L_{\min}) and the token-score threshold optimized for sequence-level AUC rather than localization quality? Given the issue raised in Weakness 1, can the authors compare different minimum-length constraints and report practically meaningful metrics, such as true-positive rates at low false-positive rates?
-
How does selecting only the longest span affect performance when multiple disjoint memorized spans occur in one output?
2 comments
Thank you and follow-up
Dear Reviewer UgQf,
Thank you for your constructive comments and the time you dedicated to reviewing our work. We hope our rebuttal has adequately addressed your concerns. We would be happy to provide any further clarification if needed. Should our response have resolved your concerns, we would appreciate it if you would consider updating your evaluation.
Best regards,
The Authors
Response to Reviewer UgQf
We agree that sequence AUC neither optimizes boundaries nor guarantees precision at low prevalence. We now tune for each goal separately and report low-FPR operating points against the baselines.
Q1: Tuning objective. The original objective was detection-oriented (§5.1). We now select ζ and Lmin on validation (50k) for either AUC or mean IoU, and freeze them for test (100k). Missing predictions count as IoU 0.
The AUC-tuned column reports newly validation-selected controls. We recomputed both objectives using the same parameter grid and evaluation protocol to provide a matched comparison of AUC and IoU tuning.
Q1: Minimum length Lmin. For each Lmin and FPR target, we selected the token threshold and sequence cutoff on validation, then fixed both for test. The table gives the FL mean over the four Hubble 100B configurations. Cells are test recall / achieved test FPR (%).
Recall is highest with long minimum spans (40–50 tokens), so performance does not depend on flagging short phrases. Longer is not uniformly better: Lmin=40 beats 50 at the 1% target.
Q1: Low-FPR reliability. We evaluate recall at test-ROC FPR caps of 1% and 0.1%. For TSL, the token threshold ζ and minimum predicted run length Lmin are jointly selected on validation, separately for each cap, and then frozen. For all methods, the final sequence decision cutoff is swept on test. The Hubble results are equal-weight means across all eight configurations, covering both 100B and 500B training settings. GPT-2 XL results use unconditional generations. Each cell reports recall (%) at 1% / 0.1% FPR.
TSL-FL achieves the highest recall among the compared methods in all four comparisons, exceeding twice the strongest baseline recall at the 1% cap. TSL-SLD ranks second in three comparisons.
To assess operational calibration separately, we also select the final decision cutoff on validation and freeze it for test. Across the same eight Hubble configurations, TSL-FL achieves 93.7% mean recall at 0.9715% achieved test FPR, with 53.8% precision, and 63.5% recall at 0.0944% FPR, with 88.7% precision. These results show the practical trade-off: reducing false positives increases precision but misses more genuine matches.
Q2: Multiple disjoint spans. At γ=50, 133 of the 1,533 positive GPT-2 outputs (8.7%) contain several disjoint matched regions, 287 regions in total. We used one-to-one region matching at IoU ≥ 0.5 against all-region references, with extraction settings fixed:
With one span per output, recall on this subset can be at most 133/287 = 46.3%. Returning all runs recovers most of the remaining regions, at a precision cost. Detection AUC is unchanged when the sequence score is kept.
W1: Common phrases. An overlap shorter than γ=50 does not by itself make an output positive. In the displayed four-configuration FL ablation, the highest mean recall occurs at Lmin=40 for the 1% target and Lmin=50 for the 0.1% target. Commonness is nonetheless real at the passage level. Our content audit finds that 73.3% of all 1,533 GPT-2 hits and 90.94% of the 1,600 sampled Hubble hits are boilerplate. These are correct matches under the benchmark definition, but, consistent with Liu et al. (ICML 2025), overlap does not establish distinctive or record-caused memorization.
W2: Practical reliability at low prevalence. The validation-frozen results under Q1 explicitly report achieved FPR and precision at the observed test prevalence. They quantify the recall–precision trade-off rather than treating ROC-AUC as a reliability guarantee. Precision will change with match prevalence.