- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper argues that a model reproducing a training sequence is not, by itself, evidence of memorization: one has to compare against how often the model reproduces matched text it was not trained on. It frames any extraction method as a fixed procedure (model, decoding policy, prompt rule, score, verification rule) and proposes setting the verification threshold from the scores of matched non-member "controls".
Critical strengths
- The paper addresses an important problem.
- The paper is solid in math.
Critical weaknesses
-
The significance of the paper is very unclear.
- As far as I understands, the paper aims to re-define ``extractable memorization'': but the focus of paper totally deviates from `extractable memorization'', where preivous work use this term to determine whether a (training data) sentence is extractable by some prefix or not. What the paper actually did is bascially membership inference attacks (MIA), which ****have been studied for almost 10 years.
- In this context, the authors did not differentiate their membership inference atttack method with baseline methods, and therefore the evaluation of the proposed method is very weak. As far as I understand, formulation of MIA using statistical tools has been very popular, e.g. [1] [2].
- There is almost no related work. Authors are strongly to encouraged to include a dedicated ``related work'' section and distingush your paper with others.
For the reasons above, the paper falls under a category that neither training data extraction nor MIA. The implication to the memorization community is embrassingly unclear.
-
Method soundness
- For consus experiments, the census threshold seems to be by a manual, one-sided decontamination step.
- The Wikipedia experiment splits members and non-members by date (2021–23 vs 2024), the same design as WikiMIA in [1]. However, Duan et al. 2024 and Das et al. (“Blind Baselines Beat Membership Inference Attacks”) showed this is inappropriate, since date splits are confounded by distribution shift, so the “matched” controls may not actually be matched.
- This paper is very hard to follow.
- The paper uses terms which don't follow standard in memorization literature. For example,
census'',population'' are almost never seen in memorization works. Population has been explored in a previos work [3] by Maini et al, and Maini et al has described as a ``dataset infernce attack". - the meaning of many words are not clear, e.g. “predictability” in introduction section,
- Unnesscary bold text across section 2, where the uahthors boldize many terms in LLM decoding steps, which are elementary (and do not exhibit high relavance to the focus on this paper) for reviewers.
- As a follow up of concern 1, the limitation of preivous work is not clear to me. please provide addtional contexnt.
I belive the paper needs a thorough polish to meet the standard of ICLR.
[1] Shi, Weijia, et al. "Detecting pretraining data from large language models." International Conference on Learning Representations. 2024.
[2] Zhang, Jingyang, et al. "Min-k%++: Improved baseline for pre-training data detection from large language models." International Conference on Learning Representations. 2025.
[3] Maini, Pratyush, et al. "LLM Dataset Inference: Did you train on my dataset?." Advances in Neural Information Processing Systems 37 (2024): 124069-124092.
Questions for authors
My main concern is concern 1 above. Please try to answer:
- How do this paper distinguish from MIA? and can you provide any comparsion between your method and state-of-the-art MIA / DIA conceptually? And experimental results of these comparsions are neccessary.
For concern 2, please answer my two subconcerns directly. For concern 3, please justify the reasons of adopting non-standard terms used in major memorization works.