- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies membership inference in LLMs, in particular observed that loss is not an discriminative features on low-frequency training instances, a well-established findings in the membership inference literature, and find the loss is discriminative when the frequency is high (i.e., likely when models already verbatim memorize the sequences), however, at which point the members is confounded by near-duplicate edits, again, another established finding in the memorization literature.
Critical strengths
None
Critical weaknesses
Novelty: Research question and observations have been well explored in the literature:
- There is a rich literature on membership inference and rigorous methodologies to measure counterfactual memorization, and it appears that this work is on the journey of rediscovering some of these well known findings in the community. The research question around when is membership inference possible and the core observations regarding repetitions, near-duplications are well known in the literature.
- While the core claim "Loss-based evidence of membership becomes strong only when a sentence has been duplicated on the order of a thousand times or more." is plausible, it is also trivial in some sense, at this repetition range, there is no need for membership inference as the model likely can verbatim output the members.
- In general, there appears to be insufficient engagement with literature. This perhaps can be explained by a surprisingly short list of reference.
Presentation: Excessive use of vague and anthropomorphizing languages that does not meet the standard of scientific writing:
- For a paper that studies membership inference, there is no mention of membership inference terminologies in the abstract, and in fact exact two mentions of "membership inference" in the main text, including one the related work. instead, the paper is filled with vague and anthropomorphizing phrases. Just use the opening sentences of the abstract as an example: "When a language model finds a sentence unusually cheap to predict, it is tempting to conclude that the sentence was in its training data. Almost every published test of that inference has had to guess which sentences were in the training data, the members, and which were not." What does "cheap to predict" mean here? And how does a probability model "finds" something cheap to predict? "Almost every published test" which published test? "of that inference" Which inference?
Questions for authors
Apart from addressing lack of novelty discussed above, it would be great if the authors can clarify a few claims made in the paper.
- "so what the model is rewarding is the author’s word choice, not memory": What does memory mean here? Why reconstructing a particular author's word choice is not memory?
- "The unit is a sentence, not a paragraph": Why sentence? Why not shorter n-grams? Why not more complicated syntactic structures.
- "Membership evidence from loss is either weak or confounded, and the transition between the two regimes is not a place where a careful test can stand": What does it mean "not a place where a careful test can stand"?