- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper investigates the connection between membership inference attacks (MIAs) and machine-generated text detection. Under idealized asymptotic assumptions, it argues that both tasks share the same optimal likelihood-ratio statistic comparing the target model’s distribution with the human-text population distribution. It interprets existing methods as approximations to this statistic and evaluates their cross-task transferability using MIMIR and RAID. The experiments report substantial correlation between method rankings and show that Binoculars, originally developed for text detection, performs strongly on membership inference as well. Additional analyses examine black-box detection and methods with limited transferability, such as Zlib. The paper also introduces MINT, a unified evaluation suite for methods from both tasks.
Critical strengths
- The paper is well written.
- The paper provides a diverse range of empirical results.
Critical weaknesses
My main concern is that the paper’s conclusions do not appear particularly surprising. Connecting these two research areas seems fairly straightforward, and I believe there is a substantial body of similar work, particularly from 2025–2026.
Questions for authors
Could you briefly explain what is novel about the paper’s conclusions and what actionable insights they offer for designing future machine-generated text detectors?
3 comments
Thank you for your rebuttal. I will keep my positive score. I think this is well written with a diverse range of empirical results. Good luck!
Thank you! We appreciate it
Response to Reviewer T5h5
Thank you for your valuable feedback. We sincerely appreciate your positive assessment of the clarity of our paper and the breadth of our empirical results.
We’ll reply to each question below.
We agree that similar methods have emerged in both MIAs and machine-generated text detection, making their connection appear intuitive. However, methodological similarities alone do not establish whether the two tasks, which have fundamentally different objectives (members vs. non-members and machine-generated vs. human-written text), share a theoretical foundation or whether methods developed for one task perform well on the other. As discussed in Section 4, prior studies have noted specific similarities, but a systematic understanding of their relationship has remained limited.
Our novelty lies in moving beyond these individual observations to establish a theoretical and empirical foundation for cross-task transferability. Specifically, (1) we show that both tasks share the same optimal likelihood-ratio statistic under idealized assumptions—and during the rebuttal, we further proved a non-asymptotic theorem showing that for any finite-capacity, single-epoch model, cross-task transfer between the exact optimal MIA and detection statistics is directly proportional to expected counterfactual memorization γ(x)/PQ(x) (2) we provide, to our knowledge, the first systematic evaluation of methods from both communities on each task's standard benchmark under a unified implementation; and (3) we reveal substantial cross-task performance correlation and demonstrate that top detectors match leading MIA methods (and vice versa in black-box detection). These results establish that the connection is not merely a methodological similarity but also has practical significance.
Our findings also offer guidance for future detector development. First, our empirical results motivate considering MIA methods as candidates for machine text detection (for instance,
Min-K%andDC-PDDoutperform several dedicated detectors in black-box detection), rather than developing methods independently within each community. Second, our likelihood ratio perspective identifies the approximation of the unknown human text distribution as a common design objective, with existing methods adopting different strategies, such as reference models or perturbation-based sampling. This provides a principled basis for exploring new approximation strategies and combining existing approaches. For instance, selecting methods that approximate this distribution differently may help exploit complementary signals in ensembling. While this direction remains to be validated experimentally, it illustrates how our framework can inform future method design.