- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies AI detection in text, builds a large corpus of generated texts using the latest models, paired with texts from 2015 (i.e., not AI since pre-ChatGPT), and trains classifiers, showing they outperform certain weak baselines from a COLING workshop.
The paper is (ironically) entirely AI-generated, and absolutely filled with low-quality "AI slop" catchphrases, with unfortunately very little conceptual clarity beyond the hypey phrases. It suffices to read the Abstract which jumps right in and provides extremely little context, to understand the little value this work would add to the community.
The fact that the work is about AI detection is particularly ironic: I wonder how their draft would be classified by the models they trained. I encourage the authors to reconsider what they submit to conferences, as submitting low-quality drafts such as these is leading to a crisis in science. Instead, I invite the authors to actually work on AI detection, which is an important topic, and take the time to distill their findings into a thoughtful draft, which will help our community better understand how AI detection works. Please do not offload your thinking (and your thinking) to AI.
Critical strengths
None, to be very frank.
Critical weaknesses
- I don't think the dataset creation proposed is valid, it is just a big shortcut: they claim to have this "2026 frontier paired dataset of human-written vs. AI-generated texts", but only the AI-generated texts (the easy part) are from 2026, which the human-written part is from 2015...
- The comparisons all involve cryptic, unknown models. There does exist high-quality AI detection models (GPTZero, Pangram, just to name a few, including recent Pangram's open-source model). Why not compare to actual, known baselines...
- The writing is absolutely atrocious, and could likely serve in a classroom for all the AI cliches for what not to do for clear, scientific communication. It would not surprise me if this paper was generated entirely autonomously by an AI agent.
Questions for authors
See weakness and summary.