- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents IdeaLens. It predicts whether a document’s ideas came from an AI regardless of who wrote the words. Documents are converted to outlines of discourse roles with content descriptions that are paraphrased for training and the model is trained on Pangram’s prose-level labels while ProseLens is trained on raw text as a control. When idea and prose provenance differ the two models disagree and IdeaLens remains competitive on standard detection.
Critical strengths
- Only the input changes so the same labels teach a different thing. ProseLens is an appropriate control. In IdeaShift the flag rate of IdeaLens falls as the human plan becomes more detailed while ProseLens and Pangram remain high.
- The evaluation is broad and the limitations are clearly acknowledged.
Critical weaknesses
- The appendix itself reports that a prose detector still separates paraphrased outlines well above chance and that renaming characters changes the scores. Detection also drops sharply on newer generators.
- The key quadrant rests on a small set of stories from one format (creative writing) and one generator. CoCoDet from ACL 2026 is evaluated on the same benchmark and is not compared.
Questions for authors
- What is IdeaLens’s accuracy on the documents that Pangram 4 misclassifies when given only the paraphrased outline (Table 9)? What is the TwiceTold flag rate after proper nouns are anonymized?
- How does IdeaLens compare with CoCoDet on CoCoNUTS in the binary setting?