- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces MixDetect, an approach for identifying and quantifying AI-edited or AI-generated text. It not only estimates the proportion of text affected by AI editing, but also measures how substantial the edits are. The method uses two separate prediction objectives to model editing occurrence and editing intensity, allowing it to capture both the scope and strength of AI editing.
Critical strengths
A key contribution of this work is that it explicitly models the magnitude of AI editing rather than only determining whether a text contains AI-generated content. By separating editing scope from editing intensity, the proposed formulation provides a more fine-grained characterization of human–AI collaborative writing and may facilitate further studies on the differences between human-written, AI-edited, and AI-generated text.
Critical weaknesses
The main concern is that the proposed editing intensity is supervised using an automatically constructed proxy rather than human-annotated edit magnitude. For substituted words, the target is defined by contextual dissimilarity between source and edited words, and inserted words are assigned maximum intensity. As a result, the current evaluation primarily demonstrates consistency with this proxy, but does not establish that the predicted intensity corresponds to human-perceived editing severity. Independent human annotation or external validation would make this claim more convincing. In addition, the related-work coverage appears incomplete. Several recent works on fine-grained human–AI authorship localization, including word-level and sentence-level detection in collaborative text, are not discussed. Since fine-grained localization itself has prior precedent, the paper should more carefully position its novelty around the explicit decomposition of editing occurrence and editing intensity.
Questions for authors
Why are and in Table~1 evaluated using partial Spearman correlation rather than standard Spearman correlation?
W1: Editing intensity lacks independent human validation.
We conducted an independent evaluation with three annotators on 120 word-level editing examples, yielding 360 ratings. The reference intensity agreed with human ratings in 296 cases (82.22%), and MixDetect's predictions agreed in 273 cases (75.83%); inter-annotator agreement was 0.64. MixDetect also showed strong correlations with external editing-degree and editing-proportion measures on EditLens and APT-Eval. On LAMP, which provides human-annotated editing regions, its regional localization AUROC was 0.769. These results support both the word-level supervision and the predicted editing intensities.
W2: Related work and novelty positioning need clarification.
We will expand the related work to cover HACo-Det (ACL 2025) and SenDetEX (EMNLP 2025). The former studies fine-grained authorship attribution in human–AI co-written text; the latter identifies AI-generated sentences in mixed text. EditLens estimates AI editing degree at the document level. MixDetect instead models word-level edit occurrence and conditional editing intensity in AI-edited text, locating edited words while separately quantifying editing scope Q, conditional intensity M, and overall magnitude S. Its central contribution is the scope–intensity decomposition, rather than word-level authorship attribution alone.
Q1: The use of partial Spearman needs justification.
Partial Spearman measures the predictive quality of Q and M separately: it controls for the reference M when evaluating Q, and for the reference Q when evaluating M. We also computed standard Spearman correlations. Under Edit-Only, standard/partial correlations were 0.886/0.878 for Q and 0.658/0.640 for M; under Full, they were 0.879/0.873 for Q and 0.671/0.617 for M. Both metrics show that MixDetect effectively predicts editing scope and conditional intensity.