acceptodds
Under review as a conference paper at ICLR 2027

Attacking AI Editing Scores with Adversarial Rewriting

Abstract

Large language models (LLMs) now generate and revise text so rapidly that verifying its provenance has become increasingly difficult across scholarship, journalism, law, and creative writing. The written record is becoming a blend of human and machine contributions, and the central question is no longer whether AI was involved, but how much of a text it shaped and how reliably that can be measured. Models designed for this task assign AI editing scores, and because such scores increasingly inform judgments about authorship, credit, and integrity, their trustworthiness under adversarial pressure is a critical concern. In practice, however, the original human-written text is usually unavailable when a text is scored, which may make AI editing scores sensitive to surface form rather than the actual extent of AI editing. We study whether meaning-preserving rewrites can substantially reduce AI editing scores in this setting. We propose Prior-guided Rewriting for Editing Score Suppression (PRESS), which uses a paired humanization prior to identify regions characteristic of AI-edited writing and a surrogate model to select candidate rewrites. PRESS requires neither target model queries nor the original human-written text paired with the input, while a perturbation budget and quality checks control the rewriting process. The experimental results show that PRESS can lower AI editing scores across four models without moving the adversarial text closer to the original human-written text, raising concerns about their reliability as measures of the extent of AI editing.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI editing extent, AI editing scores, adversarial rewriting, robustness, AI-generated text detection, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uNW6OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper attempts to attack the AI edit score of a detector, the first adversarial robustness technique aimed at suppressing AI edit scores, rather than binary AI detector probabilities. The authors identify regions characteristic of AI-edited writing using pairs of human and AI-edited text, and use that to identify rewriting regions in unseen text at inference time. They accomplish their goal

Critical strengths

The strengths of this paper lie in the metrics used. They have a robust suite of semantic, detail, and grammatical preservation metrics. Additionally, the results that Editlens score while not moving embedding distance closer to human-written text are compelling.

Critical weaknesses

There is fairly low novelty of approach. The difference between this approach and approaches designed to reduce the binary score of fully AI-generated documents is not well-established. Additionally, the conclusion that AI editing scores may be unreliable is not well-founded or discussed prior.

Questions for authors
  1. Do other adversarial attacks designed for binary AI detection methods accomplish similar goals?
  2. Please expand on why adversarial attacks not moving texts closer to human-written texts questions the reliability of edit scores.
2 comments
  1. PRESS addresses a different problem from existing attacks: reducing continuous AI editing-extent scores rather than evading binary AI-text detection. To demonstrate this distinction, we evaluated StealthRL (Ranganath and Ramesh, 2026) on 2,031 AI-edited texts from the EditLens Test set.
ScorerPRESSStealthRL
RoBERTa+0.0258−0.1998
DeBERTa-large+0.0328−0.0470
ModernBERT-large+0.0279−0.0801
Llama+0.0331+0.0226
*Mean score reduction (original − adversarial; higher is better). *

PRESS reduces all four editing-extent scores, while StealthRL increases three of them. PRESS also modifies only 14.7% of the words on average, compared with 85.5% for StealthRL. These results show that attacks designed for binary detection do not necessarily work well for editing-extent estimation, motivating the task-specific design of PRESS.

  1. A lower AI editing score should indicate less AI editing, not merely a change in wording. For an AI-edited text, moving closer to its original human-written source provides a natural reference for assessing this change. However, Table 13 shows that 29–36% of AI-edited texts receive lower scores without moving closer to their original human-written versions. Table 14 further shows that the estimation errors increase across all four models when evaluated against editing levels derived from the original human texts. These results show that lower AI editing scores do not necessarily reflect less editing, directly challenging the reliability of current editing-extent estimators under adversarial rewriting.

I think a necessary part of this work is establishing if and how binary AI detection adversarial attacks fail at the task of attacking AI editing scores. This concern is not fully addressed by benchmarking a single adversarial attack (StealthRL), so I will maintain my score.

Reviewer_v7b9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether adversarial rewriting can reduce predicted AI editing extent without making the text closer to its original human-written source. It proposes PRESS, which combines a paired-data-derived lexical and representation prior with a surrogate scorer to localize and rank rewrites. Candidate generation and grammatical repair use local language models, subject to edit-budget and quality constraints. Attack generation requires neither queries to the target scorer nor access to the current input’s human source. The evaluation covers three splits, four editing-extent scorers, and Fast-DetectGPT. In addition to score reductions, the appendix reports source-distance analyses, changes in reference-level prediction error, human controls, budget sensitivity, and component ablations.

Critical strengths

1.Studying editing-extent estimates extends the usual binary detection setting. The source-based analyses are particularly valuable: Table 13 reports that approximately 29–36% of AI-edited inputs on Test and Test_llama receive lower scores without decreasing their distance to the human source. Table 14 additionally reports increased error against source-derived editing levels, with paired confidence intervals above zero. These results provide more convincing evidence of scorer instability than a reduction in mean scores alone. 2.The same adversarial outputs are evaluated across multiple scorer architectures, a held-out generation source, and an email domain. PRESS does not use target-model feedback during generation. The human controls, perturbation-budget analysis, and component ablations provide useful additional checks, while the source-group bootstrap accounts for dependence among examples sharing a human source.

Critical weaknesses

1.Paraphrase-based detector evasion is well established by DIPPER [1], and more recent work includes detector-agnostic humanization through iterative paraphrasing [2] and transfer to held-out detector families [3]. The paper acknowledges these connections, but its main external comparison is limited to a T5 paraphraser. PRESS combines a Gemma-based generator, surrogate scoring, iterative candidate search, and a separate Qwen repair model. Matching only the maximum word-edit budget does not isolate the proposed prior and ranking mechanism from generator capability, actual edit amount, quality filtering, or computational expenditure. Table 15 is helpful, but the gains over the no-prior variant are modest, and the prior-only variant outperforms the full method on RoBERTa in mean reduction and both relative-reduction rates. Stronger comparisons under a matched generation and selection pipeline are needed to establish the method-specific contribution. 2. The main editing-score reductions are approximately 0.02–0.07, while the proportion of inputs achieving a relative reduction of at least 20% ranges from about 8% to 25%. The larger proportion receiving any decrease should therefore not be interpreted as an operational evasion rate. The reference-error analysis in Table 14 is a useful step, but the evaluation does not yet establish how often attacks cause consequential editing-level underestimation or changes at relevant decision thresholds. Moreover, reported semantic similarity is based on local checks over accepted edits, using a metric also involved in acceptance. An independent assessment of document-level meaning, factual preservation, and grammatical quality is needed to substantiate the meaning-preservation claim.

[1] Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Paraphrasing Evades Detectors of AI-Generated Text, but Retrieval Is an Effective Defense. NeurIPS 2023. [2] Yixuan Even Xu, Ziqian Zhong, Aditi Raghunathan, Fei Fang, and J. Zico Kolter. Base Models Look Human To AI Detectors. arXiv, 2026. [3] Suraj Ranganath and Atharv Ramesh. StealthRL: Reinforcement Learning Paraphrase Attacks for Multi-Detector Evasion of AI-Text Detectors. arXiv, 2026.

Questions for authors

1.Tables 3 and 11 report different Test_enron AI-edited reductions: for example, 0.0268 versus 0.0217 for Llama and 0.0224 versus 0.0180 for DeBERTa-large. Do these tables use different subsets? Please provide the sample counts and inclusion criteria. Please also clarify scorer-specific valid denominators given the 4,077 Test inputs and the 4,076 paired observations used in Table 10. 2. Equation (8) equals the difference of the two empirical marginal means and is unchanged by permuting pair assignments when both multisets and weights remain fixed. Beyond source-matched sampling and surrogate supervision, what empirically demonstrated benefit specifically depends on pairing? How often is the theorem’s separation condition informative in practice?

2 comments

W1. PRESS addresses a different problem from existing paraphrasing attacks. Rather than evading binary AI-text detection, PRESS targets continuous AI editing-extent estimation. It learns human–AI editing patterns to guide localized rewriting, without querying target detectors or accessing the original human text at inference time.

DIPPER and Base Models Look Human To AI Detectors focus on binary detection evasion, rather than editing-extent estimation. We use T5 as a direct baseline for general-purpose paraphrasing under the same inputs, scorers, and maximum edit budget. Table 15 separately examines the contributions of our prior and surrogate within the PRESS pipeline.

We further evaluated StealthRL on 2,031 AI-edited texts from the EditLens Test set.

ScorerPRESSStealthRL
RoBERTa+0.0258−0.1998
DeBERTa-large+0.0328−0.0470
ModernBERT-large+0.0279−0.0801
Llama+0.0331+0.0226

*Mean score reduction (original − adversarial; higher is better). *

PRESS reduces all four editing-extent scores, whereas StealthRL increases three of them. PRESS also modifies only 14.7% of words on average, compared with 85.5% for StealthRL. These results highlight the importance of targeting editing-extent estimation directly.

W2. Our evaluation goes beyond score reduction. Table 13 shows that 29–36% of AI-edited texts receive lower scores without moving closer to their human-written sources. Table 14 further shows increased estimation errors across all four models when compared with editing levels derived from the original human texts. These results provide direct evidence of unreliable editing-extent estimates under adversarial rewriting. PRESS also integrates semantic checks, content preservation, and grammatical repair into its rewriting process (Section 3). Appendix D (Qualitative Examples) provides four examples with highlighted changes, which we manually checked for grammar, meaning, and factual consistency.

Q1: Thank you for pointing this out. Table 3 mistakenly retained results from an earlier Test_enron batch of 992 AI-edited texts, whereas Table 11 uses the full set of 2,308. We will correct Table 3. The Test split contains 4,077 inputs. The count of 4,076 in Table 10 is a typographical error. All four editing-extent scorers use the full 4,077 inputs. We will correct the corresponding counts.

Q2: Equation (8) is invariant to permutations of fixed pairs. However, correct pairing still matters. Appendix A.5 shows that correct pairing reduces estimation variance because the text embeddings of human-written texts and their corresponding AI-edited versions are positively correlated. To verify this, we conducted an additional analysis using 22,265 human–AI-edited pairs from the training data used to construct our humanization prior. We compared correct source pairing with random pairing and found a strong correlation of 0.886 between paired text embeddings. Correct pairing reduces the variance of embedding differences from 1.8540 to 0.2119 (88.6%). The reduction remains 87.7% compared with random pairing within the same domain. These results support Appendix A.5. The theorem's separation condition is sufficient, not necessary, for exact candidate selection. Our experiments independently demonstrate the effectiveness of PRESS in practice.

Thank you for your detailed responses and additional analyses. Your explanation resolves my questions about the Enron results and sample counts, and the StealthRL comparison provides useful evidence that attacks targeting binary detection do not necessarily transfer to editing-extent estimation. However, the substantial difference in editing rates limits this comparison, and the incremental contribution of the proposed prior and ranking mechanism remains insufficiently isolated. Manually checking four examples also does not fully address my concern about independent, systematic evaluation of text quality. I therefore consider my concerns partially addressed and maintain my overall assessment.

Reviewer_pyFXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

PRESS lowers a text’s estimated AI-editing score through iterative rewriting. At each step, it selects a phrase to edit, generates candidate rewrites, and ranks them using a humanization prior and a surrogate scorer, subject to meaning-preservation checks and an edit budget.

Critical strengths

The method is intuitive and technically plausible.

Critical weaknesses

Unclear how this attack compares with the prior attack RAFT: Realistic Attacks to Fool Text Detectors [EMNLP 2024]. RAFT is also an iterative rewrite based attack against AI text detectors.

Unclear whether this attack also degrades the performance of rewriting-based detectors such as Raidar: geneRative AI Detection viA Rewriting [ICLR 2024].

Questions for authors

How does your attack compare to RAFT in terms of both novelty and performance?

How does your attack work against rewrite-based detectors?

1 comment
  1. PRESS and RAFT address different objectives. RAFT uses word-level substitutions to evade binary AI-text detectors, whereas PRESS targets continuous AI editing-extent estimation, particularly for partially AI-edited texts. RAFT validates text quality through a separate human evaluation and explicitly acknowledges the need for larger-scale human assessment. In contrast, PRESS integrates semantic checks, content preservation, and grammatical repair directly into its rewriting process, allowing automatic candidate validation without requiring manual review during generation. More importantly, Tables 13–14 show that PRESS can reduce editing scores without moving texts closer to their human-written sources, while increasing source-based estimation errors. This reliability issue is not examined by RAFT.

  2. We further evaluated PRESS against RAiDAR on 2,031 AI-edited texts from the EditLens Test set. The results are summarized below.

MetricBefore PRESSAfter PRESSRelative reduction
AUROC0.70910.68193.8%
TPR @ 10% FPR36.04%30.82%14.5%

Without access to RAiDAR during attack generation, PRESS reduces its detection rate by 14.5% at a fixed 10% false-positive rate, which shows its transferability to rewriting-based detection despite being designed for editing-extent estimation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.