acceptodds
Under review as a conference paper at ICLR 2027

Caught You, AI Reply! Rephrasing Messages to Expose AI-Generated Replies

Abstract

Detecting undisclosed AI-generated replies can help authors assess whether the feedback they receive reflects a respondent’s own reasoning and judgment. While existing work has primarily focused on improving detectors, we investigate a com- plementary question: Can authors make AI-generated replies to their writing eas- ier to detect by changing how they express the same content? We formulate this question as constrained optimization of source wording using the expected score assigned to subsequent AI replies by an existing AI-text detector. Rephrasing to Expose AI Replies (REAIR) provides a common framework for studying this in- tervention with BoN, Hill-climb, Hill-climb-ICL, and a source-specific adaptation of GEPA. The framework selects rephrasings using detection feedback from the replies they elicit, with semantic similarity and length checked against the origi- nal source. For greedy Gemma 4 31B replies generated after source selection, the three iterative instantiations achieve true-positive rates of 77.23% to 78.39% on an online discussion task and 56.46% to 63.27% on a research-proposal review task, compared with 29.39% and 12.93% for unmodified sources. These rates use the Binoculars AI-text detector at its fixed published threshold. Across the evaluated settings, gains are smaller under sampled generation, and transfer varies with the reply model, instruction, and detector. These findings establish source rephrasing as an author-controlled means of improving AI-reply detection that complements existing post hoc detectors.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI-generated text detection, source rephrasing, detector-guided optimization, semantic preservation, large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jbeHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether authors can make subsequent AI-generated replies easier to detect by rephrasing their own messages while preserving their meaning. REAIR uses detector feedback on trial replies to select source rephrasings. Experiments on discussion posts and research proposals show detection gains.

Critical strengths

The paper presents an interesting intervention that gives authors a way to influence the detectability of subsequent AI replies. The substantial gains under matched settings are supported by separate final-reply evaluation, a manual semantic audit, and analyses controlling source and reply length. The transfer experiments also provide useful evidence about the approach’s limitations.

Critical weaknesses

The human comparison does not establish how well the method distinguishes human from AI replies when both respond to the rephrased sources. AI replies are generated from rephrased sources, but the human references respond only to the original sources. Rephrasing could also change human response style and hence false-positive rates. Although the paper explicitly discloses this design, neither the reported detection rates nor AUROC establish whether human and AI replies become easier to distinguish when both respond to the rephrased sources.

The practical applicability remains limited by dependence on the reply configuration used during optimization. The tested changes of reply model or instruction yield no statistically significant improvement in fixed-threshold detection rate, and some reduce detection, although some instruction-transfer AUROC gains remain. Improvements are also substantially smaller under sampled generation. These limitations matter because authors seeking to detect undisclosed AI use generally do not know the respondent’s model or instructions.

Questions for authors

Is there evidence about how rephrasing affects human replies? In particular, would the detection gains persist when both human and AI replies are collected for rephrased sources, with false-positive rates measured in that condition?

Given the limited transfer gains in Section 4.4, what deployment setting do the authors envision? Is the approach intended primarily for cases where the respondent’s model and instructions are known, or is there evidence that it remains effective when these are unknown during source optimization?

4 comments
AuthorsOpenReview ↗

Response to Reviewer jbeH

Thank you for the review. We address the human-reply comparison and the intended deployment setting below.

W1/Q1: Does rephrasing affect human false positives, and do detection gains persist when both humans and AI respond to rewritten sources?

Our fixed-threshold AI-TPR gains do not depend on human replies: With the detector and threshold fixed, TPR measures the proportion of AI replies classified as AI. Changes in human replies therefore cannot alter our measured AI-TPR improvement.

Changes in human replies do not necessarily increase the false-positive rate: Human replies can differ across people or change when the source is rephrased. These differences in wording or style do not necessarily make the replies more likely to be classified as AI. Detection gains can therefore persist even when human replies change.

Comprehensively studying how rephrasing affects human replies requires substantial time and effort: Reliably measuring changes in the human-reply distribution requires substantive replies to both original and rewritten sources from comparable participants. This separates the effect of rephrasing from differences between people. Designing and conducting such a dedicated study requires substantial additional time and effort.

Human pilot: AUROC improves numerically when both humans and AI reply to rephrased posts: To examine whether source rephrasing affects human reply scores and false positives, we collected replies to 20 CMV posts, randomly assigned to Original or frozen BoN-8 rephrasing (10 each). One author wrote one reply per post, seeing only one version without knowing whether it was original or rephrased. We scored all 20 replies with the same Binoculars detector.

StatisticOriginalRephrasedTwo-sided p-value
Mean score0.96501.00170.2021
Median score0.95551.0088—
Standard deviation0.04360.0752—
FPR (main-table threshold)0/10 (0%)1/10 (10%)—

Human replies to rephrased posts have higher mean Binoculars scores, indicating weaker AI evidence on average. The two-sided test finds no significant difference in mean scores. Although the rephrased group has one additional false positive, this small pilot does not establish that rephrasing systematically increases human false positives.

Human replies toAI replies toAUROC ↑
Original postsOriginal posts0.8700
Rephrased postsRephrased posts0.9133

The rephrased group has a higher AUROC. The result is an encouraging signal that detection gains may persist when both humans and AI reply to rephrased posts.

W2/Q2: Where can REAIR be used in practice, and how effective is it when the respondent's model and prompt are unknown?

Envisioned deployment: authors use likely reply configurations to guide source rephrasing and improve AI-reply detection: An author preparing a discussion post or research proposal may not know the respondent's exact configuration, but can identify a set of likely models and prompts. We envision authors using REAIR to rephrase their posts or proposals based on likely reply models and prompts, with the aim of making subsequent AI replies easier to detect.

Joint selection improves AUROC numerically across the tested model set: To test the envisioned setting, we used feedback from several likely reply models to select one shared rephrasing, then evaluated it across those models.

For each of 20 CMV posts, we selected among 100 rewrites using trial replies from Qwen, Gemma and Ministral at T=0.7. A candidate had to improve the Binoculars score over Original for all three models. We chose the candidate with the largest average improvement. If none qualified, we kept Original. We then froze the selections and generated three fresh replies per model to both Original and the selected version.

Reply modelAUROC: Original → Selected ↑TPR (%): Original → Selected ↑
Qwen3.5-9B0.8910 → 0.931736.67 → 43.33
Gemma-4-E4B0.8412 → 0.876721.67 → 20.00
Ministral-3-8B0.8316 → 0.866218.52 → 18.52

The same selected rewrites improve AUROC numerically for all three models, while TPR changes are mixed. This provides preliminary evidence for selecting against a set of possible models without knowing the respondent’s exact choice.

REAIR shows numerical transfer gains and offers a starting point for methods robust to unknown models and prompts: In Qwen3-32B prompt transfer, both TPR and AUROC improve numerically in seven of eight method–temperature comparisons, and AUROC improves in all eight (Table 24). These results provide an encouraging signal for extending detection gains beyond the optimization prompt. REAIR provides a first step and a common framework for studying author-side detection enhancement. Our next work will focus on stronger methods that are robust to unknown reply models and prompts.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

Thank you again for your time and thoughtful feedback on our paper. We have provided detailed responses and additional experimental results in our rebuttal, and would appreciate hearing whether these address your questions and concerns.

If any points remain unclear or require further discussion, please let us know. We would be happy to clarify them and discuss any remaining concerns.

Thank you for your time and consideration.

Best regards, The authors of Paper 48244

I thank the authors for their response. The authors provided a new pilot, which deserves credit. There is improvement from 0.87 to 0.91 in AUROC, however only 10 replies per condition from one person provides rather little evidence about the range of human responses. It does give positive signal but falls short of fully addressing the point. I am not requesting new experiments as this is last day, but this uncertainty remains.

On transfer, the detection-rate improvements are helpful evidence. It’s still unclear whether the improvement is sufficiently reliable across plausible unknown configurations given the small held out test and mixed detection rate resultss across models.

With these concerns I view my original concern as partly addressed. Since score increments jumps directly to weak positive, I think the response is helpful though does not quite justify raising my score.

AuthorsOpenReview ↗

Response to Reviewer jbeH’s Follow-up

Thank you for your follow-up and for recognizing the positive signal from our human-reply pilot and the helpful evidence provided by the transfer experiments.

Due to the limited rebuttal timeframe, these additional studies were conducted as pilots. We plan to include a larger human-reply study and evaluations across a broader range of reply configurations in the revised manuscript to further validate these encouraging observations.

Thank you again for your time and constructive feedback.

Reviewer_yWv6OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2(+1)
Read review
Paper summary

The paper focus on whether authors can reformulate their own source texts so that the subsequent AI-generated replies become easier for an existing detector to identify. The authors propose a method named REAIR, which uses detector feedback to select source-text rephrasings that make subsequent AI-generated replies easier to detect, subject to semantic-similarity and length constraints. Experiments on ChangeMyView and AI-Researcher show improved detection under matched reply configurations, but, to be noted, gains are smaller under sampling, transfer across reply models and instructions is weak, and cross-detector results are heterogeneous.

Critical strengths
  1. Interesting Problem Angle. Shifting AI text detection from passive post hoc analysis to active pre-generation source phrasing is a neat conceptual twist
  2. Detailed experimental documentation. The appendices provide prompt templates, generation settings, and additional result tables that help readers inspect the evaluation protocol.
  3. Clear presentation. The paper presents the optimization objective and the main search procedures in an organized manner.
Critical weaknesses
  1. Questionable Motivation The motivation does not sufficiently establish the practical value of detecting AI-generated replies. The paper motivates REAIR by claiming that authors need to detect undisclosed AI replies to properly "interpret and use that feedback". This premise is unconvincing across both evaluated tasks. In argumentative discussions (ChangeMyView) and technical reviews (AI-Researcher), the utility of feedback is determined by its epistemic merit ( i.e. logical soundness, factual accuracy, and constructiveness), which a competent author can evaluate directly rather than who or what generated it. Given that frontier LLMs often provide feedback matching or exceeding average human quality, a valid critique remains valid regardless of its provenance. Conversely, if the goal in peer review is enforcing human accountability rather than assessing content quality, the authors explicitly concede in their Ethics Statement that detector outputs have high false-positive rates (7.63% on AI-Researcher) and "should not be used as the sole basis for accusations or punitive decisions". If detections cannot be used for enforcement, and authors can judge the substantive arguments on their own, spending very much tokens per source to "expose" AI replies lacks practical justification.
  2. Methodological Flaw. In practice, an author never knows which LLM, system prompt, or temperature a respondent will secretly use. Yet Sec 4.4 and Appendix D show that REAIR has zero statistically significant transfer across models or prompts. Worse, sources optimized for Gemma 4 31B actually degrade Qwen3-32B detection under greedy decoding and sampling setting. REAIR is not finding generalizable linguistic triggers; it is merely overfitting to the brittle decoding trajectory of a single known model and prompt.
  3. Unrealistic LLM Decoding Setting. Real-world LLM deployments default to stochastic sampling (T≥0.7T ≥ 0.7), not greedy decoding (T=0T=0). What’s more, Tab 7 notes Explicit reasoning trace: disabled. Modern LLMs heavily rely on reasoning/CoT before outputting a response.
Questions for authors

I rewrite the weakness above into the questions below:

  1. Actionability of Detection: Since a user can directly evaluate the logical and technical merit of a reply on their own, and your Ethics Statement notes that detections cannot be used for punitive actions, what concrete, actionable decision does an author make differently once a high-quality reply is flagged as AI-generated?
  2. Optimization for Transferability: Can optimizing a source text against an ensemble of reply models and prompts during search produce rephrasings that successfully transfer to a held-out, unseen model?
  3. Effect of Reasoning Mode: How does REAIR perform when Explicit reasoning trace is enabled on the reply model? Does the intermediate thinking step wash out the detector score gains?
5 comments
AuthorsOpenReview ↗

Response to Reviewer yWv6 (Part 1/3)

Thank you for the review. We address motivation, transfer, and decoding settings below.

W1/Q1: Why detect AI-generated replies if their quality can be judged directly, and how can detection inform action?

Feedback quality does not determine compliance with AI-use rules: ICML 2026 detected violations by reviewers who had agreed not to use LLMs and desk-rejected 497 associated submissions. Its chairs explicitly separated these decisions from review quality (official report). Thus, judging a reply's quality does not remove the need to determine whether it complied with the conditions under which feedback was requested.

Evaluating feedback manually requires time and effort: The ability to judge a reply does not make that judgment cost-free. Assessing logical soundness, factual accuracy and technical merit takes time, and the burden grows when AI-generated material can be produced at scale. Manual assessment therefore does not remove the practical role of automated checks.

REAIR gives authors a technical tool to complement detection through source rephrasing: ICML used hidden instructions in submission PDFs to identify reviews that violated agreed AI-use policies (official report). This illustrates a concrete use for detecting AI involvement beyond judging review quality. REAIR studies how authors can support this detection by changing the text they control. Authors rephrase their source texts while preserving meaning, aiming to make subsequent AI replies easier to detect. Our contribution is this author-side intervention and its experimental evaluation.

The Ethics Statement permits corroborated use of detection evidence: Our separate Ethics Statement following the main text says detection should not be the sole basis for accusations or punitive decisions. Q1 instead paraphrases this as “cannot be used for punitive actions,” which is a different claim. Consequential decisions require corroborating evidence; detection can contribute to checking compliance with agreed AI-use rules alongside other evidence. The safeguard therefore does not imply that detection has no actionable value.

AuthorsOpenReview ↗

Response to Reviewer yWv6 (Part 2/3)

W2/Q2: Do the transfer results indicate overfitting, and can ensemble selection improve transfer to unseen models and prompts?

REAIR provides a framework and empirical foundation for more transferable source-rephrasing methods: REAIR introduces source-side rephrasing and a framework comparing four search methods. It does not claim universal linguistic triggers. The matched-setting gains establish feasibility, while §4.4 reports the transfer limits of these methods. Establishing where they succeed and fail provides an empirical foundation for developing and evaluating source-text optimization methods with more robust transfer.

Transfer results do not establish single-trajectory overfitting: Qwen prompt transfer improves TPR numerically in seven of eight comparisons and AUROC in all eight (Table 24). Gemma-to-Qwen greedy BoN raises TPR from 66.76% to 67.72% and AUROC from 0.9504 to 0.9614 (Table 23). The other cross-model comparisons with a numerical gain in at least one metric are:

Transfer directionTMethodTPR (%): Original → rewriteAUROC: Original → rewriteSource
Qwen → Gemma0BoN32.18 → 34.100.9117 → 0.9098Table 22
Qwen → Gemma0Hill-climb32.18 → 31.600.9117 → 0.9169Table 22
Qwen → Gemma0Hill-climb-ICL32.18 → 32.280.9117 → 0.9112Table 22
Qwen → Gemma0GEPA32.18 → 32.760.9117 → 0.9203Table 22
Qwen → Gemma0.7Hill-climb31.70 → 33.140.9104 → 0.9135Table 22
Qwen → Gemma0.7GEPA31.70 → 31.890.9104 → 0.9069Table 22
Gemma → Qwen0GEPA66.76 → 67.340.9504 → 0.9425Table 23

Our results show numerical improvements in both cross-model and cross-prompt evaluations, alongside declines in other settings. These mixed results do not justify concluding that REAIR “is merely overfitting to the brittle decoding trajectory of a single known model and prompt.”

Ensemble-selected rewrites improve TPR numerically on a held-out model and prompt: To address Q2, we selected among 100 candidates for each of 20 CMV posts using three reply models × three prompts × two temperatures. Candidates had to improve the mean Binoculars score overall, for at least two models and two prompts, and at both temperatures. We selected the eligible candidate with the largest overall mean improvement, retaining Original if none qualified.

We froze these selections and tested Qwen3-32B with a new prompt at T=0 and T=0.7, with thinking disabled; neither this model nor prompt was used for selection. We generated three replies per post and condition at each temperature for Original and the selected version. The same posts and rewrites were used at both temperatures, without reselection. TPR uses the paper's fixed threshold.

TMethodTPR (%) ↑AUROC ↑Mean Binoculars score ↓
0Original85.000.99170.8014
0Ensemble-selected95.000.98890.7980
0.7Original75.000.98060.8166
0.7Ensemble-selected80.000.98060.8159

On this held-out model and prompt, the same frozen rewrites yield numerical TPR gains of 10.00 and 5.00 percentage points at T=0 and T=0.7, respectively. AUROC decreases slightly at T=0 and is unchanged at T=0.7. These results provide positive evidence of fixed-threshold detection gains beyond the selection configurations.

Joint selection improves AUROC numerically across the tested model set: An author may identify a set of likely reply models without knowing which one the respondent will use. We therefore also tested joint selection within such a set, separately from the held-out evaluation above.

Using the same 20 CMV posts and 100 candidates per post, we reselected rewrites using Qwen, Gemma and Ministral with a counterexample-based reply prompt at T=0.7. Candidates had to improve the Binoculars score over Original for all three models. We chose the largest average improvement, retaining Original if none qualified. We then froze these new selections and generated three fresh replies per model to both Original and the selected version.

Reply modelAUROC: Original → Selected ↑TPR (%): Original → Selected ↑
Qwen3.5-9B0.8910 → 0.931736.67 → 43.33
Gemma-4-E4B0.8412 → 0.876721.67 → 20.00
Ministral-3-8B0.8316 → 0.866218.52 → 18.52

The same selected rewrites improve AUROC numerically for all three models, while TPR changes are mixed. This provides preliminary evidence for selecting against a set of possible models without knowing the respondent’s exact choice.

AuthorsOpenReview ↗

Response to Reviewer yWv6 (Part 3/3)

W3/Q3: Do the detection gains persist under stochastic decoding and when the reply model's thinking mode is enabled?

The reported gains also occur under stochastic decoding: At T=0.7, Table 1 reports CMV TPR gains of 30.16%→43.90% for Gemma Hill-climb (+13.74 pp) and 57.25%→66.47% for Qwen Hill-climb-ICL (+9.22 pp), both with unadjusted source-level paired p < 0.001. The effect is therefore not restricted to greedy decoding.

Detection gains can persist with thinking enabled: To answer Q3, we evaluated 20 randomly sampled CMV sources with thinking enabled in Qwen3-32B at T=0.7, using budgets of 2,048 reasoning tokens and 512 final-answer tokens.

Original uses the unmodified source text. BoN (thinking on) selects from the existing eight-candidate pool using the mean detector score of three thinking-on trial replies per candidate. BoN (thinking off) uses the rewrites previously selected with thinking disabled. We generated three fresh thinking-on replies per source in each condition and scored only the final answers. TPR uses the same fixed threshold as the paper’s main tables.

MethodTPR (%) ↑AUROC ↑Mean Binoculars score ↓
Original3.330.69420.9283
BoN (thinking on)6.670.70920.9275
BoN (thinking off)3.330.72830.9256

With thinking enabled during both selection and evaluation, BoN improves TPR, AUROC, and mean detector score numerically over Original. Reusing rewrites selected without thinking also improves AUROC and mean score. These results provide preliminary evidence that source rephrasing can retain detection benefits when the reply model uses explicit reasoning, including when the rewrites were selected without thinking.

Reviewer_rPGcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors attempt to optimize a given text to make AI-generated replies generated in response to that text more detectable by an AI detector. They sweep rewording methods to attempt to increase reply detectability, and eventually raise detector TPR from 29% to 77%. They contribute the novel task to the literature (adjusting source texts to increase reply detectability) as well as their initial methods and promising results.

Critical strengths

Two strengths of this paper are the novel task formation as well as the variety of rephrasing methods attempted. It is a creative yet useful idea to be able to optimize a given text for maximally detectable replies. The authors select and explain the variety of methods they use to solve this task, such as Best of N, Hill Climb, etc. The cross-detector transfer experiment is a compelling and necessary addition, as it suggests different detectors may be optimized for in different manners. It clearly sets up future work in this field.

Critical weaknesses

Binoculars as the primary chosen detector is a weaker evaluation choice, as it neglects newer, more robust detectors.

Questions for authors
  1. Can you elaborate more on the methodology by which you validate preservation of numerical values, dates, URLs, and citation identifiers? At what rate did rephrasing disturb any of these factual anchors?
3 comments
AuthorsOpenReview ↗

Response to Reviewer rPGc

Thank you for the review. We address detector choice and factual-anchor preservation below.

W1: Why use Binoculars, and do the gains extend to newer detectors?

Binoculars provides a meaningful evaluation baseline: Binoculars is an established zero-shot detector with a published operating threshold (ICML 2024). On CMV at T=0, its Original AUROC is 0.9100 for Gemma and 0.9529 for Qwen (Table 1), supporting its use as a meaningful baseline.

The submission already evaluates newer detectors: We tested DNA-DetectLLM, LastDE Base and SpecDetect Base through both cross-detector evaluation and detector-specific BoN selection. For Qwen/CMV at T=0, DNA-guided BoN raises DNA AUROC from 0.9597 to 0.9883, and LastDE-guided BoN raises LastDE AUROC from 0.7953 to 0.8658 (Table 21). Table 4 additionally evaluates all four Binoculars-guided methods with these detectors.

Gains persist when newer DNA-DetectLLM guides both search and evaluation: To test whether the gains depend on Binoculars, we evaluated all four methods using a newer detector, DNA-DetectLLM (NeurIPS 2025), which reports robustness to editing and paraphrasing attacks. DNA guided both search and final evaluation on 49 AI-Researcher proposals (Gemma-4-E4B rewrites; Gemma-4-31B replies; T=0).

Each condition contains three fresh final replies per proposal (147 AI replies), evaluated against the same 118 human references.

MethodMean DNA score ↓AUROC ↑Score p
Original0.6097040.877234—
BoN-80.5987510.9095185.00 × 10⁻⁵
Hill-climb0.5925650.9281105.00 × 10⁻⁵
Hill-climb-ICL, K=80.5907110.9284565.00 × 10⁻⁵
GEPA0.5932540.9227495.00 × 10⁻⁵

All four methods lower mean DNA scores and increase AUROC over Original. These results directly address the concern about detector choice: source rephrasing improves detection when DNA-DetectLLM replaces Binoculars in both search and evaluation.

Q1: How are numerical values, dates, URLs, and citation identifiers checked, and how often does rephrasing alter them?

CMV: our anchor checks and contextual review identified changes in 4.50% of final rewrites: We compare each generated candidate with its original text. A fixed script extracts numbers and units, dates, URLs, and citation identifiers from both texts. It standardizes their formats before comparison. For numbers written as digits, it checks whether the original values are retained. For dates, URLs, and citations, it checks both their content and occurrence counts. The extracted anchors and differences are logged and stored.

We applied these checks to 7,838 final rewrites. GPT-6 reviewed the 860 flagged records in context, and we manually checked uncertain cases. The identified changes are reported below, with rates calculated over all 7,838 rewrites.

AnchorChanged rewritesRateFindings
Number3374.30%41 changes to values or related claims; 296 omissions or less specific descriptions
Date (also counted under Number)200.26%1 alteration; 19 omissions or less specific descriptions
URLN/AN/ANo URLs were identified
Citation160.20%Already-broken link markers removed from one source; no URL targets were present
Total3534.50%Each affected rewrite counted once

Research proposals: we put saved factual anchors back unchanged after rephrasing, and the audit found 0% anchor disturbance: Before rephrasing, we save passages containing experiment details, such as numerical specifications and citations. We mark each passage with a uniquely numbered <LOCKED> tag. After rephrasing, a fixed script checks that every tag appears once and in the original order. It then restores the saved passages word for word and verifies that they exactly match the originals. The tags are removed before reply generation.

We audited all 372 final rewrites. We extracted anchors from each complete restored text and compared them with the original. We reviewed differences in context and manually checked uncertain cases. We found 0% factual-anchor disturbance (0/372).

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

Thank you again for your time and thoughtful feedback on our paper. We have provided detailed responses and additional experimental results in our rebuttal, and would appreciate hearing whether these address your questions and concerns.

If any points remain unclear or require further discussion, please let us know. We would be happy to clarify them and discuss any remaining concerns.

Thank you for your time and consideration.

Best regards, The authors of Paper 48244

Thank you for the detailed reply. The authors meaningfully engaged with my concerns, and as a result, I will maintain my positive score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary for the Area Chair

Dear Area Chair,

Thank you for coordinating the review. We summarize the reviewers’ assessments, the additional evidence, and the contribution established by the revised paper.

Reviewer engagement and updates

Reviewer rPGc reaffirmed their weak-accept recommendation, noting that we meaningfully engaged with their concerns.

Reviewer yWv6 acknowledged that our responses addressed some concerns and raised their recommendation to weak reject, with motivation being the only concern explicitly reiterated in their follow-up.

Reviewer jbeH credited the human pilot and transfer results as positive evidence, while retaining concerns about their breadth and reliability.

Strength agreement

The reviews recognize several substantive strengths.

  • An interesting author-side intervention (all three reviewers), with rPGc highlighting the novelty of the task and describing it as “creative yet useful.”
  • Substantial detection gains supported by independent final-reply evaluation, semantic assessment, and length controls (jbeH).
  • A range of search methods and informative cross-detector evaluation (rPGc), together with detailed experimental documentation (yWv6).

The main experiments demonstrate the size of this opportunity. At the fixed Binoculars threshold, greedy Gemma TPR increases from 29.39% to as high as 78.39% on discussions and from 12.93% to 63.27% on research reviews.

Concerns without full reviewer agreement

  • Practical motivation (yWv6). We disagree that detecting AI-generated replies lacks practical value. ICML 2026 has already used input-side interventions to expose AI-generated reviews, and reported AI-review detection at ICLR 2027 further demonstrates an existing practical need. REAIR addresses this need through meaning-preserving rephrasing of the source text authors control. Our motivation is therefore well supported by existing practice. ICML report, ICLR 2027 notice.

  • Human-reply comparison (jbeH). The pilot supplies the requested comparison with humans and AI replying to corresponding source versions. AUROC increases numerically from 0.8700 to 0.9133, with ten human replies per condition from one participant. These preliminary results provide no clear evidence that human replies to rephrased sources reduce detectability or systematically increase false-positive rates. jbeH explicitly acknowledged this encouraging positive signal (Appendix E.5).

  • Unknown reply configurations (jbeH, yWv6). Joint selection on 20 posts improves AUROC numerically across all three models used for selection. This provides encouraging evidence for selecting against plausible reply models without knowing which model in the set will be used (Appendix D.1). jbeH also acknowledged the added evidence on this concern as helpful.

Overall

REAIR introduces an author-controlled intervention that uses existing detectors to guide source wording and improve the detectability of subsequent AI replies. This contribution is supported by substantial gains across two tasks, successful optimization with a second detector, and semantic and length controls. The human pilot and joint-selection experiment provide encouraging evidence in the reviewers’ requested settings. We believe this novel problem formulation and its demonstrated empirical value merit acceptance at ICLR.

Thank you again for your consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.