- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes AnchorWrite, a detector-query-free framework for rewriting LLM-generated text toward a fixed statistical reference constructed from human-authored documents. It combines cross-order n-gram statistics, length-conditioned calibration, token-level anomaly localization, and constrained LLM rewriting. Candidate rewrites are accepted based on reference-distance reduction and preservation constraints. Evaluation across 900 documents, six domains, and six external detectors reports reduced reference distance and detector separability while maintaining high semantic similarity. Additional analyses examine matched-edit baselines, human judgments, and transfer across model backbones.
Critical strengths
-
The framework provides a coherent and interpretable approach to controlled rewriting without downstream detector feedback. Separating token-level anomaly localization from document-level reference-based acceptance makes the optimization process understandable and allows explicit preservation constraints.
-
The evaluation is broad and considers both detector transfer and text quality. Multiple domains, detectors, rewriting strengths, and backbones, together with matched-edit comparisons and human evaluation, provide meaningful evidence about the method’s behavior. The manuscript also carefully acknowledges important limitations.
Critical weaknesses
-
The distinctive benefit of reference-guided optimization is not sufficiently isolated. Reference-distance reduction is explicitly required for candidate acceptance, so this reduction alone does not establish an advantage over generic constrained rewriting. At L4, Qwen Rewrite achieves lower AUROC on five of six detectors with comparable editing and semantic similarity. Controlled comparisons separating anomaly localization and reference-based acceptance are therefore important for establishing the proposed mechanism’s additional value.
-
The interpretation and usefulness of the reference objective require stronger validation. Held-out human documents reportedly have a larger mean reference distance than original AI-generated documents, indicating that lower distance does not necessarily imply closer alignment with human writing in the evaluation domains. The authors appropriately acknowledge this distribution shift, but the benefit of optimizing this particular corpus-specific reference remains insufficiently established beyond reducing the defined objective.
Post-rebuttal update Thank you for the response and the additional analyses. After reviewing the rebuttal, I still find the evidence insufficient to support the paper’s main conclusions. I consider this concern substantial enough to warrant rejection and have revised my rating from 2 to 1.
Questions for authors
-
How do the reference-distance changes achieved by Qwen Humanize, Pegasus, and Qwen Rewrite compare with AnchorWrite under comparable editing and preservation constraints?
-
Can you isolate the contributions of anomaly localization and reference-based candidate acceptance while keeping the proposal budget, rewriting backbone, and preservation constraints fixed?
-
Given that held-out human documents have larger reference distances than original AI-generated documents, what specific writing characteristics does reducing this distance capture? How should the claimed alignment benefit be interpreted?
-
Can you report inference costs, including candidate counts or generated tokens, and paired uncertainty estimates comparing AnchorWrite directly with the main matched-edit baselines?
General Response to All Reviewers and the Area Chair
Dear Reviewers and Area Chair,
Thank you so much for your time, thoughtful reviews, and constructive feedback. Your comments have helped us improve the methodological clarity, experimental validation, and presentation of AnchorWrite. We have revised the manuscript and provided detailed individual rebuttals. Below, we summarize the main changes in response to each reviewer's concerns.
Reviewer 3tDd, thank you for highlighting the need to justify the reference statistic, clarify baseline comparisons, and strengthen human evaluation. We clarified the 2–4 cross-order statistic, distinguishing token-level localization from document-level reference-based acceptance and defining D as a corpus-relative control objective, not a universal human-likeness measure (Sections 3.2–3.4; Appendix B). We added a 150-document 2×2 control study (Section 5.6; Appendix J) and a 900-document reference-distance comparison with Pegasus, Qwen Humanize, and Qwen Rewrite (Appendix J.5). We acknowledge that Qwen Rewrite and back-translation obtain lower AUROC on several detectors (Appendix H), so we do not claim universal detector superiority. We also expanded the human-evaluation analysis with Krippendorff's alpha, rating distributions, paired tests, and discussion of low agreement and ceiling effects (Appendix M.7).
Reviewer 3mVb, thank you for urging a clearer assessment of reference guidance and its computational cost. At L4, AnchorWrite reduces the fixed reference distance by +0.3630, versus −0.0426 for Pegasus, −0.0669 for Qwen Humanize, and −0.1242 for Qwen Rewrite, with paired confidence intervals (Appendix J.5). These results demonstrate optimization of the specified reference objective, not independent proof of human-likeness. Our 150-document 2×2 study compares cross-order/random localization and reference-guided/D-blind acceptance (Section 5.6; Appendix J). AnchorWrite attains BERTScore 0.96785 and MiniLM similarity 0.97615 with fewer evaluated candidates, but unequal realized edits and proposal pools prevent strict causal isolation. We clarified the held-out-human distribution shift and the corpus-relative meaning of D (Section 5.1; Appendix B.4), and added candidate counts and direct paired AUROC comparisons (Appendix J.4–J.5). Candidate counts are not GPU-hours or generated-token costs.
Reviewer VTQr, thank you for your detailed questions on statistical uncertainty, mechanism validation, reference validity, transfer, and compute. We added paired AUROC confidence intervals against Qwen Humanize and Pegasus for detectors with retained sample-level scores, including inconclusive and baseline-favoring comparisons (Appendix J.5), and the 150-document 2×2 study with its limitations (Section 5.6; Appendix J). We reported that held-out human texts have larger D than original AI texts under the fixed UMBC calibration (Section 5.1; Appendix B.4); as clarified in our rebuttal, in-domain recalibration was not performed, so its effect remains unknown. We explain that cross-order predictability is a corpus-derived control signal, not a comprehensive human-writing measure (Section 3.2). Three additional rewriting backbones were tested on 300 documents, with positive AUROC reductions in all 18 backbone–detector combinations (Section 5.8; Appendix L); this does not establish transfer to new source generators. We added candidate-level accounting (Appendix J.4), acknowledged the absence of DIPPER/MASH evaluations in our rebuttal, and discussed dual-use risks in the Ethics Statement.
Across these revisions, we distinguish optimization of the fixed reference objective from independently evaluated detector transfer and explicitly state the remaining limitations in reference-corpus sensitivity, mechanism isolation, and computation. We appreciate your time and would welcome any further questions or clarification requests.
Best regards,
Authors