- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes SpanWM, a watermark scheme placed in a few syntactic phrases of an otherwise unwatermarked draft.
It uses a secret key to select up to 4 noun, verb, or prepositional phrases, and regenerates each selected phrase with a green-list token bias. It samples several candidate replacements and chooses one that fits. The detector parses the received text, selects phrases using the same key, and tests the green-token count within them. A pretty simple idea with solid numbers (if reproducible).
Critical strengths
- Rewriting selected phrases after draft generation is a clean way to limit changes to the original text, unlike the other semantic watermarking schemes that pose strong constraints (like SemStamp or the others)
Critical weaknesses
-
The paper says in its conclusion that SpanWM remains robust to paraphrasing, but presents no real-world practical paraphrase experiment. Its evaluated attacks include token insertions, deletions, and swaps. However, an attacker can simply ask another model to rewrite the text while preserving the meaning. This is a central missing experiment for the paper’s motivation. The reported 0.1% false-positive operating points are also unsupported by only 200 negative examples -- how can 0.1% FPR be possible with 200 examples? Maybe I missed something.
-
The linked anonymous repository is empty, despite the abstract and reproducibility statement saying code is available. This is a strong red flag for me.
Questions for authors
-
Can you report detection under actual paraphrasing, rather than only token-level edits, and provide the detailed paraphrasing model / prompts, and results?
-
Can you provide the detailed implementation and scripts?
3 comments
"We apologize for the initial delay in the code release. However, the implementation has been publicly available in our anonymous repository since October 13, more than three weeks before the review was last updated on November 6. Thus, the concern regarding the empty repository no longer reflects its current status. We encourage the reviewer to check the implementation and will further improve the documentation to facilitate reproducibility."
This framing is just misleading, imo. My review was submitted on October 8, well within the official review period. The November 6 timestamp simply reflects when conference administrators released reviews to authors, not when the review was written.
Authors are responsible for ensuring all materials are ready at submission time, not weeks after the paper submission deadline. Reviewers cannot be expected to conduct a fresh audit of a completely newly uploaded repository during the brief rebuttal window.
Thank you for the clarification. We understand that the original review was submitted on October 8. Our intention was simply to clarify that the code had been made available by the November 6 update, not to suggest that the reviewer was expected to revisit the repository. We appreciate your clarification and apologize for any confusion caused by our wording.
Beyond this clarification, we sincerely hope that the additional experimental evidence provided during the rebuttal, including the paraphrasing robustness results and expanded low-FPR evaluation, as well as the methodological contributions of SpanWM's shared span-level embedding and detection framework, will also be taken into consideration in your overall assessment of the paper.
We appreciate your time and consideration.
Response to Reviewer iZUc
We thank the reviewer for recognizing SpanWM's clean localized rewriting design and strong empirical results. We have conducted additional experiments to directly address the concerns about practical robustness and statistical reliability.
W1+Q1. Real-world Paraphrasing Robustness & Low-FPR Evaluation
We additionally evaluated LLM-based paraphrasing attacks using GPT-5-mini and GPT-OSS-20B, beyond the token-level perturbations in the original submission. SpanWM achieves the highest AUROC under both attacks, outperforming all four baselines:
Regarding the reviewer's question, we clarify an important distinction: empirical FPR resolution does not limit the ability to estimate intermediate ROC operating points. With 200 negatives, [email protected]% FPR can be estimated using standard linear interpolation of the ROC curve, as done in our original evaluation.
More importantly, we have now expanded the negative set to 10k samples, enabling direct empirical evaluation at 0.1% FPR. The results are shown below.
The expanded evaluation confirms our central finding without relying on interpolation: SpanWM achieves the highest [email protected]% FPR and AUROC across all three datasets. We additionally report 95% bootstrap confidence intervals for AUROC, further strengthening the statistical evidence.
W2+Q2. Code Availability & Reproducibility
We apologize for the initial delay in the code release. However, the implementation has been publicly available in our anonymous repository since October 13, more than three weeks before the review was last updated on November 6. Thus, the concern regarding the empty repository no longer reflects its current status. We encourage the reviewer to check the implementation and will further improve the documentation to facilitate reproducibility.
We sincerely thank the reviewer for the constructive feedback, which motivated us to strengthen the empirical evaluation of SpanWM. In particular, we have added LLM-based paraphrasing experiments, expanded the negative sample set to 10k for more reliable low-FPR evaluation, and clarified the availability of our implementation.
We hope that these additional results and clarifications satisfactorily address the reviewer's concerns and would greatly appreciate reconsideration of the initial assessment in light of the new evidence.
If you have any further questions or concerns, we would be happy to address them.
Thank you very much,
Authors