- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces SS-WM, a watermark scheme with two channels: in both semantic and stylistic features rather than conventional token-level schemes like KGW and the others. During generation, it periodically reranks candidate next tokens to move the text toward those favored by the secret key selected regions (style and semantics). For detection, it extracts the same features and compares their likelihood under the candidate key with an unwatermarked baseline. The intuition is that paraphrasing may change the words while preserving enough meaning and style to retain the signal. The experiments report promising results of detection rate after paraphrasing.
Critical strengths
- Combining semantic and style features for watermarking is well motivated.
- The paper evaluates wrong-key and partial-text detection, which matter for false-positive control and broader robustness.
Critical weaknesses
-
The "attack" folder under the released repository contains only a single readme file. I can't find the actual paraphrasing implementation, which is the crucial part to evaluate the strength of the paper. Many watermarking papers make overclaims about their results on paraphrasing robustness by doing less-intensive attacks than the advertised ones. This limits my confidence in the central robustness claim.
-
Another important weakness is the limitation on forgery attack: the threat model assumes that a secret key prevents an attacker from imitating its selected constraints, despite allowing access to watermarked outputs. With enough chosen prompts and marked examples, an attacker might infer the key-dependent style feature pattern without recovering the key itself. My suggestion will be to actually collect enough samples, run an agent on style mining, and see whether it can actually recover the secret styling key or no.
Questions for authors
-
Can you release the full implementation and concrete data log? Many folders are empty.
-
Given increasing numbers of outputs under one key, how well can an attacker infer its feature pattern and make unrelated text pass that key’s detector?
6 comments
Weakness 2: Forgery through key-dependent profile inference
Another important weakness is the limitation on forgery attack: the threat model assumes that a secret key prevents an attacker from imitating its selected constraints, despite allowing access to watermarked outputs. With enough chosen prompts and marked examples, an attacker might infer the key-dependent style feature pattern without recovering the key itself. My suggestion will be to actually collect enough samples, run an agent on style mining, and see whether it can actually recover the secret styling key or no.
We thank the reviewer for identifying this important security concern. We agree that key secrecy alone may not prevent an adversary from learning and imitating the stylistic patterns induced by a watermark. Our original evaluation examined wrong-key acceptance and rewriting robustness but did not directly investigate this form of chosen-prompt profile inference. To address this limitation, we have added a targeted forgery experiment in Appendix H.8 (p. 38, Table 27) and expanded the security discussion in Section 4.4 (p. 9).
First, we investigate whether an attacker can infer a key-dependent feature profile from observed watermarked outputs. The attacker collects 10, 100, or 1,000 marked outputs for each of ten target keys and uses the public feature extractor to estimate the induced stylistic and semantic shifts. The estimated profile is then compared with the target profile. The stylistic-profile cosine similarity increases from 0.775 with ten observations to 0.96 with 1,000, indicating that additional same-key observations enable increasingly accurate approximation of the target's observable feature pattern.
Second, we evaluate whether the inferred profile can facilitate forgery of unrelated text. Using 500 unrelated C4 passages disjoint from the profile-estimation material, the attacker generates eight candidate rewrites per round with GPT-5.6 Sol. Candidates are checked for admissibility, and the rewrite closest to the inferred stylistic or joint semantic–stylistic profile is selected. This process is repeated over three rounds. Importantly, selection relies on the estimated feature profile, without access to the secret key or target detector scores.
Additional security experiment (Appendix H.8, Table 27): Targeted forgery acceptance (%) under increasing observation budgets.
| Observed outputs per key | Stylistic-profile forgery | Joint-profile forgery |
|---|---|---|
| 10 | 3.5 | 4.5 |
| 100 | 14.0 | 20.0 |
| 1,000 | 24.0 | 33.5 |
The results demonstrate that forgery becomes increasingly effective as the observation budget grows. Stylistic-profile acceptance rises from 3.5% to 24.0%, while joint-profile acceptance increases from 4.5% to 33.5%. In comparison, unmodified human-written text has an acceptance rate of 1.1%, and the different-key-profile control achieves 1.9% at the 1,000-observation setting. These controls suggest that the improvement arises from learning target-specific watermark characteristics rather than generic rewriting alone.
Regarding recovery of the secret styling key, we distinguish exact key recovery from effective profile inference. Our experiment does not reconstruct the cryptographic key or identify the exact keyed codebook selections. Instead, it tests whether an adversary can approximate the observable feature pattern sufficiently to produce text accepted by the target detector. The results show that successful profile imitation and targeted forgery do not require exact key recovery. While the experiment implements feature-guided iterative rewriting, it does not evaluate an unrestricted autonomous style-mining agent; exact key recovery therefore remains an open question.
Finally, we have clarified the security implications. Wrong-key acceptance measures whether a watermark generated under one key is accepted under another, whereas targeted forgery evaluates deliberately modified unrelated text against the intended key. These are different adversarial conditions. Accordingly, the 1% null-FPR calibration does not guarantee resistance to an attacker who learns the target's observable feature pattern. We have clarified in Section 4.4 and Appendix E.6 that SS-WM provides statistical verification under a supplied candidate key, not cryptographic proof of provenance or unconditional forgery resistance.
We appreciate the reviewer for motivating this additional experiment. The findings substantiate the concern that repeated access to marked outputs can expose exploitable key-dependent regularities, while establishing the distinction between effective profile imitation and exact secret-key recovery.
Question 2:
Given increasing numbers of outputs under one key, how well can an attacker infer its feature pattern and make unrelated text pass that key’s detector?
We thank the reviewer for this insightful question. To investigate this threat directly, we have added a chosen-prompt profile-inference and targeted forgery experiment in Appendix H.8 (p. 38, Table 27). The attacker observes 10, 100, or 1,000 watermarked outputs generated under the same target key and uses the public feature extractor to estimate the corresponding stylistic or joint semantic–stylistic profile. The estimated profile is then used to guide rewriting of 500 unrelated C4 passages from a disjoint test set, with eight candidates generated per round and the admissible candidate closest to the inferred profile selected over three rounds. This allows us to evaluate whether information learned from observed watermarked outputs can facilitate forgery without direct access to the secret key.
The reported results show that the attack becomes substantially more effective as the number of observed outputs increases. Stylistic-profile forgery acceptance rises from 3.5% to 14.0% and 24.0% for 10, 100, and 1,000 observations, respectively. The joint semantic–stylistic attack achieves 4.5%, 20.0%, and 33.5% at the same observation budgets. Consistent with this trend, the reported cosine similarity between the inferred and target stylistic profiles increases from 0.775 with ten observations to 0.96 with 1,000 observations. In comparison, unmodified human-written text has an acceptance rate of 1.1%, while the different-key profile control achieves 1.9% in the reported 1,000-observation setting.
These findings directly support the reviewer's concern: an adversary can progressively approximate the observable feature pattern associated with a secret key and exploit that information to increase the probability of making unrelated text pass the corresponding detector. Importantly, the attack does not recover the secret key or reconstruct its exact codebook selections; rather, it demonstrates that effective profile imitation can enable targeted forgery without such recovery. We have therefore clarified in Section 4.4 and Appendix E.6 that SS-WM provides statistical verification under the specified threat model, rather than an unconditional guarantee against forgery by adversaries with extensive access to marked outputs. We appreciate the reviewer for motivating this additional security analysis.
We thank the reviewer for recognizing the motivation for a joint semantic–stylistic watermark and for emphasizing wrong-key and partial-text detection. These are important to the practical interpretation of verification results.
Weakness 1:
We thank the reviewer for pointing out this issue and apologize for the oversight. During preparation of the initial anonymous repository, we inadvertently omitted the executable attack scripts from the uploaded archive, leaving only the README in the attack directory. This was an upload mistake rather than an intended restriction on implementation access. We acknowledge that this omission made it difficult for the reviewer to examine the attack procedures and assess their strength.
We have now corrected this omission and updated the anonymous repository with the attack implementations and supporting evaluation scripts. In particular, the repository includes code for GPT-3.5 paraphrasing, modern-model rewriting, style rewriting, expansion, feature-aware candidate selection, and adaptive attacks. It also provides utilities for admissibility checking, recording candidate-generation attempts, applying fallback and selection rules, and exporting detector scores. Reviewers can therefore inspect the attack prompts and configurations, follow how rewriting candidates are generated and selected, and examine how the selected outputs are passed to the watermark detector.
To further improve transparency, we have clarified the attack configurations in Appendix F.4 (pp. 25–27). The revised description explicitly distinguishes the default GPT-3.5 paraphrasing attack from the intensity-controlled experiments. The default setting uses a general paraphrasing instruction without a specified replacement percentage, whereas the intensity-controlled settings request 20%, 40%, 60%, and 80% content-word replacement. The corresponding measured token-level edit rates are approximately 18%, 37%, 55%, and 72%, respectively. We also document the semantic admissibility criteria, retry budgets, fallback procedures, and candidate-selection rules to clarify how attack strength is evaluated beyond the requested rewriting intensity.
We have additionally strengthened the attack evaluation with contemporary rewriting and more demanding attack settings. Appendix H.3 (Table 20) reports single-pass and three-stage GPT-5.6 Sol rewriting, including measured edit rates, independent semantic similarity, and admissibility. Appendix H.4 (Table 22) includes expansion, transferred-proxy B4 scrubbing, and public-feature-aware attacks. These experiments provide a broader assessment of robustness under different rewriting mechanisms and attacker capabilities.
Question 1:
We thank the reviewer for pointing out the incomplete contents of the original anonymous archive. As discussed in our response to pL6h.W1, the attack scripts were inadvertently omitted during the initial upload, leaving README files in the relevant directories. We apologize for this mistake. The updated repository includes the SS-WM implementation, supported paraphrasing and rewriting attacks, admissibility filtering, adaptive and feature-aware candidate selection, query accounting, and detector-score export. Appendix F.4 (pp. 25–27) complements the code with the attack prompts, parameter settings, fallback rules, and evaluation definitions. The updated artifact is designed to let reviewers inspect and execute the supported workflows and generate concrete source-level traces. Its logging and export utilities record the source identifier, attack configuration, generated candidates, admissibility decisions, selected rewrite, query counts, and detector scores as applicable to each run. The release is an executable implementation with logging support rather than a static archive of historical API responses. With the documented dependencies and access to the relevant models or APIs, reviewers can follow how attack outputs are produced and how detection statistics are calculated. We appreciate the reviewer for highlighting this omission, which has led to a substantially more inspectable release.