- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Redwing, which is a training-free and generation-time watermarking method for text-to-speech models that targets retokenization. It builds a graph from token substitutions that observed under codec round trips and uses the smoothest Laplacian eigenvectors as a basis. On this basis, it solves a bounded CCA-like problem for separate embedding and detection functions. At a speech quality cost comparable to KGW, the watermark stays highly detectable after repeated neural codec resynthesis, where KGW and WMAR largely underperform. It also experiments on two TTS models and remains robust across codecs.
Critical strengths
- The core observation is simple and well exploited. The method stays highly detectable under repeated resynthesis, where other baselines failed to do it. The ablations show that the Laplacian contributes clearly beyond a random or transition matrix under the same solver.
- The evaluation is good enough. Thresholds are calibrated empirically on the held out unwatermarked answers for every method and reused under attack. FPR are reported after attacks, which exposes threshold drift in some other post watermarking baselines. The derivations in the appendix are correct as far as I checked and the authors are explicit about which steps are approximations.
Critical weaknesses
- The threat model covers only signal processing and resynthesis. The design may be weak against an informed adversary, and this is not discussed yet.
- The authors claim that HiPT is the closest work but omitted because its nominal threshold is miscalibrated. However, every other method’s threshold is set empirically on the calibration set, so HiPT could have been calibrated in the same way.
Questions for authors
- Can you report HiPT with an empirically calibrated threshold?
- Can an adversary with the released model reproduce g and h? If so, how many watermarked clips are needed to recover b by majority vote on sign g(Y_t), and what TPR remains after a removal attack that biases sampling against the recovered signs?
3 comments
Reply to the Authors
Thanks for the rebuttal effort. I will maintain my score.
Thank you for your review and for considering our response. If any concern remains, we would be happy to address it.
Response to Reviewer vRKE
Dear Reviewer vRKE,
Thank you for the insightful review and for the two precise questions. We ran both experiments.
(1) HiPT with an empirical threshold. Yes. HiPT is now in the main tables (Tables 1–3). Because its nominal p < 0.01 already flags all of Moshi's unwatermarked answers and 98.5 % of CosyVoice3's, we detect it, like every other method, at a threshold calibrated on unwatermarked answers. It runs at the largest δ whose cost stays within KGW's (3.4 on Moshi, 3.9 on CosyVoice3). After eight passes of the model's own codec, it detects 12.8 % on Moshi and 45.8 % on CosyVoice3, against 8.3 and 8.7 % for KGW and 80.7 and 83.5 % for Redwing (Tables 1–2). At its documented δ = 2, it detects less (9.8 and 26.8 %; Appendix B.5). Details are in our reply to Reviewer 5RTX.
(2) Informed adversary. Thank you for raising this; we added the experiment (Appendix C.10). The adversary we test knows the method and has the public language model and its codec, our code and the strength δ, but not the key or our functions g and h. In short, it rebuilds g from its own data and recovers most of the key's signs on the TTS models. Removal with these signs, however, needs control over the sampling and succeeds only in part: the TPR falls to 56.8 % on CosyVoice3 and stays at 96.8 % on MOSS-TTS and 95.0 % on Moshi. An adversary with our exact g and h removes almost all of the watermark.
Rebuilding the functions. g and h depend on no key: they are fitted from the codec and from speech with our code. The adversary therefore fits its own embedding function g′. For this, the adversary measures the codec's substitutions on public speech that we never used (LJSpeech, a single speaker) and the remaining statistics on 150 unwatermarked clips of its own. Up to sign, g′ correlates with g at 0.75 on CosyVoice3, 0.44–0.69 on Moshi's streams and 0.11–0.98 on MOSS-TTS's.
Recovering the key. The key fixes one sign per stream and frame, and this sign does not depend on the content, so every watermarked clip carries the same sign at a given frame. The adversary estimates it as the majority of sign g′(Y_t) over N watermarked clips. A fitted function is defined only up to its sign: g′ may approximate −g instead of g, and then all signs of that stream come out flipped. We count the signs up to this common flip, because the removal below multiplies them by g′, so the flip cancels. The table gives the share of the key's signs that the adversary recovers (Table 21):
On the TTS models, the share grows with N. On Moshi it stays near 60 %.
Removal. We then designed a removal attack. With the signs b̂ recovered from 100 clips, it biases the sampling of new clips against them, subtracting the rebuilt function at the watermark's strength from the watermarked logits at every stream s and frame t: ℓ″ₛ,ₜ = ℓ′ₛ,ₜ − δ · b̂ₛ,ₜ · g′ₛ. This requires control over the model's sampling, which a service that returns only audio would not give in practice. The attack lowers our TPR from 97.7 to 56.8 % on CosyVoice3 and from 100 to 96.8 % on MOSS-TTS. It leaves Moshi unchanged (95.0 % against 94.8 % without attack; Table 22). An adversary that also knows our g and h, but not the key, recovers 96.1–98.2 % of the signs from the same clips and removes almost the whole watermark (TPR 1.0–3.8 %). An adversary that must rebuild g and h itself therefore removes much less of the watermark than one that has ours. Stronger adversaries are possible: these results use one seeded set of 100 clips, the watermark's strength without tuning, and a channel measured on one voice (Appendix C.10).
(3) Threat model. We agree and now state the threat model (Limitations; Appendix C.10): the attacks in our main results use no knowledge of the watermark, and an adversary who knows the method can do more. Our functions depend on no key, and the rules that a fixed key sets can be learned from watermarked outputs, as Jovanović et al. (2024) show for text watermarks. With control over the sampling, such an adversary can then weaken the watermark, as in (2). Existing defenses against such adversaries each come at a cost: a longer hashing context costs robustness (Kirchenbauer et al., 2024; Pang et al., 2024). For codec tokens, resynthesis would also change the context tokens themselves. Several keys can raise the number of outputs an adversary needs, but they make removal easier (Pang et al., 2024). The detector itself is a further risk: a detector that anyone can query can itself be exploited to remove the watermark (Pang et al., 2024). Protecting the key against adversaries who learn it from outputs is future work.