acceptodds
Under review as a conference paper at ICLR 2027

Tokens Change, Structure Endures: Spectral Watermarking for Generated Speech

Abstract

Watermarking is a promising tool for establishing the provenance of AI-generated speech. While many neural audio watermarking methods rely on a separately trained watermark generator, token-level watermarking is a training-free alternative that operates directly during generation. Its main weakness is retokenization: decoding generated speech to a waveform and encoding it again can change token identities and erode the watermark. To make the watermark robust to these changes, we propose Redwing, REtokenization-Durable Watermarking IN Generation. It builds a graph from the token substitutions observed under retokenization, whose Laplacian yields a basis that assigns similar values to tokens likely to substitute for one another. Over this basis, embedding and detection functions are jointly optimized to preserve watermark signal through retokenization while limiting embedding distortion and detector variability on unwatermarked speech. On the Moshi full-duplex system, after eight consecutive passes of Mimi resynthesis, Redwing achieves 80.7% TPR at a calibrated 1% FPR, compared with 8.3% for KGW and at most 7.3% for WMAR. It also has the highest TPR after eight passes through three other neural codecs (77.5–93.0%), and the gains generalize to TTS models at a speech-quality cost close to that of KGW. These results show that retokenization is not merely a source of noise: its transition structure can be exploited as a design principle for robust token-level watermarking.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Watermarking, Speech synthesis, Neural audio codecs, Generative AI, Retokenization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_vRKEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Redwing, which is a training-free and generation-time watermarking method for text-to-speech models that targets retokenization. It builds a graph from token substitutions that observed under codec round trips and uses the smoothest Laplacian eigenvectors as a basis. On this basis, it solves a bounded CCA-like problem for separate embedding and detection functions. At a speech quality cost comparable to KGW, the watermark stays highly detectable after repeated neural codec resynthesis, where KGW and WMAR largely underperform. It also experiments on two TTS models and remains robust across codecs.

Critical strengths
  1. The core observation is simple and well exploited. The method stays highly detectable under repeated resynthesis, where other baselines failed to do it. The ablations show that the Laplacian contributes clearly beyond a random or transition matrix under the same solver.
  2. The evaluation is good enough. Thresholds are calibrated empirically on the held out unwatermarked answers for every method and reused under attack. FPR are reported after attacks, which exposes threshold drift in some other post watermarking baselines. The derivations in the appendix are correct as far as I checked and the authors are explicit about which steps are approximations.
Critical weaknesses
  1. The threat model covers only signal processing and resynthesis. The design may be weak against an informed adversary, and this is not discussed yet.
  2. The authors claim that HiPT is the closest work but omitted because its nominal threshold is miscalibrated. However, every other method’s threshold is set empirically on the calibration set, so HiPT could have been calibrated in the same way.
Questions for authors
  1. Can you report HiPT with an empirically calibrated threshold?
  2. Can an adversary with the released model reproduce g and h? If so, how many watermarked clips are needed to recover b by majority vote on sign g(Y_t), and what TPR remains after a removal attack that biases sampling against the recovered signs?
3 comments
AuthorsOpenReview ↗

Response to Reviewer vRKE

Dear Reviewer vRKE,

Thank you for the insightful review and for the two precise questions. We ran both experiments.

(1) HiPT with an empirical threshold. Yes. HiPT is now in the main tables (Tables 1–3). Because its nominal p < 0.01 already flags all of Moshi's unwatermarked answers and 98.5 % of CosyVoice3's, we detect it, like every other method, at a threshold calibrated on unwatermarked answers. It runs at the largest δ whose cost stays within KGW's (3.4 on Moshi, 3.9 on CosyVoice3). After eight passes of the model's own codec, it detects 12.8 % on Moshi and 45.8 % on CosyVoice3, against 8.3 and 8.7 % for KGW and 80.7 and 83.5 % for Redwing (Tables 1–2). At its documented δ = 2, it detects less (9.8 and 26.8 %; Appendix B.5). Details are in our reply to Reviewer 5RTX.

(2) Informed adversary. Thank you for raising this; we added the experiment (Appendix C.10). The adversary we test knows the method and has the public language model and its codec, our code and the strength δ, but not the key or our functions g and h. In short, it rebuilds g from its own data and recovers most of the key's signs on the TTS models. Removal with these signs, however, needs control over the sampling and succeeds only in part: the TPR falls to 56.8 % on CosyVoice3 and stays at 96.8 % on MOSS-TTS and 95.0 % on Moshi. An adversary with our exact g and h removes almost all of the watermark.

Rebuilding the functions. g and h depend on no key: they are fitted from the codec and from speech with our code. The adversary therefore fits its own embedding function g′. For this, the adversary measures the codec's substitutions on public speech that we never used (LJSpeech, a single speaker) and the remaining statistics on 150 unwatermarked clips of its own. Up to sign, g′ correlates with g at 0.75 on CosyVoice3, 0.44–0.69 on Moshi's streams and 0.11–0.98 on MOSS-TTS's.

Recovering the key. The key fixes one sign per stream and frame, and this sign does not depend on the content, so every watermarked clip carries the same sign at a given frame. The adversary estimates it as the majority of sign g′(Y_t) over N watermarked clips. A fitted function is defined only up to its sign: g′ may approximate −g instead of g, and then all signs of that stream come out flipped. We count the signs up to this common flip, because the removal below multiplies them by g′, so the flip cancels. The table gives the share of the key's signs that the adversary recovers (Table 21):

watermarked clips N1025100250
CosyVoice378.5 %87.0 %94.5 %96.8 %
MOSS-TTS75.4 %82.5 %89.5 %92.0 %
Moshi58.7 %60.0 %61.0 %62.3 %

On the TTS models, the share grows with N. On Moshi it stays near 60 %.

Removal. We then designed a removal attack. With the signs b̂ recovered from 100 clips, it biases the sampling of new clips against them, subtracting the rebuilt function at the watermark's strength from the watermarked logits at every stream s and frame t: ℓ″ₛ,ₜ = ℓ′ₛ,ₜ − δ · b̂ₛ,ₜ · g′ₛ. This requires control over the model's sampling, which a service that returns only audio would not give in practice. The attack lowers our TPR from 97.7 to 56.8 % on CosyVoice3 and from 100 to 96.8 % on MOSS-TTS. It leaves Moshi unchanged (95.0 % against 94.8 % without attack; Table 22). An adversary that also knows our g and h, but not the key, recovers 96.1–98.2 % of the signs from the same clips and removes almost the whole watermark (TPR 1.0–3.8 %). An adversary that must rebuild g and h itself therefore removes much less of the watermark than one that has ours. Stronger adversaries are possible: these results use one seeded set of 100 clips, the watermark's strength without tuning, and a channel measured on one voice (Appendix C.10).

(3) Threat model. We agree and now state the threat model (Limitations; Appendix C.10): the attacks in our main results use no knowledge of the watermark, and an adversary who knows the method can do more. Our functions depend on no key, and the rules that a fixed key sets can be learned from watermarked outputs, as Jovanović et al. (2024) show for text watermarks. With control over the sampling, such an adversary can then weaken the watermark, as in (2). Existing defenses against such adversaries each come at a cost: a longer hashing context costs robustness (Kirchenbauer et al., 2024; Pang et al., 2024). For codec tokens, resynthesis would also change the context tokens themselves. Several keys can raise the number of outputs an adversary needs, but they make removal easier (Pang et al., 2024). The detector itself is a further risk: a detector that anyone can query can itself be exploited to remove the watermark (Pang et al., 2024). Protecting the key against adversaries who learn it from outputs is future work.

Reviewer_vRKEOpenReview ↗

Reply to the Authors

Thanks for the rebuttal effort. I will maintain my score.

Thank you for your review and for considering our response. If any concern remains, we would be happy to address it.

Reviewer_5RTXOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Redwing, an in-generation token-level watermarking method for speech synthesis that addresses watermark loss caused by token identity shifts after audio re-encoding. The core idea is to estimate token substitution patterns to construct a transition graph, and then leverage the low-frequency eigenvectors of the graph Laplacian to assign similar scalar values to mutually confusable tokens. Within this spectral representation space, the authors jointly optimize the embedding function used during generation and the scoring function used during detection, maximizing watermark signal retention after re-encoding while bounding sampling distribution shift and detection variance.

Critical strengths
  • The paper models retokenization as an estimable channel and exploits its structure. Instead of hard token clustering, it uses continuous, smooth scores and decouples the embedding function from the detection function, forming a coherent and well-motivated framework.

  • This paper models the surviving signal, KL distortion, and null variance, justifying why per-token biases must be bounded to prevent rare tokens from receiving extreme weights.

Critical weaknesses
  • Table 12 (Appendix p. 33) shows that a 40 ms time shift drops TPR from ~95% to 9% on Moshi and from ~100% to 2% on MOSS-TTS; cropping half the audio reduces TPR across all three models to ~2%–3%. Thus, robustness to repeated codec reconstruction does not translate into resistance against trivial desynchronization attacks, and a viable resynchronization mechanism is still missing.

  • No valid comparison with the closest baseline, HiPT. Despite identifying HiPT as the most related work, the paper excludes it because its default nominal 1% threshold yields high FPR. However, a miscalibrated nominal threshold does not imply poor discriminability after empirical recalibration. Since other baselines are empirically calibrated, HiPT should be evaluated under the same calibration protocol.

  • Quality evaluation omits content and speaker preservation. The paper only reports acoustic quality scores without WER, speaker similarity, or QA accuracy. Since watermarking perturbs token sampling during generation, preserved acoustic scores alone cannot confirm spoken content or speaker identity.

  • Appendix B.4 notes that 24 kHz waveforms are fed directly into the 16 kHz SpeechTokenizer model without resampling. While applied consistently across methods, this does not represent a standard SpeechTokenizer reconstruction at its intended sample rate; results with proper resampling should be included.

Questions for authors
  • Can you include practical resynchronization experiments? Under unknown time shifts and cropping locations, how much TPR can be recovered by adding sub-frame offset search or other alignment methods, and what are the resulting computational overhead and recalibrated FPR?

  • Why exclude HiPT entirely instead of recalibrating it?

  • Why does the specially optimized scoring function hh degrade noticeably under DAC16? Could you analyze the differences between the native and external codec transition matrices to explain which structural properties govern cross-codec generalization? Does incorporating transition statistics from multiple codecs improve worst-case performance without sacrificing native-codec TPR?

  • How well does the theoretical objective predict actual multi-round reconstruction performance? Since the optimization objective relies on a single-pass channel model and small-perturbation approximations, how strongly do the theoretical objective values across candidate functions correlate with empirical TPR after 1, 4, and 8 reconstruction rounds?

  • Could you add content-fidelity metrics and fix the SpeechTokenizer attack protocol? Please report WER, speaker similarity, and necessary subjective listening tests both before and after watermarking (as well as across multi-round attacks), alongside the corrected SpeechTokenizer results with proper 24→16→24 kHz24 \to 16 \to 24\text{ kHz} resampling.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 5RTX (1/2)

Dear Reviewer 5RTX,

Thank you for the detailed review and the precise questions. They led to several new experiments and to two corrections of our attack protocol. We answer in your order.

(1) Resynchronization. We added a resynchronizing detector (Appendix C.7), called the "search" below. It trims 0, 1, …, 79 ms from the start of the received audio (0–39 ms on CosyVoice3, whose frames are 40 ms), re-encodes each version and searches over the frame at which the key's signs start. A clip is flagged if this search statistic or the default statistic exceeds its threshold. Both thresholds are set together on the same 1,200 calibration answers, so that 1 % of them are flagged, and are used for every attack.

Redwing TPR (%)MoshiCosyVoice3MOSS-TTS
random 0–2 s shift: default → search2.8 → 73.75.7 → 87.07.0 → 99.5
random crop (half the clip, random position): default → search2.0 → 48.34.0 → 69.24.5 → 95.5
no attack: default → search94.8 → 93.797.7 → 96.8100 → 100

With the search, the FPR stays at or below 1.3 % under these attacks. It costs 80 tokenizations per clip (40 on CosyVoice3), about 4–18 s per clip on a shared V100, against 0.07–0.22 s for the default detector. The same search also helps KGW under shifts and crops (on CosyVoice3, it matches ours), but after eight native passes KGW stays at 5.5–8.8 % (Table 18). The search does not undo speed changes, except a 1 % speed-up on MOSS-TTS; the Limitations state this.

(2) HiPT with an empirical threshold. We agree, and HiPT is now in the main tables (Tables 1–3). It runs under our protocol, with the same test prompts, sampler and seeds, its released detector after re-encoding, and our threshold rule. Its nominal p < 0.01 flags all of Moshi's unwatermarked answers and 98.5 % of CosyVoice3's, so we use a calibrated threshold instead. Like Redwing in our protocol, HiPT runs at the largest δ whose validation cost stays within KGW's: 3.4 on Moshi and 3.9 on CosyVoice3. The authors release no cluster map for MOSS-TTS, so HiPT is not run there. The table collects Tables 1–3 and 18 and Appendix B.5:

TPR (%)KL/framecleanown codec ×8
Moshi: HiPT, documented δ = 20.7734.09.8
Moshi: HiPT, δ = 3.41.7345.312.8
Moshi: KGW1.7378.58.3
Moshi: Redwing1.5294.880.7
CosyVoice3: HiPT, documented δ = 20.3140.726.8
CosyVoice3: HiPT, δ = 3.90.6160.045.8
CosyVoice3: KGW0.6293.38.7
CosyVoice3: Redwing0.5597.783.5

After eight passes, HiPT's calibrated threshold flags up to 4.5 % of Moshi's unwatermarked answers (after SpeechTokenizer) and 6.5 % of CosyVoice3's (after its own codec) (Appendix C.8, Table 20). The stronger δ also costs content: its WER without attack rises from 21.8 to 32.7 % on Moshi and from 7.3 to 18.6 % on CosyVoice3 (Appendix B.5).

(3) DAC16, transition matrices and multi-codec statistics. Your question uncovered the cause (Appendix C.4). On every pass, the released DAC 16 and 24 kHz models return their audio 8 samples early. The offset is the same on noise, sine sweeps and speech, and through both interfaces of the official package. An open issue on the DAC repository (#118) also reports a temporal misalignment of the 16 kHz model. The attack in our original submission applied the passes one after another without re-aligning the audio, so after eight passes it was 4.0 ms (DAC16) or 2.7 ms (DAC24) early. Compensating this delay after every pass is enough to restore detection: after eight DAC16 passes, Redwing keeps 93.2 / 97.0 / 100 % (as released: 34.2 / 95.0 / 73.2 %). All methods and tables now use the compensated passes, and AudioSeal and CRAW also become strong after DAC16.

Transition matrices. We measured the channel of one pass of each codec in Moshi's token space (Appendix C.5, Table 16). One pass of DAC24 or AAC adds almost nothing to the round trip that our functions are fitted to: 40 % of the tokens survive, as after the round trip alone, against 28 % after one more Mimi pass. Across the ten codecs, the smoother h stays along a codec's one-pass channel, the more of the watermark survives eight passes (Spearman −0.79). SNAC is an exception: its pass changes the tokens about as much as Mimi's, but our TPR after eight passes is 93.0 %, against 80.7 % after Mimi.

Multi-codec statistics (Appendix C.5). Re-solving h on the statistics averaged over the native and the foreign one-pass channels, with SpeechTokenizer held out, gives almost the same function: over the 2,048 tokens of each stream, its values correlate with those of our h at 0.999 or more. The TPR changes by at most 0.5 points on any codec, native and held-out included, so it neither improves the worst case nor costs native TPR. Whether a basis or an embedding function built from several codecs helps needs new generations, which we leave for future work.

AuthorsOpenReview ↗

Response to Reviewer 5RTX (2/2)

(4) Objective against multi-pass TPR. We tested this directly (Appendix C.3, Table 14). On Moshi, we compared eight pairs (g, h) in the same basis: our solved functions, our embedding function also used for detection (h := g), solves restricted to the first 1, 2 or 4 basis functions, uniform coefficients, and two random ones. Each pair runs at the strength whose validation cost matches ours and is detected at its own 1 % threshold, on the 250 validation answers. Although the objective models a single pass, the rank correlation between the objective and the TPR is 0.88, 0.86, 0.93 and 0.88 without attack and after 1, 4 and 8 Mimi passes. With only eight pairs, these correlations describe the agreement rather than establish it. The order is also not exact: one random pair has a higher objective than h := g but a lower TPR. Still, our functions have both the highest objective and the highest TPR in every condition (80.8 % after eight passes, against 75.6 % for the next pair).

(5) Content, speaker and listening tests. We added WER and, on the TTS models, speaker similarity (Table 4, Appendix C.13), without attack and after up to eight resynthesis passes. They cover KGW and Redwing on every model and the post-hoc methods on the TTS models. On the TTS models, WER is measured against the input text (Whisper large-v3), and speaker similarity to the voice prompt with the Seed-TTS-eval WavLM verifier. Without attack (Table 4):

CosyVoice3 WER (%)MOSS-TTS WER (%)CosyVoice3 SIMMOSS-TTS SIM
Unwatermarked6.35.00.7150.694
KGW7.85.60.7080.682
Redwing, selected δ (0.8)8.17.90.7040.682
Redwing, lower δ (0.6 / 0.5)6.85.50.7100.687

Speaker similarity is barely affected, but Redwing has a content cost, and the cost grows with δ. A slightly lower δ closes this gap to KGW: at δ = 0.6 on CosyVoice3 and 0.5 on MOSS-TTS, Redwing's WER is comparable to KGW's or lower. After eight native passes, it still detects 66.7 % on CosyVoice3 and 92.8 % on MOSS-TTS, more than every baseline (at most 63.3 %). Figure 12 (Appendix C.13) shows this trade-off between WER and TPR over δ.

Moshi has no reference text, so we score its transcript against the text stream that it generates alongside the speech. Without attack, this WER is 17.7 % for unwatermarked answers, 19.3 % for KGW, and 25.4 and 23.0 % for Redwing at δ = 0.7 and 0.6 (Table 4). As on the TTS models, the lower δ narrows the gap to KGW, and Redwing still detects 71.0 % after eight Mimi passes, against 8.3 % for KGW. For Moshi, the reference is imperfect, because many transcripts begin with words that the text stream lacks, and more often for our watermark. Ignoring these words lowers all WERs but leaves the ranking unchanged (Appendix C.13, "WER on Moshi").

We did not measure QA accuracy: WER shows whether the speech preserves its content, not whether the answer is correct. In the MOS ratings collected so far, listeners rate Redwing as they rate KGW (Table 4; see also our reply to Reviewer TSts). We will post the final results before the end of the discussion.

(6) SpeechTokenizer protocol. Thank you for catching this. The attack now resamples 24 → 16 → 24 kHz for every method and model, and the columns are replaced. After eight passes, Redwing detects 86.0 / 87.2 / 51.5 % (Moshi / CosyVoice3 / MOSS-TTS; Tables 1–2), with an FPR of at most 1.5 % (Table 20). The best baseline on each model detects 29.8 % (HiPT), 38.8 % (HiPT) and 5.0 % (CRAW). Redwing remains the most robust method after SpeechTokenizer on every model.

Thanks for addressing my concerns, and I have raised my score.

Thank you for the constructive review and for raising your score. The final MOS results are now in the revised manuscript (Table 4; Appendix B.6), and we summarize them in our general comment and the reply to Reviewer TSts. If anything else comes up, we would be happy to address it.

Reviewer_TStsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

On a high level, this paper improves on previous token-bias-based watermarking methods with Redwing, a token watermark embedding and detection method that takes the retokenization problem into account. The general idea is to identify a token substitution graph through repeated re-encoding; the graph is incorporated into the optimization of the embedding process, and the detection score is also modified accordingly. The authors conducted experiments on speech-LLM and TTS models and showed that the proposed method has significantly greater robustness against repeated resynthesis at the cost of slight or no degradation in the model's perceptual quality.

Suggested References:

The retokenization issue also reminds me of the lack of idempotency in modern RVQ-based tokenizers [1], which could be a good reference for this paper. There is also a paper [2] by the same authors that tries to tackle robust watermarking against resynthesis.

[1] P. O'Reilly et al., "Code Drift: Towards Idempotent Neural Audio Codecs," ICASSP 2025.

[2] P. O'Reilly et al., "MaskMark: Robust Neural Watermarking for Real and Synthetic Speech."

Critical strengths

I applaud the study of the retokenization issue and resynthesis attacks—it's very timely given the widespread issues with audio deepfakes.

The paper is well written and easy to comprehend.

The central hypothesis is well supported by the experiments, which also show clear advantages of the proposed method.

Critical weaknesses

The issues are very minor. If the authors can provide reasonable explanations and supplemental information, I would consider raising my rating, which is 3.5, in between weak and clear accept.

(1) The baseline study includes signal-based methods such as AudioSeal and Timbre, which have limited robustness against neural codecs. However, to my knowledge, there is another method named MaskMark that can survive neural codecs, which may be a good addition to the comparison [2]. It's also worth noting that MaskMark can work with 2-second audio, with robustness increasing exponentially as the number of frames increases. Since the paper claims to be the strongest method against resynthesis (page 9, Limitations), it should be compared with watermarking schemes that have similar advantages.

(2) The paper uses automatically generated MOS scores to support the claim that the proposed method introduces minimal to no quality degradation. However, as we know in this field, all the metrics used in Table 3 are sensitive to SNR rather than the naturalness of speech. For scientific rigor, I would expect a human listening MOS test and/or PESQ scores (if possible) to back up the claim.

(3) As stated in the limitations, shorter lengths reduce robustness. It would be nice to have a graph added to Figure 2 showing TPR as a function of length. Expanding this information would be valuable because, if the proposed method is used in practical applications, the minimum length required for robust detection could be referenced from this paper.

Questions for authors

Please review the minor weaknesses I pointed out and see if you can respond to them.

I assume this method is not robust to shifts, low-pass filtering, and commonly used signal-based attacks, right? I assume so, but in any case, it's worth clarifying in the paper.

Another point that may not be directly obvious to researchers in signal-based watermarking is that the proposed method uses optimization during the embedding process, rather than a universal embedding function like AudioSeal. It's worth pointing this out for the broader audience.

5 comments
AuthorsOpenReview ↗

Response to Reviewer TSts (1/2)

Dear Reviewer TSts,

Thank you for the constructive review. We answer each point below.

(1) MaskMark and codec-robust baselines. MaskMark has no public code or weights, so we could not run it. Its authors' later study ("Deep Audio Watermarks are Shallow", 2025) finds that it detects 3–64 % after a single pass of a neural codec (e.g. 15 % after EnCodec and 4 % after SpeechTokenizer), even with a threshold set per attack. We cite both papers. As the closest codec-robust post-hoc method with public code, we added Latent-Mark (Interspeech 2026, its Latent-Cluster variant at default settings) to the main tables. On Moshi, the TPR (%) after eight passes is as follows (Tables 1 and 13; without attack, Tables 11 and 26):

cleanMimiEnCodecSpeechTok.SNACDAC16Opus
Redwing94.880.785.086.093.093.290.7
Latent-Mark94.80.014.223.781.384.51.2

Latent-Mark survives SNAC, the codec it is built on, and DAC, but not Mimi, EnCodec, SpeechTokenizer or Opus. On the TTS models, it keeps at most 8.8 % after eight passes of any codec in Table 2. Every method uses one threshold per detector, set at 1 % FPR on unwatermarked model answers (the calibrated threshold). Latent-Mark's authors instead flag a clip that scores 1.5 standard deviations above unwatermarked speech. On the TTS models, this rule flags 5.3 and 4.2 % of unwatermarked answers (CosyVoice3 and MOSS-TTS), and Latent-Mark embeds only just above it. After one CosyVoice3 pass, it therefore detects 96.3 % with its authors' rule but 20.7 % at the calibrated threshold (Appendix B.5).

(2) Human listening test and PESQ. We agree that predicted quality is not enough. A MOS test (ITU-T P.800 ACR, naturalness on a 1–5 scale) is still running on all three models. Listeners rate unwatermarked speech, KGW (δ\delta = 2), and Redwing at its selected and at a lower δ\delta, without attack and after one and eight resyntheses (Appendix B.6). Table 4 shows the ratings collected so far. So far, listeners rate Redwing as they rate KGW, and we will post the final results before the end of the discussion. PESQ needs a time-aligned reference of the same signal. The watermark changes the sampled tokens, so the unwatermarked answer to the same prompt differs in timing (and, on Moshi, in words) and cannot serve as one. We therefore report WER and speaker similarity instead (Table 4, Appendix C.13; see also our reply to reviewer 5RTX).

(3) TPR as a function of length. Added (Appendix C.9, Figure. 11). We cut every test answer that is at least L seconds long to its first L seconds and score it at the fixed threshold. After eight passes of each model's own codec, Redwing detects 66.2 % at 4 s and 95.1 % at 8 s on Moshi, 87.5 % at 5 s on CosyVoice3, and 94.8 % at 2 s on MOSS-TTS. KGW detects at most 21.6 % at any length. Our detector flags at most 2.3 % of the unwatermarked answers at any length, so the fixed threshold also holds on short audio. Without attack, 2 s already suffice for 77.8 / 81.9 / 100 % (Moshi / CosyVoice3 / MOSS-TTS).

AuthorsOpenReview ↗

Response to Reviewer TSts (2/2)

(4) Shifts, low-pass filtering and signal attacks. We answer separately for attacks that keep the timing of the audio and for attacks that move it in time (Appendix C.6, Table 17).

Signal processing that keeps the timing. Redwing is robust to most of it. After a 3.5 kHz low-pass, it detects 95 / 97 / 100 % (Moshi / CosyVoice3 / MOSS-TTS), and most attacks of this kind lower its TPR by only a few points. It loses the most under strong noise and coarse quantization on Moshi. Under pitch shifts, it keeps most of its watermark, while almost every post-hoc watermark is removed.

Sub-frame shifts and crops. A shift by a whole frame costs nothing, because our detector searches over offsets of up to two frames. A shift by a fraction of a frame lowers our TPR to 25 / 74 / 49 % (20 ms), and a crop removes almost all of the watermark. A resynchronizing detector, which re-encodes the audio at sub-frame offsets and searches a wider range of frame positions (Appendix C.7, Table 18), recovers most of it after shifts and much of it after crops. Under a random 0–2 s shift, the TPR rises from 2.8 / 5.7 / 7.0 % to 73.7 / 87.0 / 99.5 %. Under a random crop that keeps half of the clip at a random position, it rises from 2.0 / 4.0 / 4.5 % to 48.3 / 69.2 / 95.5 %. The FPR stays at or below 1.3 %, and the search costs 80 tokenizations per clip (40 on CosyVoice3).

Speed changes. A speed-up of only 1 % lowers our TPR to 17 / 66 / 62 %, and the resynchronizing detector undoes this only on MOSS-TTS (61.5 → 93.8 %). We state this in the Limitations, and a detector that is robust to speed changes is future work.

(5) Optimization during embedding. There is no optimization per clip. The functions g and h are solved once per model, offline. During generation, the watermark is a bias added to the logits, given by g and the key's sign at each frame. Embedding therefore needs access to the logits of the model, unlike a post-hoc embedder such as AudioSeal. The revision states this in Section. 3.4.

(6) References. Thank you for both references. We now cite Code Drift where we discuss codecs that are not idempotent (Section. 2). We also cite MaskMark among the post-hoc watermarks, together with its authors' later study, "Deep Audio Watermarks are Shallow" (O'Reilly et al., 2025), summarized in (1) (Section. 2). The Limitations now state where our advantage lies: neural resynthesis. After codecs that keep the waveform nearly intact, such as MP3 and Opus, the best post-hoc methods detect nearly every watermarked clip, against 91–97 % for Redwing on Moshi and CosyVoice3.

Reviewer_TStsOpenReview ↗

Thank you for the rebuttal

Thank you for the detailed response and additional experiments. I think concerns (1) and (3) have been sufficiently addressed, and I appreciate the additional baseline comparison.

Regarding (2), I agree with the explanation for not using PESQ. However, I would like to see the finalized human MOS results, particularly the comparison between Redwing and unwatermarked speech, before considering this concern resolved. You mentioned you are running MOS. Any chance we can see the numbers in your further reply?

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response: revised PDF and changes

Revised PDF. We thank all reviewers for their careful and constructive reviews. We have uploaded a revised PDF. The section and table numbers in our replies refer to it. Three corrections change numbers throughout the paper, so we list them first. The numbers in our replies already include them.

  1. The same data split on all three models. The TTS models now use the split that Moshi already used: 150 prompts for fitting, 250 for validation and 600 for testing, plus 1,200 unwatermarked calibration answers for the thresholds. We repeated the CosyVoice3 and MOSS-TTS experiments, including the post-hoc baselines, on the new answers. Every TTS number changes slightly, but no conclusion changes.
  2. DAC 16 and 24 kHz with their delay compensated (Appendix. C.4). On every pass, the released DAC 16 and 24 kHz models return their audio 8 samples early. The submitted attack chained eight passes without re-aligning them, so the audio drifted against the frame grid of the tokenizer. To compensate for this drift, we now delay the output of every pass by 8 samples, for every method and model. This changes several DAC results, both ours (Moshi after eight DAC16 passes: 34.2 → 93.2 %) and those of the baselines (AudioSeal and CRAW are now strong after DAC16). The uncompensated chain appears only in Appendix. C.4.
  3. SpeechTokenizer at its own sampling rate. As Reviewer 5RTX pointed out, the submitted attack fed 24 kHz audio to the 16 kHz SpeechTokenizer. The attack now resamples 24 → 16 → 24 kHz, for every method. After eight passes, Redwing detects 86.0 / 87.2 / 51.5 % on Moshi / CosyVoice3 / MOSS-TTS.

The methods, the rule that sets Redwing's strength (the largest δ whose validation cost stays within KGW's, which we now also apply to HiPT), and the threshold rule (one threshold per detector, at 1 % FPR on the calibration answers, for every attack) are unchanged.

New in the revision, each answered in the replies below:

  • HiPT (Tables 1–3) and Latent-Mark (Tables 1–2) as baselines in the main tables, with HiPT's documented strength and Latent-Mark's own detection rule in Appendix. B.5;
  • WER and speaker similarity (Table 4, Appendix. C.13), and a lower δ for each model (Table 3, Appendix. C.13);
  • a resynchronizing detector against shifts and crops (Appendix. C.7), TPR as a function of clip length (Appendix. C.9), the objective against multi-pass TPR (Appendix. C.3), the one-pass channels of the codecs (Appendix. C.5), and an informed adversary (Appendix. C.10) with a threat-model paragraph in the Limitations;
  • a human listening test (MOS). Table 4 shows the ratings collected so far, and we will post the final results before the end of the discussion period.
Area_Chair_1m4XOpenReview ↗

ICLR Review Discussion – Follow-up on Author Rebuttal

Dear Reviewers,

Thank you for your time and effort in reviewing this submission.

The authors have submitted their rebuttal. Could you please take a moment to review their responses and assess whether your concerns have been adequately addressed? If you have any remaining concerns or new questions, please feel free to raise them in the discussion.

If you have already responded to the authors, please kindly disregard this reminder.

Thank you again for your valuable contributions to the review process.

Best,

AC

AuthorsOpenReview ↗

Final listening-test results

The listening test has finished, and we have uploaded the manuscript with the final MOS ratings in Table 4 and Appendix B.6. At the selected δ, listeners rate Redwing within 0.08 of KGW on every model. On Moshi and CosyVoice3, both are rated on par with unwatermarked speech. On MOSS-TTS, both are rated lower, KGW by 0.12 and Redwing by 0.19. With the lower δ of 0.5, Redwing's gap on MOSS-TTS disappears. Our reply to Reviewer TSts gives the full table.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.