- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Thrive, which is a multi-bit generative watermark for autoregressive TTS model designed to survive model-driven reconstruction. This method has two main parts, RISE injects a message into intermediate representations and modulates later generation with the same message, whereas CARE decodes with a waveform expert and a spectral expert, then uses a calibrated per bit reliability selector to choose between their logits. The experiments show that Thrive achieves high bit accuracy under reconstruction attacks and covers source attribution over up to 10000 identities.
Critical strengths
- The main finding is that robustness profiles cross between backbones and methods. Dots.tts is good at codecs, while qwen3 is stronger on vocoders and enhancers. This finding is informative and gives the community a reusable reconstruction benchmark.
- I like how the authors present their figures. The drawing is nice and clear to me.
Critical weaknesses
- The novelty is overstated. RISE modifies only the codec decoder in qwen3 or the AudioVAE decoder for Dots.tts and leaves the autoregressive model and its tokens or latents untouched. As a result, Thrive is a message conditioned decoder watermark, which is close to HFGw and RIWF instead of a new watermarking method for autoregressive TTS.
Questions for authors
- How is TPR@1%FPR computed for the 16 bit payload? Is the threshold the number of matching bits against the claimed message or the best match over all registered users?
- How is the generation quality for the unwatermarked outputs from qwen3 and dots.tts? So that we know how the watermarks affect the quality of the generated output.
- Why Dots.tts STOI has a std 0.197 and 0.314 on the OOD sets? Is it catastrophic failures on some utterances?
- What is the deployment cost? e.g. inference latency and runtime compared to other methods.
5 comments
Response to Reviewer 6cPq (Part 2/3)
Q2: Unwatermarked Generation Quality
To better distinguish the intrinsic quality of the underlying generators from the impact of watermark embedding, we have extended Table I to include unwatermarked outputs from Qwen3-TTS and Dots.tts across all three datasets. We report DNSMOS and NISQA because both assess perceptual quality without a reference. These comparisons use the same evaluation samples and synthesis conditions, with watermark embedding disabled or enabled. The unwatermarked outputs also serve as references for the paired fidelity metrics, for which self-comparison would provide no additional quality information.
Table I. Fidelity.
| Method (bps) | STOI ↑ | PESQ ↑ | SSIM ↑ | MCD ↓ | DNSMOS ↑ | NISQA ↑ | VGGish ↓ | CLAP ↓ | ACC ↑ | TPR ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| LJSpeech (In-Distribution Dataset) | ||||||||||
| Qwen3-TTS | - | - | - | - | 2.711 ± 0.478 | 2.991 ± 1.173 | - | - | - | - |
| Ours [Qwen3-TTS 0.6B] (16) | 0.958 ± 0.073 | 3.565 ± 0.768 | 0.850 ± 0.087 | 1.754 ± 0.136 | 2.638 ± 0.429 | 2.807 ± 1.165 | 1.355 | 0.086 | 0.999 | 1.000 |
| Dots.tts | - | - | - | - | 3.099 ± 0.303 | 4.185 ± 0.540 | - | - | - | - |
| Ours [Dots.tts 2B] (16) | 0.999 ± 0.003 | 4.210 ± 0.259 | 0.980 ± 0.004 | 0.777 ± 0.103 | 3.066 ± 0.317 | 4.094 ± 0.548 | 0.604 | 0.037 | 0.999 | 1.000 |
| LibriTTS (OOD Dataset) | ||||||||||
| Qwen3-TTS | - | - | - | - | 2.794 ± 0.414 | 3.224 ± 1.015 | - | - | - | - |
| Ours [Qwen3-TTS] (16) | 0.969 ± 0.050 | 3.747 ± 0.566 | 0.858 ± 0.075 | 1.781 ± 0.157 | 2.271 ± 0.431 | 3.159 ± 1.007 | 1.539 | 0.089 | 0.999 | 1.000 |
| Dots.tts | - | - | - | - | 3.163 ± 0.224 | 3.936 ± 0.812 | - | - | - | - |
| Ours [Dots.tts] (16) | 0.995 ± 0.024 | 4.045 ± 0.479 | 0.980 ± 0.004 | 0.739 ± 0.135 | 3.105 ± 0.271 | 3.827 ± 0.812 | 0.412 | 0.052 | 0.999 | 0.999 |
| AISHELL-3 (OOD Dataset) | ||||||||||
| Qwen3-TTS | - | - | - | - | 3.102 ± 0.250 | 3.437 ± 0.808 | - | - | - | - |
| Ours [Qwen3-TTS] (16) | 0.980 ± 0.023 | 3.820 ± 0.439 | 0.874 ± 0.048 | 1.763 ± 0.146 | 2.667 ± 0.353 | 3.291 ± 0.798 | 1.243 | 0.083 | 0.999 | 1.000 |
| Dots.tts | - | - | - | - | 3.176 ± 0.219 | 3.481 ± 1.058 | - | - | - | - |
| Ours [Dots.tts] (16) | 0.997 ± 0.010 | 4.069 ± 0.471 | 0.979 ± 0.004 | 0.764 ± 0.138 | 3.084 ± 0.294 | 3.399 ± 1.045 | 0.551 | 0.067 | 0.999 | 1.000 |
Across all three datasets, Thrive achieves near-perfect clean recovery on both backbones, with PESQ scores of 3.565-4.210 and NISQA remaining close to the unwatermarked baselines. These results demonstrate reliable multi-bit watermarking with high fidelity across discrete-token and continuous-representation autoregressive TTS. We have updated Table 1 and the corresponding discussion in Section 5.2 of the manuscript, with the revisions highlighted in blue.
Q3: High STOI Variance on OOD Datasets
We would like to clarify that the reported variability primarily reflects STOI's handling of short evaluation clips.
Within fixed one-second windows, speech duration and onset vary, leaving some unwatermarked references with insufficient frames after silence removal. In these cases, pystoi returns a near-zero invalid marker, which inflates the standard deviation when counted as a valid score. The same references produce this return even when compared with themselves, confirming that this statistical anomaly reflects insufficient temporal support for STOI, rather than watermark-induced intelligibility collapse. STOI statistics exclude clips for which fewer than 30 STFT frames remain after silence removal, causing pystoi to return the invalid value . We compute the reported mean and standard deviation over the remaining valid clips. We have corrected the STOI statistics in Table 1 (Sec. 5.2 of the manuscript) and highlighted the updated values.
PESQ and SSIM do not share this frame requirement and measure similarity between clean and watermarked outputs. Short audio pairs can remain highly similar even when STOI is non-computable, so their higher scores are consistent with this explanation.
Response to Reviewer 6cPq (Part 3/3)
Q4: Inference Latency and Deployment Cost
We have added an efficiency analysis in Sec. 5.6, with detailed comparisons in Table II. The results show that Thrive introduces small generation overhead and completes recovery from one-second speech in approximately 4.5 ms.
All methods are benchmarked on the same GPU using FP32, batch size one, and 16 bps. Generation overhead is relative to each native host: Thrive covers text-to-waveform synthesis, while vocoder baselines start from mel spectrograms. Recovery includes each method's full extraction pipeline on identical one-second speech.
Table II. Computational efficiency.
| Method | Params. (M) ↓ | WM Size (MiB) ↓ | GFLOPs ↓ | Latency (ms) ↓ | GPU (GiB) ↓ | Throughput ↑ | RTF ↓ |
|---|---|---|---|---|---|---|---|
| Generation | |||||||
| HFGw | 13.936 / 0 | 53.162 | 52.813 / 0 (0.00%) | 4.401 / 0.003 (0.06%) | 0.122 | 226.759 | 0.004 |
| RIWF | 0 / 0.115 | 0.438 | 346.116 / 0.001 (0.0003%) | 37.377 / 3.869 (10.35%) | 0.095 | 24.488 | 0.041 |
| Groot | 0 / 227.063 | 866.177 | 346.116 / 0.454 (0.13%) | 38.916 / 0.977 (2.51%) | 1.360 | 25.320 | 0.039 |
| Ours-Q | 0 / 5.889 | 22.450 | 456.763 / 0.826 (0.18%) | 1013.676 / 6.499 (0.64%) | 4.356 | 0.941 | 1.063 |
| Ours-D | 0 / 12.196 | 46.524 | 11838.964 / 24.306 (0.21%) | 971.546 / 30.920 (3.18%) | 9.435 | 0.998 | 1.002 |
| Recovery | |||||||
| HFGw | 0.306 | 1.169 | 26.713 | 1.313 | 0.060 | 761.625 | 0.001 |
| RIWF | 4.353 | 16.604 | 9.208 | 10.447 | 0.045 | 95.725 | 0.010 |
| Groot | 23.131 | 88.239 | 0.392 | 0.844 | 0.101 | 1185.018 | 0.0008 |
| Ours-Q | 2.839 | 10.829 | 4.842 | 4.624 | 0.088 | 216.248 | 0.005 |
| Ours-D | 2.839 | 10.829 | 4.842 | 4.446 | 0.088 | 224.943 | 0.004 |
The results highlight three advantages: 1. Parameter-efficient adaptation. RISE freezes the TTS host and trains only 5.889M/12.196M watermark parameters on Qwen/Dots, 97.4%/94.6% fewer than Groot, keeping the adaptation compact. 2. Low generation overhead. Coordinating representation injection with generation modulation adds little overhead, increasing GFLOPs by only 0.18%/0.21% and end-to-end latency by 0.64%/3.18%. 3. Efficient dual-expert recovery: Full CARE remains efficient with both experts and selection included, processing one-second speech in 4.446-4.624 ms with approximately 0.088 GiB peak GPU memory and 56%-57% lower latency than RIWF.
Response to Reviewer 6cPq (Part 1/3)
Dear Reviewer 6cPq, We sincerely appreciate the reviewer's careful evaluation and constructive feedback. We address each concern in detail below and have incorporated corresponding revisions into the manuscript, with all changes highlighted for clarity.
We would like to clarify the distinct contributions of Thrive relative to prior work. In Thrive, Representation-Generation Synchronized Embedding (RISE) coordinates the Representation Injection Path and the Generation Integration Path through a shared watermark message while keeping whole autoregressive model frozen. It injects watermark residuals into intermediate representations and conditions subsequent synthesis on the same watermark. This design enables different watermarks to be embedded at inference time without retraining.
In contrast, HFGw first pretrains TBWM [1] and then uses its decoder to guide full-parameter fine-tuning of the vocoder, enabling it to generate speech carrying a fixed watermark. RIWF first learns parameter masks to identify embedding locations within convolutional kernels, then replaces the selected kernel elements with watermark encodings and normalizes the modified weights. The key distinction is that RISE jointly learns watermark formation in intermediate representations and its continued integration into synthesis, rather than realizing the watermark solely through vocoder-weight modification.
In a nutshell, by operating on synthesis representations and transformations, RISE provides a shared embedding mechanism for both discrete-token and continuous-representation autoregressive TTS. Building on this mechanism, Thrive combines our Cross-Representation Adaptive Recovery with a training strategy that separates capability learning from reliability calibration, achieving strong robustness against reconstruction attacks and scalable multi-user attribution.
We compute TPR empirically from per-speech verification outcomes. Following VINE [2], we decode each audio and count the matching bits between the recovered message w^ and its pre-specified claimed 16-bit watermark w. The threshold applies to this claimed message, not the best match over all registered users. Under the theoretical assumption that bit matches are independent with probability 0.5 for unwatermarked audio, the matching count follows Binomial(16,0.5). The smallest integer threshold with theoretical FPR at most 1% is 14 matches, corresponding to an FPR of 0.209% because the matching count is discrete.
For the empirical evaluation, we calibrate the operating threshold using real unwatermarked outputs, consistent with the recommendation in SoK [3] to complement theoretical analysis with empirical measurements. These negative samples are generated with watermark injection disabled and evaluated using the same extraction and keyed decoding procedure against messages assigned before extraction. For each detector, we select the smallest integer threshold whose false-positive fraction on the calibration set is at most 1%. Calibration and test sets have no source-content overlap.
We then freeze the calibrated threshold for clean and all attacks. For each condition, TPR is the number of watermarked speech samples whose matching count meets or exceeds this threshold, divided by the total number of watermarked test samples. Each sample therefore contributes one observed detection decision. We provide the complete calculation procedure in App. D.2.
References:
[1] Liu C, et al. Detecting voice cloning attacks via timbre watermarking. NDSS 2024.
[2] Lu S, et al. Robust watermarking using generative priors against image editing: From benchmarking to advances. ICLR 2025.
[3] Zhao X, et al. Sok: Watermarking for ai-generated content. IEEE S&P 2025.