- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to make audio watermarks robust against unseen neural codecs. First, they frame codec resynthesis via Koopman operator theory, arguing that detectors built on invariant observable functions (eigenvalue 1) achieve robustness. In practice, they train a mel-spectrogram encoder on EnCodec-resynthesized audio using reconstruction and cosine invariance losses, select the top 46 of 128 dimensions via a stability index on 6 dev codecs, and embed watermarks via AWARE-style STFT magnitude optimization. Across 22 unseen codecs, the method achieves 69.1% bit accuracy and 76.7% detection accuracy, outperforming Timbre-SA (66.7/71.5%) and AWARE (61.5/65.8%).
Critical strengths
-
Generalization to unseen neural codecs is a real and under-studied problem. The experiment on a 30-codec evaluation benchmark is a solid contribution.
-
The results show that a codec-invariant pretrained representation transfers better than end-to-end attack simulation.
Critical weaknesses
-
Overclaimed theory: Thm 3.6 is tautological by construction. The paper overlooks whether invariance holds for watermarked audio (); if the codec discards , exact invariance implies , making watermark extraction impossible. The implementation (cosine loss, , VAD) deviates from the theory, and "simulation-free" is inaccurate given the 7 codecs used during training/selection.
-
Unconvincing robustness & unfair comparison: 69% bit accuracy is modest (multiple codecs perform near chance), with only a 2.4% gain over Timbre-SA. The comparison is unfair: the method accesses 7 codecs while baselines see . Timbre-SA appears poorly tuned (PESQ 1.90) and is then dismissed for its quality, without a matched-quality comparison.
Questions for authors
-
AWARE and “w/o pretraining” both appear to use a randomly initialized encoder, yet they differ by about 10 points (61.5 vs 51.2). What is the difference between them?
-
Can you measure invariance at watermarked inputs, i.e. ψ̂(A(x_w)) vs ψ̂(x_w), not only on clean audio?
-
How do the results change if the baselines get the same codec budget (training plus dev codecs), or if Timbre-SA is compared at matched quality?
4 comments
Question 4: If a codec discards watermark information , can the watermark still be extracted?
The answer depends on how you define whether a codec discards watermark information . If "the codec discards " implies that and exact invariance holds for all audio, then indeed holds, indicating that the watermark can hardly be embedded into or extracted from even unattacked audio. However, it is unreasonable to assume : This condition itself means that theoretically, there exists no watermark extractor that can extract watermark from any watermarked audio resynthesized by codec . On the other hand, if "the codec discards " merely implies inaccurate watermark detection on codec resynthesized audio, then we cannot claim that or that watermark extraction is impossible from unattacked audio.
Question 5: Is Theorem 3.6 overclaimed?
No. Thm. 3.6 guides the design of StableMark, but we never claim that the implemented system is theoretically robust. Below we address your concerns point by point.
The implementation W_1 deviates from the theory
Our theory guides the design of our method. Here is the connection. First, in our paper, we refer to the eigenfunctions with as "robust eigenfunctions". Though Thm. 3.6 reveals that the robust eigenfunctions provide theoretically guaranteed robustness, strictly optimizing towards these robust functions significantly restrict the space for watermark embedding, leading to limited limited detection accuracy. In this regard, our methods don't aim for the robust set. Instead, it tries to find an approximation set with , which may satisfy the Weak Robustness Condition in Thm. 3.5. Second, the approximation set construction motivates the invariance loss based on cosine similarity between original audio and resynthesized audio. Compared to exactly equivalence constraints, e.g., based on L2 norm difference between original audios and resynthesized audios, cosine similarity learns more functions that reflect the close relationship between original audios and resynthesized audios, including the exact equivalence. Bitwise accuracy listed in the table below confirms this theory. The subsequent selection stage then removes the functions with or to obtain the approximation set.
Table: Bitwise accuracies with different invariance losses .
| Clean | Train | Dev | Test | |
|---|---|---|---|---|
| L2 | 89.53 | 72.05 | 67.1 | 64.51 |
| Cosine Similarity | 98.7 | 76.7 | 71.23 | 65.04 |
"simulation-free" is inaccurate given the 7 codecs used during training/selection
We would like to clarify that StableMark requires attacked audio but not attack simulation. Attack simulation takes watermarked audio as input, whereas our method only requires codec resynthesis version of non-watermarked audio during pretraining and selection. Therefore, attack simulation needs to be conducted on-the-fly during training with checkpoint of the neural codec available. In contrast, StableMark can be trained with offline paired data in CodecFake+ without accessing the actual neural codec model. Nevertheless, our answer to Question 3 confirms that StableMark still surpasses the Timbre-SA baseline under both the same codec budget and the same quality budget.
Thank you for the response and new experiments. Two follow-ups:
Q2. Regressing on full features doesn't address my question, since ψ̂(x) dominates ψ̂(x_w). What matters is whether the watermark component survives: e.g., retention of Δ = ψ̂(x_w) − ψ̂(x) after the attack, or, per Thm 3.5, the fraction of bits whose pre-attack margin exceeds |f_i(A(x_w)) − f_i(x_w)|.
Q3. With a matched codec budget, Timbre-SA** (71.25%) outperforms StableMark (69.96%), so SA's main issue is quality, not overfitting. Two operating points (0.1/1) are too coarse for a quality-matched comparison; a robustness–quality curve would be more convincing.
The related work would benefit from discussing Audio Pirates: Black-box Audio Watermark Removal via Diffusion Priors, which studies diffusion-based watermark removal, and SoK: How Robust Is Audio Watermarking in Generative AI Models?, which systematically evaluates watermark robustness against generative attacks.
The new experiments partially address my concerns, so I will raise my score. I hope the authors can address the points above in the revision.
We sincerely appreciate your positive feedback and constructive suggestions. We have further modified our manuscript per your request. The newly added contents are listed below, and we hope they can help address your follow-up concerns.
Q2. Does the watermark component survive after attacks?
Indeed, a sufficient robustness condition is:
Proof:
which implies that if pre-attack margin exceeds , then the watermark component survives. Therefore, we can measure whether the watermark component survives by the fraction of bits whose pre-attack margin exceeds , averaged across all utterances and unseen test codecs. We denote this metric as "Margin condition". Below we list the results calculated on 200 samples from LibriSpeech test-clean.
| Method | Margin condition (%) ↑ | Post-attack bit acc. (%) ↑ |
|---|---|---|
| AWARE | 61.22 | 61.22 |
| StableMark | 66.34 | 66.38 |
The results show that compared with the AWARE baseline, StableMark watermark components are more likely to survive neural codec resythesis attacks. This coincides with and explains our advantage on bitwise accuracy under unseen code resynthesis attacks.
Revision: We added the marginal condition test in Appendix D.10.
Q3: Is there any fine-grained robustness–quality curve of the Timbre-SA baseline?
Thanks for raising this issue. In fact, we have searched for the optimal attack simulation frequency so that the audio quality of Timbre-SA** could match our proposed system. Here is the full result:
| Model | SA Frequency | Test Bit Acc. (%) | PESQ | STOI | ViSQOL |
|---|---|---|---|---|---|
| Timbre-SA* | 1 | 68.72 | 1.792 | 0.872 | 2.207 |
| 0.5 | 64.42 | 2.862 | 0.974 | 3.411 | |
| 0.25 | 62.98 | 3.045 | 0.964 | 3.311 | |
| 0.1 | 58.89 | 3.692 | 0.987 | 3.742 | |
| Timbre-SA** | 1 | 71.25 | 2.036 | 0.893 | 1.951 |
| 0.5 | 63.89 | 2.679 | 0.953 | 3.445 | |
| 0.25 | 59.83 | 3.444 | 0.986 | 3.630 | |
| 0.1 | 56.14 | 3.719 | 0.991 | 3.928 | |
| Timbre | 0 | 61.51 | 3.841 | 0.989 | 3.573 |
| StableMark | N/A | 69.96 | 3.965 | 0.972 | 3.670 |
The results above indicate that setting attack simulation frequency as 0.1 best matches the audio quality of Timbre-SA** and our proposed system. This is why we report results at SA frequency=0.1 in our initial response. Following your advice, we have added the fine-grained robustness–quality curve in Appendix D.1.
Revision: We added the fine-grained robustness–quality curve of the Timbre-SA baseline in Appendix D.1.
Other Revisions:
We have included Audio Pirates (DiffErase) and SoK as related works.
Hope our additional clarifications and revisions can address your concerns. Thank you again for your valuable suggestions, which have helped a lot in improving our manuscript.
Thank you for your careful review and insigntful comments! We are now going to address all your questions and concerns.
Question 1: What is the difference between the "AWARE" baseline and the "w/o pretraining" ablation?
The difference lies in the number of output dimensions of the audio encoder: The AWARE baseline uses 1024 output dimensions whereas "w/o pretraining" uses 46. A potential reason for the robustness gap is that when the audio encoder is randomly initialized, dimension of the latent space should be sufficiently large to encode a 30-bit watermark message. Robustness thus drops in the "w/o pretraining" ablation as we uses only 46 dimensions to align with the StableMark system.
Question 2: Can you measure invariance at watermarked inputs, i.e., ψ(A(xw)) vs ψ(xw), not only on clean audio?
Thanks for this insightful question. We measure invariance by estimating how close per-dimension eigenvalues are to 1. Specifically, we linearly regress the attacked feature ψ^k(A(x)) on its corresponding clean feature ψ^k(x) with Ordinary Least Squares OLS. The slope estimates ψ^k(x)ψ^k(A(x)) and can thus serve as an approximation of per-dimension eigenvalues, whereas the normalized residual verifies how much each dimension resembles an eigenfunction. Let t denotes the frame index, c denotes the codec type, and k denotes the dimension index. The estimated eigenvalue λ^k and normalized residual rk of each dimension can be calculated as:
λ^k=Scc(k)Sca(k),rk=Scc(k)Saa(k)−λ^kSca(k)
where
Scc(k)=∑c,tψk(x)[t]2,Sca(k)=∑c,tψk(x)[t]ψk(Ac(x))[t],Saa(k)=∑c,tψk(Ac(x))[t]2
Aggregating λ^k and rk across dimensions yields two measurements assessing how close estimated eigenvalues are to 1, and the approximation error, respectively:
Δλ=d1∑kλ^k−1,rˉ=d1∑krk
where d is the total number of dimensions. Lower Δλ and rˉ indicate better approximation of stable eigenfunctions.
Below we report Δλ and rˉ on 22 test codecs, with both 100 unwatermarked and 100 watermarked audio from CodecFake+ dataset.
Compared with unwatermarked audio, watermark audio exhibits a slight increase in Δλ +19.36 and rˉ +9.01. This is within our expectation since a domain gap exists between non-watermarked and watermarked audio. Nevertheless, Thm. 3.5 indicates that despite a feature drift between ψ(A(xw)) and ψ(xw), the extracted watermark can remain correct if the resulting score change is smaller than its pre-attack decision margin. Superior bitwise accuracy reported in Tab. 1 also support that the approximate eigenfunctions remain approximately robust when the input is watermarked.
Question 3: How do the results change if the baselines get the same codec budget (training plus dev codecs), or if Timbre-SA is compared at matched quality?
We would like to respectively clarify that StableMark still surpasses the Timbre-SA baseline under both the same codec budget and the same quality budget. To align the training codec budget, we add 6 development codecs into the Timbre-SA training pipeline, forming the Timbre-SA** system. To align the quality budget, we reduce frequency of simulation attack to 0.1. Results in the following table show that adding more training codecs improves robustness of Timbre-SA against unseen test codecs, but watermarked audio quality remains unacceptible. Meanwhile, reducing simulation attack frequency significantly degrades robustness of both Timbre-SA* and Timbre-SA**. Timbre-SA** with SA frequecy=0.1, namely the system with matched codec and quality budget, lags behind StableMark by 13.82pp in robustness. This indicates the advantage of our proposed system.
(Continued in the next comment...)