acceptodds
Under review as a conference paper at ICLR 2027

StableMark: Towards Codec-Robust Audio Watermarking via Attack Decomposition

Abstract

Audio watermarking is an emerging technology designed to embed imperceptible markers in audio signals for proactive deepfake detection and source tracing. However, these embedded markers are often susceptible to watermark removal attacks, particularly those leveraging neural codec resynthesis. While attack simulation during training has been widely adopted to enhance robustness against specific neural codec attacks, this approach often fails to generalize to unseen codecs or significantly degrades watermarked audio quality. To improve robustness against unseen neural codecs, we propose StableMark, a simulation-free audio watermarking framework. Our core idea is to first decompose nonlinear codec transformations into a set of attack-invariant functions that capture stable acoustic features preserved throughout resynthesis, and then construct the watermark detector as a set of linear combinations of these stable functions. In this paper, we start by proving that watermarking systems built upon attack-invariant functions are theoretically robust. Guided by this theory, we develop a method to derive generalizable stable functions through a combination of audio encoder pretraining and stable dimension selection. Extensive experiments across diverse neural codecs demonstrate that our framework significantly improves robustness against unseen neural codec attacks while maintaining high audio fidelity.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio Watermarking, Robustness, Nueral Codec, Deepfake Detection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ekbEOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to make audio watermarks robust against unseen neural codecs. First, they frame codec resynthesis via Koopman operator theory, arguing that detectors built on invariant observable functions (eigenvalue 1) achieve robustness. In practice, they train a mel-spectrogram encoder on EnCodec-resynthesized audio using reconstruction and cosine invariance losses, select the top 46 of 128 dimensions via a stability index on 6 dev codecs, and embed watermarks via AWARE-style STFT magnitude optimization. Across 22 unseen codecs, the method achieves 69.1% bit accuracy and 76.7% detection accuracy, outperforming Timbre-SA (66.7/71.5%) and AWARE (61.5/65.8%).

Critical strengths
  1. Generalization to unseen neural codecs is a real and under-studied problem. The experiment on a 30-codec evaluation benchmark is a solid contribution.

  2. The results show that a codec-invariant pretrained representation transfers better than end-to-end attack simulation.

Critical weaknesses
  1. Overclaimed theory: Thm 3.6 is tautological by construction. The paper overlooks whether invariance holds for watermarked audio (x+δx+\delta); if the codec discards δ\delta, exact invariance implies ψ(x+δ)≈ψ(x)\psi(x+\delta) \approx \psi(x), making watermark extraction impossible. The implementation (cosine loss, W1W_1, VAD) deviates from the theory, and "simulation-free" is inaccurate given the 7 codecs used during training/selection.

  2. Unconvincing robustness & unfair comparison: 69% bit accuracy is modest (multiple codecs perform near chance), with only a 2.4% gain over Timbre-SA. The comparison is unfair: the method accesses 7 codecs while baselines see ≤1\le 1. Timbre-SA appears poorly tuned (PESQ 1.90) and is then dismissed for its quality, without a matched-quality comparison.

Questions for authors
  1. AWARE and “w/o pretraining” both appear to use a randomly initialized encoder, yet they differ by about 10 points (61.5 vs 51.2). What is the difference between them?

  2. Can you measure invariance at watermarked inputs, i.e. ψ̂(A(x_w)) vs ψ̂(x_w), not only on clean audio?

  3. How do the results change if the baselines get the same codec budget (training plus dev codecs), or if Timbre-SA is compared at matched quality?

4 comments

Thank you for your careful review and insigntful comments! We are now going to address all your questions and concerns.

Question 1: What is the difference between the "AWARE" baseline and the "w/o pretraining" ablation?

The difference lies in the number of output dimensions of the audio encoder: The AWARE baseline uses 1024 output dimensions whereas "w/o pretraining" uses 46. A potential reason for the robustness gap is that when the audio encoder is randomly initialized, dimension of the latent space should be sufficiently large to encode a 30-bit watermark message. Robustness thus drops in the "w/o pretraining" ablation as we uses only 46 dimensions to align with the StableMark system.

Question 2: Can you measure invariance at watermarked inputs, i.e., ψ^(A(xw))\widehat{\psi}(\mathcal{A}(x_w)) vs ψ^(xw)\widehat{\psi}(x_w), not only on clean audio?

Thanks for this insightful question. We measure invariance by estimating how close per-dimension eigenvalues are to 1. Specifically, we linearly regress the attacked feature ψ^k(A(x))\hat\psi_k(\mathcal A(\mathbf x)) on its corresponding clean feature ψ^k(x)\hat\psi_k(\mathbf x) with Ordinary Least Squares OLSOLS. The slope estimates ψ^k(A(x))ψ^k(x)\frac{\hat\psi_k(\mathcal A(\mathbf x))}{\hat\psi_k(\mathbf x)} and can thus serve as an approximation of per-dimension eigenvalues, whereas the normalized residual verifies how much each dimension resembles an eigenfunction. Let tt denotes the frame index, cc denotes the codec type, and kk denotes the dimension index. The estimated eigenvalue λ^k\hat\lambda_k and normalized residual rkr_k of each dimension can be calculated as:

λ^k=Sca(k)Scc(k),rk=Saa(k)−λ^kSca(k)Scc(k)\hat\lambda_k=\frac{S_{ca}^{(k)}}{S_{cc}^{(k)}}, \quad r_k=\frac{\sqrt{S_{aa}^{(k)}-\hat\lambda_kS_{ca}^{(k)}}}{\sqrt{S_{cc}^{(k)}}}

where

Scc(k)=∑c,tψk(x)[t]2,Sca(k)=∑c,tψk(x)[t]ψk(Ac(x))[t],Saa(k)=∑c,tψk(Ac(x))[t]2S_{cc}^{(k)}=\sum_{c,t}\psi_k(\mathbf x)[t]^2,\quad S_{ca}^{(k)}=\sum_{c,t}\psi_k(\mathbf x)[t]\psi_k\big(\mathcal A_c(\mathbf x)\big)[t],\quad S_{aa}^{(k)}=\sum_{c,t}\psi_k\big(\mathcal A_c(\mathbf x)\big)[t]^2

Aggregating λ^k\hat\lambda_k and rkr_k across dimensions yields two measurements assessing how close estimated eigenvalues are to 1, and the approximation error, respectively:

Δλ‾=1d∑k∣λ^k−1∣,rˉ=1d∑krk\overline{\Delta\lambda}=\frac{1}{d}\sum_{k}\big|\hat\lambda_{k}-1\big|, \quad \bar r=\frac{1}{d}\sum_{k}r_{k}

where dd is the total number of dimensions. Lower Δλ‾\overline{\Delta\lambda} and rˉ\bar r indicate better approximation of stable eigenfunctions.

Below we report Δλ‾\overline{\Delta\lambda} and rˉ\bar r on 22 test codecs, with both 100 unwatermarked and 100 watermarked audio from CodecFake+ dataset.

Δλ‾\overline{\Delta\lambda}↓rˉ\bar r↓
Unwatermarked0.24020.5968
Watermarked0.28670.6506

Compared with unwatermarked audio, watermark audio exhibits a slight increase in Δλ‾\overline{\Delta\lambda} +19.36+19.36% and rˉ\bar r +9.01+9.01%. This is within our expectation since a domain gap exists between non-watermarked and watermarked audio. Nevertheless, Thm. 3.5 indicates that despite a feature drift between ψ^(A(xw))\widehat{\psi}(\mathcal{A}(x_w)) and ψ^(xw)\widehat{\psi}(x_w), the extracted watermark can remain correct if the resulting score change is smaller than its pre-attack decision margin. Superior bitwise accuracy reported in Tab. 1 also support that the approximate eigenfunctions remain approximately robust when the input is watermarked.

Question 3: How do the results change if the baselines get the same codec budget (training plus dev codecs), or if Timbre-SA is compared at matched quality?

We would like to respectively clarify that StableMark still surpasses the Timbre-SA baseline under both the same codec budget and the same quality budget. To align the training codec budget, we add 6 development codecs into the Timbre-SA training pipeline, forming the Timbre-SA** system. To align the quality budget, we reduce frequency of simulation attack to 0.1. Results in the following table show that adding more training codecs improves robustness of Timbre-SA against unseen test codecs, but watermarked audio quality remains unacceptible. Meanwhile, reducing simulation attack frequency significantly degrades robustness of both Timbre-SA* and Timbre-SA**. Timbre-SA** with SA frequecy=0.1, namely the system with matched codec and quality budget, lags behind StableMark by 13.82pp in robustness. This indicates the advantage of our proposed system.

ModelSA FrequencyTest Bit Acc. %PESQSTOIViSQOL
Timbre-SA*168.721.7920.8722.207
0.158.893.6920.9873.742
Timbre-SA**171.252.0360.8931.951
0.156.143.7190.9913.928
StableMarkN/A69.963.9650.9723.670

(Continued in the next comment...)

Question 4: If a codec discards watermark information δ\delta, can the watermark still be extracted?

The answer depends on how you define whether a codec discards watermark information δ\delta. If "the codec discards δ\delta" implies that A(x+δ)=A(x)\mathcal A(x+\delta)=\mathcal A(x) and exact invariance holds for all audio, then ψ(x+δ)=ψ(x)\psi(x+\delta)=\psi(x) indeed holds, indicating that the watermark can hardly be embedded into or extracted from even unattacked audio. However, it is unreasonable to assume A(x+δ)=A(x)\mathcal A(x+\delta)=\mathcal A(x): This condition itself means that theoretically, there exists no watermark extractor that can extract watermark from any watermarked audio resynthesized by codec A\mathcal A. On the other hand, if "the codec discards δ\delta" merely implies inaccurate watermark detection on codec resynthesized audio, then we cannot claim that ψ(x+δ)≈ψ(x)\psi(x+\delta)\approx\psi(x) or that watermark extraction is impossible from unattacked audio.

Question 5: Is Theorem 3.6 overclaimed?

No. Thm. 3.6 guides the design of StableMark, but we never claim that the implemented system is theoretically robust. Below we address your concerns point by point.

The implementation cosineloss,cosine loss, W_1,VAD, VAD deviates from the theory

Our theory guides the design of our method. Here is the connection. First, in our paper, we refer to the eigenfunctions with λ=1\lambda = 1 as "robust eigenfunctions". Though Thm. 3.6 reveals that the robust eigenfunctions provide theoretically guaranteed robustness, strictly optimizing towards these robust functions significantly restrict the space for watermark embedding, leading to limited limited detection accuracy. In this regard, our methods don't aim for the robust set. Instead, it tries to find an approximation set with λ≃1\lambda \simeq 1, which may satisfy the Weak Robustness Condition in Thm. 3.5. Second, the approximation set construction motivates the invariance loss based on cosine similarity between original audio and resynthesized audio. Compared to exactly equivalence constraints, e.g., based on L2 norm difference between original audios and resynthesized audios, cosine similarity learns more functions that reflect the close relationship between original audios and resynthesized audios, including the exact equivalence. Bitwise accuracy listed in the table below confirms this theory. The subsequent selection stage then removes the functions with λ>>1\lambda >> 1 or λ<<1\lambda << 1 to obtain the approximation set.

Table: Bitwise accuracies with different invariance losses .

Linv\mathcal L_{inv}CleanTrainDevTest
L289.5372.0567.164.51
Cosine Similarity98.776.771.2365.04

"simulation-free" is inaccurate given the 7 codecs used during training/selection

We would like to clarify that StableMark requires attacked audio but not attack simulation. Attack simulation takes watermarked audio xw\mathbf{x}_w as input, whereas our method only requires codec resynthesis version of non-watermarked audio x\mathbf x during pretraining and selection. Therefore, attack simulation needs to be conducted on-the-fly during training with checkpoint of the neural codec available. In contrast, StableMark can be trained with offline paired data in CodecFake+ without accessing the actual neural codec model. Nevertheless, our answer to Question 3 confirms that StableMark still surpasses the Timbre-SA baseline under both the same codec budget and the same quality budget.

Thank you for the response and new experiments. Two follow-ups:

Q2. Regressing on full features doesn't address my question, since ψ̂(x) dominates ψ̂(x_w). What matters is whether the watermark component survives: e.g., retention of Δ = ψ̂(x_w) − ψ̂(x) after the attack, or, per Thm 3.5, the fraction of bits whose pre-attack margin exceeds |f_i(A(x_w)) − f_i(x_w)|.

Q3. With a matched codec budget, Timbre-SA** (71.25%) outperforms StableMark (69.96%), so SA's main issue is quality, not overfitting. Two operating points (0.1/1) are too coarse for a quality-matched comparison; a robustness–quality curve would be more convincing.

The related work would benefit from discussing Audio Pirates: Black-box Audio Watermark Removal via Diffusion Priors, which studies diffusion-based watermark removal, and SoK: How Robust Is Audio Watermarking in Generative AI Models?, which systematically evaluates watermark robustness against generative attacks.

The new experiments partially address my concerns, so I will raise my score. I hope the authors can address the points above in the revision.

We sincerely appreciate your positive feedback and constructive suggestions. We have further modified our manuscript per your request. The newly added contents are listed below, and we hope they can help address your follow-up concerns.

Q2. Does the watermark component survive after attacks?

Indeed, a sufficient robustness condition is:

mi′fi(xw)>∣fi(xw)−fi(A(xw))∣m_i' f_i(\mathbf x_w) > |f_i(\mathbf x_w)-f_i(\mathcal A(\mathbf x_w))|

Proof: mi′fi(xw)>∣fi(xw)−fi(A(xw))∣  ⟹  mi′fi(xw)>mi′[fi(xw)−fi(A(xw))]  ⟹  mi′fi(A(xw))>0m_i' f_i(\mathbf x_w) > |f_i(\mathbf x_w)-f_i(\mathcal A(\mathbf x_w))| \implies m_i' f_i(\mathbf x_w) > m_i'[f_i(\mathbf x_w)-f_i(\mathcal A(\mathbf x_w))]\implies m_i'f_i(\mathcal A(\mathbf x_w))>0

which implies that if pre-attack margin exceeds ∣fi(xw)−fi(A(xw))∣|f_i(\mathbf x_w)-f_i(\mathcal A(\mathbf x_w))|, then the watermark component survives. Therefore, we can measure whether the watermark component survives by the fraction of bits whose pre-attack margin exceeds ∣fi(xw)−fi(A(xw))∣|f_i(\mathbf x_w)-f_i(\mathcal A(\mathbf x_w))|, averaged across all utterances and unseen test codecs. We denote this metric as "Margin condition". Below we list the results calculated on 200 samples from LibriSpeech test-clean.

MethodMargin condition (%) ↑Post-attack bit acc. (%) ↑
AWARE61.2261.22
StableMark66.3466.38

The results show that compared with the AWARE baseline, StableMark watermark components are more likely to survive neural codec resythesis attacks. This coincides with and explains our advantage on bitwise accuracy under unseen code resynthesis attacks.

Revision: We added the marginal condition test in Appendix D.10.

Q3: Is there any fine-grained robustness–quality curve of the Timbre-SA baseline?

Thanks for raising this issue. In fact, we have searched for the optimal attack simulation frequency so that the audio quality of Timbre-SA** could match our proposed system. Here is the full result:

ModelSA FrequencyTest Bit Acc. (%)PESQSTOIViSQOL
Timbre-SA*168.721.7920.8722.207
0.564.422.8620.9743.411
0.2562.983.0450.9643.311
0.158.893.6920.9873.742
Timbre-SA**171.252.0360.8931.951
0.563.892.6790.9533.445
0.2559.833.4440.9863.630
0.156.143.7190.9913.928
Timbre061.513.8410.9893.573
StableMarkN/A69.963.9650.9723.670

The results above indicate that setting attack simulation frequency as 0.1 best matches the audio quality of Timbre-SA** and our proposed system. This is why we report results at SA frequency=0.1 in our initial response. Following your advice, we have added the fine-grained robustness–quality curve in Appendix D.1.

Revision: We added the fine-grained robustness–quality curve of the Timbre-SA baseline in Appendix D.1.

Other Revisions:

We have included Audio Pirates (DiffErase) and SoK as related works.

Hope our additional clarifications and revisions can address your concerns. Thank you again for your valuable suggestions, which have helped a lot in improving our manuscript.

Reviewer_9g8WOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the robustness of audio watermarks against neural-codec resynthesis attacks. It decomposes codec attacks through the Koopman-operator framework, derives conditions under which a detector is provably robust, instantiates the analysis with an encoder pretrained for invariance and diversity plus a stability-based dimension selection (USI), and builds a detector-only multi-bit watermark on top. However, the theoretical guarantees do not cover the implemented system, the core assumption of shared invariant eigenfunctions across codecs is asserted rather than established, the headline gains are largely driven by baselines not designed for codec attacks, and part of the observed robustness comes from VAD-based engineering rather than learned invariance.

Critical strengths

This problem is well-motivated. Neural-codec copy-resynthesis is a practical way to remove audio watermarks. This paper addresses this security problem and offers useful guidance for designing watermarks robust to such attacks.

Critical weaknesses
  • The theory doesn't actually cover the proposed system. Thm 3.6 requires the detector to be a linear combination of exact λ=1 eigenfunctions, but the encoder is only trained with a cosine-similarity loss (λ≈1 at best), and the stable dimensions are picked by a heuristic USI score with no formal link to Thm 3.5/3.6. So the theory motivates the design but guarantees nothing about the final watermark. Thus, calling it “theoretically robust” may overstate the claim.
  • The headline gains mostly come from weak baselines. The only apples-to-apples baseline is Timbre-SA (same training codec), and the margin there is small: 69.05 vs 66.49 bit accuracy and 76.66 vs 71.51 existence accuracy on the 22 unseen codecs, and Timbre-SA is actually better on existence accuracy under AI attacks (90.55 vs 88.87). AudioSeal/WavMark dropping to chance level just shows they weren't designed for codec attacks, and mixing 16-bit and 30-bit systems confounds capacity with robustness. "Significant improvement" needs toning down.
  • Part of the robustness is engineered, not learned. The VAD does real work: removing it costs several points, and the system drops badly under energy-type edits (limiter ≈71%, Table 11), a fragility the authors themselves blame on the VAD, so restricting embedding/detection to voiced segments buys a chunk of the numbers. The evaluation is also narrow: it covers speech only, uses a single training codec, and tests “unseen” codecs only from the same ecosystem. Music, other codec families, and stochastic regeneration remain untested, despite the general-defense framing.
Questions for authors
  • How close are the selected stable dimensions to actual λ=1 eigenfunctions? Reporting the empirical eigenvalue distribution of the learned features per test codec would show whether Thm 3.5/3.6 has any purchase on the deployed system.
  • Can you provide a matched-capacity comparison (e.g., 16-bit StableMark and Timbre-SA) so that robustness is not confounded with payload size?
  • How much of the unseen-codec gain remains with the VAD disabled at both embedding and detection? This would separate the contribution of the voicing restriction from that of the learned invariant features.
3 comments

We sincerely appreciate your careful review and constructive feedback. Below we provide answers to your questions and concerns.

Question 1: How close are the selected dimensions to exact λ=1\lambda=1 eigenfunctions?

To assess how well each dimension approximates robust eigenfunction, we estimate per-dimension eigenvalues by linearly regressing the attacked feature ψ^k(A(x))\hat\psi_k(\mathcal A(\mathbf x)) on its corresponding clean feature ψ^k(x)\hat\psi_k(x). Specifically, we apply Ordinary Least Squares OLSOLS without intercept, which is the regression method with least squared residual. The slope estimates ψ^k(A(xn))ψ^k(xn)\frac{\hat\psi_k(\mathcal A(\mathbf x_n))}{\hat\psi_k(\mathbf x_n)} and can thus serve as an approximation of per-dimension eigenvalues, whereas the normalized residual verifies how much each dimension resembles an eigenfunction. Let tt denotes the frame index and kk denotes the dimension index. The estimated eigenvalue λ^k\hat\lambda_k and normalized residual rkr_k of each dimension can be calculated as:

λ^k=Sca(k)Scc(k),rk=Saa(k)−λ^kSca(k)Scc(k)\hat\lambda_k=\frac{S_{ca}^{(k)}}{S_{cc}^{(k)}}, \quad r_k=\frac{\sqrt{S_{aa}^{(k)}-\hat\lambda_kS_{ca}^{(k)}}}{\sqrt{S_{cc}^{(k)}}}

where

Scc(k)=∑tψk(x)[t]2,Sca(k)=∑tψk(x)[t]ψk(A(x))[t],Saa(k)=∑tψk(A(x))[t]2S_{cc}^{(k)}=\sum_{t}\psi_k(\mathbf x)[t]^2,\quad S_{ca}^{(k)}=\sum_{t}\psi_k(\mathbf x)[t]\psi_k\big(\mathcal A(\mathbf x)\big)[t],\quad S_{aa}^{(k)}=\sum_{t}\psi_k\big(\mathcal A(\mathbf x)\big)[t]^2

Aggregating λ^k\hat\lambda_k and rkr_k across dimensions yields two measurements assessing how close estimated eigenvalues are to 1, and the approximation error, respectively:

Δλ‾=1d∑k∣λ^k−1∣,rˉ=1d∑krk\overline{\Delta\lambda}=\frac{1}{d}\sum_{k}\big|\hat\lambda_{k}-1\big|, \quad \bar r=\frac{1}{d}\sum_{k}r_{k}

where dd is the total number of dimensions. Lower Δλ‾\overline{\Delta\lambda} and rˉ\bar r indicate better approximation of stable eigenfunctions.

Below we report Δλ‾\overline{\Delta\lambda} and rˉ\bar r calculated on 22 test codecs, each with 100 unwatermarked audio from CodecFake+ dataset.

CodecΔλ‾\overline{\Delta\lambda} AWARE↓Δλ‾\overline{\Delta\lambda} StableMark↓rˉ\bar r AWARE↓rˉ\bar r StableMark↓
A0.20530.19130.60080.568
C0.20290.19060.59480.5729
E10.24410.22460.66290.6218
E20.28490.27190.70650.6674
E30.36720.36060.78080.7531
F20.10180.08410.43490.4251
G0.14710.13990.51580.4949
H0.24750.24250.67610.6428
I20.21380.19820.61230.5819
K0.12530.11420.47720.4529
L10.15220.13770.52450.4920
L20.13120.11750.48930.4556
M0.89810.97221.06191.0250
P10.23130.21860.64260.6214
P20.20090.19840.62120.5886
Q10.31770.30480.72360.6916
R0.31400.33150.71930.6757
S0.17240.16100.55530.5353
T10.34400.34220.74520.7225
T30.27680.27140.68290.6512
U0.11790.12050.47370.4729
V0.10030.09050.43520.4161
mean0.24530.24020.62440.5968

These results show that compared to the AWARE baseline, StableMark better approximate stable eigenfunctions, indicating that the cosine-similarity loss and USI-based selection promote StableMark to better satisfy Thm. 3.5.

Question 2: Can you provide a matched-capacity comparison (e.g., 16-bit StableMark and Timbre-SA) so that robustness is not confounded with payload size?

To begin with, we have already compared our system with Timbre, Timbre-SA and AWARE under the same 30-bit payload size. Since 30-bit AudioSeal / WavMark checkpoints are not available, we compare them against 16-bit StableMark in the following table.

ModelnbitsClean Acc.TrainDevTestAI AttacksPESQSTOIVISQOLNISQA
AudioSeal16100.0 / 99.5863.48 / 51.2460.16 / 61.4347.60 / 50.2858.79 / 58.984.4110.9984.3054.006
WavMark16100.0 / 99.6253.83 / 50.7558.05 / 57.7353.82 / 50.2157.82 / 51.654.1870.9973.6864.054
Timbre-SA16100.00 / 94.9093.40 / 94.6672.90 / 72.8662.98 / 65.3288.26 / 84.563.2780.9853.1013.842
StableMark1699.99 / 94.9086.29 / 92.4074.39 / 74.3270.04 / 74.5188.97 / 85.923.9580.9723.6843.605

(bitwise accuracy (%)/ existence accuracy (%))

The results above confirm that StableMark outperforms AudioSeal, WavMark and Timbre-SA under the same 16-bit payload size.

(Continued in the next comment...)

Question 3: How much of the unseen-codec gain remains when VAD is disabled at both embedding and detection?

As introduced in Sec. 4.2, VAD is only applied during watermark detection in all StableMark variants. The "w/o VAD" system in Tab. 1 shows robustness against unseen codecs when VAD is disabled during detection. It achieves bitwise and existence accuracy comparable to those of Timbre-SA, and surpasses all other baselines on robustness against unseen neural codecs. However, the watermarked audio quality of Timbre-SA is unacceptable. Therefore, StableMark maintains superior robustness even without VAD at embedding or detection.

Question 4: Can the sensitivity of VAD to energy-based edits be reduced?

Yes. We find that applying Root Mean Square Normalization (RMSNorm) before WebRTC VAD effectively prevents accuracy degradation caused by Dynamic Range Compression DCDC, Dynamic Range Expansion DEDE and Limiter LMLM. Meanwhile, robustness against neural codec resynthesis attacks also becomes slightly better.

Below we compare the bitwise accuracy of StableMark with and without RMSNorm under dynamic range edits and neural codec resynthesis attacks. Results on DC, DE and LM are tested on RawBench, whereas results on codecs are tested on 200 samples from LibriSpeech test clean. Adding RMSNorm improves robustness against both sides, alleviating the system's sensitivity to global energy adjustments.

RMSNormEditsCodecs
DCDELMTrainDevTest
×88857778.0372.5966.14
✓99979679.5272.7666.83

Question 5: How sensitive is the robustness to the codec used for encoder pretraining?

We mentioned in Sec. 5.4 that the choice of training codec significantly influences the system's robustness and generalization ability. Instead of using a single training codec (EnCodec-6b24k), we report robustness of StableMark variants using different types of training codecs. We find that switching to other types of training codecs like EnCodec-12b24k can further improve robustness +5.15pponsharedunseencodecs+5.15pp on shared unseen codecs. We report system trained on EnCodec-6b24k in the main experiment just for a fair comparison with baselines.

Question 6: Does robustness generalize to other codec families and stochastic regeneration?

As listed in Tab. 6 in Appendix C.1, the testing codecs covers different types of codec families. Except two codecs from seen ecosystems F2andI2F2 and I2, all 20 other test codecs are real unseen codecs with different model architectures, configs and parameters. As for stochastic regeneration, we have reported robustness against diffusion regeneration in Tab. 9 in Appendix D.2. The result shows that StableMark achieves 99.57% bitwise accuracy and 98.49% existence accuracy, indicating that StableMark is robust against stochastic regeneration.

Question 7: Does StableMark generalize to music?

To begin with, RAW-Bench, the benchmark we used to evaluate robustness against conventional audio edits, contains music data. Therefore, results in Tab. 10 in Appendix D.3 shows that on music data, StableMark at least maintains comparable robustness agaist conventional audio edits. For neural codec resynthesis attacks, we report bitwise accuracy of StableMark against neural codec resynthesis attacks on 542 music samples from RAW-Bench in the following table.

CleanTrainDevTestAI
Speech99.8682.0074.0369.0588.14
Music99.5778.9968.2563.1479.69

The results show that on music data, StableMark does not perform as well as on speech data. This is within our expectation since we only train StableMark on speech data. Another potential reason is that some neural codecs does not perform well on music. To illustrate this, we list the per-codec difference in bitwise accuracy and audio quality between music and speech data.

IDcodecMusic bit Acc.Speech bit Acc.Δbit Acc.Music SI-SNR dBdBSpeech SI-SNR dBdBΔSI-SNRdBdB
F1FunCode-en68.492.3-23.90.69.8-9.2
GFACodec56.576.6-20.1-9.56.4-15.9
RAudioDec58.477.3-18.9-18.1-14.6-3.5
ASpeechTokenizer56.171.9-15.8-1.93.6-5.5
SBigCodec54.770.3-15.6-11.33.9-15.2
……………………
BDAC99.199.9-0.8-0.3-4.7+4.4
WSQ-Codec51.250.7+0.5-30.8-36.2+5.4
JLanguage-Codec50.049.5+0.5-49.4-43.9-5.5
P2SNAC74.573.6+0.97.25.5+1.7
E2SemantiCodec-0.70k67.559.7+7.8-29.9-30.7+0.8

The following table shows a positive correlation between audio quality degradation and robustness degradation. Indeed, a lot of neural codecs are designed to compress speech and performs poorly on music. Neural codec resynthesis thus damages both audio quality and watermark on music.

Summary of Changes

Added Appendix D.7 and Table 14 to estimate how close are the selected dimensions to exact λ=1\lambda=1 eigenfunctions.

Thanks again for your insightful comment. To further address your concern, we would like to make some additional clarification on Weakness 2. Considering the low audio quality (PESQ=1.898, MOS=1.406) of the Timbre-SA baseline, the advantage of StableMark is not marginal. In the table below, we also tried to compare Timbre-SA and our proposed StableMark under matched audio quality by reducing simulation frequency in Timbre-SA to 0.1.

ModelSA FrequencyTest Bit Acc. (%)PESQSTOIViSQOL
Timbre-SA168.721.7920.8722.207
0.158.893.6920.9873.742
StableMarkN/A69.963.9650.9723.670

The results show that under matched quality budget, StableMark surpasses Timbre-SA by 13.82pp in robustness, indicating that the advantage of our proposed system is not modest.

Please let us know if you have further questions or concerns. Any feedback would be highly appreciated.

Reviewer_Cp2dOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses multi-bit audio watermarking under neural codec resynthesis attacks, where an attacker re-synthesizes the watermarked waveform with a neural codec and destroys the payload. The standard defense, attack simulation with a differentiable codec in the training loop, overfits to the seen codec and destabilizes training. The paper frames attacks as Koopman operators on waveform space: a robust detector must live in the span of attack-invariant eigenfunctions. Three theorems formalize sufficient conditions (exact invariance, or a margin condition against the fragile residual). The practical realization, StableMark, has three stages: (1) pretrain a frame-wise convolutional encoder inside a Vocos autoencoder with a cosine invariance loss between clean and codec-resynthesized latents plus a reconstruction loss; (2) rank the 128 latent dimensions by a universal stability index aggregating four Wasserstein-distance statistics across seen and unseen codecs and keep the top 46; (3) embed by directly optimizing the short-time Fourier transform amplitude under a six-decibel budget (inherited from prior embedder-free work), with detection via a frozen encoder, a linear layer, and a bitwise readout head preceded by voice activity detection.

Critical strengths

The strongest attack-simulation baseline achieves higher accuracy on the seen codec but renders audio unusable (perceptual evaluation of speech quality 1.9, mean opinion score 1.4). StableMark achieves the best unseen-codec accuracy among quality-preserving systems (bitwise 69.05 versus 61.49 for the best such baseline; above sixty percent on 17 of 22 unseen codecs) while keeping perceptual quality in a usable range, and it does so with detector-only training on a frozen encoder. The component ablations are informative: removing pretraining collapses accuracy to chance, and removing the reconstruction loss is catastrophic, documenting a real invariance-capacity tension.

Critical weaknesses
  1. The only theorem with a real guarantee requires the detector to use exactly attack-invariant eigenfunctions; the method retains 46 approximately stable dimensions, so the guarantee never applies, and the weaker margin condition is enforced by nothing in any objective. No spectral decomposition is ever computed — the Koopman machinery is used only to name the concept, and the entire construction could be restated as "learn codec-invariant features and read the watermark out of them linearly" without loss. Meanwhile the invariance loss is trained on actual codec outputs and the dimension selection uses six development codecs, so the method uses attack information exactly as simulation does; what is avoided is only the differentiable attack layer inside the loop.
  2. The strongest competitor is excluded on quality grounds, but no matched-quality variant of it (weaker simulation strength, fewer augmentations) is evaluated, even though the paper itself provides exactly such a trade-off curve for two other baselines; without it, "robustness at preserved quality" is only half-shown. Baselines see one codec while the proposed method's selection procedure uses statistics from seven, so the comparison is not budget-matched. There is no false-positive rate, receiver operating curve, or operating-point reporting for the existence metric (existence accuracy of 76.66 percent is uninterpretable without it), no test-set size, no confidence intervals, no adaptive or white-box attack (an attacker optimizing against the frozen, published encoder), no robustness evaluation under cropping or resampling before the codec, and no ablation of the four universal-stability-index weights against simpler selectors. The mel-spectrogram front-end and the Vocos decoder used in pretraining are plausible confounds: mel features are intrinsically stable to spectrogram-preserving codecs, and the two test codecs built on Vocos score among the highest.
Questions for authors
  1. Can you provide the robustness-versus-quality trade-off curve for the attack-simulation baseline at matched perceptual quality, and a budget-matched baseline whose simulation uses the same seven codecs your selection procedure sees?
3 comments

We sincerely appreciate your constructive feedback and insightful comments. Below we address each of your questions in detail.

Question 1: Can you provide the robustness-versus-quality trade-off curve for the attack-simulation baseline at matched perceptual quality?

Thanks for your question! The table below shows the robustness-versus-quality trade-off for the attack-simulation baseline Timbre−SATimbre-SA. We try to improve audio quality of Timbre-SA by gradually reducing the frequency of EnCodec simulated attack (SA Frequency) from 1 Timbre−SATimbre-SA to 0 (Timbre). All metrics are calculated on 200 samples from LibriSpeech test-clean.

SA FrequencyTest Bit Acc. %↑PESQ↑STOI↑ViSQOL↑
168.721.7920.8722.207
0.564.422.8620.9743.411
0.2562.983.0450.9643.311
0.158.893.6920.9873.742
061.513.8410.9893.573
StableMark69.963.9650.9723.670

It can be obseved that the attack-simulation variants cannot reach comparable quality until the simulation attack frequency goes down below 0.1, where StableMark outperforms Timbre-SA variants by about 10pp on bitwise accuracy averaged among all test codecs. Therefore, StableMark still outperforms attack-simulation baselines at matched audio quality.

Question 2: Can you provide a budget-matched baseline whose simulation uses the same seven codecs your selection procedure sees?

Sure! In the following table, we report robustness and watermarked audio quality of Timbre-SA variants trained with the same seen codec budget 1traincodec+6developmentcodecs1 train codec + 6 development codecs, denoted as Timbre-SA**. All metrics are calculated on 200 samples from LibriSpeech test-clean. As required in Q1, we also tuned the attack simulation frequency so that audio quality could be matched.

ModelSA FrequencyTest Bit Acc. %↑PESQ↑STOI↑ViSQOL↑
Timbre-SA**171.252.0360.8931.951
Timbre-SA**0.563.892.6790.9533.445
Timbre-SA**0.2559.833.4440.9863.630
Timbre-SA**0.156.143.7190.9913.928
Timbre-SA**061.513.8410.9893.573
StableMarkN/A69.963.9650.9723.670

As listed in the first row, Timbre-SA** does achieve silghlty better bitwise accuracy than StableMark when simulated attack frequency is 1. However, its watermarked audio quality is still unacceptable. The baseline with both matched seen codec budget and audio quality budget is Timbre-SA** with SA Frequency=0.1, where bitwise accuracy on test codecs is 13.82pp behind StableMark. This indicates that StableMark still outperforms the attack-simulation baseline at matched seen codec budget.

(Continued in the next comment...)

Question 3: Is the implemented StableMark theoretically robust?

As mentioned in Sec. 4, the implemented system is guided but not certified by theorems in Sec. 3. Below we address your concerns point by point.

The only theorem with a real guarantee requires the detector to use exactly attack-invariant eigenfunctions;

In fact, Thm 3.4 (Eqivalent Robustness Condition), 3.5 (Weak Robustness Condition) and 3.6 (Strong Robustness Condition) are all sufficient robustness conditions providing real guarantee. However, building upon strict robust eigenfunctions as required in Thm. 3.6 may restrict the space for watermark embedding, making it difficult to satisfy the first condition in eq. 1111. Therefore, our methods don't aim for the exact robust set. Instead, it tries to find an approximation set with λ≃1\lambda \simeq 1 that may satisfy the Weak Robustness Condition in Theorem 3.5 (Weak Robustness Condition).

the method retains 46 approximately stable dimensions, so the guarantee never applies, and the weaker margin condition is enforced by nothing in any objective.

The selection phase is introduced to find approximated stable eigenfunction that may satisfy the weaker margin condition. Specifically, compared to exact attack-invariant constraints, the learnt functions reflects close relationship between original audios and resynthesized audios. The subsequent selection stage then removes the functions with λ>>1\lambda >>1 or λ<<1\lambda <<1 to obtain the approximation set.

No spectral decomposition is ever computed — the Koopman machinery is used only to name the concept, and the entire construction could be restated as "learn codec-invariant features and read the watermark out of them linearly" without loss.

Direct spectral decomposition of nonlinear codec attack A\mathcal A is intractable. To lift A\mathcal A to a liear function space where spectral decomposition can be conducted, we introduce Koopman operator KA\mathcal K_{\mathcal A}. Eq. 55 shows the decomposition process. It is the decomposition of the Koopman operator and the subsequent derivation of robustness condition that motivates us to learn robust eigenfunctions and build the watermark detector upon them in the implemented system.

Question 4: Why call the method "simulation-free" when pretraining and selection use actual codec outputs?

Thanks for pointing out this ambiguity. It should be noticed that attack simulation takes watermarked audio xw\mathbf{x}_w as input, whereas our method only requires codec resynthesis version of non-watermarked audio x\mathbf x during pretraining and selection. Therefore, attack simulation needs to be conducted on-the-fly during training with checkpoint of the neural codec available. In contrast, StableMark can be trained with offline paired data in CodecFake+ without accessing the actual neural codec model.

Question 5: What are the FPR, ROC, and operating point behind the reported 76.66% existence accuracy? Are there any confidence intervals?

As mentioned in Sec. 5.1 and Appendix C.5, the threshold τ\tau for each system (aka operating point) is tuned on 500 audio samples from LibriSpeech dev-clean with training and development codecs, so that optimal binary accuracy could be achieved while preserving FPR and FNR<0.1. The operating point, FPR, bitwise and existence accuracies under unseen test codecs, as well as their 95% confidence intervals CICI are listed in the following table. The ROC curves can be found at Fig. 5 in Appendix D.1 of the revised manuscript. Results show that: 1) StableMark surpasses all baselines on AUC except Timbre-SA, which suffer from significant quality degradation; 2) There is no overlapping of reported intervals, confirming our robustness gain on unseen test codecs.

ModelOperating Point (%)FPR %AUCTest Bit Acc (%)95% CITest exist. Acc (%)95% CI
AudioSeal81.30.000.475847.52
473˙5,476˙947\.35, 47\.69

|50.28|

502˙5,503˙150\.25, 50\.31

| |WavMark|75.0|0.49|0.5404|53.83|

536˙9,539˙853\.69, 53\.98

|50.23|

501˙9,502˙950\.19, 50\.29

| |Timbre|73.4|0.00|0.8257|59.63|

595˙1,597˙459\.51, 59\.74

|54.31|

542˙0,544˙354\.20, 54\.43

| |Timbre-SA|66.7|0.08|0.9015|65.54|

654˙0,656˙765\.40, 65\.67

|69.68|

694˙4,699˙269\.44, 69\.92

| |AWARE|63.4|3.50|0.7457|60.08|

598˙4,603˙359\.84, 60\.33

|64.18|

637˙7,646˙063\.77, 64\.60

| |StableMark|63.4|2.80|0.8810|67.46|

672˙4,676˙867\.24, 67\.68

|75.71|

753˙2,761˙175\.32, 76\.11

|

Question 6: How many recordings and speakers are included in the testset?

The evaluation is conducted on all 2620 recordings of LibriSpeech test-clean, covering 40 speakers.

(Continued in the next comment...)

Question 7: Is StableMark robust against white-box attacks?

Thanks for raising this issue. Previous work [1] reported that most existing audio watermarking methods are not robust to white-box watermark-removal attacks given sufficient number of iterations and audio quality budget. Therefore, in this work, we follow the common threat modeling of audio watermarking not to public the detector parameters, and opt not to pursue robustness against white-box attacks.

That said, we still evaluate the resilience of our frozen detector under a white-box adversarial attack with a tight audio quality budget SNR=40SNR=40. We follow the default setting of [1] and test on 200 samples from LibriSpeech test clean. Compared with WavMark and Timbre, our method achieves a relatively high bitwise accuracy and a low attack success rate ASRASR, demonstrating better resilience against white-box adversarial attacks.

Bitwise Accuracy↑ASR↓
AudioSeal0.6740
WavMark0.0910.73
Timbre0.0910.99
StableMark0.6690.06

[1] Liu, Hongbin, et al. "Audiomarkbench: Benchmarking robustness of audio watermarking." The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. 2024.

Question 8: Is StableMark robust when cropping or resampling is applied before the codec?

Yes. We apply the CutSamplesAttack and ResamplingPolyAttack attacks in Deepmark benchmark before each neural codec. The table below shows the bitwise accuracy of AWARE and StableMark under these cascading attacks. The results below show that StableMark retains about 66% bitwise accuracy when cropping, resampling, or both precede unseen codec resynthesis, consistently surpassing AWARE by about 5pp.

ModelCodec TrainCodec DevCodec TestAI
CodecAWARE69.4268.6961.4986.81
StableMark82.0074.0369.0588.14
Crop->CodecAWARE68.2067.4060.7786.10
StableMark78.8071.6566.6486.89
Resample->CodecAWARE71.2769.0161.8587.06
StableMark79.3571.9366.3986.41
Crop->Resample->CodecAWARE69.2767.9960.9585.69
StableMark79.0071.6466.0486.14

Question 9: Is there any ablation of the four universal-stability-index USIUSI weights against simpler selectors?

To demonstrate the specific contribution of the stability index, we compare the system with compelete USI dimension selection and the ones with random selection or invariance score selector. The bitwise accuracy listed in the following table confims the gain of the USI compared to simple selectors.

TrainDevTest
Random Selection79.3772.5866.5
Invariance Score79.672.9966.24
StableMark79.873.3667.05

Question 10: Is robustness restricted to Vocos or spectrogram-preserving codecs?

No. Below we report mean bitwise and existence accuracy on testing codecs without Vocos codecs w/oVocosw/o Vocos and without spectrogram-preserving codecs w/oFreqw/o Freq. These results show that our proposed method maintains superiority even when Vocos and spectrogram are excluded from the testing set. Additionally, as listed in Tab. 1, other systems built upon Mel frontend like AWARE 614˙961\.49% test bitwise accuracy do not perform as well as StableMark 690˙569\.05% test bitwise accuracy, indicating that the Mel frontend is not the main source of robustness.

Methodw/o Vocosw/o Freq
AudioSeal47.82 / 50.3048.14 / 50.09
WavMark53.92 / 50.2353.43 / 50.13
Timbre60.49 / 55.8058.98 / 55.15
Timbre-SA65.37 / 69.0865.29 / 68.59
AWARE60.43 / 63.6961.02 / 64.74
StableMark67.65 / 74.6868.58 / 76.68

Summary of Changes

  • Updated Section 5.1 to report the main evaluation set size.

  • Added Table 7 and Figure 5 in Appendix D.1 to report detection thresholds, false-positive rates (PFRs), AUCs, and ROC curves.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.