acceptodds
Under review as a conference paper at ICLR 2027

Randomized Representation Smoothing for Audio Watermarking

Abstract

With the rapid development of audio generators, audio watermarking has become an important technique for content authentication and provenance verification. In this work, we propose RRS-AW, a method for embedding watermarks into randomized transformation-averaged robust representations of audio during generation. The method encodes binary messages through pairwise coordinate ordering in representations averaged over random audio transformations and optimizes the generation latent while keeping both the generator and watermark extractor fixed. We provide a theoretical analysis of the robustness of the resulting representation and complement it with an empirical analysis of its local sensitivity. Experiments on AudioLDM-generated music demonstrate strong robustness across individual, composite, and codec transformations: RRS-AW achieves the best or tied-best bit accuracy on 9 of 13 composite attacks and 9 of 13 codec settings among the competitors. Additional experiments demonstrate robustness across different audio generators, low false-positive detection, and high perceptual audio quality.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
audio watermarking, music generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_46mnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RRS-AW, an audio watermarking framework that embeds multi-bit messages into randomized transformation-averaged audio representations during generation. The method uses a pretrained auxiliary encoder to extract representations from randomly transformed versions of an audio signal and averages these representations to improve stability under perturbations. Binary messages are encoded through the relative ordering of secret pairs of representation coordinates.

RRS-AW keeps the pretrained audio generator and auxiliary encoder fixed, and embeds watermarks by optimizing the generation latent for each audio sample. The paper also presents theoretical robustness bounds relating representation perturbations to potential bit errors.

Experiments are primarily conducted on AudioLDM-generated music, with additional evaluations on other audio generators. The authors compare RRS-AW with several existing audio watermarking methods under individual attacks, composite transformations, and audio codec distortions. The results show competitive bit extraction accuracy, particularly under composite and codec attacks. The paper also includes audio quality evaluations, false-positive analysis, computational cost measurements, and an ablation study of randomized representation averaging.

Critical strengths

S1. A well-motivated representation-level approach to robust multi-bit audio watermarking.

The paper presents an interesting approach that embeds watermark information through pairwise ordering constraints in transformation-averaged audio representations. Rather than requiring the individual representation coordinates to remain unchanged, the method aims to preserve their relative ordering under perturbations. This is a reasonable design for improving robustness and allows multi-bit payloads to be embedded without retraining the pretrained generator or auxiliary encoder.

S2. Broad empirical evaluation and a meaningful ablation of the proposed smoothing mechanism.

The experiments consider individual, composite, and codec-based transformations, together with additional audio generators. The authors also evaluate audio quality, false positives, and computational costs. Importantly, the ablation in Table 14 isolates randomized representation averaging while keeping the other major components unchanged. The reported mean bit accuracy across attacked conditions increases from 0.7449 without averaging to 0.8413 with the complete method, providing direct empirical evidence for the usefulness of the main proposed mechanism.

Critical weaknesses

W1. The reported improvements in bit accuracy do not consistently translate into reliable watermark detection, while the method incurs substantial computational and audio-fidelity costs.

Although RRS-AW achieves competitive bit accuracy under many transformations, the corresponding watermark detection performance is considerably less convincing. According to Table 6, the method achieves an average TPR of only 0.5459 across all attacked conditions, with particularly low TPRs of 0.4062 under composite attacks and 0.3680 under EnCodec compression. TraceMark achieves a slightly higher overall attacked TPR of 0.5680.

This distinction is important because the paper motivates audio watermarking primarily through content authentication and provenance verification. High average bit accuracy does not necessarily translate into reliable authentication under the adopted detection rule.

Furthermore, different baselines are evaluated using method-specific detection rules and thresholds, including different Hamming-distance thresholds and native detection gates. This complicates direct comparisons of detection performance. The empirical false-positive evaluation is also limited to 512 natural audio recordings, on which several methods achieve zero observed false positives. Consequently, the paper does not sufficiently establish a detection advantage under a common false-positive operating point. These concerns are especially relevant given the substantial computational and audio-quality costs of the proposed method. Table 13 reports an average embedding time of approximately 358.63 seconds for a three-second audio sample, compared with milliseconds for the post-hoc baselines. Table 1 also shows lower signal-level fidelity than several competing methods, with an SNR of 20.23 dB. The authors acknowledge the computational and quality limitations, and the robustness improvements in bit accuracy are meaningful. Nevertheless, the current evidence does not sufficiently demonstrate that the practical detection benefits justify the substantial computational overhead and fidelity trade-off.

W2. The theoretical robustness analysis is insufficiently connected to the implemented normalization procedure and the empirical operating regime.

A central contribution of the paper is its theoretically motivated robustness analysis. However, the relationship between the theoretical formulation and the implemented watermark embedding procedure requires further clarification.

In Algorithm 1, the estimated representation is explicitly normalized by dividing it by its L2 norm, and the watermark loss is evaluated using this normalized representation. In contrast, Eq. (5) defines the watermark loss using the unnormalized representation, and Lemma 2 derives its bit-flip guarantee from a minimum coordinate margin of delta in this unnormalized representation.

If the optimization is indeed performed on the normalized representation, achieving zero watermark loss implies a margin of at least delta in normalized coordinates. However, the corresponding margin in the unnormalized representation is at least delta multiplied by the L2 norm of the original representation. Without an additional assumption or bound on this norm, the minimum unnormalized margin of delta used in the theoretical argument does not follow directly.

Although positive normalization preserves the coordinate ordering used for decoding, it does not preserve the absolute coordinate margins that determine the stated robustness condition. Therefore, the applicability of Lemma 2 and Eq. (11) to the actual implementation remains unclear.

Moreover, the theoretical guarantee depends on Lipschitz and perturbation-sensitivity bounds. The appendix reports substantial local sensitivity, including an observed maximum of approximately 2.55 × 10^5. However, the paper does not demonstrate whether the sufficient condition in Eq. (11) is satisfied for representative perturbations at the experimental watermark margin.

The theoretical analysis also explicitly assumes parametric transformations of the same type as those used during embedding, whereas codec attacks are evaluated empirically without being included in latent optimization.

These observations do not invalidate the experimental robustness results. However, they weaken the connection between the theoretical contribution, the implemented method, and the practical attack settings evaluated in the paper.

Questions for authors

Q1. Detection performance under a unified false-positive operating point

Table 6 reports a mean attacked TPR of 0.5459 for RRS-AW, despite its relatively high bit accuracy. Furthermore, the compared methods use different detection thresholds and method-specific decision rules.

Could the authors provide detection performance at matched false-positive rates, or report ROC curves for the main competing methods? How does RRS-AW compare with the baselines under a common detection operating point?

Additionally, could the authors clarify whether the negative-set evaluation includes unwatermarked outputs from the target generator and other audio generators, beyond the 512 natural recordings reported in Table 5?

Q2. Consistency between normalized embedding and the theoretical margin

Algorithm 1 computes the watermark loss on the normalized representation h(s)h(s), whereas Eq. (5) and Lemma 2 formulate the embedding margin using the unnormalized representation γ^(s)\hat{\gamma}(s).

Which representation is actually used to compute the watermark loss in the implementation?

If the normalized representation is used, how is the minimum unnormalized coordinate margin of δ\delta in Lemma 2 justified? Does the theoretical result require an additional bound on ∥γ^(s)∥2\|\hat{\gamma}(s)\|_2, or should the robustness analysis instead be formulated directly for the normalized representation?

Q3. Practical significance of the theoretical robustness bound

Could the authors provide a numerical evaluation of the sufficient robustness condition in Eq. (11) under representative experimental transformations, using the actual embedding margin and suitable sensitivity estimates?

In particular, does the bound yield a non-vacuous guarantee for any of the evaluated perturbation strengths? If not, what specific practical robustness property is established by the theoretical analysis beyond the general relationship between representation perturbations and bit flips?

Q4. Computational cost and robustness trade-off

Table 13 reports an average embedding time of approximately 358.63 seconds for a three-second audio sample. The appendix evaluates reduced transformation-sampling budgets, but does not fully characterize the corresponding trade-off between runtime, watermark detection, and audio quality.

Could the authors clarify how the embedding time, attacked TPR, bit accuracy, and audio quality change when the number of sampled transformations and the optimization budget are reduced?

Such results would help determine whether the reported robustness gains can be retained at a computational cost more suitable for practical audio generation.

3 comments
AuthorsOpenReview ↗

Response to 46mn - part 1/3

We thank the reviewer for their detailed feedback on our work. Below, we provide our answers to the weaknesses and questions raised by the reviewer. We address Q1 in this comment, Q2 and Q3 in part 2/3, and Q4 in part 3/3. We realize that the discussion phase itself is coming to an end, and posting our answers shortly before the deadline may feel inconvenient. However, we motivate it by the necessity of additional experiments and revisions. We would appreciate the reconsideration of the assessment of our work if the concerns of the reviewer are addressed sufficiently.

Q1. Detection performance under a unified false-positive operating point

For RRS-AW, AudioSeal, SilentCipher, WavMark, and Timbre, payload-matching thresholds are selected using the same criterion: the largest Hamming-distance threshold whose false-match probability under independent, uniformly random bits is below (10−3)(10^{-3}). This gives (H≤1)(H\leq1) for 16-bit payloads, (H≤4)(H\leq4) for SilentCipher’s 24-bit comparison, and exact matching for Timbre’s 10-bit payload. The numerical thresholds therefore differ with payload length. Native detector gates also remain method-specific. The original negative-set evaluation used 512 independent natural recordings from the Freesound dataset. We additionally evaluated 450 unwatermarked outputs from the target AudioLDM generator, obtaining:

MethodMessage-match false positivesCombined false positives, including native gates
RRS-AW0/450 (0.000%)0/450 (0.000%)
AudioSeal1/450 (0.222%)0/450 (0.000%)
SilentCipher0/450 (0.000%)0/450 (0.000%)
WavMark0/450 (0.000%)0/450 (0.000%)
Timbre2/450 (0.444%)2/450 (0.444%)

To improve the current methods' robustness and audio quality, we modified the optimization process. Instead of Adam optimization, the modified method uses two-stage L-BFGS. The first stage includes 600 warmup updates, followed by up to 300 refinement updates in the second stage. Latent optimization uses 120 augmented views during warmup and 200 during refinement, compared to the original methods' 1000 views per update. On top of that, the modified version uses speed and composite augmentations. The loss adds normalized waveform error and a soft 23.1 dB SNR constraint, reduces the ETV penalty weight from 5000 to 500, and applies waveform energy projection targeting at least 23.05 dB. These changes improve robustness and audio quality.

Metric - full 50-prompt benchmarkOriginalModified method
Mean SNR20.23 dB23.11 dB
Attacked BAC84.10%87.15%
Clean BAC100.00%100.00%
Attacked TPR54.65%60.37%
STOI0.94570.9340
PESQ4.04374.0864
False positives on freesound recordings - shared unchanged decoder0/5120/512

To ensure similar time cost, runtime improvements were introduced, including: grouped encoder evaluation, selective activation recomputation, and FFT-plan retention. So, although the modified version uses a more complex optimization algorithm, the time cost of the method rises by only 20%.

AuthorsOpenReview ↗

Response to 46mn - part 2/3

Q2. Consistency between normalized embedding and the theoretical margin

We are grateful for spotting the inconsistency and confirm that the method we propose uses the normalized version presented in Algorithm 1. Both the experimental results presented in the main text of the article and the code provided in the supplementary materials use this normalized implementation. Equation 5 and Lemma 2 use notation from an earlier, unnormalized version of the algorithm, which was included by mistake. In the final version of the paper, we will update both to align with Algorithm 1. We indicate that no additional theoretical derivations are needed for the unnormalized sample mean representation (since the ones are already included in assumptions on Lipschitz continuity).

Q3. Practical significance of the theoretical robustness bound

We want to clarify that the theoretical robustness bound explicitly connects together three entities: the quantitative robustness of the auxiliary encoder function, the methodological property of the proposed watermarking scheme (namely, that changes in the empirical vector representation yield detection of the watermarking signal after post-processing), and the number of bit flips that can occur in the extracted message under watermark removal perturbations. Although the encoder tested in our experiments does not necessarily have nice Lipschitz properties, we speculate that the ones can be added at zero-cost (for example, via using 1-Lipschitz networks).

For some perturbations, the deterministic margin condition in Lemma 2 guarantees zero bit errors for the measured representations. Lemma 2 gives a bit-specific guarantee. Let Δi\Delta_i be the original margin of bit ii, and let ε\varepsilon be the largest absolute change in any coordinate of the normalized representation after perturbation. Each bit is encoded by the ordering of two coordinates, so their difference can change by at most 2ε2\varepsilon. Every bit with Δi>2ε\Delta_i > 2\varepsilon therefore retains its original value.

We tested one clip whose normalized aggregate had zero watermark loss in all ten independent reference extractions. The smallest bit margin across these extractions was 0.23287. Under additive Gaussian noise with standard deviation 10−410^{-4}, the largest observed 2ε2\varepsilon was 0.18152 across 50 comparisons (five noise realizations and ten extractions per realization). Since 0.18152<0.232870.18152 < 0.23287, the predicted upper bound was zero bit errors in every comparison. All 16 bits were recovered in all 50 cases. A 1% volume reduction also satisfied the condition in all ten comparisons, with 2ε≤0.030702\varepsilon \leq 0.03070 and zero observed errors.

For Eq. (11), the estimated concentration term alone was 0.24288 for Gaussian noise and 0.19175 for volume reduction. Both exceeded the threshold δ/2=0.1125\delta/2 = 0.1125 before adding the sensitivity term. Thus, the positive result above applies to the measured representation pairs. Failure of the sufficient condition does not imply that any bit must flip, let alone that all bits will flip. Even at the stronger Gaussian noise level of 10−310^{-3}, the decoder recovered 83.13% of bits on the same clip.

We also want to address the question related to absence of codec transformation during embedding of the watermark. We initially omitted codec augmentations because real encode/decode steps introduce substantial computational overhead into the embedding loop, and our differentiable augmentation suite already achieved 100% bit accuracy on MP3 at 32-128 kbps and >94% on AAC at 32-128 kbps. To address your question, we incorporated MP3 augmentation into sample-specific latent optimization and evaluated two strategies without modifying the extractor:

  • Strategy A: Assigns weight 10/11 to the original 10-family objective and 1/11 to a single-view watermark hinge after real MP3 16 kbps encode/decode at every optimization step.
  • Strategy B: Applies the full original augmentation family after MP3 16 kbps compression on 1 seeded step out of every 11.

Applying MP3 improves bit accuracy at 16 kbps by 13.12% for Strategy A and 10.00% for Strategy B, while maintaining 100% clean bit accuracy and target detection. In addition to that, there is a gain in bit accuracy for other low-bitrate codecs, like AAC and EnCodec. Detection at Hamming distance at most one rose from 3/10 to 5/10 for both strategies; its paired 95% interval includes zero. The downside of this solution comes from audio quality metrics - both SNR and PESQ dropped from 20 dB to 17.31 and 17.60 dB and from 3.94 to 3.52 and 3.70 points respectively.

AuthorsOpenReview ↗

Response to 46mn - part 3/3

Q4. Computational cost and robustness trade-off

To better characterize the trade-off between computational cost and robustness, we evaluated reduced transformation-sampling budgets for both embedding and extraction.

We evaluated watermark bit accuracy on a subset of test audio clips across 37 attacks. Each embedding step used 1,000 transformations. We varied the maximum number of embedding steps (RR) and the number of transformations used during extraction (kk). Each cell reports mean attacked bit accuracy, with speedup relative to R=700R = 700 and k=1,000k = 1,000 in parentheses. Speedup is based on the mean time to embed from the initial latent and extract the watermark once.

Embedding step limit RRk=0k = 0k=10k = 10k=100k = 100k=1,000k = 1,000
046.62% (8763.90$\times$)47.74% (2296.85$\times$)47.91% (2174.70$\times$)47.77% (995.57$\times$)
1059.83% (35.00$\times$)60.03% (34.63$\times$)60.03% (34.60$\times$)60.00% (33.99$\times$)
5073.48% (7.22$\times$)73.18% (7.20$\times$)73.48% (7.20$\times$)73.31% (7.17$\times$)
70083.18% (1.001$\times$)84.16% (1.001$\times$)84.35% (1.001$\times$)84.10% (1.000$\times$)

R=0R = 0 is the unwatermarked control. At k=0k = 0, the encoder processes the input once without augmentation. Unlike the naive variant in Table 14, k=0k = 0 retains transformation averaging during embedding. Embedding uses the original early-stopping rule, so RR is an upper limit.

At k=1,000k = 1,000, reducing RR from 700 to 50 yielded a 7.17$\times$ speedup, while mean attacked bit accuracy fell from 84.10% to 73.31%. At R=700R = 700, reducing kk from 1,000 to 10 changed accuracy by only 0.06 percentage points and total time by less than 0.1%.

Embedding step limit RSNR (dB)
0 (unwatermarked control, measured)47.80
1013.64
5016.18
70020.23

We also conducted experiments varying both embedding and extraction transformation counts, while keeping the maximum number of embedding steps R=700R = 700.

Embedding transformations \ Extraction501002505001,000
5082.83%82.83%82.90%82.51%82.43%
10083.47%83.51%83.53%83.07%83.10%
25083.79%83.90%83.88%83.52%83.57%
50083.79%83.89%83.93%83.61%83.65%
1,00084.22%84.35%84.28%83.97%84.10%

Attacked bit accuracy remains stable across all transformation-sampling budgets. Across every embedding budget, varying the extraction transformation count between 50 and 1,000 shifts attacked bit accuracy by less than 0.5 percentage points.

Embedding transformations \ Extraction501002505001,000
5050.65%50.54%50.70%50.59%50.65%
10051.19%51.41%51.62%51.46%51.62%
25053.03%53.24%53.24%53.35%53.62%
50053.62%53.57%53.84%53.62%53.84%
1,00054.43%54.38%54.32%54.22%54.65%

The table above shows little variation in mean attacked TPR across extraction budgets. Reducing extraction transformations from 1000 down to 50 results in a performance shift of less than 0.6 percentage points across all embedding configurations. This indicates that watermark extraction and detection can be made faster in practice while preserving similar observed bit accuracy and TPR.

Embedding countSNR, dBSTOIMean step-count reduction factor
5020.260.9511.47$\times$
10020.010.9501.33$\times$
25020.060.9481.20$\times$
50020.110.9481.11$\times$
1,00020.230.9461.00$\times$

Lowering the embedding transformation count reduces both the per-step gradient computation cost and the mean number of required optimization steps, while keeping close values for audio quality metrics. For the experiment with embedding count equal to 50, the mean number of optimization steps is 1.47$\times$ less than the original methods.

Setting the embedding transformation count to 250 and the extraction transformation count to 50 provides an effective operating point, reducing per-clip mean optimization steps by 1.20$\times$ while maintaining 100.0% untouched detection, 20.06 dB SNR, an attacked TPR of 53.03%, and an attacked bit accuracy of 83.79%.

Reviewer_Zc2XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a novel way of extracting audio watermarks from multiple augmented representations of the waveform, for additional robustness to attacks, which they also motivate mathematically. They are able to embed 16-bit payloads onto 3 sec waveforms with competitive quality compared to the baselines.

Critical strengths

In general, the paper is easy to follow. Specifically:

  1. The robustness proofs are simple and informative, and the authors empirically measure the Lipschitz constant, which was the only strong assumption.
  2. The authors present a perceptual evaluation of their watermark against other baselines.
  3. Clean and reproducible code
  4. Augmentations during inference (randomized representation) is a novel idea for watermark extraction, and are ablated

Despite the good presentation, the results are not strong: little detectability gain for decreased quality and orders of magnitude slower speed

Critical weaknesses
  1. As the authors admit, the method is rather slow (orders of magnitude slower than baselines)
  2. The results are not very strong in terms of quality
  3. There is no clear motivation for the index-based key scheme and the corresponding loss
Questions for authors
  1. Line 322: Why were codec attacks not applied? AudioSeal does MP3 as a training augmentation with a straight-through estimator, and thus Table 4 reveals it's much more robust to MP3 (especially low bitrate).
  2. Can you please explain intuitively how your key scheme works and how Eq. 5 is trying to embed the key information in the representation space?
3 comments
AuthorsOpenReview ↗

Response to Zc2X - part 1/3

We thank the reviewer for their detailed feedback on our work. Below, we provide our answers to the weaknesses and questions raised by the reviewer. We address W1 in this comment, W2 and Q1 in part 2/3 and W3/Q2 in part 3/3. We realize that the discussion phase itself is coming to an end, and posting our answers shortly before the deadline may feel inconvenient. However, we motivate it by the necessity of additional experiments and revisions. We would appreciate the reconsideration of the assessment of our work if the concerns of the reviewer are addressed sufficiently.

W1. As the authors admit, the method is rather slow (orders of magnitude slower than baselines)

The primary motivation was the connection of empirical methods with theoretical robustness guarantees, which requires randomized smoothing-like processing, which is computationally expensive. As shown in Table 14, a smoothed representation improves bit accuracy compared to a naive embedding without averaging.

For practical applications, one can lower the computational budget (e.g., to several hundreds) to make watermarking faster, while still retaining the partial benefit of the randomized representation's robustness to smoothing. We investigated the time-quality trade-off and concluded that part of the computational cost can be reduced without a proportional loss in robustness.

To better characterize the trade-off between computational cost and robustness, we evaluated reduced transformation-sampling budgets for both embedding and extraction.

We evaluated watermark bit accuracy on a subset of test audio clips across 37 attacks. Each embedding step used 1,000 transformations. We varied the maximum number of embedding steps (RR) and the number of transformations used during extraction (kk). Each cell reports mean attacked bit accuracy, with speedup relative to R=700R = 700 and k=1,000k = 1,000 in parentheses. Speedup is based on the mean time to embed from the initial latent and extract the watermark once.

Embedding step limit RRk=0k = 0k=10k = 10k=100k = 100k=1,000k = 1,000
046.62% (8763.90$\times$)47.74% (2296.85$\times$)47.91% (2174.70$\times$)47.77% (995.57$\times$)
1059.83% (35.00$\times$)60.03% (34.63$\times$)60.03% (34.60$\times$)60.00% (33.99$\times$)
5073.48% (7.22$\times$)73.18% (7.20$\times$)73.48% (7.20$\times$)73.31% (7.17$\times$)
70083.18% (1.001$\times$)84.16% (1.001$\times$)84.35% (1.001$\times$)84.10% (1.000$\times$)

R=0R = 0 is the unwatermarked control. At k=0k = 0, the encoder processes the input once without augmentation. Unlike the naive variant in Table 14, k=0k = 0 retains transformation averaging during embedding. Embedding uses the original early-stopping rule, so RR is an upper limit.

At k=1,000k = 1,000, reducing RR from 700 to 50 yielded a 7.17$\times$ speedup, while mean attacked bit accuracy fell from 84.10% to 73.31%. At R=700R = 700, reducing kk from 1,000 to 10 changed accuracy by only 0.06 percentage points and total time by less than 0.1%.

Embedding step limit RSNR (dB)
0 (unwatermarked control, measured)47.80
1013.64
5016.18
70020.23

We also conducted experiments varying both embedding and extraction transformation counts, while keeping the maximum number of embedding steps R=700R = 700.

Embedding transformations \ Extraction501002505001,000
5082.83%82.83%82.90%82.51%82.43%
10083.47%83.51%83.53%83.07%83.10%
25083.79%83.90%83.88%83.52%83.57%
50083.79%83.89%83.93%83.61%83.65%
1,00084.22%84.35%84.28%83.97%84.10%

Attacked bit accuracy remains stable across all transformation-sampling budgets. Across every embedding budget, varying the extraction transformation count between 50 and 1,000 shifts attacked bit accuracy by less than 0.5 points.

Embedding countSNR, dBSTOIMean step-count reduction factor
5020.260.9511.47$\times$
10020.010.9501.33$\times$
25020.060.9481.20$\times$
50020.110.9481.11$\times$
1,00020.230.9461.00$\times$

Lowering the embedding transformation count reduces both the per-step gradient computation cost and the number of required optimization steps, while keeping close values for audio quality metrics. For the experiment with embedding count equal to 50, the mean number of optimization steps is 1.47$\times$ less than the original methods.

Setting the embedding transformation count to 250 and the extraction transformation count to 50 provides an effective operating point, reducing per-clip mean optimization steps by 1.20$\times$ while maintaining 20.06 dB SNR and 83.79% attacked bit accuracy.

AuthorsOpenReview ↗

Response to Zc2X - part 2/3

W2. The results are not very strong in terms of quality

While our method performs not so noticeably better than competitors under regular non-compositional attacks, RRS-AW shows better robustness against compositional attacks and also demonstrates improvements against compression attacks.

To improve the current methods' robustness and audio quality, we modified the optimization process. Instead of Adam optimization, the modified method uses two-stage L-BFGS. The first stage includes 600 warmup updates, followed by up to 300 refinement updates in the second stage. Latent optimization uses 120 augmented views during warmup and 200 during refinement, compared to the original methods' 1000 views per update. On top of that, the modified version uses speed and composite augmentations. The loss adds normalized waveform error and a soft 23.1 dB SNR constraint, reduces the ETV penalty weight from 5000 to 500, and applies waveform energy projection targeting at least 23.05 dB. These changes improve robustness and audio quality.

Metric - full 50-prompt benchmarkOriginalModified method
Mean SNR20.23 dB23.11 dB
Attacked BAC84.10%87.15%
Clean BAC100.00%100.00%
Attacked TPR54.65%60.37%
STOI0.94570.9340
PESQ4.04374.0864
Unwatermarked false positives - shared unchanged decoder0/5120/512

To ensure similar time cost, runtime improvements were introduced, including: grouped encoder evaluation, selective activation recomputation, and FFT-plan retention. So, although the modified version uses a more complex optimization algorithm, the time cost of the method rises by only 20%.

Q1. Line 322: Why were codec attacks not applied? AudioSeal does MP3 as a training augmentation with a straight-through estimator, and thus Table 4 reveals it's much more robust to MP3 (especially low bitrate).

We initially omitted codec augmentations because real encode/decode steps introduce substantial computational overhead into the embedding loop, and our differentiable augmentation suite already achieved 100% bit accuracy on MP3 at 32-128 kbps and >94% on AAC at 32-128 kbps. To address your question, we incorporated MP3 augmentation into sample-specific latent optimization and evaluated two strategies without modifying the extractor::

  • Strategy A: Assigns weight 10/11 to the original 10-family objective and 1/11 to a single-view watermark hinge after real MP3 16 kbps encode/decode at every optimization step.
  • Strategy B: Applies the full original augmentation family after MP3 16 kbps compression on 1 seeded step out of every 11.

Applying MP3 improves bit accuracy at 16 kbps by 13.12% for Strategy A and 10.00% for Strategy B, while maintaining 100% clean bit accuracy and target detection. In addition to that, there is a gain in bit accuracy for other low-bitrate codecs, like AAC and EnCodec. Detection at Hamming distance at most one rose from 3/10 to 5/10 for both strategies; its paired 95% interval includes zero. The downside of this solution comes from audio quality metrics - both SNR and PESQ dropped from 20 dB to 17.31 and 17.60 dB and from 3.94 to 3.52 and 3.70 points respectively.

AuthorsOpenReview ↗

Response to Zc2X - part 3/3

W3, Q2. Can you please explain intuitively how your key scheme works and how Eq. 5 is trying to embed the key information in the representation space?

The index-based key schema was chosen for its simplicity. To investigate whether this particular design is preferable to alternative encoding strategies, we additionally evaluated projection/repetition and Reed-Muller-based schemes, but all of them appeared inferior to the presented one in attacked bit accuracy and detection rate. The comparison between different schemes is shown in the table below:

MethodUntouched payload BACUntouched TPRAttacked payload BACAttacked TPRMean untouched SNR
Original pairwise reference100.00%100.00%84.10%54.65%20.23 dB
Projection/repetition, factor 199.25%98.00%80.48%42.38%20.98 dB
Initial projection/Reed-Muller82.25%54.00%59.23%11.41%24.77 dB
Geometry-matched Reed-Muller98.25%90.00%67.44%28.38%20.09 dB

The main idea of watermark embedding is to define a set of coordinate pairs in the smoothed representation, with one pair assigned to each watermark bit. Rather than directly modifying these coordinates, we optimize the audio/music generator's latent representation zz. Then, the audio vocoded from zz, augmentations are applied to it, and all obtained audio is passed into the audio encoder gg to obtain randomized representation embedding γ^\hat{\gamma}, which should satisfy the bit-dependent pairwise constraints with a margin delta. For example:

  • If w0=0w_0 = 0 and (l0,u0)=(10,80)(l_0, u_0) = (10, 80), we enforce γ10≥γ80+δ\gamma_{10} \geq \gamma_{80} + \delta.
  • If w1=1w_1 = 1 and (l1,u1)=(5,170)(l_1, u_1) = (5, 170), we enforce γ5≤γ170−δ\gamma_5 \leq \gamma_{170} - \delta.

During watermarking extraction/decoding, we have a query audio yy, which, in a similar manner (but without the need to generate audio), we augment and embed using gg to obtain γ^(y)\hat{\gamma}(y) and to extract the watermark, having the secret key (list of pairs).

We expected such strategies would make encoding and decoding be more robust against common corruptions; namely, we augment/smooth the gamma with the expected augmentations. More formally, it is shown in Lemmas 1 and 2. The motivation behind this watermark encoding and decoding approach is based on randomized smoothing (RS) methods for certified robustness on neural networks against additive/parametric perturbations, e.g. it is shown that l(x)=Eϵ∼N(0,σ2)g(x+ϵ)l(x) = \mathbb{E}_{\epsilon \sim \mathcal{N}(0, \sigma^2) } g(x+\epsilon) is a Lipschitz function - thus, the l’s output is linearly bounded by the input’s perturbation. One can vary the smoothing distribution (e.g., Rayleigh instead of normal) or make certification against not only norm-bounded perturbations, but also against some parametric ones, such as x∗αx*\alpha, xαx^{\alpha}, and others and some of their combinations:

These articles might be helpful

  • Cohen, Jeremy, Elan Rosenfeld, and Zico Kolter. "Certified adversarial robustness via randomized smoothing." international conference on machine learning. PMLR, 2019.
  • Li, Linyi, et al. "Tss: Transformation-specific smoothing for robustness certification." Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 2021.
Reviewer_Lp1EOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

his paper proposes RRS-AW, a generative audio watermarking method that uses randomized transformation-averaged representations to improve robustness against audio processing and compression. Experiments demonstrate improved robustness under various attacks, alongside trade-offs in audio quality and computational cost.

Critical strengths
  1. The paper addresses an important problem in generative audio provenance and explores an alternative to conventional post-hoc watermarking. Its representation-based approach offers a distinct perspective on embedding robust watermarks without retraining the generator.
  2. The paper presents a broad empirical evaluation covering multiple audio transformations, codecs, generators, false-positive rates, and human perceptual quality. This provides a useful overall picture of the method’s performance and practical trade-offs.
Critical weaknesses
  1. The trade-off between robustness and audio quality remains unclear. Table 1 shows that RRS-AW achieves an SNR of 20.23 dB and a PESQ score of 4.0433, both lower than AudioSeal and Timbre. In Table 15, RRS-AW also receives a lower mean perceptual rating of 4.48/5 and a lower “Cannot tell” rate of 54% than AudioSeal. Furthermore, Table 2 shows that RRS-AW has lower bit accuracy than AudioSeal or Timbre under several individual transformations. Without comparisons at similar audio quality or distortion levels, it remains unclear how much of the robustness gain comes from the proposed representation itself versus a different robustness-quality trade-off.
  2. The computational cost substantially limits practical applicability. Table 13 reports an average watermark embedding time of 358.63 seconds for a 3-second audio clip, whereas post-hoc baselines typically require only milliseconds. The authors describe their setting as “the watermarking experiments in the music generation setting,” yet the evaluation uses only 3-second clips. Music generation commonly involves audio lasting tens of seconds or even several minutes, and the embedding cost and practical feasibility for longer music remain untested. Such high per-sample optimization costs raise serious concerns about the method’s practicality for long-form music generation and high-throughput applications.
Questions for authors
  1. Does RRS-AW retain its robustness advantage over other methods when audio quality or distortion levels are comparable?
  2. How does varying the quality-loss weight affect the trade-off between watermark robustness and audio quality?
  3. How does embedding time scale to music lasting tens of seconds or several minutes?
  4. Why does RRS-AW achieve a lower overall TPR than TraceMark despite higher bit accuracy under several attacks?
3 comments
AuthorsOpenReview ↗

Response to Lp1E - part 1/3

We thank the reviewer for their detailed feedback on our work. We provide our answers to the weaknesses and questions raised by the reviewer. We address Q1 and Q2 in this comment, Q3 in part 2/3 and Q4 in part 3/3. We realize that the discussion phase itself is coming to an end, and posting our answers shortly before the deadline may feel inconvenient. However, we motivate it by the necessity of additional experiments and text rework. We would appreciate the reconsideration of the assessment of our work if the concerns of the reviewer are addressed sufficiently.

Q1. Does RRS-AW retain its robustness advantage over other methods when audio quality or distortion levels are comparable?

We have conducted additional experiments to evaluate how our method performs at a matched audio quality level. The optimization process was modified to enforce SNR≥26dB\mathrm{SNR} \geq 26 \mathrm{dB}, using the local correction of the proposed latent update and backtracking. We selected a 26 dB SNR restriction value because AudioSeal and Timbre achieve mean SNR of 25.76 and 26.11 dB, respectively. All 50 watermarked audio clips satisfy the constraint. Except for adding optimization constraints, all other hyperparameters remained the same as in the original experiment. We additionally evaluated AudioSeal and Timbre at a mean SNR of 20.23 dB, matching the mean SNR of our method. The table below presents robustness metrics at matched audio quality :

MethodMean SNR (dB)Attacked bit accuracy (%)Attacked TPR (%)
RRS-AW20.2384.1054.65
AudioSeal20.2375.6339.35
Timbre20.2381.2949.14
RRS-AW26.3471.3020.65
AudioSeal25.7674.0534.92
Timbre26.1178.6944.43

Our method retains higher robustness than AudioSeal and Timbre when their mean SNRs are matched to 20.23 dB, while the SNR restricted version of our method loses this advantage at approximately 26 dB.

Q2. How does varying the quality-loss weight affect the trade-off between watermark robustness and audio quality?

We varied the λqual\lambda_{\mathrm{qual}} while keeping all other hyperparameters fixed, the experiment included 5 different values 1250,2500,5000,10000,20000\\{1250, 2500, 5000, 10000, 20000\\}. Increasing the quality weight improves the mean SNR, but reduces the overall robustness of the method. Below are the results of the experiment set for 10 audio samples, including the comparison with competitors on the same 10 audio samples.

Methodλqual\lambda_{\mathrm{qual}}Mean SNR (dB)Clean bit accuracy (%)Clean TPR (%)Attacked bit accuracy (%)Attacked TPR (%)
RRS-AW125012.27100.00100.0087.4363.78
RRS-AW250016.25100.00100.0085.9660.00
RRS-AW500020.09100.00100.0083.1655.41
RRS-AW1000023.9693.7590.0078.0247.57
RRS-AW2000026.5391.8880.0073.7836.49
AudioSeal-27.04100.00100.0074.6837.57
SilentCipher-49.25100.00100.0042.3627.57
WavMark-39.1088.7580.0034.8132.70
Timbre-25.59100.00100.0081.6850.54

Relative to the default weight of 5000, increasing the quality weight to 20000 increases the mean SNR by 6.44 dB, while mean bit accuracy and TPR decrease by 9.38 and 18.92 percentage points, respectively. On the other hand, reducing the quality weight to 1250, increases the mean bit accuracy and TPR by 4.27 and 8.37 percentage points, respectively, at the cost of 7.82 dB mean SNR drop. Relative to the default weight of 5000, STOI decreases by 0.045 at weight 1250 and increases by 0.033 at weight 20000.

Notably, the results of our method with λqual\lambda_{\mathrm{qual}} equal to 20000 show comparable transformation robustness and audio quality to Audioseal. Overall, the observed means show that stronger quality regularization reduces waveform distortion while sacrificing watermark recovery and detection.

AuthorsOpenReview ↗

Response to Lp1E - part 2/3

Q3. How does embedding time scale to music lasting tens of seconds or several minutes?

We agree that computational cost substantially limits practical applicability. The primary motivation was the connection of empirical methods with theoretical robustness guarantees, which requires a randomized smoothing-like process which is computationally expensive.

To evaluate long-form feasibility directly, we extended our evaluation to 30-second music-clips and compared them against our 3-second baseline. Scaling from 3 seconds to 30 seconds required almost the same number of optimization steps (slightly more for 30 seconds), doubled audio generation time, and nearly tripled average watermark extraction time. The most time-consuming operation is watermark embedding, which, unfortunately, takes approximately 5.5 times longer on average. On 30-second audio, clean bit accuracy remains at 100%, while average bit accuracy under attacks is 88.34%, compared to 87.84% for 3-second audio. However, perceptual quality slightly decreases to 17.92 dB SNR, compared with 19.75 dB for short clips.

To better characterize the trade-off between computational cost and robustness, we evaluated reduced transformation-sampling budgets for both embedding and extraction.

We evaluated watermark bit accuracy on a subset of test audio clips across 37 attacks. Each embedding step used 1,000 transformations. We varied the maximum number of embedding steps (RR) and the number of transformations used during extraction (kk). Each cell reports mean attacked bit accuracy, with speedup relative to R=700R = 700 and k=1,000k = 1,000 in parentheses. Speedup is based on the mean time to embed from the initial latent and extract the watermark once.

Embedding step limit RRk=0k = 0k=10k = 10k=100k = 100k=1,000k = 1,000
046.62% (8763.90$\times$)47.74% (2296.85$\times$)47.91% (2174.70$\times$)47.77% (995.57$\times$)
1059.83% (35.00$\times$)60.03% (34.63$\times$)60.03% (34.60$\times$)60.00% (33.99$\times$)
5073.48% (7.22$\times$)73.18% (7.20$\times$)73.48% (7.20$\times$)73.31% (7.17$\times$)
70083.18% (1.001$\times$)84.16% (1.001$\times$)84.35% (1.001$\times$)84.10% (1.000$\times$)

R=0R = 0 is the unwatermarked control. At k=0k = 0, the encoder processes the input once without augmentation. Embedding uses the original early-stopping rule, so RR is an upper limit.

At k=1,000k = 1,000, reducing RR from 700 to 50 yielded a 7.17$\times$ speedup, while mean attacked bit accuracy fell from 84.10% to 73.31%. At R=700R = 700, reducing kk from 1,000 to 10 changed accuracy by only 0.06 percentage points and total time by less than 0.1%.

Embedding step limit RSNR (dB)
0 (unwatermarked control, measured)47.80
1013.64
5016.18
70020.23

We also conducted experiments varying both embedding and extraction transformation counts, while keeping the maximum number of embedding steps R=700R = 700.

Embedding transformations \ Extraction501002505001,000
5082.83%82.83%82.90%82.51%82.43%
10083.47%83.51%83.53%83.07%83.10%
25083.79%83.90%83.88%83.52%83.57%
50083.79%83.89%83.93%83.61%83.65%
1,00084.22%84.35%84.28%83.97%84.10%

Attacked bit accuracy achieves high stability across all transformation-sampling budgets. Across every embedding budget, varying the extraction transformation count between 50 and 1,000 shifts attacked bit accuracy by less than 0.5 percentage points. Extraction budgets of 50 and 100 achieve similar or slightly higher mean attacked bit accuracy than 1000 across the tested embedding budgets.

Embedding countSNR, dBSTOIMean step-count reduction factor
5020.260.9511.47$\times$
10020.010.9501.33$\times$
25020.060.9481.20$\times$
50020.110.9481.11$\times$
1,00020.230.9461.00$\times$

Reducing the embedding transformation budget yields reductions in optimization overhead while keeping close values for audio quality metrics. Lowering the embedding transformation count reduces both the per-step gradient computation cost and the mean number of optimization steps required to reach the target watermark loss - for the experiment with embedding count equal to 50, the mean number of optimization steps is 1.47$\times$ less than the original methods. Setting the embedding transformation count to 250 and the extraction transformation count to 50 provides an effective operating point, reducing per-clip mean optimization steps by 1.20$\times$ while maintaining 100.0% untouched detection, high perceptual fidelity at 20.06 dB SNR and 0.948 STOI, and an attacked bit accuracy of 83.79%.

AuthorsOpenReview ↗

Response to Lp1E - part 3/3

Q4. Why does RRS-AW achieve a lower overall TPR than TraceMark despite higher bit accuracy under several attacks?

This difference arises because bit accuracy and TPR capture different aspects of watermark detection. RRS-AW can achieve higher average bit accuracy due to a larger number of samples with highly accurate bit recovery, while some samples still fall below the detection threshold. In contrast, TraceMark may produce more samples with bit accuracy only slightly above the threshold, resulting in a higher TPR despite lower average bit accuracy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.