- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RRS-AW, an audio watermarking framework that embeds multi-bit messages into randomized transformation-averaged audio representations during generation. The method uses a pretrained auxiliary encoder to extract representations from randomly transformed versions of an audio signal and averages these representations to improve stability under perturbations. Binary messages are encoded through the relative ordering of secret pairs of representation coordinates.
RRS-AW keeps the pretrained audio generator and auxiliary encoder fixed, and embeds watermarks by optimizing the generation latent for each audio sample. The paper also presents theoretical robustness bounds relating representation perturbations to potential bit errors.
Experiments are primarily conducted on AudioLDM-generated music, with additional evaluations on other audio generators. The authors compare RRS-AW with several existing audio watermarking methods under individual attacks, composite transformations, and audio codec distortions. The results show competitive bit extraction accuracy, particularly under composite and codec attacks. The paper also includes audio quality evaluations, false-positive analysis, computational cost measurements, and an ablation study of randomized representation averaging.
Critical strengths
S1. A well-motivated representation-level approach to robust multi-bit audio watermarking.
The paper presents an interesting approach that embeds watermark information through pairwise ordering constraints in transformation-averaged audio representations. Rather than requiring the individual representation coordinates to remain unchanged, the method aims to preserve their relative ordering under perturbations. This is a reasonable design for improving robustness and allows multi-bit payloads to be embedded without retraining the pretrained generator or auxiliary encoder.
S2. Broad empirical evaluation and a meaningful ablation of the proposed smoothing mechanism.
The experiments consider individual, composite, and codec-based transformations, together with additional audio generators. The authors also evaluate audio quality, false positives, and computational costs. Importantly, the ablation in Table 14 isolates randomized representation averaging while keeping the other major components unchanged. The reported mean bit accuracy across attacked conditions increases from 0.7449 without averaging to 0.8413 with the complete method, providing direct empirical evidence for the usefulness of the main proposed mechanism.
Critical weaknesses
W1. The reported improvements in bit accuracy do not consistently translate into reliable watermark detection, while the method incurs substantial computational and audio-fidelity costs.
Although RRS-AW achieves competitive bit accuracy under many transformations, the corresponding watermark detection performance is considerably less convincing. According to Table 6, the method achieves an average TPR of only 0.5459 across all attacked conditions, with particularly low TPRs of 0.4062 under composite attacks and 0.3680 under EnCodec compression. TraceMark achieves a slightly higher overall attacked TPR of 0.5680.
This distinction is important because the paper motivates audio watermarking primarily through content authentication and provenance verification. High average bit accuracy does not necessarily translate into reliable authentication under the adopted detection rule.
Furthermore, different baselines are evaluated using method-specific detection rules and thresholds, including different Hamming-distance thresholds and native detection gates. This complicates direct comparisons of detection performance. The empirical false-positive evaluation is also limited to 512 natural audio recordings, on which several methods achieve zero observed false positives. Consequently, the paper does not sufficiently establish a detection advantage under a common false-positive operating point. These concerns are especially relevant given the substantial computational and audio-quality costs of the proposed method. Table 13 reports an average embedding time of approximately 358.63 seconds for a three-second audio sample, compared with milliseconds for the post-hoc baselines. Table 1 also shows lower signal-level fidelity than several competing methods, with an SNR of 20.23 dB. The authors acknowledge the computational and quality limitations, and the robustness improvements in bit accuracy are meaningful. Nevertheless, the current evidence does not sufficiently demonstrate that the practical detection benefits justify the substantial computational overhead and fidelity trade-off.
W2. The theoretical robustness analysis is insufficiently connected to the implemented normalization procedure and the empirical operating regime.
A central contribution of the paper is its theoretically motivated robustness analysis. However, the relationship between the theoretical formulation and the implemented watermark embedding procedure requires further clarification.
In Algorithm 1, the estimated representation is explicitly normalized by dividing it by its L2 norm, and the watermark loss is evaluated using this normalized representation. In contrast, Eq. (5) defines the watermark loss using the unnormalized representation, and Lemma 2 derives its bit-flip guarantee from a minimum coordinate margin of delta in this unnormalized representation.
If the optimization is indeed performed on the normalized representation, achieving zero watermark loss implies a margin of at least delta in normalized coordinates. However, the corresponding margin in the unnormalized representation is at least delta multiplied by the L2 norm of the original representation. Without an additional assumption or bound on this norm, the minimum unnormalized margin of delta used in the theoretical argument does not follow directly.
Although positive normalization preserves the coordinate ordering used for decoding, it does not preserve the absolute coordinate margins that determine the stated robustness condition. Therefore, the applicability of Lemma 2 and Eq. (11) to the actual implementation remains unclear.
Moreover, the theoretical guarantee depends on Lipschitz and perturbation-sensitivity bounds. The appendix reports substantial local sensitivity, including an observed maximum of approximately 2.55 × 10^5. However, the paper does not demonstrate whether the sufficient condition in Eq. (11) is satisfied for representative perturbations at the experimental watermark margin.
The theoretical analysis also explicitly assumes parametric transformations of the same type as those used during embedding, whereas codec attacks are evaluated empirically without being included in latent optimization.
These observations do not invalidate the experimental robustness results. However, they weaken the connection between the theoretical contribution, the implemented method, and the practical attack settings evaluated in the paper.
Questions for authors
Q1. Detection performance under a unified false-positive operating point
Table 6 reports a mean attacked TPR of 0.5459 for RRS-AW, despite its relatively high bit accuracy. Furthermore, the compared methods use different detection thresholds and method-specific decision rules.
Could the authors provide detection performance at matched false-positive rates, or report ROC curves for the main competing methods? How does RRS-AW compare with the baselines under a common detection operating point?
Additionally, could the authors clarify whether the negative-set evaluation includes unwatermarked outputs from the target generator and other audio generators, beyond the 512 natural recordings reported in Table 5?
Q2. Consistency between normalized embedding and the theoretical margin
Algorithm 1 computes the watermark loss on the normalized representation , whereas Eq. (5) and Lemma 2 formulate the embedding margin using the unnormalized representation .
Which representation is actually used to compute the watermark loss in the implementation?
If the normalized representation is used, how is the minimum unnormalized coordinate margin of in Lemma 2 justified? Does the theoretical result require an additional bound on , or should the robustness analysis instead be formulated directly for the normalized representation?
Q3. Practical significance of the theoretical robustness bound
Could the authors provide a numerical evaluation of the sufficient robustness condition in Eq. (11) under representative experimental transformations, using the actual embedding margin and suitable sensitivity estimates?
In particular, does the bound yield a non-vacuous guarantee for any of the evaluated perturbation strengths? If not, what specific practical robustness property is established by the theoretical analysis beyond the general relationship between representation perturbations and bit flips?
Q4. Computational cost and robustness trade-off
Table 13 reports an average embedding time of approximately 358.63 seconds for a three-second audio sample. The appendix evaluates reduced transformation-sampling budgets, but does not fully characterize the corresponding trade-off between runtime, watermark detection, and audio quality.
Could the authors clarify how the embedding time, attacked TPR, bit accuracy, and audio quality change when the number of sampled transformations and the optimization budget are reduced?
Such results would help determine whether the reported robustness gains can be retained at a computational cost more suitable for practical audio generation.
3 comments
Response to 46mn - part 2/3
Q2. Consistency between normalized embedding and the theoretical margin
We are grateful for spotting the inconsistency and confirm that the method we propose uses the normalized version presented in Algorithm 1. Both the experimental results presented in the main text of the article and the code provided in the supplementary materials use this normalized implementation. Equation 5 and Lemma 2 use notation from an earlier, unnormalized version of the algorithm, which was included by mistake. In the final version of the paper, we will update both to align with Algorithm 1. We indicate that no additional theoretical derivations are needed for the unnormalized sample mean representation (since the ones are already included in assumptions on Lipschitz continuity).
Q3. Practical significance of the theoretical robustness bound
We want to clarify that the theoretical robustness bound explicitly connects together three entities: the quantitative robustness of the auxiliary encoder function, the methodological property of the proposed watermarking scheme (namely, that changes in the empirical vector representation yield detection of the watermarking signal after post-processing), and the number of bit flips that can occur in the extracted message under watermark removal perturbations. Although the encoder tested in our experiments does not necessarily have nice Lipschitz properties, we speculate that the ones can be added at zero-cost (for example, via using 1-Lipschitz networks).
For some perturbations, the deterministic margin condition in Lemma 2 guarantees zero bit errors for the measured representations. Lemma 2 gives a bit-specific guarantee. Let be the original margin of bit , and let be the largest absolute change in any coordinate of the normalized representation after perturbation. Each bit is encoded by the ordering of two coordinates, so their difference can change by at most . Every bit with therefore retains its original value.
We tested one clip whose normalized aggregate had zero watermark loss in all ten independent reference extractions. The smallest bit margin across these extractions was 0.23287. Under additive Gaussian noise with standard deviation , the largest observed was 0.18152 across 50 comparisons (five noise realizations and ten extractions per realization). Since , the predicted upper bound was zero bit errors in every comparison. All 16 bits were recovered in all 50 cases. A 1% volume reduction also satisfied the condition in all ten comparisons, with and zero observed errors.
For Eq. (11), the estimated concentration term alone was 0.24288 for Gaussian noise and 0.19175 for volume reduction. Both exceeded the threshold before adding the sensitivity term. Thus, the positive result above applies to the measured representation pairs. Failure of the sufficient condition does not imply that any bit must flip, let alone that all bits will flip. Even at the stronger Gaussian noise level of , the decoder recovered 83.13% of bits on the same clip.
We also want to address the question related to absence of codec transformation during embedding of the watermark. We initially omitted codec augmentations because real encode/decode steps introduce substantial computational overhead into the embedding loop, and our differentiable augmentation suite already achieved 100% bit accuracy on MP3 at 32-128 kbps and >94% on AAC at 32-128 kbps. To address your question, we incorporated MP3 augmentation into sample-specific latent optimization and evaluated two strategies without modifying the extractor:
- Strategy A: Assigns weight 10/11 to the original 10-family objective and 1/11 to a single-view watermark hinge after real MP3 16 kbps encode/decode at every optimization step.
- Strategy B: Applies the full original augmentation family after MP3 16 kbps compression on 1 seeded step out of every 11.
Applying MP3 improves bit accuracy at 16 kbps by 13.12% for Strategy A and 10.00% for Strategy B, while maintaining 100% clean bit accuracy and target detection. In addition to that, there is a gain in bit accuracy for other low-bitrate codecs, like AAC and EnCodec. Detection at Hamming distance at most one rose from 3/10 to 5/10 for both strategies; its paired 95% interval includes zero. The downside of this solution comes from audio quality metrics - both SNR and PESQ dropped from 20 dB to 17.31 and 17.60 dB and from 3.94 to 3.52 and 3.70 points respectively.
Response to 46mn - part 3/3
Q4. Computational cost and robustness trade-off
To better characterize the trade-off between computational cost and robustness, we evaluated reduced transformation-sampling budgets for both embedding and extraction.
We evaluated watermark bit accuracy on a subset of test audio clips across 37 attacks. Each embedding step used 1,000 transformations. We varied the maximum number of embedding steps () and the number of transformations used during extraction (). Each cell reports mean attacked bit accuracy, with speedup relative to and in parentheses. Speedup is based on the mean time to embed from the initial latent and extract the watermark once.
| Embedding step limit | ||||
|---|---|---|---|---|
| 0 | 46.62% (8763.90$\times$) | 47.74% (2296.85$\times$) | 47.91% (2174.70$\times$) | 47.77% (995.57$\times$) |
| 10 | 59.83% (35.00$\times$) | 60.03% (34.63$\times$) | 60.03% (34.60$\times$) | 60.00% (33.99$\times$) |
| 50 | 73.48% (7.22$\times$) | 73.18% (7.20$\times$) | 73.48% (7.20$\times$) | 73.31% (7.17$\times$) |
| 700 | 83.18% (1.001$\times$) | 84.16% (1.001$\times$) | 84.35% (1.001$\times$) | 84.10% (1.000$\times$) |
is the unwatermarked control. At , the encoder processes the input once without augmentation. Unlike the naive variant in Table 14, retains transformation averaging during embedding. Embedding uses the original early-stopping rule, so is an upper limit.
At , reducing from 700 to 50 yielded a 7.17$\times$ speedup, while mean attacked bit accuracy fell from 84.10% to 73.31%. At , reducing from 1,000 to 10 changed accuracy by only 0.06 percentage points and total time by less than 0.1%.
| Embedding step limit R | SNR (dB) |
|---|---|
| 0 (unwatermarked control, measured) | 47.80 |
| 10 | 13.64 |
| 50 | 16.18 |
| 700 | 20.23 |
We also conducted experiments varying both embedding and extraction transformation counts, while keeping the maximum number of embedding steps .
| Embedding transformations \ Extraction | 50 | 100 | 250 | 500 | 1,000 |
|---|---|---|---|---|---|
| 50 | 82.83% | 82.83% | 82.90% | 82.51% | 82.43% |
| 100 | 83.47% | 83.51% | 83.53% | 83.07% | 83.10% |
| 250 | 83.79% | 83.90% | 83.88% | 83.52% | 83.57% |
| 500 | 83.79% | 83.89% | 83.93% | 83.61% | 83.65% |
| 1,000 | 84.22% | 84.35% | 84.28% | 83.97% | 84.10% |
Attacked bit accuracy remains stable across all transformation-sampling budgets. Across every embedding budget, varying the extraction transformation count between 50 and 1,000 shifts attacked bit accuracy by less than 0.5 percentage points.
| Embedding transformations \ Extraction | 50 | 100 | 250 | 500 | 1,000 |
|---|---|---|---|---|---|
| 50 | 50.65% | 50.54% | 50.70% | 50.59% | 50.65% |
| 100 | 51.19% | 51.41% | 51.62% | 51.46% | 51.62% |
| 250 | 53.03% | 53.24% | 53.24% | 53.35% | 53.62% |
| 500 | 53.62% | 53.57% | 53.84% | 53.62% | 53.84% |
| 1,000 | 54.43% | 54.38% | 54.32% | 54.22% | 54.65% |
The table above shows little variation in mean attacked TPR across extraction budgets. Reducing extraction transformations from 1000 down to 50 results in a performance shift of less than 0.6 percentage points across all embedding configurations. This indicates that watermark extraction and detection can be made faster in practice while preserving similar observed bit accuracy and TPR.
| Embedding count | SNR, dB | STOI | Mean step-count reduction factor |
|---|---|---|---|
| 50 | 20.26 | 0.951 | 1.47$\times$ |
| 100 | 20.01 | 0.950 | 1.33$\times$ |
| 250 | 20.06 | 0.948 | 1.20$\times$ |
| 500 | 20.11 | 0.948 | 1.11$\times$ |
| 1,000 | 20.23 | 0.946 | 1.00$\times$ |
Lowering the embedding transformation count reduces both the per-step gradient computation cost and the mean number of required optimization steps, while keeping close values for audio quality metrics. For the experiment with embedding count equal to 50, the mean number of optimization steps is 1.47$\times$ less than the original methods.
Setting the embedding transformation count to 250 and the extraction transformation count to 50 provides an effective operating point, reducing per-clip mean optimization steps by 1.20$\times$ while maintaining 100.0% untouched detection, 20.06 dB SNR, an attacked TPR of 53.03%, and an attacked bit accuracy of 83.79%.
Response to 46mn - part 1/3
We thank the reviewer for their detailed feedback on our work. Below, we provide our answers to the weaknesses and questions raised by the reviewer. We address Q1 in this comment, Q2 and Q3 in part 2/3, and Q4 in part 3/3. We realize that the discussion phase itself is coming to an end, and posting our answers shortly before the deadline may feel inconvenient. However, we motivate it by the necessity of additional experiments and revisions. We would appreciate the reconsideration of the assessment of our work if the concerns of the reviewer are addressed sufficiently.
Q1. Detection performance under a unified false-positive operating point
For RRS-AW, AudioSeal, SilentCipher, WavMark, and Timbre, payload-matching thresholds are selected using the same criterion: the largest Hamming-distance threshold whose false-match probability under independent, uniformly random bits is below (10−3). This gives (H≤1) for 16-bit payloads, (H≤4) for SilentCipher’s 24-bit comparison, and exact matching for Timbre’s 10-bit payload. The numerical thresholds therefore differ with payload length. Native detector gates also remain method-specific. The original negative-set evaluation used 512 independent natural recordings from the Freesound dataset. We additionally evaluated 450 unwatermarked outputs from the target AudioLDM generator, obtaining:
To improve the current methods' robustness and audio quality, we modified the optimization process. Instead of Adam optimization, the modified method uses two-stage L-BFGS. The first stage includes 600 warmup updates, followed by up to 300 refinement updates in the second stage. Latent optimization uses 120 augmented views during warmup and 200 during refinement, compared to the original methods' 1000 views per update. On top of that, the modified version uses speed and composite augmentations. The loss adds normalized waveform error and a soft 23.1 dB SNR constraint, reduces the ETV penalty weight from 5000 to 500, and applies waveform energy projection targeting at least 23.05 dB. These changes improve robustness and audio quality.
To ensure similar time cost, runtime improvements were introduced, including: grouped encoder evaluation, selective activation recomputation, and FFT-plan retention. So, although the modified version uses a more complex optimization algorithm, the time cost of the method rises by only 20%.