acceptodds
Under review as a conference paper at ICLR 2027

MARC: Multi-Bit Watermarking for Autoregressive Audio Generation against Codec Attacks

Abstract

Generated audio is now used in a range of applications, creating a need to verify its origin after distribution and signal processing. This task is particularly challenging for autoregressive audio generation because codec processing can alter the token sequence recovered from the waveform. Such changes reduce the reliability of watermark detection and payload decoding. Existing methods construct token groups using either intrinsic token representations or substitution patterns caused by transformations. As a result, intrinsic token relationships and codec induced substitutions are modeled separately. In addition, most methods support only zero bit detection. They can determine whether a watermark is present but cannot distinguish individual generated outputs. We propose MARC, a multi-bit generative watermarking method for autoregressive audio generation. MARC integrates intrinsic token representations with confusion patterns obtained through retokenization and multiple codecs, forming a codec-aware token-cluster space. Within this space, payload-driven cluster scheduling is used to embed a multi-bit watermark, while detection and payload decoding are performed on retokenized observations. Experiments on speech, dialogue, and music generation show that MARC achieves an average of 97.3% bit extraction accuracy on unmodified watermarked audio and the watermark can still be extracted under diverse codec attacks. MARC also demonstrates robustness to overwriting attacks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio Watermarking, Autoregressive Audio Generation, Multi-bit Watermarking

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BQvFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents MARC, a multi-bit watermarking scheme for autoregressive audio generation. It tackles token substitution and temporal desynchronization caused by the audio detokenization, compression, and retokenization pipeline. To address this, MARC groups tokens into clusters by combining embedding similarity with cross-codec confusion patterns so that mutually confusable tokens share the same cluster.

Critical strengths
  • Beyond watermark detection, the paper addresses how to recover multi-bit payloads after audio re-encoding. It separates token substitution from temporal misalignment, making the research problem much better defined than simply boosting detection rates on clean audio.

  • The evaluation covers speech, dialogue, and music generation across 21 audio perturbations and watermark overwriting attacks. This helps delineate the method's effective operating regime.

Critical weaknesses
  • Section 4.3 states that the verifier takes a candidate message m⋆m^\star as input to guide the alignment search, and the match-rate parameter during decoding is also computed using m⋆m^\star. The reported results reflect recovery performance aided by a candidate message and auxiliary checksum information, rather than blind message extraction solely from the audio without prior knowledge of the payload. This paper needs to clarify how much these auxiliary inputs actually contribute to the reported accuracy.

  • "The paper states that during verification, the verifier can use the provided attack type and processing parameters to construct waveform variants. It is necessary to distinguish between recovery when the audio processing history is known and blind recovery when the transformation history is completely unknown. The current presentation does not clearly specify which codecs are used during cluster construction versus which test codecs are held out as unseen, leaving the method's generalization to unknown codecs unclear.

Questions for authors
  • How much information can still be recovered without providing the ground-truth candidate message? Could you remove the guidance of m⋆m^\star from both alignment and scoring, and report the bit accuracy and full-message recovery rate before and after checksum correction? Also, since the default payload is only 12 bits, how much information can a trivial enumeration baseline recover using only the public checksum and secret suffix without reading the audio at all?

  • How is the FPR calibrated for the full detection pipeline? After searching over multiple windows, offsets, and waveform variants, is the final decision threshold recalibrated to account for multiple testing? Could you compare the TPR across all methods on the same negative sample set at a fixed FPR, rather than only comparing −log⁡(p)-\log(p)?

  • How does the method perform when the attack type and parameters are unknown? Could you provide unified decoding results without using attack labels, and clearly separate the codecs seen during cluster construction from unseen test codecs? What is the computational time overhead required for this search?

7 comments
AuthorsOpenReview ↗

Response to Reviewer BQvF (Part 1/4)

Dear Reviewer BQvF,

Thank you for emphasizing the connection between codec robustness, payload recovery, and the verification protocol. We explain the receiver step by step and provide common-pipeline detection, construction-exposure, and computational evidence.

W1 & Q1: Clarify extraction, auxiliary information, and payload recovery.

Recovery uses the received waveform and registered keys, without a supplied ground-truth message. Section 4.3 now makes this procedure explicit. The receiver applies the same fixed steps to every input:

  1. Observed evidence: encode the received waveform into tokens, cluster observations, and context seeds.
  2. Candidate-wise alignment: score all 16 symbol values at each position over fixed windows and offsets, comparing candidate target clusters with observed clusters.
  3. Repeated-observation scoring: aggregate hard matches and bounded soft compatibility across laps. Pub-key-compatible audio-supported candidates supply detection features.
  4. Presence decision: compare scores with frozen unwatermarked references and apply the branch-union decision in Eq. (6).
  5. Payload selection: rank candidates using their own compatibility estimate, audio evidence, pri-key consistency, and the pub-key bonus in Eq. (7). Emit the message after final key validation.

The estimated rate η^a(m)\widehat\eta_a(m) is computed from the candidate's audio support, not agreement with the embedded reference. The pri-key constrains combinations and supplies consistency evidence. The pub-key supplies a fixed ranking bonus and final validation. The reference bitstring is compared only after the output is fixed. No attack label or actual processing parameter is an input to this procedure.

Auxiliary-information contribution. Thank you for asking us to distinguish the contribution of auxiliary validation. D3 in revised Table 4 isolates the pub-key ranking contribution: removing its bonus reduces bit accuracy from 98.17% to 80.75%, while TPR remains 94% and the cluster statistic remains 37.08. Section 4.3 distinguishes candidate selection from final validation. Audio evidence and key consistency determine the selected candidate. The final check determines whether that candidate is emitted. This comparison quantifies the benefit to payload selection separately from detection performance. Section 5.5 explains the corresponding recovery mechanism.

Role of the metadata-only baseline. Given both registered values, enumeration of the short 12-bit space can identify the registered target. The verification task also requires audio evidence supporting that target: identifying a message from metadata is not a positive watermark decision for the received waveform. The complete multi-bit verification protocol combines audio-evidence screening and key-assisted recovery, with an observed 0.60% complete-search FPR on unwatermarked inputs. We therefore describe the protocol as registered-key-assisted extraction and verification. Confidentiality, when required, is a deployment-layer property of the selected encryption/access-control mechanism and key management. Section 3 specifies the watermark threat model separately from cryptographic secrecy.

Revision: Section 4.3 now explicitly defines the candidate-specific rate, audio score, two key roles, and final validation in Eq. (7). Table 4 and Section 5.5 add the public-check ranking ablation.

AuthorsOpenReview ↗

Response to Reviewer BQvF (Part 2/4)

W2: Construction codecs and unseen test codecs.

Section 4.1 distinguishes ordinary retokenization from codec-processed confusion and combines them in the codec union. The test suite includes reconstruction methods not used to construct the cluster map. The verified model construction split is:

Response Table R1. Construction exposure.

TransformationUsed in the cluster construction
Ordinary model retokenization✓
MP3✓
Opus✓
EnCodec✓
DAC✓
External SpeechTokenizer×
FACodec×
SoCodec×
SemantiCodec×
VoiceCraft–EnCodec pipeline×

Ordinary model retokenization differs from the external SpeechTokenizer attack. VoiceCraft–EnCodec is held out as a reconstruction pipeline while sharing the EnCodec component with a construction codec. This makes the exposure definition precise at the codec/pipeline level.

Main Table 3 reports 98% bit accuracy under FACodec and 97% under SemantiCodec on SpiritLM/FinanceQA. Appendix A.2, Tables 9–11 further evaluate codec parameters and waveform perturbations. These results support transfer beyond the construction transformations. The construction split concerns offline representation learning. The same receiver policy is used at test time without being told which transformation was applied.

AuthorsOpenReview ↗

Response to Reviewer BQvF (Part 3/4)

Q2: Complete-pipeline FPR and comparable detection metrics.

We have made the metric choice and its operating-point interpretation explicit in Section 5.1.

Why report −log⁡(clusterp)-\log(\mathrm{cluster_p})? In the evaluated 1%-target-FPR condition, the cluster detection TPR is already 100% for all four methods. Thresholded TPR therefore has little remaining ability to distinguish their detection evidence. Following HiPT [1], −log⁡(clusterp)-\log(\mathrm{cluster_p}) provides an additional description of how strongly the observed token/cluster statistics support watermarking.

Why also report multi-bit TPR? Following VINE [2], the multi-bit score is the fraction of extracted bits matching the target message. It evaluates recovered-message agreement, beyond detecting a statistical watermark signal. The zero-bit baselines output no dd-bit payload estimate, so this particular message-matching TPR cannot be computed from their native outputs. Their ordinary presence-detection performance is compared at the statistical-detection level, as shown below.

Response Table R2. Distinct evaluation quantities.

MetricComputationPurpose
Cluster TPRFraction of watermarked samples passing the statistical detector's thresholdPresence detection
−log⁡(clusterp)-\log(\mathrm{cluster_p})Negative log of the median cluster p-valueDistinguish evidence when thresholded TPR saturates
Multi-bit TPRFraction whose extracted-bit matching score passes its target-FPR thresholdMessage verification
bitacc\mathrm{bit_acc}Average fraction of correctly recovered bitsPayload recovery quality

Response Table R3. Saturated statistical detection in the evaluated setting.

MethodTarget FPRCluster TPR
AR speech WM1%100%
Aligned-IS1%100%
HiPT1%100%
MARC1%100%

All methods in Table R3 use the same unwatermarked evaluation set, with method-specific thresholds calibrated at the common 1% target FPR.

For the VINE-style score u(x,m)=d−1∑i1[m^i(x)=mi]u(x,m)=d^{-1}\sum_i\mathbf1[\widehat m_i(x)=m_i], matching TPR and FPR are the fractions of watermarked and unwatermarked inputs satisfying the same rule u≥τu\ge\tau. The extracted message is fixed before its reference is used to compute this score. Thus, multi-bit TPR counts accepted samples, whereas bitacc\mathrm{bit_acc} averages bit agreement.

Complete-search calibration. Section 4.3 uses the same feature construction for received inputs and frozen null references, and Eq. (6) defines the final OR decision. The independent measurement is:

Response Table R4. Empirical FPR after the complete search.

MethodUnwatermarked inputsFalse positivesEmpirical FPR
MARC50030.60%

The final count includes every enabled branch, waveform variant, window, and offset. Table R4 reports the empirical FPR of the complete multi-bit verification protocol on unwatermarked inputs. The same fixed verification rule is applied to watermarked inputs to measure the corresponding multi-bit TPR. The 1% target and 0.60% empirical test rate are consistent: the target specifies the operating level, while the independent test count reports the frozen protocol's observed behavior. Table R3 separately concerns cluster/statistical detection.

Revision: Section 5.1 now states why cluster TPR saturation motivates −log⁡(clusterp)-\log(\mathrm{cluster_p}), cites VINE for message-matching verification, and explains why the zero-bit baselines have no corresponding payload-matching output.

References

[1] Georgios Milis, Yubin Qin, Yihan Wu, and Heng Huang. Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audio. ICML 2026.

[2] Shilin Lu, Zihan Zhou, Jiayou Lu, Yuanzhi Zhu, and Adams Wai-Kin Kong. Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances. ICLR 2025.

Reviewer_LBXoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MARC, a multi-bit generative watermarking framework for autoregressive audio generation that aims to improve robustness against codec-induced token substitutions. The method combines intrinsic token embedding similarities with empirical confusion patterns collected through retokenization and multiple audio codecs to construct a codec-aware token-cluster representation.

MARC embeds a 12-bit payload through cluster-based probability reweighting and a repeated, payload-driven symbol schedule. For watermark verification and recovery, it employs alignment search, candidate scoring, and aggregated decoding to mitigate token substitutions and temporal misalignment.

The method is evaluated on SpiritLM and MusicGen across dialogue, speech, and music generation tasks. Experiments include codec and neural reconstruction attacks, conventional signal-processing distortions, desynchronization, and watermark overwriting. The authors report high bit extraction accuracy and favorable detection statistics compared with existing zero-bit watermarking methods. Component ablations are also provided to investigate the effects of clustering, embedding schedules, and decoding mechanisms.

Critical strengths

S1. A meaningful and well-motivated problem.

The paper addresses a practically relevant challenge in autoregressive audio watermarking. Codec processing can change the token sequence recovered from a waveform even when its perceptual content is largely preserved. Extending token-level watermarking from zero-bit detection to multi-bit payload recovery is meaningful for applications that require distinguishing individual generated outputs. The motivation is clear, and the proposed codec-aware representation provides a reasonable approach to addressing this challenge.

S2. Broad robustness evaluation with useful component ablations.

The evaluation covers two autoregressive audio generation models, multiple generation tasks, and a diverse set of audio transformations. The authors also investigate overwriting attacks and provide separate ablations for clustering, embedding schedules, and decoding. These experiments offer useful evidence regarding the behavior of the proposed framework and the contributions of its major components.

Critical weaknesses

W1. The evaluation protocol does not sufficiently establish independent multi-bit payload recovery.

My primary concern is the role of the claimed payload m⋆m^\star during watermark verification and decoding.

Section 3 explicitly states that the verifier uses m⋆m^\star to search over candidate alignments before estimating the payload. Section 4.3 further specifies that the expected cluster sequence used for alignment scoring is determined by m⋆m^\star, and that the match rate against m⋆m^\star is incorporated into the decoding procedure.

Although the decoder subsequently enumerates possible symbol values, the selection and scoring of candidate observations are conditioned on a supplied payload hypothesis. Crucially, the paper does not clearly explain how m⋆m^\star is obtained in the experiments, whether it is set to the ground-truth embedded message, or how recovery performs when the embedded message is unknown.

This distinction is consequential. Verification of a claimed payload and independent extraction of an unknown payload are different tasks, especially when the claimed payload influences alignment selection. Without a clearly specified evaluation protocol, it is difficult to determine whether the reported results support the paper's broader claims regarding multi-bit recovery and traceability of individual generated outputs.

I therefore consider the insufficiently specified evaluation protocol and the unclear distinction between claimed-payload verification and independent payload recovery to be a major weakness, particularly given the paper's claims regarding multi-bit watermark recovery.

W2. The independent contribution of codec-aware clustering is not sufficiently established, and inconsistencies in the ablation results complicate its assessment.

The paper provides useful clustering and embedding ablations. In particular, Table 5 compares multiple clustering strategies under a fixed cluster count and construction-data budget. However, the reported improvements in average post-codec bit accuracy are relatively modest. The accompanying text reports 97.77% for the complete clustering method, compared with 97.24% for GMM with retokenization and 97.41% for geometric GMM.

In contrast, Table 7 reports a much larger performance decrease when the lap pipeline and HiPT grid search are jointly removed, reducing the average post-codec bit accuracy from 97.77% to 74.57%. These results indicate that alignment-aware decoding and repeated observations make substantial contributions to the final performance. While the clustering ablations provide some supporting evidence, the paper does not sufficiently quantify the statistical reliability of the relatively small improvements attributed to codec-aware clustering.

More importantly, there are inconsistencies in the reported ablation results. Table 5(b) reports 95.00% bit accuracy for the complete method under Opus at 16 kbps, whereas Tables 6(b) and 7(b) report 99.00% for the corresponding full configuration. The values in Table 5(b), taken literally, yield an average of approximately 97.27%, rather than the 97.77% reported in the text. This discrepancy directly affects the magnitude of the claimed clustering improvement.

In addition, the main baselines are zero-bit watermarking methods. The paper lacks end-to-end, capacity-matched multi-bit comparisons using comparable embedding and decoding protocols. Such comparisons are important for distinguishing the benefit of the new clustering representation from that of the overall multi-bit framework.

Finally, Section 4.1 states that the GMM and spectral-clustering centers are combined in their returned index order. Since cluster labels are arbitrary up to permutation, the theoretical justification and stability of this center-fusion procedure remain unclear.

Taken together, these issues weaken the experimental and methodological evidence for the paper's central representation-level contribution.

Questions for authors

Q1. Payload hypothesis and blind recovery

How is the claimed payload m⋆m^\star obtained in the evaluation? Is it set to the actual embedded payload? If so, could the authors report recovery results when the verifier knows the appropriate keys but is not provided with the ground-truth payload?

Additionally, how does recovery accuracy change when the supplied payload hypothesis is incorrect?

Q2. Contribution of codec-aware clustering

Could the authors provide multi-bit baselines?

Given the relatively small improvements reported in the clustering ablations, could the authors also provide results across multiple random seeds or confidence intervals to establish the reliability of these gains?

Q3. Validity of cluster-center fusion

Section 4.1 states that the GMM and spectral-clustering centers are blended according to their returned index order. Since cluster labels are permutation-invariant, what justifies directly combining centers with the same numerical index?

How sensitive are the resulting cluster assignments and watermark performance to different initializations or permutations of cluster labels?

Q4. False positive rate and attack-aware verification

Section 4.3 describes a search over multiple candidate offsets, observation windows, and waveform variants. How is the detection threshold calibrated to achieve the reported FPR of 1% over the complete search procedure?

The paper also states that the evaluation can use supplied attack types and processing parameters to construct waveform variants. Were such parameters provided during the reported robustness experiments? If so, how does the method perform when the attack type and parameters are unknown?

Q5. Inconsistencies in the ablation results

Could the authors reconcile the numerical inconsistencies in Appendix A.1?

Specifically, Table 5(b) reports 95.00% bit accuracy for the full method under Opus at 16 kbps, whereas Tables 6(b) and 7(b) report 99.00%. Using the values in Table 5(b) gives an average bit accuracy of approximately 97.27%, rather than the reported 97.77%.

Additionally, the text describing the clustering ablation reports a clean bit accuracy of 98.17%, whereas Table 5(a) reports 99.17%.

Which values are correct, and do any corrections affect the conclusions regarding the benefit of codec-aware clustering?

8 comments
AuthorsOpenReview ↗

Response to Reviewer LBXo (Part 1/3)

Dear Reviewer LBXo,

Thank you for recognizing the practical motivation and breadth of the evaluation. The revision separates the representation contribution from scheduling and decoding, and makes the extraction and operating-point definitions explicit.

W1 & Q1: Payload hypothesis and blind recovery.

The receiver does not take the ground-truth payload as an inference input. It selects and validates a message using the received waveform and registered keys. Sections 3 and 4.3 now make this procedure explicit:

  1. Observed evidence: encode the received waveform into tokens, cluster observations, and context seeds.
  2. Candidate-wise alignment: score all 16 symbol values at each position over fixed windows and offsets, comparing candidate target clusters with observed clusters.
  3. Repeated-observation scoring: aggregate hard matches and bounded soft compatibility across laps. Pub-key-compatible audio-supported candidates supply detection features.
  4. Presence decision: compare scores with frozen unwatermarked references and apply the branch-union decision in Eq. (6).
  5. Payload selection: rank candidates using their own compatibility estimate, audio evidence, pri-key consistency, and the pub-key bonus in Eq. (7). Emit the message after final key validation.

Every input follows this same fixed pipeline, without supplied attack labels or actual attack parameters. The revised text states: “All rates and rankings use received observations and registered key values.” Candidate-specific η^a(m)\widehat\eta_a(m) is estimated from the candidate's observed support, so neither alignment nor rate estimation requires the true message.

The role of a supplied message is now confined to evaluation: changing that comparison target changes its matching score, while the extracted output remains fixed. Registered-key interventions are evaluated separately in Appendix Table 5. Main Table 4 additionally shows that removing the pub-key ranking bonus changes bit accuracy from the reported 98.17% to 80.75%, with the presence TPR remaining 94%, demonstrating the importance of candidate selection for recovery.

Revision: Sections 3–4 specify the waveform/key inputs, candidate-based alignment, and post-extraction reference comparison. Section 4.3 states: “The receiver emits the selected message only if it passes final key validation, otherwise recovery fails.”

AuthorsOpenReview ↗

Response to Reviewer LBXo (Part 2/3)

W2 & Q2: Contribution of codec-aware clustering.

We isolate the representation contribution in a capacity-matched 12-bit framework. The evaluated zero-bit baselines establish the presence-detection comparison. The component controls below retain MARC's payload, embedding schedule, and decoder to compare clustering alternatives directly.

Only the geometric algorithm inside fusion is changed for K-means/agglomerative controls. A separate set changes the codec information used to construct the clusters. Together, these two groups distinguish geometric-center selection from the use of codec-confusion evidence.

Response Table R1. Clustering design under the same multi-bit framework.

ControlConstructionTPR (%) ↑−log⁡(clusterp)-\log(\mathrm{cluster_p}) ↑Clean bitacc\mathrm{bit_acc} (%) ↑FAD ↓NISQA ↑DNSMOS ↑
FullGMM + codec union (MARC)94.0037.0898.170.26224.5033.868
Geometric solverK-means + codec union90.0034.7998.060.25174.4853.848
Geometric solverAgglomerative + codec union87.0035.9598.250.26494.4993.839
Codec evidenceGMM + retokenization96.0032.8499.000.27914.5083.889
Codec evidenceGeometric GMM95.0036.9698.750.37994.5123.800
Codec evidenceGMM + retokenization + CNBP91.0035.2098.000.41374.4623.855

The tabulated clean results are from main Table 4. Comparisons after codec attacks are discussed separately below, using Appendix Table 6.

Replacing GMM reduces TPR by 4 and 7 percentage points, with lower cluster evidence. The codec-evidence comparison targets recovery after reconstruction. Appendix Table 6 reports 97.77% mean accuracy after codec attacks for codec-union clustering, compared with 97.24% using retokenization alone and 97.41% using geometric GMM, even though those alternatives have higher clean accuracy. This supports a design selected for codec robustness, rather than clean accuracy alone.

Thank you for emphasizing the reliability of the clustering gains. Confidence intervals for the clustering gain characterize one component of MARC, while the complete contribution also includes structured multi-bit embedding and alignment-aware recovery. We therefore interpret uncertainty estimates alongside the controlled, capacity-matched comparisons, which distinguish the roles of these components. The reported differences are observed gains under the evaluated configuration. All variants use the same prompt set, 12-bit payload, embedding schedule, and decoder, with only the designated clustering component changed. The geometric-solver controls show TPR differences of 4 and 7 percentage points. Appendix Table 6 provides complementary recovery comparisons after codec attacks, with observed gains of 0.53 and 0.36 percentage points. We report these differences directly and connect them to the corresponding component interventions in Section 5.5. Section 5.5 links these complementary gains to their mechanisms: codec-aware grouping addresses token substitution, repeated bursts/laps provide observation redundancy, and alignment restores temporal correspondence. Removing the lap/grid pipeline lowers mean recovery accuracy after codec attacks to 74.57% (Appendix Table 8). The contribution is the combination of codec-stable representation and structured multi-bit recovery.

AuthorsOpenReview ↗

Response to Reviewer LBXo (Part 3/3)

Q3: Validity of cluster-center fusion.

Thank you for raising the correspondence issue. MARC combines geometric centers and codec-informed spectral centers in the same reduced feature space, followed by balanced assignment. The implemented rule pairs centers in their returned index order. It does not assume that equal indices have an intrinsic semantic correspondence. We evaluate this pairing as a construction choice, keeping the underlying evidence and downstream assignment fixed.

To test whether explicit center matching improves the downstream watermark task, Table 4 compares direct fusion with Hungarian-matched fusion and single-source centers:

Response Table R2. Center-construction ablation.

IDCenter constructionGeometricSpectralHungarianTPR (%) ↑−log⁡(clusterp)-\log(\mathrm{cluster_p}) ↑bitacc\mathrm{bit_acc} (%) ↑
F0Direct fusion✓✓×9437.0898.17
G1Geometric only✓××8934.8498.25
G2Spectral only×✓×8331.9598.05
G3Hungarian before fusion✓✓✓8433.9198.10

Direct fusion gives higher TPR and cluster evidence than either center source alone or Hungarian-matched fusion. With the remaining construction fixed, G3 tests an explicit alternative to returned-index pairing. The results support retaining the implemented pairing based on downstream watermark performance. Revision: Section 5.5 and Appendix A.4 explain this empirical construction choice without assigning semantic meaning to cluster indices.

Q4: False positive rate and attack-aware verification.

The receiver is not given attack labels or actual attack parameters. Every audio input follows the same fixed compensation, observation-window, alignment, scoring, and decoding policy. Audio-dependent candidate selection follows predefined rules. It does not use the applied attack's identity or settings.

Section 4.3 specifies that the input score and frozen unwatermarked references use the same feature construction, and that reference scores and thresholds are fixed before evaluation. The empirical test below measures the final OR decision after all enabled searches, including waveform variants, offsets, and windows.

Response Table R3. Complete-search empirical FPR.

MethodUnwatermarked inputsFalse positivesEmpirical FPR
MARC50030.60%

This evaluates the complete multi-bit verification protocol after the full search. The unwatermarked references calibrate branch scores. The independent 500-input set measures the final pipeline's observed false-positive behavior. Appendix A.4 explicitly specifies the common receiver policy without attack labels or actual attack parameters.

The codec evaluation also extends beyond the construction set: the verified SpiritLM construction uses MP3, Opus, EnCodec, and DAC confusion, while external SpeechTokenizer, FACodec, SoCodec, and SemantiCodec are held out from that construction. VoiceCraft–EnCodec is held out as a pipeline sharing the EnCodec component. Table 3 reports 98% and 97% bit accuracy under FACodec and SemantiCodec, respectively, supporting transfer beyond the construction transformations.

Q5: Inconsistencies in the ablation results.

Thank you for highlighting these inconsistencies. We have checked the full manuscript for numerical and textual consistency, including repeated table entries, associated averages, notation, and cross-references. Revised Tables 6–8 consistently report the full-method Opus and clean-accuracy entries highlighted in your review. The corresponding main Table 1 displays clean bit accuracy to two decimal places.

These corrections preserve the reported mean accuracy after codec attacks and the interpretation of the component comparisons. Section 5.5 now links the clustering, embedding, and decoding conclusions explicitly to their corresponding appendix results.

Reviewer_43CwOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper combines past approaches in token-level audio watermarking to create token groups that are informed both by the token structure itself and the retokenization and audio editing errors. It extends the token-level watermark from zero-bit to a multi-bit payload, and claims very strong p-values and robustness to attacks.

Critical strengths

The paper starts with a good and principled motivation to interpolate the tokenizer's embedding space with the actual attack errors, combining the strengths of different methods from the literature. Also, the evaluation is very thorough in terms of attack robustness.

Critical weaknesses
  1. Overall lack of clarity: The paper generally is not transparent about the inner workings of the multi-bit embedding mechanism. The terms "lap" and "bust" are used throughout but they are not defined or indicated clearly in Fig. 1.
  2. It introduces a lot of notation without explaining it or reusing it. For instance, Eq. 4 includes a first-time-mentioned "base mapping".
  3. The notation is inconsistent in general, e.g. token ids are named y_t in lines 151, 157, and 1019, but the u/v symbols are used in paragraph starting in line 195. The term CNBP (line 635) is not explained. Clustering maps are sometimes is noted with capital C and sometimes with lowercase.
  4. The two key construction is not explained well. Table 2 presents overwriting attacks, where an attacker tries to falsify attribution, but the pvalue is high in all cases (which is not explained). Is this because the watermark is robust to overwriting? Table 4 also implies that when both keys are incorrect, TPR is high.

Weaknesses 1-3 are mostly writing and presentation, and can be partially addressed by my Question 1. Weakness 4 is revisited in Question 3.

Questions for authors

In order of importance:

  1. Please explain decoding clearly, and what Eq. 5 is expressing.
  2. Please report the empirical FPR on unwatermarked audio.
  3. What exactly is the functionality of the pub-key and priv-key? Table 4 implies that even when BOTH keys are incorrect, TPR@FPR=1% is 50%, and -log(p) is 5! Shouldn't the both wrong keys imply that the TPR should be around 1%?
  4. What do you mean that the verifier receives the claimed payload m*? The verifier should only know the key and extract everything they need from the waveform. This affects Eq. 6.
  5. Why is GMM necessary? The similarity in line 192 could be calculated without it, and it is not ablated anywhere.
  6. The values of K=20 for the cluster number should be ablated.
5 comments
AuthorsOpenReview ↗

Response to Reviewer 43Cw (Part 1/3)

Dear Reviewer 43Cw,

Thank you for recognizing the motivation for combining embedding geometry with codec-error information. We have clarified the receiver and notation and added targeted clustering and complete-detector evaluations.

W1 & W2 & W3 & Q1 & Q4: Decoding procedure and interpretation of Eq. (5).

Thank you for highlighting these presentation issues. The rewritten Section 4.3 is summarized below, followed by the changes corresponding to your individual concerns.

A. Revised Section 4.3: Extraction and Detection

Inputs and outputs. The receiver takes received audio, fixed watermark parameters, and registered pub-key/pri-key values and outputs a presence decision and recovered message. Sections 3–4 now consistently exclude the raw reference payload from inference. Its role is post-extraction evaluation.

Cluster mapping and alignment. Speech encoding produces tokens z~t\widetilde z_t and cluster observations C(z~t)C(\widetilde z_t), with their context seeds. For each fixed window and step offset, the receiver evaluates all 16 values at each of the three symbol positions. A candidate symbol determines its target cluster through the fixed mapping and observed seed. That cluster is compared with the audio observation. Offsets change lap/symbol assignments while retaining the observed clusters and seeds.

Multi-segment lap scoring. Repeated observations provide hard cluster matches and bounded soft compatibility from RR. Audio-supported candidates passing the pub-key check provide global, symbol-wise, and segment-wise consistency features. A frozen function combines them into each branch score SbS_b.

Calibrated detection. Equation (6) compares SbS_b with frozen unwatermarked reference scores and accepts when a valid branch exceeds its prescribed evidence threshold:

\qquad D(\widetilde y)=\bigvee_b\{\mathrm{valid}_b\land p_b\le\alpha_b\}.$$ Reference scores and thresholds are fixed before evaluation. **All audio inputs use the same fixed search and decision rules, without attack labels or actual attack parameters.** The unwatermarked references are prepared offline. **Key-assisted decoding.** Before inference, registration fixes $k_{\mathrm{pri}}=s_0\oplus s_1\oplus s_2$ and $k_{\mathrm{pub}}=\operatorname{CRC16}(m\Vert k_{\mathrm{pri}})$. For each alignment, hard and soft rules separately retain the top 12 values per symbol position and form candidate set $\mathcal C_a$. The decoder scores each candidate using its own observed compatibility $h_a(m)$, rather than a match rate against the true message: $$\widehat{\eta}_a(m)=\operatorname{clip}\left(\frac{h_a(m)+1}{N_a+2},\frac{1}{K},0.995\right).$$ This rate defines audio likelihood-ratio evidence $A_a(m)$. Equation (7) combines audio evidence, pri-key consistency, and a fixed pub-key bonus: $$\widehat m=\arg\max_{a,\,m\in\mathcal C_a} \left[A_a(m)+P_{\rm pri}(m)+\lambda\mathbf1\{\operatorname{CRC16}(m\Vert k_{\rm pri})=k_{\rm pub}\}\right].$$ Selection compares windows, alignments, and hard/soft rules. A message is emitted only if final key validation passes. **Inference uses the audio evidence and registered keys. The reference bitstring is reserved for evaluation.** **B. Specific changes** **W1 — mechanism, lap, and burst.** We added module input/output overviews in Sections 4.1–4.3. Section 4.2 defines a burst as $B=8$ consecutive steps for one symbol position and a lap as an interval containing complete symbol cycles, with $L=16$ configured laps. Figure 2 now states: “A burst contains consecutive steps for one symbol, a lap comprises complete symbol cycles.” This makes local and cross-lap repetition explicit. **W2 — notation and base mapping.** Section 4.2 now explicitly identifies $b$ in Eq. (4) as a “fixed symbol-to-cluster base mapping” and $H$ as a domain-separated hash. The base offset and context/lap-dependent hash together determine the target cluster modulo $K$. Section 4.1 defines the token affinity and cluster map before they are used. Section 4.3 distinguishes observed clusters from candidates' computed targets. **W3 — inconsistent symbols and CNBP.** The method distinguishes the map $C$, observed tokens $\widetilde z_t$, and cluster observations $C(\widetilde z_t)$. The sequences $u$ and $v$ in Section 4.1 are paired for confusion collection. $R$ denotes the decoder's cluster affinity rather than the token affinity $A$. New Table 4 expands CNBP as “confusion-based boundary polishing of token assignments.” These definitions clarify the roles of the previously overloaded symbols and abbreviation.
AuthorsOpenReview ↗

Response to Reviewer 43Cw (Part 2/3)

Q1 — Eq. (5). We added the explanation: “This equation scales each token probability by its cluster's new-to-original mass ratio.” For VkV_k, Pt,k=∑v∈Vkpt(v)P_{t,k}=\sum_{v\in V_k}p_t(v). Multiplying each token probability by P^t,C(v)/Pt,C(v)\widehat{P}_{t,C(v)}/P_{t,C(v)} gives each cluster its prescribed new mass while preserving within-cluster relative probabilities. The target mass becomes min⁡(KPt,kt,1)\min(KP_{t,k_t},1), and the residual rule distributes the remaining mass. This encourages a target cluster without forcing an individual token.

Q4 — claimed payload and Eq. (6). Sections 3 and 4.3 now consistently describe candidate-wise alignment without a supplied reference payload. The revised text states: “For each candidate symbol value, the receiver computes its target cluster using the fixed mapping and observed context seed, and compares it with the cluster recovered from the audio.” Revised Eq. (6) specifies the calibrated presence decision, while Eq. (7) specifies candidate selection. Reference bits enter only post-extraction evaluation, not alignment, rate estimation, or selection.

W4 & Q3: The two-key construction is not explained well.

Thank you for this question. The registered values have complementary roles: the pri-key constrains candidate symbol combinations, and the pub-key assists ranking and final validation. Section 4.3 now specifies both in Eq. (7), while Section 3 identifies the authorized verifier's access to the registered values.

Appendix Table 5 separates the key conditions:

pub-keypri-keyTPR (%)−log⁡(clusterp)-\log(\mathrm{cluster_p})bitacc\mathrm{bit_acc} (%)
CorrectCorrect9437.0898
CorrectIncorrect588.0666
IncorrectCorrect7615.3278
IncorrectIncorrect495.0852

The correct-key combination supports reliable recovery, whereas the mismatched combination produces near-chance bit agreement (52%). The 49% reports acceptance on already watermarked inputs under mismatched identity parameters, rather than false acceptance on unwatermarked audio. The 1% FPR target applies to the latter null distribution. It does not prescribe a 1% acceptance rate for watermarked inputs tested with different keys. Together, the 49% acceptance and 52% bit agreement show that mismatched keys do not support reliable verification and recovery of the target owner's message.

For overwriting, Table 2 reports 90% TPR and 88% bit accuracy with the owner's key, compared with 68% and 75% when the owner's watermark is evaluated with the attacker key. This supports the value of key-conditioned recovery. A large −log⁡(p)-\log(p) represents a small cluster p-value: the cluster statistic describes surviving watermark evidence, while message accuracy evaluates the recovered attribution information.

Revision: Section 4.3 states: “The pri-key constrains symbol combinations, while the pub-key assists candidate ranking and final validation.” Appendix A.1 explains their roles consistently with the revised decoder.

Q2: Report the empirical FPR on unwatermarked audio.

The complete fixed receiver accepts 3 of 500 unwatermarked inputs, giving an empirical FPR of 0.60%. Calibration audio is separated from this independent test set. Section 4.3 specifies the common feature construction, frozen reference scores, and decision rule in Eq. (6).

Response Table R1. Empirical FPR of the complete receiver.

MethodUnwatermarked inputsFalse positivesEmpirical FPR
MARC50030.60%

All 500 inputs run the complete fixed waveform, window, and alignment search, with any accepting branch making the final decision positive. The inputs are unattacked unwatermarked audio. The full search is still executed. The observed FPR is 0.60%, below the 1% target. Reference scores and thresholds are fixed before this confirmation evaluation.

AuthorsOpenReview ↗

Response to Reviewer 43Cw (Part 3/3)

Q5: Why is GMM necessary?

Thank you for asking us to clarify this distinction. GMM supplies the geometric centers for MARC's codec-aware fusion, complementing the empirical token substitutions captured by codec confusion. Its role extends beyond pairwise cosine similarity: these centers contribute to the fused representation used for balanced assignment.

To test the geometric solver’s contribution beyond the shared embedding and decoder, we replace only GMM inside fusion with K-means or agglomerative clustering. Codec evidence, fusion, balanced assignment, embedding, and decoding remain fixed.

Response Table R2. Geometric algorithm inside fixed fusion (manuscript Table 4).

IDGeometric algorithmTPR (%) ↑−log⁡(clusterp)-\log(\mathrm{cluster_p}) ↑bitacc\mathrm{bit_acc} (%) ↑FAD ↓NISQA ↑DNSMOS ↑
F0GMM (MARC)94.0037.0898.170.26224.5033.868
M1K-means90.0034.7998.060.25174.4853.848
M2Agglomerative87.0035.9598.250.26494.4993.839

GMM provides the strongest detection performance among the tested geometric solvers, with comparable bit accuracy and higher NISQA/DNSMOS than both replacements. These controls support its selection within MARC’s fusion design. Revision: Table 4 reports the controlled comparison, and Section 5.5 discusses detection, recovery, and fidelity together.

Q6: The cluster count K=20K=20 should be ablated.

We have added K∈{10,15,25}K\in\{10,15,25\} alongside the full configuration K=20K=20 in main Table 4. The controlled comparison is:

Response Table R3. Cluster-count selection.

IDKTPR (%) ↑−log⁡(clusterp)-\log(\mathrm{cluster_p}) ↑bitacc\mathrm{bit_acc} (%) ↑FAD ↓NISQA ↑DNSMOS ↑
K11088.0025.7898.050.25924.5013.853
K21590.0030.4197.550.31344.4613.854
F02094.0037.0898.170.26224.5033.868
K32589.0038.9898.050.26624.4843.844

K=20K=20 gives the highest TPR among the tested counts, retaining 98.17% bit accuracy and strong speech-quality scores. K=25K=25 produces a larger cluster statistic but lower TPR, illustrating why cluster significance alone is not the selection criterion. Section 5.5 connects the selected granularity to the joint detection–recovery–fidelity trade-off.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response and Revision Summary

We sincerely appreciate all three reviewers for their constructive feedback and for recognizing the motivation for codec-aware multi-bit audio watermarking. We have revised the manuscript and provide additional measurements in the responses below. All section, figure, and table numbers refer to the revised PDF. The main updates are:

  1. Codec robustness and structured multi-bit recovery (43Cw Q5–Q6, LBXo W2/Q2–Q3, BQvF W1/Q1). The expanded ablations distinguish the contributions of codec-aware representation, repeated embedding, and alignment-aware recovery. Table 4 brings together 19 controlled configurations covering clustering, embedding, and decoding. Replacing GMM with K-means or agglomerative clustering reduces TPR from 94% to 90% and 87%. Section 5.5 connects these controls with Appendix Tables 6–8 to explain the complementary roles of codec-aware representation and structured multi-bit recovery.

  2. Verification without attack information and receiver cost (43Cw Q2, LBXo Q4, BQvF Q2–Q3). Every input follows the same fixed receiver policy without supplied attack labels or actual attack parameters, as specified in Section 4.3 and Appendix A.4. The responses report 3 false positives among 500 unwatermarked inputs (0.60%) after the complete search and a mean end-to-end receiver time of 31.690 s on watermarked clean audio. They also distinguish construction codecs from held-out test transformations.

  3. Key-assisted recovery (43Cw Q1/Q3–Q4, LBXo W1/Q1, BQvF W1/Q1–Q2). The receiver selects and validates a message using received-audio evidence and registered keys. Sections 3–4 explain candidate-wise alignment, scoring, and the two key roles. The reference message is used only after extraction for evaluation. Section 5.1 distinguishes statistical detection, message-matching TPR, and bit accuracy, with references to HiPT [1] and VINE [2].

  4. Improved presentation and manuscript consistency (43Cw W1–W3, LBXo Q5). We added module input/output overviews, defined the base mapping, burst/lap schedule, and CNBP, and explained the probability-reweighting rule. We also checked numerical reporting, notation, and cross-references throughout the manuscript. The revisions are detailed in the point-by-point responses below.

We thank all reviewers again for their time and constructive feedback. We hope that our responses and revisions adequately address their concerns.

References

[1] Georgios Milis, Yubin Qin, Yihan Wu, and Heng Huang. Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audio. ICML 2026.

[2] Shilin Lu, Zihan Zhou, Jiayou Lu, Yuanzhi Zhu, and Adams Wai-Kin Kong. Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances. ICLR 2025.

Area_Chair_Ni1oOpenReview ↗

ICLR Review Discussion – Follow-up on Author Rebuttal

Dear Reviewers,

Thank you for your time and effort in reviewing this submission.

The authors have submitted their rebuttal. Could you please take a moment to review their responses and assess whether your concerns have been adequately addressed? If you have any remaining concerns or new questions, please feel free to raise them in the discussion.

If you have already responded to the authors, please kindly disregard this reminder.

Thank you again for your valuable contributions to the review process.

Best,

AC

AuthorsOpenReview ↗

Summary of Revisions and Reviewer Feedback

Dear AC,

We sincerely appreciate the time and effort you are dedicating to our submission, and thank all three reviewers for their valuable comments.

We propose MARC, a codec-aware multi-bit watermarking framework for key-assisted verification and payload recovery in autoregressive audio generation. Reviewers noted the following strengths:

  1. Multi-bit verification and recovery. MARC embeds a 12-bit identifier and uses structured scheduling and alignment-aware recovery to trace individual outputs after re-encoding. Reviewers LBXo and BQvF recognized the importance of multi-bit recovery and distinguishing individual generated outputs.
  2. Codec robustness and multi-bit recovery. MARC combines embedding geometry with retokenization and multi-codec confusion evidence. Repetition and alignment support payload recovery under substitutions and temporal misalignment. Reviewer 43Cw highlighted the principled motivation for combining geometry with actual attack errors. Reviewer LBXo also recognized the codec-aware motivation.
  3. Robustness under watermark overwriting. Our experiments cover speech, dialogue, music, and watermark overwriting. We evaluate a wide range of conventional and neural codecs. The original owner's watermark retains 90% TPR and 88% bit accuracy in the reported overwriting setting. Reviewers LBXo and BQvF recognized the broad evaluation, including overwriting. Reviewer 43Cw described the robustness evaluation as thorough.

We have provided point-by-point responses to every question, including follow-ups, and revised the manuscript throughout, with changes highlighted in blue. The main additions follow.

(A). Additional Experiments and Analyses

  1. Controlled ablations (Section 5.5 and Appendix A.1). Following reviewers 43Cw and LBXo, Table 4 now includes 19 configurations covering clustering, embedding, and decoding. Replacing only GMM with K-means or agglomerative clustering reduces TPR from 94% to 90% and 87% in the same 12-bit framework. These controls isolate representation, repetition, and alignment contributions.
  2. Center-fusion sensitivity (Section 4.1 and Appendix A.4). Following reviewer LBXo, we evaluated 10 additional GMM seeds and 10 spectral-center permutations in total. These controls complement direct-versus-Hungarian fusion and support stability under the tested changes.
  3. Detection, empirical FPR, and receiver cost. We added an attack-wise statistical-detection comparison covering 21 conditions on Finance QA, with MARC reaching 100% TPR across the reported conditions. The responses also report 3 false positives among 500 unwatermarked inputs (0.60%) after complete search and final union acceptance, and a mean complete receiver time of 31.690 s on clean watermarked audio.

(B). Response to Individual Reviewers

  1. Response to Reviewer 43Cw. Sections 3–4 clarify decoding, probability reweighting, burst/lap scheduling, and notation. Section 4.3 and Appendix A.1 explain the registered values' roles as one verifier credential and the key-component controls. We added GMM/cluster-count ablations (see (A).1) and empirical FPR evidence (see (A).3).
  2. Response to Reviewer LBXo. We clarified audio-based alignment and scoring, added component controls (see (A).1), and evaluated fusion sensitivity (see (A).2). Sections 4.1 and 5.5 connect the construction to codec robustness and structured recovery. We also checked numerical reporting throughout.
  3. Response to Reviewer BQvF. We clarified registered-target identification and audio-dependent verification, revised the Abstract to explicitly state key-assisted verification and recovery, and provided a no-registered-key control reporting bit accuracy and exact-match rate. We added the attack-wise detection comparison, distinguished it from multi-bit verification TPR, and explained branch calibration and complete-search FPR (see (A).3). Section 5.1 and Appendix A.5 clarify the evaluation metrics and receiver protocol.

During the discussion, reviewer 43Cw raised the score to weak acceptance, citing positive soundness evidence on empirical FPR, GMM, and cluster count. Reviewer LBXo stated that most earlier concerns had been addressed, and that the sensitivity results helped address the remaining fusion-stability concern. We also answered BQvF's latest follow-up.

We believe these revisions address the full set of reviewer concerns and strengthen the evidence for multi-bit recovery, codec robustness, and resilience under overwriting. Thank you again for your time and consideration.

Best regards,

The authors of submission 4765

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.