acceptodds
Under review as a conference paper at ICLR 2027

SparseMark: Routing-Aware Differential Parameter Watermarking for Mixture-of-Experts LLMs

Abstract

Parameter watermarking for sparse mixture-of-experts (MoE) language models faces a placement challenge: evidence concentrated in a few experts is vulnerable to localized damage, while embedding must preserve model utility. We introduce SparseMark, a post-training multi-bit watermark that distributes evidence across experts and limits the influence of individual parameter pairs during recovery. SparseMark combines balanced expert allocation with response-based carrier selection, encodes a secret codeword through antisymmetric parameter updates, and spreads each bit across experts using keyed block ordering. Verification recovers the codeword from stored baseline contrasts through unit sign voting, without model forward passes. Controlled experiments show that balanced allocation improves recovery under targeted expert damage at comparable utility cost, while unit voting outperforms mean aggregation under pruning at fixed embedding energy. SparseMark achieves exact clean recovery of 256-bit codewords across three MoE backbones with small measured changes in perplexity. Detailed OLMoE evaluations demonstrate watermark retention under the evaluated pruning, quantization, and fine-tuning settings, while identifying limited recovery margin under feedback-guided attacks after quantization. These findings support distributed carrier placement and bounded pairwise influence as design principles for parameter watermarking in sparse MoE models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mixture-of-Experts, Model Watermarking, Ownership Verification

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rKYjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a whitebox parameter watermark for MoE LLMs. The watermark uses multibit paired signals that distribute across experts and layer, signal carriers are allocated evenly across experts and selected within each expert using statistics from a small calibration set. Moreover, the method does not require fine tuning or training. Detection is straight forward by comparing original weight with the watermarked model's weight, the procedure first takes a sign vote per pair, decodes each bit by majority, then compares statistic with a threshold to decided whether its watermarked. Experiment on OLMoE Qwen GPT-OSS demonstrated the performance and robustness of the method.

Critical strengths

The balanced carrier allocation is a well-motivated, MoE-specific design and ablation shows its effectiveness.

The method is lightweight, it only requires value change of a small set of the parameters.Despite the simplicity,it still demonstrated great detectability compared to baselines. The method is also robust to attacks and have small effect on model performance.

Critical weaknesses

Limited novelty. in cited work (Uchida, Lin 2024), they have proposed paired difference, majority voting for identification. Moreover, EmMark (https://arxiv.org/abs/2402.17938), which is not cited, proposed a framework with similar procedures for dense networks: importance-based allocation, additive signature, verification by comparison to original model.

Questions for authors

Could the authors compare SparseMark with EmMark if we transfer the method of EmMark to MoE models?

On OLMoE, routing statistics only exclude 3 of 1,024 experts, could the authors test additional configurations that vary this allocation and see how it works?

Neural Mark showed am abnormal PPL in Table 1 for OLMoE and Qwen, but normal value for GPT-OSS, could the authors explain this discrepancy?

2 comments
AuthorsOpenReview ↗

Response to Reviewer rKYj

We thank Reviewer rKYj for the constructive feedback. Before addressing the individual questions, we clarify the distinction between generic watermarking primitives and our architectural contribution.

Q1. What is new, and how does SparseMark compare with prior work?

R1: SparseMark is built upon three novel, tightly coupled algorithmic designs for robust parameter watermarking in sparse Mixture-of-Experts (MoE) models. First, it leverages calibration-time routing statistics during enrollment to enforce balanced carrier allocation across eligible experts, preventing carrier concentration within a small subset of experts. Second, keyed block ordering mixes carrier–reference pairs across experts and MoE layers before partitioning them into codeword blocks, making the block assignment depend on the secret key rather than public routing or response scores. Third, for a fixed embedding-energy budget, SparseMark applies antisymmetric paired updates to each carrier–reference pair, maximizing the shift in pair contrast while preserving the pair sum to strengthen the embedded signal against parameter perturbations and removal attacks. These design choices distinguish SparseMark from several related parameter-space approaches. Uchida et al. (2017) exploit differences between parameter coordinates, yet their differential construction primarily compensates for embedding-induced energy variations. In contrast, SparseMark uses antisymmetric differential updates to enlarge the pair contrast between a carrier and its reference coordinate under a fixed squared-update budget, thereby improving recovery margins against erasure and quantization noise. Li et al. (2024) achieve robust verification by aggregating votes across repeated verifier evaluations on perturbed parameters, whereas SparseMark aggregates unit sign votes across disjoint carrier–reference pairs within a single static candidate checkpoint without requiring model forward passes. Finally, while EmMark (Zhang & Koushanfar, 2024) selects watermark coordinates via importance scores, SparseMark combines within-expert gate-weighted response selection with balanced expert quotas and keyed cross-layer block spreading, preventing the watermark structure from being exposed or erased through public-score targeting.

Allocation rule experiments validate the superior robustness of our method over EmMark. Given EmMark’s public response ranking, zeroing out the down-projections of 32 experts compromises 86.82% of pairs under EmMark’s global response selection, compared to only 3.12% under SparseMark’s allocation rule. Consequently, the watermark extraction rate (WER; higher is better) drops to 71.56% with EmMark’s rule, whereas SparseMark maintains 100.00% WER (see Section 4.5 and Appendix A.1, Tables 4 and 6).

Direct comparison with EmMark, as suggested by the reviewer. Unlike SparseMark and our baselines (NeuralMark, PathMark, ELLMark), which are blind schemes, EmMark is non-blind: verification requires comparing the suspect model against the full pre-watermarked checkpoint M0M_0, incurring prohibitive storage and I/O overhead per query at LLM scale. Moreover, EmMark embeds watermarks via discrete integer additions and subtractions, a mechanism exclusively tailored to quantized weight grids. Coupled with its assumption of neither subsequent pruning nor fine-tuning, this design cannot operate on floating-point MoE checkpoints throughout a full post-training lifecycle.​

Notwithstanding these fundamental differences in watermarking paradigms, we adapted EmMark to a 256-bit MoE setting and evaluated it alongside SparseMark on OLMoE. The table reports WER (%) under this unified evaluation protocol.

OLMoE transformationEmMark (MoE adaptation)SparseMark
Gaussian noise99.69 ± 0.31100.00 ± 0.00
10% pruning91.72 ± 1.59100.00 ± 0.00
Fine-tuning, 200 steps95.12 ± 1.63100.00 ± 0.00
Merged LoRA, 200 steps99.45 ± 0.56100.00 ± 0.00

ΔPPL\Delta\mathrm{PPL} is +0.0011 ± 0.0020 for the adapted EmMark and −0.0004 ± 0.0048 for SparseMark. SparseMark has the higher observed mean WER in these four conditions.

AuthorsOpenReview ↗

Response to Reviewer rKYj (cont.)

Q2. Why use a 32-token eligibility threshold?

R2: We retain expert ee when ∣Te∣≥τ\lvert T_e\rvert\geq\tau, where TeT_e is its set of routed calibration tokens. On OLMoE, only 3 of 1,024 experts receive fewer than 32 tokens. This is therefore an observed count for this calibration set, not the number of experts that the method is designed to exclude.

Targeted quantization. The threshold sweep below compares two targeted transformations: 4-bit quantization of the 880 highest-frequency experts and 3-bit quantization of the 192 highest-frequency experts. Values are WER (%).

Threshold (tokens)Experts excluded880 experts, 4-bit192 experts, 3-bit
0, 16097.40 ± 0.9088.75 ± 1.74
32, 48398.05 ± 0.3990.62 ± 1.66
64797.53 ± 0.8189.69 ± 1.96
128997.27 ± 0.5589.49 ± 1.81
2562197.07 ± 0.2888.98 ± 1.54
5125295.31 ± 1.9588.83 ± 0.92
1,02410895.70 ± 0.6888.52 ± 2.08

Note. Thresholds 0 and 16 exclude no experts, and thresholds 32 and 48 exclude the same three experts; rows are merged because the resulting carrier sets are identical.

The 32/48 row has the highest observed mean under both transformations. Moving to 64 excludes four more experts without improving either mean.

Q3. Why does NeuralMark affect perplexity differently across backbones?

R3: We respectfully note that this issue was already investigated in Section 4.2 and Appendix E.3 through an α\alpha sweep over:

α∈{1,2,4,8,16,32}\alpha \in \{1, 2, 4, 8, 16, 32\}.

The sweep shows that NeuralMark’s high PPL on OLMoE (minimum 406.00) and Qwen1.5-MoE (minimum 27.29) is not caused by evaluating only an unsuitable value of α\alpha.

The cross-backbone discrepancy is instead explained by the treatment of internal activations in the MoE SwiGLU blocks. At its native setting α=32\alpha = 32, NeuralMark introduces relatively large LoRA-based weight perturbations. GPT-OSS-20B applies built-in activation clamping to its MoE SwiGLU gating/intermediate activations, with a threshold of ±7.0\pm 7.0. This provides a numerical buffer that prevents the perturbations from propagating into extreme internal outliers and preserves normal PPL (ΔPPL=−0.96\Delta\mathrm{PPL} = -0.96).

By contrast, OLMoE and Qwen1.5-MoE use standard unclamped SwiGLU projections. The same perturbation scale can therefore cascade through sparse layers into severe activation outliers, causing teacher-forced PPL to increase sharply. This architectural explanation is consistent with the α\alpha-sweep results.

Planned manuscript changes

  • Section 5, opening weight-based methods paragraph: distinguish shared primitives from the routing-aware allocation rule and add the EmMark citation. In Appendix A.1, immediately after Table 4, distinguish the controlled allocation ablation from the direct comparison.
  • Appendix E.2, after Table 22: describe the MoE adaptation, verifier inputs, and comparison budget. Appendix E.3, before Appendix F: report the four transformation results separately from the existing clean cross-backbone comparison.
  • New Appendix B.4, immediately after B.3: add the carrier-row stability table and the targeted-quantization grid. Cross-reference it at the eligibility definition in Appendix D.1.
  • Section 4.2, after Table 1, and Appendix E.3, after the NeuralMark strength sweep: distinguish paired changes from absolute PPL and document the carrier and adaptation scope used on each backbone.

We sincerely hope that our clarifications and planned revisions have thoroughly addressed your concerns, and if so, we would be deeply grateful if you would kindly consider a more favorable evaluation of our work.

Reviewer_M3c2OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduced SparseMark which is a post-training multi-bit parameter watermarking framework for sparse Mixture-of-Experts (MoE) language models.It combines balanced expert allocation, response-based carrier selection, antisymmetric parameter-pair embedding, and unit sign voting. Experiments on three MoE models demonstrate exact recovery of 256-bit watermarks with negligible utility degradation and robustness against various parameter transformations.

Critical strengths

1. Clear Motivation and Novel MoE-Aware Design.

The paper addresses the vulnerability of concentrated watermark carriers in MoE models. Balanced allocation and response-based selection are well motivated and novel

2. Strong Empirical Results.

The method achieves 100% clean recovery across three MoE models with minimal utility impact. It also demonstrates robustness against pruning, quantization, and fine-tuning

Critical weaknesses

1. Insufficient Evaluation Against Adaptive Attacks.

Although the authors evaluate adaptive attacks targeting down-projection parameters, each attack proposal randomly selects only one expert for perturbation, while SparseMark distributes its watermark across approximately 1,021 experts. Since the watermarking algorithm is public, an adversary could use a publicly available proxy dataset to re-estimate the locations of high-response parameters and simultaneously perturb potential watermark carriers across multiple experts. The current experiments do not sufficiently evaluate robustness against such distributed, response-aware attacks.

2. Potential Pair Synchronization Vulnerability.

SparseMark relies on fixed parameter pairs and their original contrasts for watermark extraction. Similar to the synchronization problem in classical Patchwork watermarking, function-preserving parameter permutations may change the locations of watermarked parameters and disrupt pair alignment.

For example, consider a MoE expert with gate, up, and down projections. Given a permutation matrix PP, the following transformations preserve the expert's functionality:

W_{\mathrm{gate}}' &= PW_{\mathrm{gate}}, W_{\mathrm{up}}'&=PW_{\mathrm{up}}, W_{\mathrm{down}}' &=W_{\mathrm{down}}P^\top. \end{aligned}$$ Since $P^\top P=I$ and element-wise operations are permutation-equivariant, the transformed expert produces the same output: $$E'(x)=E(x)$$ However, the parameter coordinates have changed, potentially invalidating the registered parameter pairs and baseline contrasts used for watermark extraction.
Questions for authors

1. Why Is Embedding Restricted to Down Projections?

Why do the authors exclusively embed watermarks into down-projection parameters rather than up or gate projections? Could the authors provide a comparison across these projection types under the same embedding budget and model utility constraints?

2. Adaptive Attacks Targeting Carrier Selection.

An adversary could use a publicly available proxy dataset to estimate the locations of high-response parameters and perturb potential watermark carriers across multiple experts, rather than randomly targeting individual experts. Could the authors evaluate such an attack? To what extent can the actual carrier coordinates be inferred from public proxy data?

3. Pair Synchronization Under Parameter Permutations.

How does SparseMark maintain correct pair alignment under function-preserving parameter permutations? Could the authors evaluate robustness against such transformations and clarify whether an additional parameter alignment mechanism is required?

2 comments
AuthorsOpenReview ↗

Response to Reviewer m3c2

We thank Reviewer m3c2 for the constructive feedback.

Q1. Why use the down-projection for embedding?

R1: The rationale for selecting down-projection is that each row of down_proj corresponds to an expert output dimension, so we can use the row response score ρu\rho_u directly and select a reference coordinate from the same tensor. To evaluate how recovery varies under different choices, we additonally tested other projection options, while holding B=256B=256, N=8192N=8192, and the embedding energy unchanged.

ProjectionClean WER (%)INT4 WER (%)Pruning WER (%)Noise WER (%)ΔPPL\Delta\mathrm{PPL}
gate_proj100.00 ± 0.0087.77 ± 1.46100.00 ± 0.00100.00 ± 0.00−0.0009 ± 0.0024
up_proj100.00 ± 0.0091.99 ± 1.21100.00 ± 0.00100.00 ± 0.00−0.0013 ± 0.0033
down_proj (default)100.00 ± 0.0092.30 ± 2.23100.00 ± 0.00100.00 ± 0.00−0.0004 ± 0.0048

WER denotes watermark extraction rate; higher is better. Since down_proj yields the highest observed INT4 WER of the three, we adopt it as our default carrier. We will summarize this additional comparison in Section 4.5 (“Carrier selection”) and add the projection-specific settings and results to Appendix A.3.

Q2. Can an adaptive attacker exploit public scores across multiple experts?

R2: In the original evaluation (Section 4.4, Table 3; Appendix C.4), each proposal perturbs one randomly sampled expert down-projection group. The 200 queries comprise one initial evaluation and 199 proposals; accepted changes can accumulate across experts. In the new evaluation, public proxy scores guide expert selection, a proposal can perturb multiple experts simultaneously, and the verifier-query budget increases to 1,000. The attacker knows the public scores but not the watermark keys.

Additional experimental results across 1,000 verifier queries confirm that our method maintains consistent performance. Values are means ± standard deviations. “Clean” describes the starting checkpoint.

Starting conditionFinal WER (%)ΔPPL\Delta\mathrm{PPL}
Clean (ε=10−3\varepsilon=10^{-3})100.00 ± 0.00+0.228 ± 0.223
Calibrated packed INT498.48 ± 0.68+0.419 ± 0.040
RTN INT493.91 ± 1.20+0.332 ± 0.015
GPTQ INT492.62 ± 1.59+0.255 ± 0.068
Per-channel fake INT4 (ε=2×10−3\varepsilon=2\times10^{-3})99.45 ± 0.35+0.194 ± 0.166

Why does public-score guidance have limited effect here? Public scores identify active or high-response experts, but they do not specify the keyed reference coordinates or tell the attacker which pairs vote on the same payload bit. The verifier returns an aggregate score, not a list of affected pairs. A perturbation that lowers that score need not flip enough votes in any one block to change its decoded bit.

The matched expert-zeroing control provides evidence for this explanation (Section 4.5, “Structural dispersion”; Appendix A.1, Table 4). Under the same public ρu\rho_u ranking and the same budget of 32 experts, global response-only selection exposes 86.82% of pairs and yields 71.56% WER. SparseMark’s balanced allocation exposes only 3.12%; each 32-vote block retains at least 26 correct votes, yielding 100.00% WER (100.00% accepted). We will report the new results alongside Table 3 in Section 4.4 and document the multi-expert protocol in Appendix C.4. The existing experiments without verifier feedback remain separate: expert zeroing is reported in Appendix A.1 (Tables 4 and 6), and row perturbations are reported in Appendix C.3 (Table 16).

AuthorsOpenReview ↗

Response to Reviewer m3c2 (cont.)

Q3. What happens when an equivalent expert representation changes the registered coordinates?

R3: Permuting parameters lies outside the standard white-box watermarking threat model—we assume the candidate is presented in the registered coordinate order, as do NeuralMark, ELLMark, and EmMark. Under such a permutation, the extraction rate of all evaluated baselines (NeuralMark, ELLMark, EmMark) drops below the recovery threshold because their carrier coordinates no longer align. SparseMark’s pairs can be realigned with a key-free preprocessing step: using the publicly released watermarked checkpoint MkM_k as the alignment reference, this preprocessing restores exactly the same carrier–reference pairing and achieves 100.00% alignment accuracy, after which verification proceeds unchanged. We will clarify both the threat-model assumption and this alignment procedure in the revision.

Table: Pair synchronization under parameter permutations.

MethodClean MkM_kPermuted, no alignmentΔ (pp)
EmMark100.00%± 0.00%6.02 ± 0.74%−93.98
NeuralMark98.91 ± 2.34%51.09 ± 3.49%−47.81
ELLMark88.28 ± 0.00%63.63 ± 2.17%−24.65
SparseMark w/o realignment100.00%± 0.00%42.03 ± 2.77%−57.97
SparseMark w/ realignment100.00%± 0.00%100.00%± 0.00%0.00

We sincerely hope that our clarifications and planned revisions have thoroughly addressed your concerns, and if so, we would be deeply grateful if you would kindly consider a more favorable evaluation of our work.

Reviewer_ip56OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SparseMark, a post-training, white-box parameter watermarking framework for sparse Mixture-of-Experts (MoE) language models. SparseMark addresses uneven expert utilization by combining balanced expert allocation, routing-aware carrier selection, and keyed block spreading. It embeds multi-bit ownership signatures through antisymmetric parameter updates and recovers them using baseline-referenced contrasts and unit sign voting without model inference. The framework incorporates codeword-specific statistical verification and a wrong-key false-acceptance bound.

Critical strengths

The paper addresses the architectural challenges of parameter watermarking in sparse MoE models, particularly uneven expert utilization and vulnerability to localized modifications. Its combination of balanced carrier allocation, antisymmetric embedding, and unit sign voting provides a coherent framework for distributing ownership evidence across experts. The parameter-only verification procedure avoids inference overhead, while the statistical tests offer an explicit acceptance criterion.

Critical weaknesses

W1. The system background and practical threat model require clearer explanations. Section 2 directly introduces a white-box adversary without sufficiently explaining the deployment environment or how an attacker obtains complete access to MoE parameters. Although adversarial capabilities and knowledge are specified, a concrete real-world scenario involving model distribution, unauthorized reuse, and ownership disputes would better justify the assumptions and practical relevance of the proposed framework.

W2. The research problem and design objectives are not formally defined. Sections 1–2 motivate the challenges of sparse MoE watermarking but do not provide an explicit problem formulation. The authors should formally specify the watermark embedding and ownership verification objectives, including robustness, utility preservation, embedding capacity, and verification reliability, to clarify the intended optimization criteria and evaluation requirements.

W3. The methodological presentation lacks clarity regarding the differential watermarking mechanism. Section 3.1 introduces routing-aware carrier selection, keyed spreading, and antisymmetric parameter updates, but their relationships are difficult to follow. In particular, the term differential watermarking appears in the title without a sufficiently explicit conceptual definition. The authors should clarify how carrier-reference parameter differences encode ownership information and distinguish this mechanism from conventional parameter watermarking.

W4. The robustness evaluation should consider a broader range of recent attacks and common perturbations. Section 4.4 evaluates pruning, quantization, fine-tuning, parameter noise, and several targeted modifications. However, additional contemporary watermark removal, overwriting, and adaptive attacks would provide stronger evidence of practical security. More diverse perturbation settings would also clarify the limits of watermark retention against adversaries attempting to evade ownership verification. W5. The robustness of SparseMark under adaptive white-box attacks remains insufficiently established. Section 2 assumes that attackers have full parameter access but cannot query the verifier. Although Section 4.4 introduces feedback-guided attacks, it considers only 200 verification queries. Table 3 shows that an adaptive attack after INT4 quantization already causes one registration to fail. Stronger adaptive attacks and larger query budgets would better establish security limits.

Questions for authors

refer to the weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer IP56

We thank the reviewer for the detailed and constructive comments.

W1.Unclear background and threat model

R1:In the original submission, we have already defined the white-box adversary, private enrollment assets, and forward-pass-free parameter verification in Section 2 (Threat Model, p. 2) and Section 3.2 (p. 4).

To make clearer explanation, we will provide the following clarification for revision. Because these operational elements were previously distributed across Section 2, Section 3.2, and the Ethics Statement rather than summarized in one place, we will consolidate them at the opening of Section 3 (Method, before Section 3.1) with the following overview paragraph:

Consider an owner who distributes a sparse MoE checkpoint for local deployment or adaptation. Before release, the owner embeds a registered codeword and retains the key, baseline contrasts, and enrollment record. A recipient has white-box access to the checkpoint and can modify or redistribute it, but does not receive the private verification assets. If a suspect checkpoint comes to light, the owner tests its parameters in the registered coordinate order without running a candidate forward pass. The decision provides evidence about the registered watermark; license and provenance records provide the context for an ownership claim.

W2. Unclear problem formulation and design objectives

R2: In the original submission, all mathematical definitions, statistical bounds, and empirical metrics for our four design objectives are already established and evaluated, though introduced across separate sections:

  1. Retention under transformation. For each tested transformation TT, assess the fraction of correctly recovered blocks (watermark extraction rate, WER), the registered acceptance decision on T(Mk)T(M_k), and the utility cost. High WER alone does not establish acceptance (as specified in Section 4.4; Appendix C).
  2. Utility and detectability. The L2L_2 embedding budget applies to each pair’s update (Section 3.1, Eqs. 1–2). We measure the marked–unmarked perplexity (PPL) gap; Appendix E.1, Eq. 21, measures how a transformation changes that gap. Separately, Appendix C.5 tests two learned detectors on held-out keys. Near-chance AUC for those detectors is not a general stealth guarantee.
  3. Payload and coexistence. Each default registration spreads B=256B=256 bits over N=8192N=8192 disjoint pairs, or K=N/B=32K=N/B=32 pairs per bit (Section 3.1; Fig. 1). Appendix D.4 separately demonstrates 16 sequential registrations on one backbone with occupied coordinates excluded. This is a tested setting, not a capacity ceiling.
  4. Verification reliability. Section 3.2, Eq. 8, requires all three conditions: WER≥227/256\mathrm{WER}\ge 227/256, prel≤0.05p_{\mathrm{rel}}\le 0.05, and pwk≤0.05p_{\mathrm{wk}}\le 0.05. The minimum correct-block count t⋆=227t^\star=227 meets the 2−1282^{-128} tail bound under the independent wrong-code null (Section 3.3, Eq. 9; Appendix D.3). This is not a bound on adaptive attack success.

To make clearer presentation, we will provide focused problem statement and research objective for revision. We will close Section 2 (Threat Model) with a consolidated problem statement: given an original checkpoint M0M_0, a secret key kk, and a codeword of BB bits, the owner releases a marked checkpoint MkM_k and keeps a private enrollment record. Given a candidate MM, the verifier must decide whether the registered watermark remains detectable. The adversary seeks rejection at low utility cost.

AuthorsOpenReview ↗

Response to Reviewer IP56 (cont.)

W3. How do the components work together?

R3: In the original submission, Sections 3.1–3.2 and Fig. 1 give explanations of individual operations. We define differential parameter watermarking as a paradigm that encodes multi-bit ownership signatures into baseline-referenced relative differences (contrasts) between paired scalar weights, rather than into individual absolute parameter values or runtime activation/routing trajectories. Conventional weight-based watermarking methods (e.g., Uchida et al., 2017; GaussMark; NeuralMark) typically embed signatures into absolute weight values, linear projections, or global parameter statistics under the assumption of a uniform dense weight substrate.

The following justifications are already provided in the original submission:

Why use differential? For pair ii in block jj, let ai=(ε/2)sicja_i=(\varepsilon/\sqrt{2})s_i c_j. Eqs. 1–2 add aia_i to coordinate uiu_i and subtract aia_i from viv_i. The sum is unchanged, but the contrast shifts by 2ai=2εsicj2a_i=\sqrt{2}\varepsilon s_i c_j. Subtracting the stored baseline bi=M0[ui]−M0[vi]b_i=M_0[u_i]-M_0[v_i] from the candidate contrast gives did_i. The vote is zi=sign⁡(sidi)z_i=\operatorname{sign}(s_i d_i); zero votes abstain and tied blocks are unrecovered (Section 3.2; Fig. 1(c)–(d)).

Why not write to one coordinate? At equal embedding energy, Appendix A.4 (Eqs. 12–14) reports a measured signal-to-noise ratio of 1.288±0.0281.288\pm0.028 relative to single-coordinate embedding under the tested INT4 condition. The gain depends on the carrier and reference error statistics. Appendix A.1, Tables 4–5, examine exposure to expert damage and within-block dispersion; Appendix A.3, Table 7, compares carrier selection at fixed expert quotas.

To provide a structured introduction, we will present these functions as an integrated whole at the outset of Section 3.1, following the deployment overview described in R1:

Balanced allocation determines how many carriers each eligible expert receives. Row response scores guide the choice of scalar carrier coordinates within each expert. A keyed procedure pairs each carrier with an unused reference in that expert and orders the pairs into blocks across experts and layers. Each block encodes one bit. At verification, the owner reads each change in contrast, removes its spreading sign, and counts unit votes within the block. Allocation limits exposure to localized damage; spreading distributes evidence for each bit; unit voting prevents a single large error from dominating a block.

W4. Removal, overwriting, and retention limits

R4: The targeted removal and overwrite tests in the first table are already included in the original submission. WER is reported in percent; higher is better.

Existing testSettingWER (%)Already included in Original submission
Expert removal10% / 50% of experts100.00 / 99.92Section 4.4, Table 3; Appendix C.3, Table 15
Routing-frequency noiseFour tested perturbation budgets100.00 at each budgetAppendix C.3, Table 16
Pair decorrelationSmallest to largest tested budget100.00 to 90.82Appendix C.3, Table 16 (sign vote)
Random overwriteOne million scalars99.92 ± 0.16Appendix C.2, Table 13
Model fusionMixing coefficient 0.25100.00 ± 0.00Appendix C.2, Table 13
Sequential registration16 registrations; clean / shared INT4100.00 / 96.02Section 4.2; Appendix D.4

Sequential registration tests coexistence when the owner excludes occupied coordinates, not hostile overwriting by an independent party. After shared INT4 quantization, all 160 marks were accepted.

Supplementary overwrite controls (new to be added in revision). A denotes the owner’s mark and B a second mark embedded without A’s key. The columns distinguish survival of A from recovery of B; a dash means there is no second mark. We will report these additional controls in Appendix C.2.

ControlSettings testedA: WER (%)B: WER (%)
Row overwriteRandom rows; proxy-selected rows100.0—
Second mark: change amplitudeB/A amplitude ratio: 0.5, 1, 2, 4; equal pair counts100.0100.0
Second mark: change pair countB/A pair-count ratio: 2, 4; equal amplitude100.0100.0
Repeated embedding1, 2, 4, 8 embedding operations100.0100.0

Modern quantization (new to be added in revision). We will add RTN and GPTQ to Section 4.4 and Appendix C.1 alongside the existing packed INT4 result.

MetricCalibrated packed INT4 (existing, Table 12)RTN INT4 (new)GPTQ INT4 (new)
WER (%, mean ± SD)99.61 ± 0.3798.09 ± 0.4395.82 ± 1.37

Extended adaptive attacks are detailed in Response to W5.

As shown in the experimental results reported in the original submission and new results to be added to the revision, SparseMark demonstrates consistent robustness across a wide variety of attacks.

AuthorsOpenReview ↗

Response to Reviewer IP56 (cont.)

W5. Adaptive white-box robustness

R5: Following the reviewer's suggestion, we increased the verifier-query budget from 200 to 1,000. Unlike the original single-group proposals, the new proposals can modify several experts at once, guided by public proxy scores without the owner’s key. We will summarize the results alongside Table 3 in Section 4.4 and document the protocol in Appendix C.4. “Clean” describes the starting checkpoint, not the absence of an attack.

Starting checkpointFinal WER (%, mean ± SD)ΔPPL\Delta\mathrm{PPL} (mean ± SD)
Clean (ε=10−3\varepsilon = 10^{-3})100.00 ± 0.00+0.228 ± 0.223
Calibrated packed INT498.48 ± 0.68+0.419 ± 0.040
RTN INT493.91 ± 1.20+0.332 ± 0.015
GPTQ INT492.62 ± 1.59+0.255 ± 0.068
Per-channel fake INT4 (ε=2×10−3\varepsilon = 2 \times 10^{-3})99.45 ± 0.35+0.194 ± 0.166

Additional experimental results across 1,000 verifier queries confirm that our method maintains consistent performance.

We sincerely hope that our clarifications and planned revisions have thoroughly addressed your concerns, and if so, we would be deeply grateful if you would kindly consider a more favorable evaluation of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.