- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LegoMark, an efficient black-box watermarking framework for tracing model leakage in federated fine-tuning. It constructs client-specific watermarks from shared watermark bases, reducing injection and verification costs from to . The method introduces composition-aware refinement to address interference between watermark bases and constant-weight coding to prevent misattribution under a Z-channel assumption. Experiments across multiple tasks, models, and post-processing attacks demonstrate high attribution accuracy with minimal utility loss and no observed misattribution, achieving 4.4× faster injection and 12.3× faster verification than TraMark at 100 clients.
Critical strengths
-
The use of shared, composable watermark bases reduces watermark injection and verification costs from to , addressing a key scalability bottleneck in existing methods. The substantial empirical speedups further demonstrate its practical value.
-
The proposed composition-aware refinement effectively addresses interference between independently trained watermark bases. The ablation studies demonstrate its importance in maintaining reliable attribution, supporting the effectiveness of the compositional design.
-
The constant-weight coding scheme provides a theoretical guarantee against misattribution under the Z-channel assumption. Extensive experiments across multiple tasks, models, and post-processing attacks demonstrate high attribution accuracy, minimal utility degradation, and no observed misattribution.
Critical weaknesses
-
The paper does not sufficiently investigate potential cross-client watermark contamination during federated aggregation. Since clients perform local training on already-watermarked adapters, their updates may contain watermark-related information that propagates to other clients through task-arithmetic aggregation. Although the experiments show no false activations under the proposed aggregation scheme, the paper lacks a theoretical analysis or targeted evaluation of this potential leakage, raising concerns about attribution reliability in practical federated settings.
-
The robustness evaluation primarily considers standard post-processing operations, rather than adversaries explicitly optimizing to remove or manipulate watermarks while preserving model utility. This limits the evidence for security against strategically motivated model leakage.
Questions for authors
see weakness
3 comments
Response to Reviewer RxSc (2/3)
Continuing our response to [W1].
To quantify how applying the aggregated task update and reinjection each change the inactive-bit signal, we report the changes in mean target-response log-probability below:
| Round | Applying the aggregated task update | Reinjection | Net change |
|---|---|---|---|
| 1 | — | +3.11 | — |
| 2 | +0.66 | −1.27 | −0.61 |
| 3 | +0.21 | −0.97 | −0.76 |
| 4 | +0.16 | −0.86 | −0.70 |
| 5 | +0.17 | −0.57 | −0.40 |
Applying the aggregated task update increases the mean inactive-bit target log-probability by 0.16–0.66 nats per token in rounds 2–5. In round 1, the first injection raises the inactive-bit mean by 3.11 nats relative to the unwatermarked adapter; this reflects newly learned bases generalizing to other triggers rather than any transfer across clients, and inactive bits remain 5.63 nats below active bits. In rounds 2–5, reinjection decreases the inactive-bit mean by 0.57–1.27 nats, exceeding the preceding increase in every round. The effect therefore does not accumulate: the net change is negative in each of these rounds.
3. Does this affect attribution reliability? A nonzero likelihood for an inactive-bit target response does not itself imply activation, which requires its generated-response match rate to reach the verification threshold. Final adapters distributed after reinjection achieve 100% SAR and 0% MAR on AG-News, DBPedia-14, SNLI, and SLAKE under both IID and non-IID data (Table 1).
Moreover, contamination produces only errors, which give and are rejected by the weight check. We have generalized Proposition 1 accordingly (cf. Proposition 3 in Appendix [A.8]). Consistently, even the ablation without task arithmetic, with 63.52% FAR, has 0.00% MAR (Table 4).
In summary, aggregation leaves a small and transient trace on inactive bits, as the reviewer anticipated, but it does not accumulate across rounds, it is reduced by the subsequent injection, and it remains far from the activation threshold. Even if it were larger, it could not by itself lead to misattribution. We have added this step-by-step analysis and the results to Appendix [A.12].
Response to Reviewer RxSc (3/3)
[W2] The robustness evaluation primarily considers standard post-processing operations, rather than adversaries explicitly optimizing to remove or manipulate watermarks while preserving model utility. This limits the evidence for security against strategically motivated model leakage.
We thank the reviewer for suggesting a broader evaluation of watermark-removal and manipulation attempts. We extend our experiments to examine whether an attacker without access to the secret watermark information can weaken attribution while preserving task accuracy, and what kinds of attribution failures occur.
1. What a leaking client can do. A leaking client may know that its model has been watermarked, but the triggers, target responses, base watermark datasets, and codebook remain private to the server. With access to its adapter, its local data, and public data, the client can attempt to remove the watermark in two ways: directly modifying the adapter parameters without training (e.g., pruning or quantization), or modifying them through further training (e.g., fine-tuning). These two types of operations therefore provide natural ways to evaluate robustness against watermark removal and are used in the federated watermarking literature (Shao et al., 2024; Zhao et al., 2026; Xu et al., 2026). Our original experiments evaluate both types (Section [4.2] and Appendix [A.9.6] and [A.9.7]). Here, we extend this evaluation with one additional attack from each category.
2. Additional removal attacks. Without training, an attacker can go beyond pruning or quantization by exploiting the adapter's low-rank structure. We therefore test rank truncation, which discards smaller singular components in an attempt to remove watermark information while retaining task-relevant information. With further training, an attacker can attempt to overwrite the watermark using substantially more data and epochs than in our original experiments, while mixing in main-task data to preserve task accuracy. We test this through extended fine-tuning on a mixture of main-task and auxiliary-task data.
-
Rank truncation (parameter modification). We apply SVD to each LoRA update and retain only the top singular directions. We truncate the adapters from the default rank to on SNLI, where the unattacked MTA is 92.54%. Attribution is fully preserved down to rank 2, i.e., 6% of the original rank. Even at rank 1, SAR remains 90.00%, and the remaining cases are abstentions. MAR and NWR are 0% and omitted.
Rank MTA SAR AbR 4 92.41 100 0 2 92.26 100 0 1 91.70 90 10 -
Utility-preserving fine-tuning (continued training). We fine-tune each leaked adapter on a mixture of 500 SNLI and 2,500 GSM8K samples for 20 epochs without server reinjection, i.e., six times as many training samples and four times the maximum number of fine-tuning epochs used in our original study. We evaluate every 5 epochs at the default verification threshold and two lower thresholds. Each cell reports SAR / AbR / NWR; MAR is 0 in every cell.
Epoch 5 10 15 20 0.5 91.67 / 8.33 / 0 85 / 10 / 5 78.33 / 16.67 / 5 71.67 / 21.67 / 6.67 0.1 100 / 0 / 0 95 / 5 / 0 90 / 10 / 0 90 / 10 / 0 0.01 100 / 0 / 0 100 / 0 / 0 95 / 5 / 0 96.67 / 3.33 / 0 SNLI / GSM8K Acc. 92.49 / 77.07 92.20 / 75.74 91.82 / 75.72 91.44 / 75.40 At the default threshold of , 20 epochs of fine-tuning reduce SAR from 100.00% to 71.67%, while SNLI accuracy decreases from 92.54% to 91.44%, a drop of 1.10 percentage points. This shows that removal attempts can reduce successful attribution while largely preserving task accuracy. For the same attacked models, lowering the verification threshold to raises SAR to 96.67%. We have added these results to Appendix [A.9.7].
3. Manipulation. To be attributed to another client, an adapter must activate bits outside its own message, i.e., produce secret target responses for secret triggers, which a client cannot target without the server's watermark materials. Consistently, no inactive bit is activated under any attack above, even at , and unwatermarked adapters produce no target response in 180,000 trigger queries (Appendix [A.8]). The one exception we have identified is collusion, which lies outside our threat model: merging three or more clients' adapters can produce an innocent client's message (see our response to Reviewer dZsV [W2] and Appendix [A.11]).
We sincerely thank the reviewer again for your time and constructive comments. We hope these clarifications and additional results have fully resolved your concerns, and we remain open to any further discussion.
Response to Reviewer RxSc (1/3)
Dear reviewer RxSc,
We sincerely appreciate your time in carefully reading our work and providing comments and suggestions to improve the quality of the manuscript. Below, we address your questions and concerns.
We thank the reviewer for highlighting this potential source of cross-client watermark interference. We explain the update process and then examine its effect on watermark responses and attribution.
1. Can watermark-related effects propagate through aggregation? Each client's identity specifies which watermark bases its adapter should contain. We call the corresponding bits active for that client, and the unselected bits inactive.
Local training. Let θit−1 denote client i's starting adapter in round t. The client trains this adapter on its local task data, and the server computes its task update by subtracting the starting adapter from the locally trained adapter.
Aggregation. Denoting the weighted average of all clients' task updates by τˉt, the server constructs client i's adapter before watermark reinjection as
θ^it=θit−1+τˉt.
When local training starts from a watermarked adapter, its gradients and training trajectory depend on that starting point. Subtracting the starting adapter avoids directly averaging the existing watermark offsets, but does not remove this dependence from the task updates. Their average can therefore introduce watermark-related effects from other clients into θ^it, including effects associated with bits that are inactive for client i.
Watermarking. The server next learns and refines watermark bases using its watermark datasets. Bootstrap learning starts from a common anchor obtained by averaging the adapters θ^it. Composition-aware refinement trains each base within a client-specific combination of selected bases and that client's adapter. Adding the refined bases selected by client i's identity to θ^it produces the distributed adapter θit. This step reinforces the client's active watermark behaviors, but does not explicitly optimize a loss that suppresses its inactive bits. We therefore compare the same client's watermark responses before applying the aggregated task update (θit−1), after applying it (θ^it), and after reinjection (θit), as reported below.
2. Does this effect accumulate across rounds? On IID SNLI, we measure how likely the model is to produce the target response for each watermark bit, using mean per-token log-probability in nats. Values closer to zero indicate higher likelihood. The table places the three stages side by side for each round. Each cell reports active / inactive averages over clients and the corresponding bits. The dashes in round 1 mark the initial round-0 adapter, which has not yet been watermarked.
After the first watermark injection, the mean target-response log-probability is −0.01 for active bits and −5.64 for inactive bits, showing a clear separation. In rounds 2–5, this separation remains after applying the aggregated task update: active-bit means stay close to zero, while inactive-bit means remain substantially lower.
(Our response to [W1] continues in the next comment, Part 2/3.)