acceptodds
Under review as a conference paper at ICLR 2027

LegoMark: Efficient and Traceable Black-Box Watermarks for Federated Fine-Tuning

Abstract

Federated fine-tuning (FFT) enables clients to collaboratively adapt a large language model to downstream tasks without sharing their local data, but clients may redistribute the trained adapters without authorization. Existing traceable black-box watermarks identify the source of a leaked model through its outputs, but injecting and verifying a distinct watermark for every client incurs costs that grow linearly with the client population . We propose LegoMark, a watermarking method for FFT that constructs client-specific watermarks by composing shared watermark bases. The server encodes each client's identity with a unique non-zero binary message, learns one watermark base for each bit position, and combines the bases selected by the active positions of the message, reducing both the number of injection procedures per round and the number of watermark datasets queried per suspect model from to . To address interference when independently trained bases are combined, we introduce composition-aware refinement that optimizes each watermark base to survive when combined with other bases. Motivated by an empirical asymmetry in decoding errors that is well approximated by a Z-channel, whereby active bits may be lost while inactive bits are essentially never activated, we adopt a constant-weight codebook in which all messages share the same number of active bits. Under this approximation, a damaged message cannot match any valid one, so active-bit losses lead to withheld attribution rather than misattribution. Extensive experiments show that LegoMark preserves main-task accuracy while achieving a high attribution rate and never misattributes under the evaluated post-processing attacks. At , it reduces the wall-clock injection time per round by and the verification time per suspect adapter by compared to the state-of-the-art method TraMark.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
watermark, traceability, efficiency

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_RxScOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LegoMark, an efficient black-box watermarking framework for tracing model leakage in federated fine-tuning. It constructs client-specific watermarks from shared watermark bases, reducing injection and verification costs from O(n)O(n) to O(log⁡n)O(\log n). The method introduces composition-aware refinement to address interference between watermark bases and constant-weight coding to prevent misattribution under a Z-channel assumption. Experiments across multiple tasks, models, and post-processing attacks demonstrate high attribution accuracy with minimal utility loss and no observed misattribution, achieving 4.4× faster injection and 12.3× faster verification than TraMark at 100 clients.

Critical strengths
  1. The use of shared, composable watermark bases reduces watermark injection and verification costs from O(n)O(n) to O(log⁡n)O(\log n), addressing a key scalability bottleneck in existing methods. The substantial empirical speedups further demonstrate its practical value.

  2. The proposed composition-aware refinement effectively addresses interference between independently trained watermark bases. The ablation studies demonstrate its importance in maintaining reliable attribution, supporting the effectiveness of the compositional design.

  3. The constant-weight coding scheme provides a theoretical guarantee against misattribution under the Z-channel assumption. Extensive experiments across multiple tasks, models, and post-processing attacks demonstrate high attribution accuracy, minimal utility degradation, and no observed misattribution.

Critical weaknesses
  1. The paper does not sufficiently investigate potential cross-client watermark contamination during federated aggregation. Since clients perform local training on already-watermarked adapters, their updates may contain watermark-related information that propagates to other clients through task-arithmetic aggregation. Although the experiments show no false activations under the proposed aggregation scheme, the paper lacks a theoretical analysis or targeted evaluation of this potential leakage, raising concerns about attribution reliability in practical federated settings.

  2. The robustness evaluation primarily considers standard post-processing operations, rather than adversaries explicitly optimizing to remove or manipulate watermarks while preserving model utility. This limits the evidence for security against strategically motivated model leakage.

Questions for authors

see weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer RxSc (1/3)

Dear reviewer RxSc,

We sincerely appreciate your time in carefully reading our work and providing comments and suggestions to improve the quality of the manuscript. Below, we address your questions and concerns.

[W1] The paper does not sufficiently investigate potential cross-client watermark contamination during federated aggregation. Since clients perform local training on already-watermarked adapters, their updates may contain watermark-related information that propagates to other clients through task-arithmetic aggregation. Although the experiments show no false activations under the proposed aggregation scheme, the paper lacks a theoretical analysis or targeted evaluation of this potential leakage, raising concerns about attribution reliability in practical federated settings.

We thank the reviewer for highlighting this potential source of cross-client watermark interference. We explain the update process and then examine its effect on watermark responses and attribution.

1. Can watermark-related effects propagate through aggregation? Each client's identity specifies which watermark bases its adapter should contain. We call the corresponding bits active for that client, and the unselected bits inactive.

  • Local training. Let θit−1\theta_i^{t-1} denote client ii's starting adapter in round tt. The client trains this adapter on its local task data, and the server computes its task update by subtracting the starting adapter from the locally trained adapter.

  • Aggregation. Denoting the weighted average of all clients' task updates by τˉt\bar{\tau}^t, the server constructs client ii's adapter before watermark reinjection as

    θ^it=θit−1+τˉt.\hat{\theta}_i^t=\theta_i^{t-1}+\bar{\tau}^t.

    When local training starts from a watermarked adapter, its gradients and training trajectory depend on that starting point. Subtracting the starting adapter avoids directly averaging the existing watermark offsets, but does not remove this dependence from the task updates. Their average can therefore introduce watermark-related effects from other clients into θ^it\hat{\theta}_i^t, including effects associated with bits that are inactive for client ii.

  • Watermarking. The server next learns and refines watermark bases using its watermark datasets. Bootstrap learning starts from a common anchor obtained by averaging the adapters θ^it\hat{\theta}_i^t. Composition-aware refinement trains each base within a client-specific combination of selected bases and that client's adapter. Adding the refined bases selected by client ii's identity to θ^it\hat{\theta}_i^t produces the distributed adapter θit\theta_i^t. This step reinforces the client's active watermark behaviors, but does not explicitly optimize a loss that suppresses its inactive bits. We therefore compare the same client's watermark responses before applying the aggregated task update (θit−1\theta_i^{t-1}), after applying it (θ^it\hat{\theta}_i^t), and after reinjection (θit\theta_i^t), as reported below.

2. Does this effect accumulate across rounds? On IID SNLI, we measure how likely the model is to produce the target response for each watermark bit, using mean per-token log-probability in nats. Values closer to zero indicate higher likelihood. The table places the three stages side by side for each round. Each cell reports active / inactive averages over clients and the corresponding bits. The dashes in round 1 mark the initial round-0 adapter, which has not yet been watermarked.

RoundBefore aggregation update (active / inactive)After aggregation update (active / inactive)After reinjection (active / inactive)
1— / —−8.75 / −8.75−0.01 / −5.64
2−0.01 / −5.64−0.03 / −4.98−0.00 / −6.25
3−0.00 / −6.25−0.00 / −6.04−0.00 / −7.01
4−0.00 / −7.01−0.00 / −6.85−0.00 / −7.71
5−0.00 / −7.71−0.00 / −7.54−0.00 / −8.11

After the first watermark injection, the mean target-response log-probability is −0.01 for active bits and −5.64 for inactive bits, showing a clear separation. In rounds 2–5, this separation remains after applying the aggregated task update: active-bit means stay close to zero, while inactive-bit means remain substantially lower.

(Our response to [W1] continues in the next comment, Part 2/3.)

AuthorsOpenReview ↗

Response to Reviewer RxSc (2/3)

Continuing our response to [W1].

To quantify how applying the aggregated task update and reinjection each change the inactive-bit signal, we report the changes in mean target-response log-probability below:

RoundApplying the aggregated task updateReinjectionNet change
1—+3.11—
2+0.66−1.27−0.61
3+0.21−0.97−0.76
4+0.16−0.86−0.70
5+0.17−0.57−0.40

Applying the aggregated task update increases the mean inactive-bit target log-probability by 0.16–0.66 nats per token in rounds 2–5. In round 1, the first injection raises the inactive-bit mean by 3.11 nats relative to the unwatermarked adapter; this reflects newly learned bases generalizing to other triggers rather than any transfer across clients, and inactive bits remain 5.63 nats below active bits. In rounds 2–5, reinjection decreases the inactive-bit mean by 0.57–1.27 nats, exceeding the preceding increase in every round. The effect therefore does not accumulate: the net change is negative in each of these rounds.

3. Does this affect attribution reliability? A nonzero likelihood for an inactive-bit target response does not itself imply activation, which requires its generated-response match rate to reach the verification threshold. Final adapters distributed after reinjection achieve 100% SAR and 0% MAR on AG-News, DBPedia-14, SNLI, and SLAKE under both IID and non-IID data (Table 1).

Moreover, contamination produces only 0→10\to1 errors, which give h(m′)>wh(\mathbf{m}')>w and are rejected by the weight check. We have generalized Proposition 1 accordingly (cf. Proposition 3 in Appendix [A.8]). Consistently, even the ablation without task arithmetic, with 63.52% FAR, has 0.00% MAR (Table 4).


In summary, aggregation leaves a small and transient trace on inactive bits, as the reviewer anticipated, but it does not accumulate across rounds, it is reduced by the subsequent injection, and it remains far from the activation threshold. Even if it were larger, it could not by itself lead to misattribution. We have added this step-by-step analysis and the results to Appendix [A.12].

AuthorsOpenReview ↗

Response to Reviewer RxSc (3/3)

[W2] The robustness evaluation primarily considers standard post-processing operations, rather than adversaries explicitly optimizing to remove or manipulate watermarks while preserving model utility. This limits the evidence for security against strategically motivated model leakage.

We thank the reviewer for suggesting a broader evaluation of watermark-removal and manipulation attempts. We extend our experiments to examine whether an attacker without access to the secret watermark information can weaken attribution while preserving task accuracy, and what kinds of attribution failures occur.

1. What a leaking client can do. A leaking client may know that its model has been watermarked, but the triggers, target responses, base watermark datasets, and codebook remain private to the server. With access to its adapter, its local data, and public data, the client can attempt to remove the watermark in two ways: directly modifying the adapter parameters without training (e.g., pruning or quantization), or modifying them through further training (e.g., fine-tuning). These two types of operations therefore provide natural ways to evaluate robustness against watermark removal and are used in the federated watermarking literature (Shao et al., 2024; Zhao et al., 2026; Xu et al., 2026). Our original experiments evaluate both types (Section [4.2] and Appendix [A.9.6] and [A.9.7]). Here, we extend this evaluation with one additional attack from each category.

2. Additional removal attacks. Without training, an attacker can go beyond pruning or quantization by exploiting the adapter's low-rank structure. We therefore test rank truncation, which discards smaller singular components in an attempt to remove watermark information while retaining task-relevant information. With further training, an attacker can attempt to overwrite the watermark using substantially more data and epochs than in our original experiments, while mixing in main-task data to preserve task accuracy. We test this through extended fine-tuning on a mixture of main-task and auxiliary-task data.

  • Rank truncation (parameter modification). We apply SVD to each LoRA update and retain only the top singular directions. We truncate the adapters from the default rank r=32r=32 to r∈{4,2,1}r \in \{4,2,1\} on SNLI, where the unattacked MTA is 92.54%. Attribution is fully preserved down to rank 2, i.e., 6% of the original rank. Even at rank 1, SAR remains 90.00%, and the remaining cases are abstentions. MAR and NWR are 0% and omitted.

    Rank rrMTASARAbR
    492.411000
    292.261000
    191.709010
  • Utility-preserving fine-tuning (continued training). We fine-tune each leaked adapter on a mixture of 500 SNLI and 2,500 GSM8K samples for 20 epochs without server reinjection, i.e., six times as many training samples and four times the maximum number of fine-tuning epochs used in our original study. We evaluate every 5 epochs at the default verification threshold ν=0.5\nu=0.5 and two lower thresholds. Each cell reports SAR / AbR / NWR; MAR is 0 in every cell.

    ν\nuEpoch 5101520
    0.591.67 / 8.33 / 085 / 10 / 578.33 / 16.67 / 571.67 / 21.67 / 6.67
    0.1100 / 0 / 095 / 5 / 090 / 10 / 090 / 10 / 0
    0.01100 / 0 / 0100 / 0 / 095 / 5 / 096.67 / 3.33 / 0
    SNLI / GSM8K Acc.92.49 / 77.0792.20 / 75.7491.82 / 75.7291.44 / 75.40

    At the default threshold of ν=0.5\nu=0.5, 20 epochs of fine-tuning reduce SAR from 100.00% to 71.67%, while SNLI accuracy decreases from 92.54% to 91.44%, a drop of 1.10 percentage points. This shows that removal attempts can reduce successful attribution while largely preserving task accuracy. For the same attacked models, lowering the verification threshold to ν=0.01\nu=0.01 raises SAR to 96.67%. We have added these results to Appendix [A.9.7].

3. Manipulation. To be attributed to another client, an adapter must activate bits outside its own message, i.e., produce secret target responses for secret triggers, which a client cannot target without the server's watermark materials. Consistently, no inactive bit is activated under any attack above, even at ν=0.01\nu=0.01, and unwatermarked adapters produce no target response in 180,000 trigger queries (Appendix [A.8]). The one exception we have identified is collusion, which lies outside our threat model: merging three or more clients' adapters can produce an innocent client's message (see our response to Reviewer dZsV [W2] and Appendix [A.11]).


We sincerely thank the reviewer again for your time and constructive comments. We hope these clarifications and additional results have fully resolved your concerns, and we remain open to any further discussion.

Reviewer_wTzuOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

LegoMark aims to identify which client's copy of a federated LoRA adapter was redistributed by querying the resulting model. The server learns a small pool of reusable watermark components and combines them according to a binary code assigned to each client. It refines these components to remain detectable after combination and uses codes with the same number of active bits so that lost watermark bits lead to withheld attribution under its assumed error model. Experiments report lower watermark training and verification costs than per-client watermarking, with limited changes in main-task performance in the evaluated settings.

Critical strengths
  1. The method provides a measurable efficiency improvement: Sharing watermark bases reduces the number of watermark training procedures and verification datasets from linear to logarithmic in the client population, with clear injection and verification speedups compared to TraMark.
  2. The paper conducts extensive ablation studies and addresses interference among combined watermark components: The role of composition-aware refinement is supported by Table 2, where removing this component reduces SAR.
Critical weaknesses
  1. The practical protection scope and need for repeated injection require clearer justification: The introduction identifies unauthorized redistribution as a risk, but provides little concrete explanation of the intended collaboration and the redistribution permissions assumed among participants. The protection scope should be clarified for collectively trained adapters, locally retained checkpoints, and full models containing merged adapters. Section 2 focuses on redistribution of final adapters after training, so the additional protection provided by injecting watermarks every communication round remains insufficiently explained relative to watermarking only at final distribution.
  2. Applicability beyond full client participation remains untested: All clients participate in every round, and the default training configuration uses one local epoch between server updates. Since the method maintains personalized adapters and refines shared watermark bases within client-specific compositions, intermittent participation raises unresolved questions about stale adapters and clients absent from the current refinement stage. The paper provides neither an operational account nor experimental evidence showing how attribution and task performance behave under these conditions.
  3. The attribution results provide limited evidence for persistent tracing beyond the evaluated conditions: The uniformly perfect SAR values in Table 1 establish successful identification of the tested adapters but offer little separation among methods or evidence about failure rates in deployment. Although the pruning, quantization, and further fine-tuning experiments are useful, the fine-tuning study uses only 500 AG-News examples for at most five epochs, leaving persistence under substantially longer training without server reinjection uncertain. The empirical verification threshold would also benefit from attribution tests on independently trained, unwatermarked adapters to characterize the risk of falsely identifying an unrelated model.
Questions for authors
  1. Regarding protection scope and repeated injection: What concrete collaborative deployment and redistribution rules motivate the threat model? Please clarify which retained checkpoints and full-model deployment formats are covered, and explain what additional protection per-round injection provides compared with watermarking only the final distributed adapters.
  2. Regarding partial participation: Can LegoMark support a fixed client population in which only a subset participates in each round, including clients returning after several missed rounds? Please explain how inactive clients’ adapters are synchronized and report task accuracy, attribution outcomes, and server cost as participation decreases.
  3. Regarding attribution reliability and persistence: How do attribution outcomes change with substantially longer local fine-tuning and larger adaptation datasets, without watermark reinjection and while retaining useful task performance? Please report successful attribution, misattribution, abstention, and non-detection together, and include false-attribution tests on independently trained, unwatermarked adapters across verification thresholds.
5 comments
AuthorsOpenReview ↗

Response to Reviewer wTzu (1/3)

Dear reviewer wTzu,

We sincerely appreciate your time in carefully reading our work and providing comments and suggestions to improve the quality of the manuscript. Below, we address your questions and concerns.

[W1] The practical protection scope and need for repeated injection require clearer justification: The introduction identifies unauthorized redistribution as a risk, but provides little concrete explanation of the intended collaboration and the redistribution permissions assumed among participants. The protection scope should be clarified for collectively trained adapters, locally retained checkpoints, and full models containing merged adapters. Section 2 focuses on redistribution of final adapters after training, so the additional protection provided by injecting watermarks every communication round remains insufficiently explained relative to watermarking only at final distribution.

[Q1] Regarding protection scope and repeated injection: What concrete collaborative deployment and redistribution rules motivate the threat model? Please clarify which retained checkpoints and full-model deployment formats are covered, and explain what additional protection per-round injection provides compared with watermarking only the final distributed adapters.

We thank the reviewer for highlighting the need to clarify the threat model and the role of per-round injection.

1. Reason of repeated injection. LegoMark is designed to support attribution of client-held adapters that may be redistributed without authorization during or after training. An intermediate model can already be valuable once it achieves useful task performance, giving an attacker an incentive to deploy or redistribute it before training finishes. Since the defender cannot predict when this will occur, watermarking only at final distribution leaves earlier models without client-specific watermarks. LegoMark therefore embeds and refreshes these watermarks throughout training. To also cover the first round, LegoMark can additionally watermark the adapters before they are first distributed (an optional initial injection, not used in our main experiments). The table below reports per-round results with initial injection, averaged over non-IID SNLI and AG-News; MAR and NWR are 0.00% in every round.

RoundMTASARAbR
185.2096.673.33
287.1799.170.83
388.22100.000.00
489.57100.000.00
589.16100.000.00

Adapters are traceable from the very first round (SAR 96.67%) and reach 100% SAR from round 3, whereas under final-only injection none of them would be traceable. In every round, failed attributions are abstentions rather than misattributions. Since injection is performed in every round, its cost recurs in every round, which is why reducing it from Θ(n)\Theta(n) to Θ(log⁡n)\Theta(\log n) matters. We have added this discussion and these results to Appendix [A.9.10].

2. Collaboration rules. We consider a collaboration in which participants, such as hospitals jointly fine-tuning a medical VQA model, may use the jointly trained adapters for their own applications, while transferring, selling, or publicly releasing the model weights outside the collaboration requires authorization. When an unauthorized copy is accessible through an inference API, the server performs black-box verification to identify its source client. We have added these rules to the introduction (Lines [46–52]).

3. Protection scope. We agree with the reviewer. Our original statement in threat model is narrower than what our design targets: clients legitimately retain every model they hold, so any of these models may be leaked. We have revised the threat model in Section [2] (Lines [161–168]) to state that clients may redistribute any adapter they obtain at any point during or after training. The protection scope therefore covers:

  • (i) the watermarked adapter θit\theta_i^t received in every round, including the final one;
  • (ii) locally retained checkpoints (adapters) produced by local training;
  • (iii) models derived from them. A full model with the adapter merged into the backbone (W′=W+BAW' = W + BA) is functionally identical to the adapter-equipped model, so black-box verification applies unchanged. For adapters modified by further training or other post-processing, attribution depends on how well the watermark persists, which we examine in our robustness evaluations (Section [4.2], Appendix [A.9.7]).

Because the codebook and base watermark datasets are fixed across rounds, the verifier does not need to know which round a leaked model comes from; the same bb queries apply.

AuthorsOpenReview ↗

Response to Reviewer wTzu (2/3)

[W2] Applicability beyond full client participation remains untested: All clients participate in every round, and the default training configuration uses one local epoch between server updates. Since the method maintains personalized adapters and refines shared watermark bases within client-specific compositions, intermittent participation raises unresolved questions about stale adapters and clients absent from the current refinement stage. The paper provides neither an operational account nor experimental evidence showing how attribution and task performance behave under these conditions.

[Q2] Regarding partial participation: Can LegoMark support a fixed client population in which only a subset participates in each round, including clients returning after several missed rounds? Please explain how inactive clients’ adapters are synchronized and report task accuracy, attribution outcomes, and server cost as participation decreases.

Yes. LegoMark allows partial participation within a fixed client population. In each round, only the participating clients perform local training and contribute task vectors. The server aggregates their task vectors, with weights renormalized over the participating clients.

1. Synchronizing inactive clients' adapters. Each client's adapter can be represented by a common component and a combination of accumulated watermark bases selected by its fixed identity code (cf. Appendix A.7.2). Using this shared representation, the server can incorporate the aggregated task update into the common component and reconstruct each client's current personalized adapter on demand. Bootstrap learning and composition-aware refinement continue to use the full client codebook and the corresponding server-side adapter representations. During refinement, compositions associated with any client in the full codebook remain eligible, including clients not participating in local training; this stage requires only server-side models and watermark data.

2. Clients returning after missed rounds. When a client returns, its current personalized adapter can be reconstructed from the updated common component and accumulated watermark bases according to its unchanged identity code. The client receives this adapter before resuming local training, without replaying missed rounds. In the revised manuscript, we state Algorithm 3 for general participation, with full participation as the default setting in our experiments (Lines [184-187]).

3. Results under partial client participation. We evaluate this setting with n=100n=100 clients on SNLI (non-IID) for T=10T=10 communication rounds, with participation rates ρ∈{0.1,0.2,0.3,0.4}\rho\in\{0.1,0.2,0.3,0.4\}. We simulate partial client participation by randomly sampling a fraction ρ\rho of the clients in each round. Attribution is evaluated on the final adapters of all 100 clients. AbR, MAR, and NWR are 0.00% in all settings.

ρ\rhoFedAvg MTALegoMark MTASAR
0.190.7590.41100.00
0.289.9290.47100.00
0.391.0691.26100.00
0.491.1291.40100.00

At every tested participation rate, LegoMark achieves 100% SAR, with MTA within 0.55 percentage points of FedAvg. We have added these results and results under IID data, which show the same trend, to Section [4.3].

4. Server cost. Watermarking trains bb bases per round regardless of ρ\rho, while aggregation and communication scale with the ∣St∣|\mathcal{S}_t| sampled clients. Verification cost is independent of participation and remains Θ(log⁡n)\Theta(\log n).

AuthorsOpenReview ↗

Response to Reviewer wTzu (3/3)

[W3] The attribution results provide limited evidence for persistent tracing beyond the evaluated conditions: The uniformly perfect SAR values in Table 1 establish successful identification of the tested adapters but offer little separation among methods or evidence about failure rates in deployment. Although the pruning, quantization, and further fine-tuning experiments are useful, the fine-tuning study uses only 500 AG-News examples for at most five epochs, leaving persistence under substantially longer training without server reinjection uncertain. The empirical verification threshold would also benefit from attribution tests on independently trained, unwatermarked adapters to characterize the risk of falsely identifying an unrelated model.

[Q3] Regarding attribution reliability and persistence: How do attribution outcomes change with substantially longer local fine-tuning and larger adaptation datasets, without watermark reinjection and while retaining useful task performance? Please report successful attribution, misattribution, abstention, and non-detection together, and include false-attribution tests on independently trained, unwatermarked adapters across verification thresholds.

We sincerely appreciate the reviewer's thoughtful and detailed comments, which help us substantially strengthen the reliability evaluation of LegoMark. We address each point below.

1. Purpose of Table 1 (W3). Table 1 evaluates whether each method preserves fidelity while achieving traceability; perfect SAR there is the expected baseline rather than the point of comparison. The methods differ instead in fidelity (Table 1), in efficiency, which is the focus of our paper (Figure 2), and in how they fail under attack: under pruning, BEMark and TraMark misattribute up to 26.7% and 71.11% of adapters, whereas LegoMark never misattributes (Figures 3 and 6). We have revised Lines [376–399] to clarify this purpose of Table 1.

2. Attribution under longer fine-tuning (W3 and Q3). We fine-tune each leaked adapter for 20 epochs on 3,000 samples (500 SNLI and 2,500 GSM8K), i.e., six times as many training samples and four times the maximum number of fine-tuning epochs used in our original study, without any server reinjection. We include SNLI examples to preserve main-task utility and evaluate attribution and task accuracy every 5 epochs. For each verification threshold, the cells below report SAR / MAR / AbR / NWR (%); the final row reports task accuracy.

ν\nuEpoch 5Epoch 10Epoch 15Epoch 20
0.591.67 / 0.00 / 8.33 / 0.0085.00 / 0.00 / 10.00 / 5.0078.33 / 0.00 / 16.67 / 5.0071.67 / 0.00 / 21.67 / 6.67
0.1100.00 / 0.00 / 0.00 / 0.0095.00 / 0.00 / 5.00 / 0.0090.00 / 0.00 / 10.00 / 0.0090.00 / 0.00 / 10.00 / 0.00
0.01100.00 / 0.00 / 0.00 / 0.00100.00 / 0.00 / 0.00 / 0.0095.00 / 0.00 / 5.00 / 0.0096.67 / 0.00 / 3.33 / 0.00
SNLI / GSM8K Acc.92.49 / 77.0792.20 / 75.7491.82 / 75.7291.44 / 75.40

After 20 epochs, SNLI accuracy decreases from 92.54% to 91.44%, a drop of 1.10 percentage points, while SAR at the default threshold ν=0.5\nu=0.5 decreases from 100.00% to 71.67%. Of all evaluated adapters, 21.67% yield abstention, 6.67% yield non-detection, and none is misattributed. Thus, continued training can weaken attribution while largely preserving task accuracy, with failures mainly taking the form of abstention. For the same attacked models, lowering the verification threshold to ν=0.01\nu=0.01 raises SAR to 96.67%, with the remaining 3.33% yielding abstention. MAR remains 0.00% across all evaluated checkpoints and thresholds. We have added these results to Appendix [A.9.7].

3. False attribution on unwatermarked adapters (Q3). We train unwatermarked FedAvg adapters on SNLI, AG-News, and DBPedia-14 under both IID and non-IID data, each with three random seeds, and query each of the 18 adapters with all 100 of our watermark datasets, i.e., 10,000 trigger queries per adapter and 180,000 in total. None of them produces its target response. Hence every bit has a hit rate of 0, and every adapter is decoded as non-detection at any threshold ν∈(0,1]\nu \in (0, 1], giving 0.00% false attribution across all thresholds. With zero activations in 1,800 bit decisions (18 adapters × 100 datasets), the per-bit false-activation probability is below 0.17% (95% upper confidence bound by the rule of three); at the query level, no target response appears in 180,000 queries. We have added these results and analysis to Section [4.3] and Appendix [A.8].


We sincerely thank the reviewer again for your time and constructive comments. We hope these clarifications and additional results have fully resolved your concerns, and we remain open to any further discussion.

Reviewer_dZsVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LegoMark traces leaked LoRA adapters to their source clients using black-box queries. The proposed method constructs client-specific watermarks from a small set of shared watermark bases, reducing injection and verification overhead. Composition-aware training helps the bases work together, while constant-weight identities allow the verifier to abstain when watermark bits disappear.

Critical strengths
  1. The method addresses a practical problem with a clear improvement in watermarking efficiency.
  2. It reports substantial injection and verification speedups as client count increases.
  3. It preserves task accuracy while achieving strong attribution results across multiple datasets and backbones.
  4. It includes useful ablations showing why composition-aware refinement and task-arithmetic aggregation are needed.
Critical weaknesses
  1. The protection against incorrect attribution depends on the assumption that watermark bits disappear but inactive bits do not become active. The experiments support this behavior under the tested modifications, but do not test it under broader attacks.

  2. The threat model excludes collusion between clients. This is a meaningful limitation because the method deliberately shares watermark bases across client identities. Combining adapters could mix these bases and potentially produce an innocent client’s identity. A small collusion study would help show how the method behaves outside its intended setting.

  3. The attack evaluation mainly covers pruning, quantization, and additional fine-tuning. These are useful tests, but do not fully test an attacker deliberately trying to remove the watermark while preserving performance. Testing targeted removal methods that operate without knowing the secret triggers would make the method more robust.

  4. The efficiency claim considers watermark optimization and verification, rather than the entire server workload. The server still maintains personalized adapters and constructs one for each client. The reported timing gains are valuable, but memory use, adapter construction, and performance at larger client populations would give a fuller picture of scalability.

  5. The evaluation covers several datasets and includes useful component ablations, but the main ablation is limited to SNLI. Running it across different tasks and comparing against BEMark with the same constant-weight codebook can better isolate the contributions. The dataset coverage is reasonable, although effectiveness on open-ended generation remains unexplored.

Questions for authors
  1. How does attribution change across verification thresholds and smaller query budgets?
  2. Can targeted removal reduce attribution while keeping task accuracy close to its original level?
  3. Does the method remain effective under partial client participation and longer training runs?
4 comments
AuthorsOpenReview ↗

Response to Reviewer dZsV (1/4)

Dear reviewer dZsV,

We sincerely appreciate your time in carefully reading our work and providing comments and suggestions to improve the quality of the manuscript. Below, we address your questions and concerns.

[W1] The protection against incorrect attribution depends on the assumption that watermark bits disappear but inactive bits do not become active. The experiments support this behavior under the tested modifications, but do not test it under broader attacks.

[W3] The attack evaluation mainly covers pruning, quantization, and additional fine-tuning. These are useful tests, but do not fully test an attacker deliberately trying to remove the watermark while preserving performance. Testing targeted removal methods that operate without knowing the secret triggers would make the method more robust.

[Q1] How does attribution change across verification thresholds and smaller query budgets?

[Q2] Can targeted removal reduce attribution while keeping task accuracy close to its original level?

We thank the reviewer for suggesting a broader evaluation of watermark-removal attacks. We extend our experiments to examine whether an attacker without access to the secret watermark information can weaken attribution while preserving task accuracy, and whether these attacks induce inactive-bit activation or misattribution.

1. What a leaking client can do. A leaking client may know that its model has been watermarked, but the triggers, target responses, base watermark datasets, and codebook remain private to the server. With access to its adapter, its local data, and public data, the client can attempt to remove the watermark in two ways: directly modifying the adapter parameters without training (e.g., pruning or quantization), or modifying them through further training (e.g., fine-tuning). These two types of operations therefore provide natural ways to evaluate robustness against watermark removal and are used in the federated watermarking literature (Shao et al., 2024; Zhao et al., 2026; Xu et al., 2026). Our original experiments evaluate both types (Section [4.2] and Appendix [A.9.6] and [A.9.7]). Here, we extend this evaluation with one additional attack from each category.

2. Additional removal attacks (W1, W3). Without training, an attacker can go beyond pruning or quantization by exploiting the adapter’s low-rank structure. We therefore test rank truncation, which discards smaller singular components in an attempt to remove watermark information while retaining task-relevant information. With further training, an attacker can attempt to overwrite the watermark using substantially more data and epochs than in our original experiments, while mixing in main-task data to preserve task accuracy. We test this through extended fine-tuning on a mixture of main-task and auxiliary-task data.

  • Rank truncation (parameter modification). We apply SVD to each LoRA update and retain only the top singular directions. We truncate the adapters from the default rank r=32r=32 to r∈{4,2,1}r \in \{4,2,1\} on SNLI, where the unattacked MTA is 92.54%. Attribution is fully preserved down to rank 2, i.e., 6% of the original rank. Even at rank 1, SAR remains 90.00%, and the remaining cases are abstentions. MAR and NWR are 0% and omitted.

    Rank rrMTASARAbR
    492.411000
    292.261000
    191.709010
  • Utility-preserving fine-tuning (continued training). We fine-tune each leaked adapter on a mixture of 500 SNLI and 2,500 GSM8K samples for 20 epochs, evaluating every 5 epochs. To also answer Q1, we sweep the verification threshold ν\nu from 0.7 down to 0.01. Each cell reports SAR / AbR / NWR (%); MAR is 0.00% in every cell and is omitted.

    ν\nuEpoch 5101520
    0.775/25/058.33/35/6.6756.67/33.33/1041.67/41.67/16.67
    0.591.67/8.33/085/10/578.33/16.67/571.67/21.67/6.67
    0.395/5/093.33/6.67/085/15/083.33/11.67/5
    0.1100/0/095/5/090/10/090/10/0
    0.05100/0/098.33/1.67/090/10/093.33/6.67/0
    0.01100/0/0100/0/095/5/096.67/3.33/0
    SNLI/GSM8K Acc.92.49/77.0792.20/75.7491.82/75.7291.44/75.40

    At the default threshold of ν=0.5\nu=0.5, 20 epochs of fine-tuning reduce SAR from 100.00% to 71.67%, while SNLI accuracy decreases from 92.54% to 91.44%, a drop of 1.10 percentage points. This shows that removal attempts can reduce successful attribution while largely preserving task accuracy (Q2). For the same attacked models, lowering the verification threshold to ν=0.01\nu=0.01 raises SAR to 96.67%. Across the evaluated checkpoints, lowering ν\nu increases or maintains SAR by retaining weakened active bits, whereas higher thresholds increase the combined proportion of abstentions and non-detections (Q1).

(Our response to [W1], [W3], [Q1], and [Q2] continues in the next comment, Part 2/4.)

AuthorsOpenReview ↗

Response to Reviewer dZsV (2/4)

Continuing our response to [W1], [W3], [Q1], and [Q2].

These additional attacks can reduce successful attribution while largely preserving task accuracy. Nevertheless, we observe no inactive-bit activation across the tested rank-truncation and fine-tuning settings (FAR = 0.00%). All attribution failures therefore take the form of abstention or non-detection, with MAR remaining 0.00%. These results extend the empirical support for the Z-channel approximation to the additional removal attacks evaluated here (W1). We have added these results and analysis to Appendix [A.9.7].

3. Query budget (Q1). Our standard verification uses 100 triggered inputs per bit, i.e., 100b100b queries in total. We repeat verification after 20 epochs of utility-preserving fine-tuning, the most degraded setting above, using only zz randomly selected inputs per bit (bzbz queries in total). The table reports SAR (%) on the 20 adapters of one seed, averaged over 100 random subsets per adapter; MAR is 0.00% in every cell.

zzν=0.7\nu=0.7ν=0.5\nu=0.5ν=0.1\nu=0.1ν=0.01\nu=0.01
158.6058.6058.6058.60
548.9566.3585.7085.70
1054.2572.1589.8589.85
2051.7572.7589.6592.80
5048.3073.8589.6596.70
10045.0075.0090.00100.00

The budget can be reduced tenfold at little cost: with z=10z=10, SAR is 72.15% at the default ν=0.5\nu=0.5, compared with 75.00% for the full budget. Overall, attribution under attack depends far more on the threshold than on the budget, although a single query per bit is insufficient (58.60%). Reducing the budget never introduces misattribution, since inactive bits produce no target response on any input. We have added these results and a detailed analysis to Appendix [A.9.7].


[W2] The threat model excludes collusion between clients. This is a meaningful limitation because the method deliberately shares watermark bases across client identities. Combining adapters could mix these bases and potentially produce an innocent client’s identity. A small collusion study would help show how the method behaves outside its intended setting.

We appreciate the reviewer's insightful suggestion, which helps us better characterize the behavior of LegoMark beyond its intended threat model. As discussed in Section [2] and Appendix [A.5], our threat model follows prior work (Shao et al., 2024; Xu et al., 2026) and assumes that clients leak their adapters independently. As suggested, we conduct a small collusion study to show how LegoMark behaves outside this setting.

We use n=20n=20 clients. A coalition of C∈{2,3,4,5}C\in\{2,3,4,5\} colluders merges their watermarked adapters by parameter averaging. For C=2C=2 we evaluate all (202)=190\binom{20}{2}=190 pairs; for each C∈{3,4,5}C\in\{3,4,5\} we randomly sample 200 coalitions. Since several clients are involved, SAR now denotes attribution to one of the colluders, and MAR denotes attribution to an innocent client. Values outside parentheses use the default codebook (b=6b=6, w=3w=3); values in parentheses use a longer codebook (b=8b=8, w=4w=4).

CCSARMARAbRNWR
20.00 (0.00)0.00 (0.00)100.00 (100.00)0.00 (0.00)
317.67 (17.50)11.17 (4.17)71.17 (78.33)0.00 (0.00)
416.17 (10.50)6.50 (2.00)77.33 (87.50)0.00 (0.00)
528.33 (13.33)7.00 (2.17)64.67 (84.50)0.00 (0.00)

For all evaluated two-client coalitions, the merged adapter leads to abstention. For C∈{3,4,5}C\in\{3,4,5\}, most cases still end in abstention, but 6.50%–11.17% are attributed to an innocent client under the default codebook. Merging can produce a decoded message that differs from every colluder’s message while matching another assigned codeword. The default codebook is fully occupied: all (63)=20\binom{6}{3}=20 weight-3 messages are assigned, so any decoded message of weight 3 points to a client, even when that client is outside the coalition.

A longer codebook leaves more decoded messages unassigned, allowing the verifier to abstain on additional mixed watermark patterns. With b=8b=8 and w=4w=4, only 20 of the 70 weight-4 messages are assigned. In our experiments with C∈{3,4,5}C\in\{3,4,5\}, MAR decreases from 6.50%–11.17% to 2.00%–4.17%. This reduction comes with lower successful attribution to colluders and more abstentions; for example, at C=5C=5, SAR decreases from 28.33% to 13.33%, while AbR increases from 64.67% to 84.50%. The longer codebook uses eight watermark bases instead of six. These results demonstrate an empirical mitigation, while establishing formal collusion resistance remains future work. These results and analysis are added to Appendix [A.11].

AuthorsOpenReview ↗

Response to Reviewer dZsV (3/4)

[W4] The efficiency claim considers watermark optimization and verification, rather than the entire server workload. The server still maintains personalized adapters and constructs one for each client. The reported timing gains are valuable, but memory use, adapter construction, and performance at larger client populations would give a fuller picture of scalability.

We thank the reviewer for suggesting a fuller evaluation of server-side costs. We report persistent adapter-weight storage, the time required to construct all client adapters, and performance at larger client populations. With a fixed codebook, the common aggregated task updates and message-dependent watermark injection in Algorithm 3 (Lines 6 and 17) allow each client's adapter to be represented as

θit=Gt+1wi∑k∈I1(mi)Γkt.\theta_i^t=G^t+\frac{1}{w_i}\sum_{k\in\mathcal{I}_1(\mathbf{m}_i)}\Gamma_k^t.

Here, GtG^t is the shared adapter component, Γkt\Gamma_k^t is the accumulated watermark base at position kk, and wi=h(mi)w_i=h(\mathbf{m}_i) is the number of bases selected by client ii.

The server can therefore represent the personalized adapter weights using one shared component GtG^t and bb accumulated bases {Γkt}k=1b\{\Gamma_k^t\}_{k=1}^{b}. For adapters with dd parameters, this reduces persistent adapter-weight storage from Θ(nd)\Theta(nd) to Θ((b+1)d)\Theta((b+1)d), with b=Θ(log⁡n)b=\Theta(\log n). Each client's adapter can be reconstructed on demand through a weighted sum of its selected bases and the shared component, requiring Θ(wid)\Theta(w_i d) operations and no gradient computation. Constructing all nn adapters still requires Θ(nwd)\Theta(nwd) operations for a constant-weight codebook, as discussed in Appendix [A.7].

nnbbExplicit storage of all client adapters (MiB)Shared adapter-weight storage (MiB)Construction time, all clients (s)MTA (%)SAR (%)
206525.00183.75 (2.9×)0.6192.54100.00
4081,050.00236.25 (4.4×)1.2692.44100.00
6081,575.00236.25 (6.7×)1.8992.29100.00
8092,100.00262.50 (8.0×)2.8592.20100.00

LegoMark's storage grows only when bb increases (it is unchanged from n=40n=40 to n=60n=60), reducing storage by 8.0× at n=80n=80, with the gap widening as nn grows. Constructing an adapter takes about 0.03 s per client, which is negligible within a communication round. MTA and SAR remain stable as nn increases. We have added these results to Section [4.3] (Table [3]).


[Q3] Does the method remain effective under partial client participation and longer training runs?

Yes. Our additional experiments show that LegoMark remains effective under partial client participation and longer training runs.

Partial participation changes which clients contribute task updates, while server-side watermark learning continues to use the fixed codebook of all clients. Using a shared representation, the server can incorporate the aggregated task update into a common adapter component and reconstruct each client's current adapter from this component and the accumulated watermark bases selected by its identity code (cf. Appendix A.7.2).

We evaluate this setting with n=100n=100 clients on SNLI (non-IID) for T=10T=10 communication rounds, twice our default training length of 5 rounds for language tasks. We simulate partial client participation by randomly sampling a fraction ρ∈{0.1,0.2,0.3,0.4}\rho\in\{0.1,0.2,0.3,0.4\} of the clients for local training in each round. Attribution is evaluated on the final adapters of all 100 clients. AbR, MAR, and NWR are 0.00% in all settings.

ρ\rhoFedAvg MTALegoMark MTASAR
0.190.7590.41100.00
0.289.9290.47100.00
0.391.0691.26100.00
0.491.1291.40100.00

At every client participation rate ρ\rho, LegoMark's MTA stays within 0.55 points of FedAvg and all clients are attributed correctly. We have added these results and results under IID data, which show the same trend, to Section [4.3].

AuthorsOpenReview ↗

Response to Reviewer dZsV (4/4)

[W5] The evaluation covers several datasets and includes useful component ablations, but the main ablation is limited to SNLI. Running it across different tasks and comparing against BEMark with the same constant-weight codebook can better isolate the contributions. The dataset coverage is reasonable, although effectiveness on open-ended generation remains unexplored.

We thank the reviewer for suggesting broader component ablations, a comparison using the same codebook, and evaluation on generation tasks. We address these three points below.

1. Ablation across tasks. We repeat the ablation of task-arithmetic aggregation (TA), bootstrap learning (BS), and composition-aware refinement (CAR) on DBPedia-14 and AG-News. The table below averages the results over these two datasets and SNLI (non-IID).

MethodMTASARMARAbRNWRFAR
w/o TA88.613.890.0096.110.0063.52
w/o CAR91.995.000.0094.440.560.00
w/o BS92.2495.560.004.440.000.00
LegoMark92.66100.000.000.000.000.00

The conclusions on SNLI hold across tasks. Without TA, every adapter retains traces of bases its message does not select (FAR 63.52%). Without CAR, composed bases interfere and SAR collapses to 5.00%. Without BS, SAR drops to 95.56%. Only the full method attains 100% SAR. Notably, MAR is 0.00% in every variant: even when components are removed, failures become abstentions thanks to the constant-weight codebook. We have revised the ablation study in Section [4.3] with these new results.

2. BEMark with the constant-weight codebook. LegoMark combines two components: the constant-weight codebook (CWC) and composable watermark bases. To isolate their contributions, we equip BEMark with CWC on IID SNLI with n=20n=20 clients (times in seconds).

MethodMTASARInjection timeVerification time
BEMark92.57100.00615.0313.77
BEMark w/ CWC92.33100.00693.3316.23
LegoMark92.54100.00149.8216.15

The two components contribute different properties. CWC provides reliability: by Proposition 1, it rules out misattribution under bit loss, whereas BEMark without it misattributes up to 26.7% of adapters under pruning (Figure 3). Its cost is small: one more bit (b=6b=6 instead of 5) raises BEMark's injection time from 615.03 s to 693.33 s and verification time from 13.77 s to 16.23 s. Composable bases provide efficiency: with the same codebook, LegoMark cuts injection time from 693.33 s to 149.82 s (4.6×) at the same verification time. We have added these results to Appendix [A.9.9].

3. Open-ended generation. We additionally evaluate LegoMark on two commonly used open-ended generation tasks: GSM8K (math reasoning, measured by final-answer accuracy) and Dolly (instruction following, measured by ROUGE-L). Both tasks are trained for 10 rounds, and Dolly uses 5 local epochs per round.

MethodGSM8K Acc.GSM8K SARDolly ROUGE-LDolly SAR
FedAvg76.84—36.72—
LegoMark76.77100.0036.49100.00

LegoMark preserves task performance, with drops of only 0.07 points on GSM8K and 0.23 ROUGE-L points on Dolly relative to FedAvg, while attributing every client correctly (AbR, MAR, and NWR are 0.00%). This shows that LegoMark extends beyond classification to open-ended generation. We have added these results to Appendix [A.9.11].


We sincerely thank the reviewer again for your time and constructive comments. We hope these clarifications and additional results have fully resolved your concerns, and we remain open to any further discussion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses and Revisions

Dear Reviewers, Area Chairs, and Senior Area Chairs,

As the discussion period draws to a close, we would like to thank you again for your time and constructive feedback, which has improved the quality and completeness of our manuscript. We have responded to all reviewer comments and incorporated additional experiments, analyses, and clarifications into the revised manuscript, with changes marked in blue. For ease of reference, we summarize the main updates below.

  1. Scope and practical use. We clarified the intended collaboration setting, the threat model, and the protection scope for intermediate checkpoints and deployed models, explained why watermarks are injected throughout training, and added per-round attribution results (Reviewer wTzu: W1, Q1).

  2. Attribution reliability and robustness. We added two new removal attacks designed to preserve task accuracy: rank truncation and substantially longer fine-tuning with six times more data (Reviewer dZsV: W1, W3, Q2; Reviewer wTzu: W3, Q3; Reviewer RxSc: W2). We report successful attribution, misattribution, abstention, and non-detection together (Reviewer wTzu: W3, Q3), across verification thresholds (Reviewer dZsV: Q1; Reviewer wTzu: Q3) and smaller query budgets (Reviewer dZsV: Q1). Long fine-tuning can weaken attribution, but every failure is an abstention or a non-detection; no adapter is misattributed in any of these settings. Independently trained, unwatermarked adapters produce no false attribution at any threshold (Reviewer wTzu: W3, Q3).

  3. Practicality and broader evaluation. We clarified how the method supports partial client participation and returning clients, and added experiments under partial participation and longer training runs (Reviewer dZsV: Q3; Reviewer wTzu: W2, Q2). We extended the server-cost analysis to adapter storage and construction as the client population grows (Reviewer dZsV: W4). We also added generation tasks, ablations across datasets, and a baseline comparison using the same codebook (Reviewer dZsV: W5).

  4. Assumptions and limitations. We added a step-by-step analysis of potential watermark contamination during aggregation and extended the theoretical analysis to show that inactive-bit activation alone cannot lead to misattribution under the constant-weight codebook (Reviewer RxSc: W1). We also evaluated client collusion, which lies outside our threat model, documented the misattribution that can arise with three or more colluders, and examined a mitigation (Reviewer dZsV: W2).

We hope these additions clarify the contribution, supporting evidence, and limitations of LegoMark. We would be grateful to know whether our responses address your concerns, and we are happy to provide further clarification before the discussion closes.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.