acceptodds
Under review as a conference paper at ICLR 2027

EmbTracker: Traceable Black-box Watermarking for Federated Language Models

Abstract

Federated language model (FedLM) training enables collaborative learning without sharing raw data. However, every client receives valuable model instances that may be redistributed without authorization, raising important intellectual property (IP) protection challenges. Most existing federated watermarking methods require white-box access or client-side cooperation, and provide only group-level ownership evidence rather than client-level attribution. Moreover, many methods are designed for image classification tasks and have limited utility for language models. We propose EmbTracker, a server-side framework for traceable black-box watermarking tailored for FedLMs. EmbTracker embeds a backdoor-based watermark that can be verified through API queries and assigns each client a distinct registered trigger through token-embedding replacement, avoiding per-client watermark retraining. Experiments on language and vision-language models across classification, question-answering and visual question-answering tasks show verification rates near 100% in most settings, primary-task changes typically within 1–2 percentage points, and resilience to fine-tuning, pruning, quantization, noise, and watermark overwriting under the evaluated conditions.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Watermark, Federated Learning, Language Model, Black-box, Traceable

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_THFTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies black-box tracing of leaked client copies of federated language models. It trains a shared trigger embedding and assigns it to client-specific token rows while clients fine-tune adapters. Verification compares responses to registered triggers. Experiments span 11 datasets and language and vision-language models, reporting verification rates of approximately 95.2% to 100% with generally modest task degradation. The main contribution is a reusable embedding-based marking mechanism that reduces the need for separate watermark training for each client.

Critical strengths
  1. Reusing a trained embedding across client-specific trigger tokens is a simple and potentially useful way to reduce per-client marking effort in adapter-based federated fine-tuning.
  2. The evaluation covers multiple task types, language and vision-language models, and several common model modifications. This provides meaningful evidence of basic functionality beyond a single task.
Critical weaknesses
  1. The evaluation does not establish whether the mark survives separating the adapters from the marked embeddings. The client receives model parameters, the identity mark is concentrated in embedding rows, and useful fine-tuning occurs in adapters. Restoring public base embeddings or attaching the adapters to a clean base is therefore a directly relevant removal scenario. The current perturbation tests do not resolve it. This is an untested concern, not a demonstrated successful attack.
  2. The identification evidence does not match the decision rule. The rule requires other clients’ triggers to remain below a threshold, whereas low average cross-client response does not bound the worst competing identity or false identification on innocent models. Without threshold calibration and suitable null models, high verification rates alone do not establish reliable leakage tracing.
Questions for authors
  1. Under the stated threat model, what happens to task utility and verification when public base embeddings are restored or client adapters are attached to an unmarked base? If this operation is excluded, please clarify the restriction and its practical implications.
  2. What threshold is actually used for identification, and what is the probability of accusing any wrong client or an independently trained innocent model at that threshold? Please distinguish the maximum cross-client response from the reported average.
Reviewer_o1e1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies model ownership tracing in federated language model training. Instead of only proving that a leaked model came from a federation, it aims to identify which client leaked it. The proposed method, EMBTRACKER, first learns a single universal watermark by tuning one trigger token embedding. The server then copies that embedding into a different client-specific trigger token before sending the model to each client. To verify a suspicious model, the server queries it with each client’s trigger and checks which one activates the target behavior, so verification is done in a black-box way.

The main claims are that the method supports client-level tracing with only black-box access, works entirely on the server side without modifying client training, and fits PEFT settings like LoRA and prefix tuning. Experiments on classification, QA, and VQA with BERT, Llama-2-7B, and Qwen2.5-VL show high verification rates and generally small drops in task performance. The paper also compares against prior methods and tests robustness under non-IID data, different PEFT methods, pruning, quantization, noise, and overwrite attacks.

Critical strengths

The paper tackles an important problem. Client-level leak tracing is more practically useful than just proving ownership, and the paper focuses on that directly. The core idea is simple and effective. Reusing one learned watermark embedding across clients is much cleaner and cheaper than training a separate watermark for every client. The black-box verification setup is also a strong point, since that is often the only realistic access available in practice. The experimental coverage is fairly broad, with multiple tasks, models, and attack settings, which makes the results more convincing.

Critical weaknesses

My main concern is that the method depends heavily on the embedding table staying fixed during client training. That works for the PEFT setups studied here, but it makes the method less general than the paper sometimes suggests. If embeddings are updated, the mechanism may weaken substantially. The security analysis also feels incomplete. In particular, the paper assumes no collusion, but collusion seems like an important case for a tracing method. If clients compare models, they may be able to identify the modified trigger embeddings. The overwrite attack results are also weaker than the paper suggests. If both the old and new watermarks remain valid, attribution can become ambiguous. I also wanted clearer analysis of collisions and false attribution. High verification rate is encouraging, but I would like to know more directly how often a model could match multiple client triggers. Finally, some of the baseline comparisons are useful mainly for context rather than strict apples-to-apples evaluation, and the writing could be tighter in places.

Questions for authors

What happens if clients update the embedding layer? Can you report direct collision or false attribution rates? Have you tested any collusion setting? How sensitive is the method to the choice of trigger tokens? For generation tasks, how often does the target output appear without the trigger? How should overwrite cases be handled when multiple watermarks remain valid? Does the method still work well at larger client counts?

Reviewer_GzPeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies watermark-based tracing for federated language models, aiming to identify which client leaked a model. The authors propose EmbTracker, which first trains a universal backdoor watermark embedding on the server. The learned embedding is then copied to the embedding positions of different client-specific trigger tokens. In this way, each client receives a unique watermark without separate watermark training. The server can detect the watermark and identify the leaking client through black-box queries. Experimental results show that the method achieves high watermark verification rates, has a limited impact on the main task performance, and is robust against some model modification attacks.

Critical strengths
  1. The method is simple and easy to understand. By reusing a universal watermark embedding, it assigns a unique watermark to each client without separate training, reducing the computational cost.
  2. The experiments are relatively comprehensive, covering different models, tasks, and federated learning settings. The authors also evaluate robustness against several attacks. The results show high watermark verification rates with limited impact on the main task performance.
Critical weaknesses
  1. The motivation could be better explained. The paper does not sufficiently discuss why clients would leak their models in real-world federated language model settings, or how important this problem is in practice.
  2. The technical novelty seems limited. The method mainly combines existing techniques, including backdoor watermarking, embedding replacement, and black-box verification. The specific technical improvements and unique contributions of this work are not entirely clear.
  3. The paper focuses on federated LoRA fine-tuning. However, a malicious client might only leak the trained LoRA adapter instead of the full model containing the watermark. In this case, the proposed method may fail to identify the leaking client. It would be helpful if the authors could discuss or evaluate this scenario.
  4. The paper mainly reports the watermark verification rate. However, a high verification rate does not necessarily mean that the leaking client can be correctly identified. I suggest that the authors also report the correct attribution rate and false attribution rate to better evaluate the actual tracing performance.
Questions for authors

Please see Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.