- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies client-level training-data watermark attribution in federated LLM fine-tuning under secure aggregation (SA). The key challenge is that SA prevents direct inspection of individual client updates and therefore obscures which clients were responsible. The paper proposes FedAttr, which repeatedly queries secure aggregates over random client subsets that include or exclude a target client, and uses the difference of these subset averages to construct an unbiased estimate of that client’s update. FedAttr then applies differential watermark scoring, comparing the watermark score before and after adding the estimated update, and combines evidence across communication rounds using Stouffer aggregation. The empirical evaluation shows that FedAttr can successfully attribute to the client while maintaining a low FPR.
Critical strengths
- The paper targets a clear and underexplored attribution problem created by secure aggregation.
- FedAttr is technically well structured and supported. The methodology is reasonable, and the experimental results validate its effectiveness.
Critical weaknesses
-
FedAttr deliberately constructs an unbiased estimator of each client's update from repeated overlapping aggregate queries, which appears to conflict with the purpose of secure aggregation. Moreover, the estimator variance decreases as the number of queries M increases, suggesting that stronger attribution is obtained by progressively recovering more information about the individual update. The current mutual-information analysis does not fully resolve this concern. It is therefore unclear in what meaningful sense FedAttr preserves client-update privacy beyond avoiding exact plaintext disclosure.
-
FedAttr requires 2MKT=500 SA queries even in the default setting with only 10 clients, yet the paper reports only about five minutes of SA-query overhead and does not describe a concrete cryptographic SA implementation. To my knowledge, if these queries were implemented using an actual SA protocol, client-side computation, communication, cryptographic setup, synchronization, and repeated participation could introduce substantially higher costs.
Questions for authors
-
What SA protocol was actually used in the experiments? Were the reported SA queries implemented using a real cryptographic SA protocol?
-
The KGW experiments use client-specific secret keys , with one key assigned to each watermarked client. Since the violating clients are assumed to be unknown, how does the verifier determine which key should be used for an arbitrary candidate client, particularly for benign clients that have no assigned watermark key?
-
Would FedAttr remain effective if a violating client attempted simple watermark-removal or evasion strategies, such as paraphrasing the protected documents before local training?
-
The current protocol sends both global-model checkpoints and client-specific update estimates to the corpus owner. Since both evaluated watermark detectors operate through black-box model queries, is transferring these model artifacts actually necessary?