acceptodds
Under review as a conference paper at ICLR 2027

Guarding Federated LLM Finetuning: Filtering Unsafe Clients That Break Alignment

Abstract

Federated finetuning of LLMs is a promising solution to collaboratively specialize an LLM for domain specific tasks without sharing private data. However, most prior works do not consider the risk of malicious clients attempting to jailbreak the safety of LLM. Traditional malicious client detection approaches in federated learning typically rely on statistical outlier detection using client updates. These approaches are less effective in federated LLM finetuning where benign clients can be highly heterogeneous due to real-world data distribution differences across clients and the ability of LLMs to be finetuned for diverse tasks within a unified generative framework. Existing safety-oriented defenses for federated LLMs also often rely on an impractical assumption about knowledge of client dataset distribution or introduce unfavorable trade-offs between improvement in safety and degradation in task performance. To address this, we propose a safety-aware method for detecting unsafe clients rather than relying solely on raw statistical deviations between client updates. Specifically, we evaluate the displacement in weights brought by each client relative to a reference safety degrading parameter movement defined between a safety-aligned and an unsafe model. By relying on a publicly available harmful dataset alone to obtain the unsafe model, we make no assumption about knowledge of client's private data. Experiments across different poisoning, heterogeneity levels and architectures show that our approach can effectively distinguish unsafe clients and obtain the best safety-utility tradeoff after federated finetuning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Safety, LLM Finetuning, Federated Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_x7FMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the problem of malicious clients undermining safety alignment during federated LLM finetuning. Motivated by the limitation of conventional defenses that rely on statistical or geometric outliers in client updates, the authors propose a safety-aware client filtering method that is intended to better handle heterogeneous client data. The method constructs a reference safety-degrading direction from the parameter difference between a safety-aligned model and a harmful-finetuned model, scores each client update using layer-wise cosine similarity with this direction, and applies a GMM-based filtering procedure. Experiments cover different poisoning ratios, levels of client heterogeneity, model architectures, and attack settings, and evaluate the resulting safety–utility tradeoff.

Critical strengths
  1. The problem is practically relevant. Client data heterogeneity in federated LLM finetuning can indeed make conventional robust aggregation methods less reliable. Detecting unsafe clients based on safety-relevant parameter changes is a reasonable and well-motivated direction.
  2. The experimental evaluation is fairly comprehensive. The paper considers different poisoning ratios, levels of client heterogeneity, and model architectures, and compares against multiple robust aggregation and safety-oriented baselines. Overall, the results provide useful empirical support for the proposed approach.
  3. The additional analyses help support the main hypothesis. The transferability study, computational cost analysis, and adaptive attack experiments are useful. In particular, the observation that removing the component aligned with the reference direction also weakens the safety-degrading effect of the malicious update is interesting.
Critical weaknesses
  1. The methodological novelty is somewhat limited. The proposed method mainly combines a pre-computed safety-degrading direction, cosine-similarity-based scoring, and GMM-based client filtering. These individual components are relatively standard, and the main contribution appears to be their application and combination in the federated LLM safety setting rather than a substantially new mechanism.
  2. The generality of a single reference direction remains unclear. Although the paper evaluates transfer across different harmful datasets, the experiments mainly focus on explicit harmful instruction finetuning. It is not clear whether a fixed reference direction would remain effective for safety degradation caused by stealthier poisoning strategies or other training behaviors.
  3. The two-component GMM introduces an additional assumption. The method assumes that the client risk scores can be separated into two components in each communication round. In practice, however, a round may contain no malicious clients or only a very small number of them. In such cases, forcing a two-component decomposition may incorrectly reject benign clients.
  4. The analysis of heterogeneity is not yet fully convincing. The paper argues that task variations orthogonal to the safety-degrading direction do not increase the risk score. However, cosine similarity is also affected by the magnitude of the client update through its denominator. A more careful analysis is needed to explain how heterogeneous update magnitudes affect client scoring and the separation between benign and malicious clients.
  5. The adaptive attack evaluation is still limited. The current adaptive attack mainly removes the projection of the malicious update onto the reference direction. While the experiments show that this weakens the attack itself, they do not rule out the possibility that an attacker could jointly optimize for safety degradation and evasion, potentially finding another harmful update direction with a low detection score.
Questions for authors
  1. Applicability of the GMM. If a communication round contains no malicious clients, or only one malicious client, can the two-component GMM still work reliably? It would be helpful to include corresponding experiments and report the false rejection rate of benign clients.
  2. Generality of the reference direction. Beyond explicit harmful instruction finetuning, can the proposed method detect safety degradation caused by stealthier poisoning or seemingly benign training data? Are the current transfer results across harmful datasets sufficient to support a more general conclusion?
  3. Has the paper considered an attacker that directly and jointly optimizes the safety-degradation objective and the detection-evasion objective, rather than only removing the projection of the existing update onto the reference direction? Would the defense remain effective in this case?
  4. When a benign client update has only a small projection onto the reference direction but a substantially larger orthogonal component, how are the cosine similarity score and client detection performance affected? The authors are encouraged to further clarify this point.
  5. Does the proposed method remain effective when the LoRA rank, target modules, or local optimization settings are changed? Is it necessary to reconstruct the reference safety-degrading direction for different finetuning configurations? Please provide further validation.
Reviewer_mj7sOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies safety during federated LoRA fine-tuning with malicious clients. It scores each update against a reference direction obtained through harmful fine-tuning, then uses a two-component GMM to retain clients confidently assigned to the lower-scoring component. Medical-task experiments show substantial safety gains with generally competitive utility across poisoning levels, data distributions, and models.

Critical strengths
  1. The scoring rule has a clear motivation: benign clients can train on very different tasks, so their updates need not be close to one another. The results under heterogeneous data support this choice.

  2. Tables 1–3 report results over three seeds. The appendices examine detection errors, transfer to other harmful datasets, IID data, and larger numbers of clients.

Critical weaknesses

1. There is no theoretical robustness guarantee. Equations 8–10 offer intuition, but do not establish when benign and harmful updates remain separable under adaptive attacks.

2. Appendix N tests one projection-based attack. It achieves detector evasion without observed safety degradation, but does not test an attacker that preserves harmful behavior while matching benign scores.

3. The GMM is fitted to all client scores, including those submitted by attackers. Its sensitivity to coordinated score manipulation is not examined, leaving its reliability against colluding clients unclear.

4. Table 13 omits utility and benign rejection, and has no comparison using the same attack without filtering. These measurements would help distinguish ineffective attacks from protection provided by the defense.

5. All-benign and single-attacker settings are not directly evaluated. Fitting two components in every round may discard useful benign updates, and no fallback is specified if no client passes the threshold (Eqs. 11–15).

6. The contribution appears straightforward and limited. Similar ideas for scoring client updates against a reference have been explored in Byzantine-robust FL, for example in FLTrust.

Questions for authors

Please also refer to the weakness. Additionally:

1. With 0, 1, and 6–7 malicious clients out of 10, what are the per-round benign rejection rates and final safety and utility? How does the server handle scores that do not form two distinct groups, or rounds where no client passes the threshold?

2. Could you report FedAvg using only the true benign clients, particularly where filtering errors occur? For Llama-3.1, could you explain the lower mean utility and larger variation in Table 3 despite perfect client identification in Table 7?

3. Does the harmful reference use the same rank-32 query/value LoRA configuration described in Appendix E? Please also specify the GMM implementation, initialization, number of restarts, variance regularization, convergence criterion, and handling of degenerate fits.

4. Can an attacker optimizing the same-rank LoRA factors jointly minimize its harmful-training loss and the Eq. 7 risk score while remaining effective after aggregation?

Reviewer_V8teOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies malicious clients that break the safety alignment of the global model in federated LLM finetuning. The server only sees LoRA updates. Before training, it builds a reference unsafe direction by finetuning model on a public harmful dataset. Each round, the server scores each client by the sum over layers of cosine score. It then fits a GMM on the scores, keeps clients with high posterior probability, and aggregates them. Experiments on Llama-2-7B, Llama-3.1-8B, Qwen3-4B and Gemma2-2B on medical tasks show success across poisoning ratios, heterogeneity levels and architectures.

Critical strengths
  1. The paper uses public harmful datasets as extra prior knowledge, reframing the problem from unsupervised outlier detection to supervised direction matching. It works better than purely statistical Byzantine defenses under high heterogeneity.
  2. Experimental design is reasonably complete, covering different poisoning ratios, heterogeneity levels, model architectures, and client scales, with FPR/FNR reported.
Critical weaknesses
  1. No no-attack baseline and no oracle upper bound are provided. Without FedAvg on only the 7 benign clients, we cannot quantify how much utility is lost to false rejections versus other effects.
  2. The claim that the method "does not require benign clients to be a majority" (Sec 2.2) is untested.
  3. Novelty is limited compared to Hsu et al. (2024).
  4. The score is a plain layer-wise cosine sum (Eq. 7) with equal layer weights, lack ablation against global cosine or norm-weighted scoring, besides, the cosine similarity ignores update magnitude.
  5. Only one attack mechanism is studied. The adaptive attack in Appendix N tests only one trick, and Table 13 reports only FPR and S, so we cannot tell whether a fooled detector still degrades utility.
Questions for authors
  1. Can the authors report an oracle baseline, i.e. FedAvg with only the benign clients? This shows how much utility the detector loses compared with perfect filtering.
  2. Can the authors report the result when all clients are benign? Does the GMM still reject honest clients at τ = 0.8, and how much does U drop?
  3. Can the authors justify the sum of per-layer cosines in Eq. 7, and compare it with a global cosine and a norm-weighted sum?
  4. In Table 13, can the authors also report U and HMean(S, U) for the adaptive attacker?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.