- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes the CoPR algorithm, a server-side backdoor defense method for federated learning. The defender server performs cross-client mean centralization on the parameters selected by each client, calculates the cosine deviation fraction of the residual direction relative to the median residual direction for each coordinate, then filters out suspicious clients based on a threshold for the absolute deviation from the median, and finally trims and weight-aggregates the complete updates from the remaining clients according to the median norm. Furthermore, the paper analyzes the sufficient conditions under which the residual direction can distinguish malicious clients, proves that a subset of parameters can preserve all filtering decisions where the confidence margin exceeds the fractional perturbation caused by that subset, and provides upper bounds on single-round aggregation error under false acceptance, false rejection, and clipping scenarios.
Critical strengths
- This paper addresses a meaningful and specific problem: When the defender’s server lacks clean reference data and does not know the number of malicious clients, is it possible to obtain the same filtering decision as would be achieved by observing all model parameters by observing only a small number of them, thereby filtering out backdoor updates at a lower detection cost?
- The motivation regarding residual directions is compelling: when malicious clients are few in number, their updates share a common offset, and benign residual directions are relatively concentrated, mean centralization may cause the residuals of the two groups to point in opposite directions. The paper not only provides this explanation but also explicitly states the conditions required for directional separation and subset decision stability.
- The evaluation not only covered various poisoning strategies but also simultaneously examined attack suppression and the quality of normal tasks. The study achieved good results for the five attacks on the tested CIFAR-10 diffusion models and the four attacks in the IID classification experiments, and truthfully reported the failure of the default small-parameter slicing in non-IID classification.
Critical weaknesses
- The experimental results in the paper do not yet fully validate the effectiveness of the proposed residual direction mechanism. Table 6 in the ablation experiments only examines ablation for filtering and clipping; it does not separately test the necessity of mean centralization, coordinate-wise median reference directions, and cosine scores under the same thresholds and aggregation processes. Consequently, it is impossible to determine whether the performance stems from the newly proposed mechanism or from the general filtering process and experimental configuration.
- Although the authors claim that CoPR does not require prior knowledge of the number of malicious clients, its detection theory explicitly relies on conditions such as the proportion of malicious clients being less than 1/2 and the directional concentration of normal clients. Furthermore, the fact that the defender does not require knowledge of the reference set or triggers is consistent with the assumptions of most current backdoor defense methods (e.g., [1], [2], [3], [4], [5]); therefore, this cannot be considered an innovation of this paper. [1]Nguyen et al., “FLAME: Taming Backdoors in Federated Learning,” USENIX Security, 2022. [2]Rieger et al., “DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection,” NDSS, 2022. [3]Jebreel and Domingo-Ferrer, “FL-Defender: Combating Targeted Attacks in Federated Learning,” Knowledge-Based Systems, 2023. [4]Huang et al., “Multi-Metrics Adaptively Identifies Backdoors in Federated Learning,” ICCV, 2023. [5]Ma et al., “A Stability-Enhanced Dynamic Backdoor Defense in Federated Learning for IIoT,” IEEE Transactions on Industrial Informatics, 2024.
- This paper adopts a hard filtering strategy based on residual direction statistics and a MAD threshold to directly exclude clients whose cosine deviation scores exceed the threshold. This strategy may be relatively sensitive to heterogeneity in client data. However, the non-IID scenarios tested in this paper are relatively simple, primarily simulating Dirichlet partitions, i.e., label skewness. More realistic non-IID scenarios—such as feature distribution shifts, imbalanced client sample sizes, temporal heterogeneity, conceptual shifts, or partial client participation per round—were not tested.
- The non-IID classification comparison only evaluated the default 5,120-parameter slice against the full version with approximately 6.57 million parameters; intermediate scales were not tested. This does not sufficiently prove that non-IID classification must observe all parameters.
- The scope of the attacks does not fully cover the dataset. The main table of the five diffusion attacks is based solely on CIFAR-10, and the cross-dataset results for CelebA primarily focus on the AdaSCP diffusion attack; therefore, they cannot be generalized to other classification datasets or models.
Questions for authors
See weaknesses.
Response to Reviewer GRuY
We thank the reviewer for the detailed comments. Our responses follow.
W1. Table 6 shows that the filtering stage is needed: adding it to clipping alone improves both FID and Stolen-MSE. We agree that it compares filtering against no filtering and does not isolate the score. On the coordinate-wise median: after mean-centering, the residuals sum to zero, so a mean reference is identically zero and its cosine is undefined; the median is the natural robust nonzero reference. To test centering and the cosine, we kept the MAD threshold (λ = 2), clipping, aggregation, and the observed subset fixed, and replaced only the score. On CIFAR-10 diffusion under AdaSCP (three training seeds), the attacker's margin is large (Fig. 6): all four scores reject it in every round with comparable FID and Stolen-MSE, and the norm-only score has a higher benign false-rejection rate (5.08 ± 0.44%) than the other three (3.58–3.89%). The differences are larger under non-IID classification (Table 3b protocol, seed 0, one fixed subset of 2% of the weights; ASR in %):
Clean accuracy is 82.5–84.8% in all cells. Only the centered cosine suppresses all four attacks.
W2. (a) These are two different things. CoPR does not take the number of attackers as an input: the retained set is determined by the score distribution of each round through the MAD rule, whereas Krum and Multi-Krum require this number. A benign majority and the angular concentration of benign residuals are assumptions of Theorem 1, which gives a sufficient condition under which detection is guaranteed; the method runs without them. A benign majority is also assumed by defenses that do not require the attacker count, e.g., FLAME (Nguyen et al., 2022).
(b) We do not present the absence of clean reference data, trigger knowledge, or the attacker count as a novelty. The sentence in our contribution list states the conditions under which CoPR operates, which it shares with the defenses the reviewer lists. Our contributions are the residual-direction score and the characterization of when a fixed parameter subset preserves the filtering decisions of full observation (Theorem 2).
W3. We agree that hard filtering is sensitive to heterogeneity; Table 3b and Fig. 5 show this. On partial participation: CoPR decides from the uploads of each round and keeps no client-specific state, so it applies when clients are sampled. With 100 clients, 20 sampled per round, and 20% attackers (IID partition, BadNet, 200 rounds, three seeds), CoPR-full reduces ASR from 62.21 ± 11.44 to 2.81 ± 0.88. Beyond this, our evaluation of heterogeneity is limited to Dirichlet label skew (β from 0.1 to 1.0 in Fig. 5), as in the closest prior defenses; feature shift, sample-size imbalance, temporal heterogeneity, and concept drift are outside the scope of this paper.
W4. We agree. We ran intermediate budgets under the Table 3b protocol (seed 0), with fixed subsets drawn at random from the convolutional weights (ASR in %):
Observing 2% or 5% of the convolutional weights matches CoPR-full on all four attacks.
W5. Our evaluation covers two task families, federated diffusion generation and federated classification. Within each task, the scope is as the reviewer describes: the five-attack comparison follows the DataStealing protocol on CIFAR-10; on CelebA we evaluate only AdaSCP because federated diffusion training is costly; and classification uses CIFAR-10 with ResNet-9.