- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces FedHarden, a server-side defense mechanism against persistent backdoor attacks in federated learning. Prior to each training round, the server constructs two auxiliary datasets from public out-of-distribution (OOD) images: "flood data" with random labels and "shadow data" labeled by the current model. The server then leverages gradients from these datasets to select specific convolutional kernels and associated Batch Normalization (BN) parameters for additional robustification training, followed by standard client training and norm-clipped aggregation. The core motivation is to prevent attackers from exploiting parameters that receive insufficient benign training signal.
Critical strengths
-
The focus on persistent poisoning is timely and relevant. The reported performance gains are substantial, particularly against single-client attacks, and the evaluation covers a diverse range of attack types, model architectures, client data distributions, and poisoning strengths. The proposed server-side intervention is an interesting approach because it does not rely on identifying malicious clients, which is often a bottleneck in practical FL settings.
-
The ablation studies provide useful insights into the design choices. Table 5 clearly demonstrates the advantage of kernel-wise selection over parameter-wise selection under the same parameter budget. Additionally, Table 7 effectively separates the contribution of robustification from norm clipping, preventing the attribution of all improvements to the proposed training procedure alone. The inclusion of sensitivity analysis regarding OOD data sources and server-side runtime is also valuable for assessing the practicality of the defense.
Critical weaknesses
-
There is a conceptual disconnect between the theoretical motivation and the algorithmic implementation. Definition 1 identifies "low-activity parameters" based on benign training gradients, whereas Algorithm 2 selects kernels based on their response to flood data. The paper does not establish a rigorous relationship between these two quantities. Specifically, it remains unclear whether the kernels selected by Algorithm 2 are indeed underutilized during benign training. Theorem 1 does not bridge this gap, as its bound relies on assumed malicious-update constraints and Lipschitz continuity without incorporating the specific selection procedure. While the empirical improvements are significant, they could potentially be attributed to the auxiliary training acting as a general regularizer or feature disruptor rather than specifically targeting underutilized parameters. A comparison with random kernel selection, along with measurements of benign-gradient activity before and after robustification, would be necessary to validate the proposed mechanism.
-
The evaluation of adaptive attacks is insufficient for assessing robustness against a persistent adversary. The current adaptive attacks either avoid selected kernels or increase update norms, but neither directly optimizes the attack to survive subsequent rounds of FedHarden’s randomized server-side training. Given that persistent robustness is the primary objective, a defense-aware attack that accounts for the stochastic nature of the server-side robustification is crucial. Furthermore, the client participation protocol is ambiguous. Section 5 mentions 10 participating clients out of 100, while Appendix B.1 states that malicious clients participate in every poisoning round. While these statements are compatible if malicious clients are deliberately included, this should be explicitly stated. Without a clearer participation model and a stronger defense-aware attack, it is difficult to determine how much of the reported advantage would remain against an informed persistent adversary.
Questions for authors
-
Are malicious clients guaranteed to be selected in every poisoning round, or are they sampled from a fixed population? In Table 4, do the reported malicious-client ratios refer to the total client population or only the active participants in each round?
-
Have you measured whether the kernels selected by Algorithm 2 correspond to parameters with low benign-gradient activity? How does the proposed selection strategy compare with random kernel selection under the same parameter budget?
-
What information about the server-side defense is assumed to be available to the attacker? Could you evaluate an attacker that explicitly optimizes for backdoor persistence in the presence of FedHarden’s subsequent robustification steps, rather than merely avoiding selected kernels or increasing update norms?
-
How many independent runs are used for the main results, and at which specific point within each round is backdoor accuracy measured? I noticed that the Neurotoxin blended results in Tables 2 and 3 report identical backdoor accuracies but different main-task accuracies. Additionally, Tables A9 and A10 report different effective lengths for apparently similar 400-sample settings. Are these results from independent runs or different configurations?
2 comments
We appreciate the time and care you have already devoted to reviewing our work. As the discussion deadline approaches, we would be grateful to hear whether the additional experiments and clarifications in our latest response address your remaining concerns. Thank you again for your thoughtful feedback and consideration.
Considering the upcoming deadlines, we would be immensely grateful if you could provide any additional feedback or clarifications at your earliest convenience. Your guidance is crucial for the improvement of our work, and we deeply appreciate the time and effort you have invested in reviewing our responses.