- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes FedSHIELD, a honeypot-guided defense framework for robust federated learning against backdoor attacks. FedSHIELD constructs adversarially perturbed honeypot samples to detect malicious client updates based on their impact on honeypot loss. Suspicious updates are filtered out, while the remaining updates are aggregated with gradient clipping and differential privacy noise. Experiments across multiple datasets and FL settings demonstrate its effectiveness in reducing backdoor attack success rates while maintaining clean accuracy.
Critical strengths
-
The paper provides extensive ablation studies across different federated learning settings, demonstrating the contributions of individual components.
-
The paper presents a relatively comprehensive mathematical analysis, including a convergence proof.
Critical weaknesses
Major Weaknesses
-
Outdated baselines: The evaluated attacks and defense baselines are relatively outdated. More recent methods should be considered to better assess the effectiveness of FedSHIELD against advanced attacks and defenses.
-
Limited dataset scale: The experiments are restricted to relatively small datasets, such as MNIST, FashionMNIST, SVHN, and CIFAR-10. Evaluations on larger and more challenging datasets, such as ImageNet, would strengthen the claims of generalizability.
-
Dependence on auxiliary data: FedSHIELD requires clean, labeled server-side data for honeypot construction. This assumption may limit its applicability in realistic FL scenarios where such data are unavailable.
-
Missing hyperparameter sensitivity analysis: Although the hyperparameter settings are provided, sensitivity analyses for key parameters, such as the attack objective weight λ and the detection threshold parameter β, are missing. Their impact on defense performance should be investigated.
Minor Weaknesses
-
Experimental clarity: The authors should provide clearer descriptions of the baseline implementations, experimental configurations, and threat model to improve reproducibility.
-
Citation formatting: Numerous citations are incorrectly formatted. For example, in Line 42, "... evading standard evaluation Wan et al. (2024)." should be revised to "... evading standard evaluation (Wan et al., 2024)."
-
Figure quality: Figure 1 appears to be a rasterized image with small, difficult-to-read text. A properly scaled vector graphic would improve readability and presentation quality.
Questions for authors
See critical weaknesses.
FedSHIELD Generalizes to Tiny ImageNet, β Is Robust, and λ Is the Attacker’s Parameter
We thank Reviewer for the detailed assessment. We provide new experimental results and clarifications for each major weakness.
Ddataset scale
We run FedSHIELD and four baselines on Tiny ImageNet with CBA (50 clients, 20% malicious):
FedSHIELD achieves the lowest ASR and highest OPS across all methods on Tiny ImageNet. The absolute ASR is higher than on CIFAR-10, as expected on a harder dataset, but FedSHIELD maintains a clear lead over all baselines. The behavioral gap captured by the honeypot mechanism generalizes to larger datasets and architectures. We will include these result in the revised paper.
Hyperparameter sensitivity (Major Weakness 4).
First, a clarification: λ is the attacker’s loss weight in the backdoor objective (Section 3.1), not a FedSHIELD hyperparameter. It controls attack strength and is not under the defender’s control. FedSHIELD’s key hyperparameter is β, the threshold sensitivity parameter τ = μ_s + βσ_s.
We ran a β sensitivity study on CIFAR-10 with CBA:
ASR stays within 0.6%–1.1% and clean accuracy within 0.4 points across β ∈ [0.3, 1.5]. FedSHIELD is robust to β selection over a wide range. We will include this analysis in the revision.
Auxiliary data dependency
FedSHIELD requires one clean public sample per class for honeypot construction. This data comes from public datasets (CIFAR-100, DeepFashion) — not from clients — and need not match the clients’ distribution. Appendix A demonstrates that cross-dataset honeypots maintain AUROC ≥ 0.978, confirming detection does not depend on distributional alignment. This assumption is shared by FLTrust and Sageflow, which also require server-side auxiliary data. FedSHIELD’s requirement is weaker than defenses that need labeled examples from the target attack class. We will clarify this explicitly in the revised threat model section.
**Outdated baselines **
We agree that comparisons with 2025–2026 defenses would better position FedSHIELD. We will survey recent literature and incorporate the most relevant methods into the revised experimental comparison.
Minor weaknesses.
We will add clearer descriptions of baseline implementations, experimental configurations, and the threat model to improve reproducibility. All citation formatting errors will be corrected in the revision. Figure 1 will be replaced with a properly scaled vector graphic.