- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Eraser, a post-training backdoor defense method for federated learning designed to mitigate the impact of backdoor triggers on model predictions without modifying the original federated training process or the potentially compromised global model. Unlike existing defense methods that rely on malicious client identification, robust aggregation, or model parameter repair, Eraser employs a semantic reconstruction strategy to eliminate trigger-related effects before inputs reach the global model. Specifically, each client independently trains a featurizer to extract semantic representations that are relatively insensitive to triggers, while feature alignment across clients is facilitated by fixed hyperspherical semantic anchors. Subsequently, clients collaboratively train a lightweight generator that utilizes classification supervision from the frozen global model to reconstruct latent semantic features into inputs suitable for processing by the global model. During inference, each client sequentially employs its local featurizer, the shared generator, and the frozen global model to generate predictions. The authors evaluated Eraser across various datasets, backdoor attacks, data heterogeneity scenarios, client scales, federated learning algorithms, and model architectures, while also investigating adaptive attacks, reconstruction behavior, and extensions to text and speech modalities. Experimental results demonstrate that the method significantly reduces the success rate of backdoor attacks while maintaining accuracy on benign tasks, all with minimal additional communication overhead.
Critical strengths
Strength 1: It proposes a post-training backdoor defense method that offers a degree of novelty. This paper introduces a defense strategy distinct from traditional approaches such as malicious client detection, robust aggregation, and model parameter repair. Eraser employs an independently trained local featurizer to extract semantic features relatively insensitive to backdoor triggers and reconstructs inputs via a shared generator; this enables defense implementation without altering the original federated training process or the poisoned global model.Shifting the defense mechanism to a pre-inference semantic reconstruction stage is an innovative design choice that also allows the method to be integrated with existing federated learning algorithms. I consider this approach to be the paper's most significant contribution. Strength 2: The experimental validation is comprehensive, demonstrating strong backdoor suppression capabilities across a wide range of settings. The authors not only conducted systematic comparisons across multiple datasets and typical backdoor attacks but also evaluated performance under various scenarios involving data heterogeneity, client scales, federated learning algorithms, and model architectures, as well as against adaptive attacks and in cross-modal extensions. Experimental results show that Eraser significantly reduces the success rate of backdoor attacks across diverse settings while maintaining high accuracy on benign tasks. The authors also provided ablation studies, visualizations of the reconstruction process, and an analysis of computational overhead, ensuring the completeness of the experimental validation. These results offer valuable empirical support for the method's effectiveness and potential applicability.
Critical weaknesses
Weaknesses 1: Existing evaluation metrics are insufficient to fully demonstrate the restoration of correct prediction capabilities for triggered samples. The paper primarily uses the backdoor failure rate to evaluate defense effectiveness. However, this metric merely indicates that triggered samples are not predicted as the attacker-specified target class; it fails to distinguish between samples correctly predicted as their true labels and those incorrectly predicted as other non-target classes. This issue is particularly critical for Eraser, as the method's core objective is to suppress the influence of the trigger through semantic reconstruction while preserving information relevant to the normal task. While a high backdoor failure rate demonstrates the suppression of the attack target's prediction, it does not directly prove that the reconstructed triggered samples have regained the correct classification result. Although the authors report favorable clean accuracy and high backdoor failure rates, these metrics address distinct evaluation goals—clean samples versus triggered samples—and cannot fully substitute for accuracy based on the true labels of triggered samples. Consequently, the current experimental validation of the core defense efficacy remains incomplete. Weaknesses 2: There is a lack of sufficient analysis regarding cross-client feature compatibility and applicability under conditions of local data scarcity. Eraser relies on a featurizer trained independently by each client to extract semantic features, while all clients share a common generator. Thus, the ability of different clients to maintain consistent feature representations is a crucial prerequisite for the effective operation of the shared reconstruction mechanism. The authors introduce hyperspherical semantic anchors and experimentally demonstrate a reduction in class-level feature discrepancies. However, class-level alignment does not necessarily imply consistency in the fine-grained semantic structures of samples within the same class across different clients. The paper currently lacks a direct analysis of this issue. Furthermore, Table 16 in Appendix J shows that, with the total training data volume held constant, Eraser's accuracy on the normal task drops significantly as the number of clients increases. Since the local featurizer does not participate in federated aggregation, this design may render performance heavily dependent on the data scale of individual clients. However, the paper does not adequately distinguish whether this accuracy decline stems primarily from insufficient training of the local featurizer or from performance fluctuations inherent to the original global model itself.
Questions for authors
Q1. Ability to recover correct labels for triggered samples The paper primarily uses the "backdoor failure rate" to measure defense effectiveness. However, this metric only reflects whether triggered samples avoid being predicted as the attacker-specified target class; it cannot distinguish between samples being correctly classified with their true labels and those being misclassified into other non-target classes. Consequently, a high backdoor failure rate does not necessarily imply that the model has recovered the ability to correctly predict triggered samples. Could the authors provide further results—specifically under typical backdoor attacks that preserve original sample semantics—regarding the classification accuracy of triggered samples relative to their original true labels after Eraser processing? This metric would help differentiate between the suppression of backdoor target predictions and the recovery of correct task predictions, thereby allowing for a more comprehensive evaluation of the defense's effectiveness. Q2. Compatibility of fine-grained feature representations across clients Eraser facilitates feature alignment across clients using fixed hyperspherical semantic anchors. The authors evaluate alignment effectiveness via feature differences between class means (Figure 4 and Appendix D) and demonstrate the shared Generator's generalization capability through experiments with new clients (Appendix J). However, class-level feature alignment does not necessarily guarantee sufficient consistency in the fine-grained feature distributions of samples belonging to the same class across different clients. Given that all clients utilize a shared Generator, could the authors further analyze intra-class feature distribution differences across clients, or explain why class-level semantic anchor alignment suffices to support stable reconstruction by the shared Generator? This would help further elucidate the method's feature compatibility and underlying mechanisms in data-heterogeneous environments. Q3. Trade-off between inference overhead and defense performance Tables 30 and 31 in Appendix S indicate that Eraser increases the computational cost of single-sample inference from 1.1158 GFLOPs to 2.2319 GFLOPs and inference latency from 3.1577 ms to 5.7539 ms—approximately 1.82 times that of the original global model. Although the additional communication cost introduced by this method is low, the inference phase still requires the execution of a local featurizer with a computational scale comparable to that of the global model; this may limit its practical deployment in resource-constrained or latency-sensitive scenarios. Given that the additional inference overhead stems primarily from the local featurizer, could the authors further analyze the relationship between the featurizer's computational scale and its backdoor defense performance? For instance, could employing a more lightweight feature extraction network significantly reduce inference overhead while maintaining comparable defense efficacy? Such an analysis would help clarify the trade-off Eraser makes between defense robustness and deployment efficiency. Q4. Sources of performance degradation under conditions of local data scarcity Table 16 in Appendix J shows that, while the total volume of training data remains constant, Eraser's accuracy on the main task drops from 84.95% to 74.18% as the number of clients increases from 100 to 300 (reducing the average number of training samples per client from approximately 449 to 149). The authors attribute this phenomenon to the reduction in training data available to the local featurizer; however, the current results do not sufficiently distinguish between the impact of diminished local feature-learning capabilities and changes in the underlying global model's performance. Could the authors provide the accuracy of the original global model on the main task—prior to applying Eraser—for the same client configurations, and compare these figures with the results obtained after applying Eraser? Such a comparison would help quantify the portion of performance degradation attributable to Eraser itself, thereby allowing for a more accurate assessment of the method's suitability in scenarios characterized by local data scarcity.
4 comments
Response (2/3)
Q2: Fine-grained feature compatibility across clients
Response: Class-level anchors do more than align class means. in Eq. (4) is applied to every sample, pulling each feature toward its class anchor, and because all clients share the same anchors, the samples of each class are drawn toward the same region on every client.
We conduct this analysis with 20 clients on CIFAR-10, using the same data partitions across the two conditions. Each Featurizer undergoes 300 epochs of clean local training and is then frozen, followed by 50 communication rounds of shared Generator training. The two Generators start from identical weights and use the same frozen global model. Evaluation uses a common, class-balanced held-out set containing 100 images per class.
To verify this, we compare training with and without the anchors using the class-conditional between clients, computed on raw features and, to remove class-mean offsets, on features centered by each client’s class mean. With anchors, the centered falls by 33.60% (38.75% on classes seen by both clients), close to the 40.53% drop of the raw , so the alignment extends to the within-class distributions rather than stopping at the class means. To see whether this matters for the shared Generator, we measure the joint correct rate, the fraction of images that two clients both classify correctly when each passes the image through its own Featurizer and the shared Generator. With anchors it rises by 3.02%, so the same image is more often handled correctly no matter which client processes it.
Table: Effect of shared semantic anchors on cross-client feature compatibility under BadNet on CIFAR-10 (relative change for feature distances and absolute change in % for joint correct rate)
| Metric | Raw | Centered | Centered (classes seen by both clients) | joint correct rate |
|---|---|---|---|---|
| Change with anchors | −40.53% | −33.60% | −38.75% | +3.02 |
We add this analysis to Appendix D.
Response (3/3)
Q3: Featurizer scale versus defense performance
Response: Lightweight Featurizers can substantially reduce computational cost while maintaining a similar B in this experiment, although the latency benefit depends on the architecture. We replaced the default ResNet-18 Featurizer with five lighter networks, keeping the Generator and the global model unchanged.
Table: Lightweight Featurizers under BadNet on CIFAR-10 (differences from the default ResNet-18 Featurizer)
| B (%) | GFLOPs | latency (ms) | |
|---|---|---|---|
| CNN | +0.51 | -1.1052 | -0.8474 |
| MobileNet-V2 ×0.5 | +0.43 | -1.0939 | +1.2904 |
| MobileNet-V2 | +0.29 | -1.0606 | +1.2980 |
| ShuffleNet-V2 ×0.5 | -0.25 | -1.0880 | +1.5944 |
| ResNet-8 | +0.75 | -1.0858 | -0.5577 |
CNN and ResNet-8 reduce the full pipeline’s computational cost by 1.1052 and 1.0858 GFLOPs and its latency by 0.8474 and 0.5577 ms, respectively, while maintaining B. MobileNet-V2 and ShuffleNet-V2 also reduce FLOPs but increase measured latency. Inference time depends on memory access, operator overhead, and hardware utilization in addition to FLOPs, particularly for single-sample inference. These results show that the choice of lightweight architecture matters for reducing actual inference latency. We have added this analysis to Appendix S (Table 37). Tables 30 and 31 of the submission are now Tables 35 and 36.
Q4: Source of the accuracy drop at larger client scales
Response: Thank you for suggesting this comparison. The drop comes from the reduced amount of data per client, which affects both the global model and the local Featurizer. The table below adds the original global model (FedAVG, before Eraser) under the same client configurations as Table 16.
Table: FedAVG vs. Eraser at different client scales under BadNet on CIFAR-10 (%)
| 100 clients | 200 clients | 300 clients | |||||||
|---|---|---|---|---|---|---|---|---|---|
| A | B | RA | A | B | RA | A | B | RA | |
| FedAVG | 84.63 | 0.00 | 0.00 | 76.30 | 0.00 | 0.00 | 71.73 | 0.00 | 0.00 |
| +Eraser | 84.95 | 98.94 | 82.09 | 77.35 | 96.93 | 74.07 | 74.18 | 96.64 | 69.40 |
The global model degrades even without Eraser: from 100 to 300 clients, FedAVG loses 12.90% in accuracy, more than the 10.77% lost with Eraser. Applying Eraser therefore never costs accuracy relative to the global model at any of these client scales. Eraser stays above FedAVG at every scale, and its margin grows from 0.32% to 2.45% as local data shrinks. The defense also holds at every scale, with B above 96.6% and RA within 5% of A. We add these results to Appendix J, where Table 16 of the submission is now Table 17 and includes FedAVG.
Dear Reviewer ABjo,
Thank you for your constructive feedback. We have completed the supplementary experiments addressing your questions, including evaluations of reconstruction accuracy, cross-client feature compatibility, computational overhead, and scalability, and have incorporated the results into the revised manuscript.
Before the discussion period closes, we would appreciate your feedback on whether these results and explanations address your concerns. Please let us know if any points require further clarification.
Thank you again for your constructive review.
Best regards,
The authors
Response (1/3)
We sincerely thank you for the insightful comments. Each one is answered below with new experiments. Unless otherwise specified, all experiments follow the default settings described in the manuscript.
Q1: Recovery of true labels for triggered samples
Response: Thank you for raising this important point. Eraser does more than divert triggered samples from the target class: it largely restores their true labels. To show this, we additionally report the recovery accuracy (RA), i.e., the accuracy of triggered samples with respect to their ground-truth labels. "Ideal" denotes a FedAVG global model trained without malicious clients and evaluated on the same triggered inputs, so its RA shows how much the trigger itself damages the input.
Table: RA (%) on CIFAR-10
FedAVG's RA is near zero on seven of the eight attacks, whereas Eraser's RA exceeds even the Ideal model's on all eight, by 1.74% to 13.13%. On BadNet and DBA, Eraser's RA is within 1.7% of its clean accuracy (Table 2), so the trigger costs it almost no accuracy. RA is lower on Signal, Blended, BppAttack, and Kelvin because these are global triggers that alter the content of the entire image, which lowers the Ideal model's RA as well. Varying the blending ratio χ of the Blended trigger in Eq. (20) confirms this: as the trigger grows stronger, the Ideal model's RA falls from 21.46% to 13.69%, while Eraser stays 10.8% to 19.0% above it.
Table: RA (%) under Blended triggers with blending ratio χ
As requested, we also evaluate ISSBA [R1], whose invisible, sample-specific triggers preserve the original sample semantics. Eraser reaches an RA of 86.26%, within 2 percentage points of its clean accuracy and above the Ideal model (83.54%), while FedAVG drops to 0.13%.
Table: ISSBA on CIFAR-10 (%)
We define RA in Appendix B and report these results, including ISSBA, in Appendix G (Table 12) of the revised manuscript. ISSBA is described in Appendix A.
[R1] Li et al. Invisible Backdoor Attack with Sample-Specific Triggers. ICCV 2021.