- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies federated unlearning, which is the process of filtering out the learned signals from poisoned clients. The deletion request when covering multiple groups of clients may result in conflicting signals. The proposed method, is an ADMM-style consensus method. The disagreement between clients and with the servers tracked and carried across rounds to be used in the unlearning process. Forget clients minimize a bounded suppression loss and retain clients minimize task loss. Per-client disagreement states are carried across rounds and also correct each client’s local proximal target. Experiments are conducted using small CNNs.ViTs on image classification tasks.
Critical strengths
-
Problem is well-motivated and the paper is well-organized.
-
Comparisons are extensive (10+ baselines) and ablations/experiment details are clear.
Critical weaknesses
-
Experiments are conducted with standard federated learning datasets such as CIFAR but more challenging tasks (or other domains such as language) could be considered.
-
Headline results and main technique illustration could be presented more cleanly. For example, based on Fig1 , the proposed technique looks heavily engineered and hard to implement. (Please don't use informal sketch style). In Table 1, only present headline setting results and leave the details to other subsections or appendix.
Questions for authors
- The proposed technique introduces many hyper parameters. How should we tune them (considering that we may want to use the technique in OOD tasks)? Is there any aspectt of the proposed technique that is task/domain-specific or agnostic?
3 comments
Response to Reviewer p73A (2/2)
Response to Question 1: For a new task, we recommend starting from the closest tested configuration and prioritizing the regularization strength and the initial unlearning rate . The table shows the sweep in our main setting (non-IID CIFAR-10, CNN, five sources, three seeds). Appendix E gives tuning guidance and the full sweep.
- Settings that met the deletion criterion throughout the sweep. Every tested value of , and , and every positive value tested for , left no surviving sources in all three seeds. Retained accuracy and worst-source ASR still varied across these settings. With , sources crossed back above threshold after recovery.
- Parameters that need no separate tuning. The server update depends on and only through their ratio . Doubling both reproduced the original results exactly, so it suffices to tune .
- Parameters to prioritize on a new task. Start with and . On the CNN, small values of left surviving sources (the rows with ). We recommend varying one parameter at a time and using held-out data to select a setting with lower worst-source ASR among those that meet the retained-accuracy requirement. BERT-small reused the three-source NF-ResNet-18 values without further adjustment.
The server computations are task-independent. We use the same across-round disagreement, consensus and sensitivity calculations for images and text. The task-specific choices on the client side are the trigger applied to inputs (an image pattern or a text token), the target class and the unlearning loss, which is designed for classification. To adapt the computation to another type of task, only the clients’ unlearning loss needs to be replaced.
Parameter sweep · CNN · CIFAR-10 · non-IID · · 50 unlearning + 50 recovery rounds
Variants change one parameter from the first-row configuration, except for the final row, which doubles both and to keep their ratio fixed. Rows marked * are new runs; the others come from the paper's Appendix E. The remaining sweep values are in Appendix E.
| Parameter | Value | Surv ↓ | Worst ASR ↓ (%) | R-Acc ↑ (%) |
|---|---|---|---|---|
| FAC | 0.00 | 1.98±1.38 | 50.83±0.32 | |
| 1 | 2.67 | 67.84±16.54 | 47.57±2.81 | |
| 10 | 1.33 | 85.05±19.35 | 46.27±1.45 | |
| 50 | 0.00 | 0.99±0.35 | 50.30±0.79 | |
| 200 | 0.00 | 2.52±1.42 | 50.80±0.36 | |
| 1 | 0.00 | 2.82±1.39 | 51.00±0.46 | |
| 10 | 0.00 | 1.22±0.70 | 49.63±0.49 | |
| 0.01 | 0.00 | 5.18±3.33 | 51.93±0.49 | |
| 0.50 | 0.00 | 2.21±1.26 | 48.73±0.90 | |
| * | 0.01 | 0.00 | 2.36±1.62 | 50.33±0.31 |
| * | 1 | 0.00 | 1.22±0.47 | 50.57±0.31 |
| 0 | 1.33 | 15.32±1.65 | 52.93±0.35 | |
| * | 0.01 | 0.00 | 5.34±1.51 | 52.90±0.44 |
| * | 0.00 | 1.98±1.38 | 50.83±0.32 |
I thank the authors for the detailed rebuttal. They address my concerns and I raise the rating to 3.
Response to Reviewer p73A (1/2)
Thank you for the careful review. We added a text experiment and a parameter sweep, redrew Figure 1 and reorganized Table 1. Changes in the revised manuscript are marked in orange.
Response to Weakness 1: Thank you for suggesting that we extend the evaluation beyond image classification. We chose topic classification on 20 Newsgroups because we can assign each source a distinct trigger word and target topic. This lets us evaluate FAC on text using the same source-level criterion as in the image experiments. On BERT-small, FAC reused the configuration from the three-source NF-ResNet-18 experiment. None of the three sources exceeded its retraining threshold in any of the three seeds. Of the 15 baselines tested with their default settings, only SalUn also met this criterion in all three runs. FAC achieved 67.2% mean retained accuracy, 3.1 percentage points higher than SalUn’s 64.1%. The new Appendix D.3 describes the dataset and full experimental protocol.
BERT-small · 20 Newsgroups · K=3
Survivor counts are listed in seed order (1/2/3). Bold marks the lowest ASR and every survivor-free entry among approximate methods.
Response to Weakness 2: Figure 1 now shows one federated round as a vector diagram, and Table 1 now keeps only the results for three, five and seven sources. The figure follows the server broadcast, local training and server aggregation. On the client side, it shows where local correction occurs. On the server side, it shows how the previous round’s state enters the current aggregation and how the current round’s difference carries forward. Appendix A gives the formulas and pseudocode. The single-source results are now in Appendix C. Table 1 retains all 18 methods.