- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PDF, a path-dependent forgetting method in federated learning. The paper is well-organized, the motivation is good. However, the impact of unlearning is not clearly validated in benign scenarios.
Critical strengths
- The idea is well-motivated and clearly stated. Though real practice has several gap with the theoretical analysis, the method is well-designed to align with the analysis.
- The method is lightweight and straight.
Critical weaknesses
I'm not quite familiar with Federated Unlearning (FU), but I have the following concerns:
-
The experiments are mainly conducted in malicious scenarios where the FL framework are attacked by a label-flip-based attacker. Here, its quite confusing what is desired in such a scenario. General FU framework emphasize consistency with the retrained model. It would be helpful to clarify this distinction in metric choices as PDF treats lower FA and higher RA as better.
-
If it is possible, please validate the PDF in benign FU scenarios. It is still not clear that if the model actually got rid of the historical influence of the target client, or if it mainly just fixed the performance bias caused by wrong labels.
-
Label flip is a weak and outdated attack strategy. If the intention of malicious FU is to erase the influence of malicious clients, I think straight local tuning or continual federated training with same budget are also very effective.
Questions for authors
Please see weakness above. I'm willing to raise my score if main concerns are addressed.
8 comments
Response to Reviewer dbmP, part 2
Q2: If it is possible, please validate the PDF in benign FU scenarios. It is still not clear that if the model actually got rid of the historical influence of the target client, or if it mainly just fixed the performance bias caused by wrong labels.
As Reviewer suggested, we have conducted experiments on a benign setting, where we try to unlearn a correctly labelled client. Table B below includes new results.
TABLE B. Benign target results on CIFAR-10 with over four independent random seeds. FA is the accuracy on the target's own held-out data, whose labels are correct.
| Method | Participants | Unlearning client-rounds | FA (%) | RA (%) | Target-only MIA AUC (%) |
|---|---|---|---|---|---|
| Original (before removal) | 16 of 64 per round, target included, 150 rounds | – | |||
| Retrain | 16 retained per round, 150 rounds from scratch | 2400 | |||
| Continual FedAvg | 16 retained, redrawn each round, 20 rounds | 320 | |||
| Fresh-random | 25 retained, redrawn each round, 20 rounds | 500 | |||
| Fixed-random | 25 retained, drawn once, 20 rounds | 500 | |||
| Top 25 by Eq. (6), 20 rounds | 500 | ||||
| Top 25 by Eq. (6) with , 20 rounds | 500 |
a. The effect is not only wrong label repair. With the same unlearning budget, achieves 52.87% RA, compared with 47.32% and 51.35% for fixed-random and fresh-random selection. It also preserves the original trained model’s 52.94% RA. These results demonstrate that historical client selection provides a retained-utility benefit even when there are no corrupted labels to correct.
b. Target behavior and shared knowledge. Retrain achieves 59.88% target accuracy using retained clients alone. reaches 22.99%, indicating excessive suppression of benign-target performance. Removing target decoupling () raises target accuracy to 50.60% and RA to 59.21%, reducing the target-accuracy gap to Retrain from 36.89 to 9.28 points. The resulting target accuracy is also near the original model’s 50.33%, showing that this ablation primarily preserves shared-class performance. Therefore, this shows that provides flexibility through , which balances historical global alignment and target decoupling tailored for each task (e.g., increasing for benign settings).
c. Target-specific membership inference. For , mean target-specific AUC decreases from 51.82% to 50.67%, narrowing its gap to Retrain (i.e., 47.82%) from 4.00 to 2.85 percentage points. With , AUC is 52.16%, showing that the gains in RA and FA accompany higher membership distinguishability.
These experiments establish that 's utility benefit extends beyond label correction and clarify how target decoupling affects benign removal.
Response to Reviewer dbmP, part 3
Q3: Label flip is a weak and outdated attack strategy. If the intention of malicious FU is to erase the influence of malicious clients, I think straight local tuning or continual federated training with same budget are also very effective.
As clarified in Q1, we use label flipping as a controlled diagnostic of agreement with an injected label mapping. Our focus is retained-client selection for unlearning. Continual federated training provides a direct baseline because continues FedAvg from the trained model and changes which retained clients participate through Eq. (6). Since the server holds no training data, we interpret straight local tuning as fine-tuning on retained clients’ local data using the same FedAvg protocol, with random participation instead of 's CLP-weighted selection.
We agree with Reviewer that continual training without selection is also effective, as shown in Table A. Particularly, they lower FA from 7.38% to between 2.93% (i.e., fresh-random) and 4.58% (i.e., fixed-random), which removes 38% to 60% of the flipped-label behaviour. Besides, Continual FedAvg with 16 sampled clients per round achieves 3.89% FA using 320 participations. Therefore, contributes two things:
-
Target decoupling completes the removal. reaches 0.00% FA across all four seeds, compared with 3.98% for without target decoupling, under the same participation budget and training protocol. This ablation directly demonstrates the term’s contribution to suppressing flipped-label predictions. Relative to , which achieves 0.92% FA, the absolute FA gap decreases from 3.06 to 0.92 points.
-
Historical global alignment preserves utility. At the same 500 client participations, mean RA is 54.33% for and 59.98% for , compared with 49.13% for fixed-random and 52.58% for fresh-random selection. Both PDF variants also exceed the original model’s 51.24% RA. The result shows that selection based on historical global alignment remains useful without target decoupling, while controls the balance between retained utility and suppression of the injected mapping.
Revision. We will add the benign setting and the three same-budget fine-tuning baselines to Section 5.3, and discuss the role of for benign and label-flipped targets in Sections 4.2 and 5.3.
Response to Reviewer dbmP, part 1
We thank Reviewer for the constructive suggestions and comments. We address each question below.
a. The objective remains to approach retained-only retraining while preserving retained utility. Eq. (4) requires proximity to Retrain on the target objective and limits degradation on the retained objective. Accordingly, we have assessed FA by its absolute difference from Retrain, while retaining RA as a higher-is-better utility metric.
b. Role of label flipping. FA measures accuracy on the target’s held-out examples against the injected mapping y↦(y+1)modC, whereas RA uses retained clients’ correct labels. Label flipping makes agreement with a known target-associated mapping observable. Retrain can still have nonzero flipped-label FA through ordinary misclassification, despite never using the target’s data.
TABLE A. Label-flipping target results on CIFAR-10 with α=0.1 over four independent random seeds. Retrain references train from scratch for 150 rounds, excluding the target. All remaining unlearning methods start from the same Original trained model, exclude the target, and perform 20 FedAvg rounds with the same local training. Continual FedAvg samples 16 retained clients per round, matching the original training protocol. Fresh-random selection resamples 25 retained clients each unlearning round, whereas fixed-random selection draws one 25-client subset and reuses it throughout all the unlearning rounds. PDF0.5 uses the top 25 by Eq. (6), and PDF1.0 sets β=1, which switches off the target-decoupling term. Pooled MIA uses all 64 clients, as in the paper. Target-only MIA uses the removed client's training records as members and its held-out records as non-members.
Retrain16 follows Section 3.2 (from scratch, 150 rounds, 16 retained clients sampled per round) and is our reference below. Table 1 instead used all 63 retained clients per round, called Retrain63, because it is the strongest reference on retained utility (RA 55.28% vs. 46.05%). It costs about four times higher communication overhead, so we report both for the label-flipping target.
c. Interpretation of the results. As shown in Table A, on CIFAR-10 with α=0.1, PDF0.5 reduces flipped-label FA from 7.38% to 0.00% and increases RA from 51.24% to 54.33%. Its FA gaps are 0.92 percentage points to Retrain16 and 6.93 points to Retrain63. Its RA is 8.28 points higher than Retrain16 and 0.95 points lower than Retrain63. Thus, PDF achieves zero accuracy against the flipped labels while preserving retained accuracy near the full-participation Retrain level. We report the 6.93-point FA difference from that reference as a gap in target-data accuracy, rather than treating the lower FA as evidence of better unlearning.
Revision. We will replace the lower-is-better FA ranking with the smallest absolute FA gap to the specified Retrain reference, retain higher-is-better RA, and report pooled and target-specific MIA AUC with their retraining gaps. We assess these metrics jointly to characterize target-data accuracy, retained utility, and measured membership leakage. Results for correctly labelled targets (i.e., a benign setting) are provided in our response to Reviewer's Q2.