- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces the f-fum, which is an active federated unlearning methods on teacher-student optimization. This method treat original model as a fixed teacher while letting clients maximize f-divergence between student and teacher predictive distributions on the forget examples. It then minimize the KL disntance between two models. It is only launched locally. The author evaluate the forget-side divergence choices in client-level and data-level unlearning and stated that they achieve better results.
Critical strengths
- Federated unlearning is a trending and practical research direction.
- The paper makes good definition for active federated unlearning (server initialzted unlearning) and passive unlearning (reconstruct the model without access to the forgetting client).
- From reading this paper (e.g., related work and the definition of unlearning), I can see that the author has a deep understanding of the FU. Thus, the position of the work in interesting.
- The unlearning dynamics is helpful for understanding the method.
Critical weaknesses
- I think the active federated unlearning is an important setting and agree with the paper. However, this paper reproduces a limited baselines in the active FU scenarios. It would be nice if the paper could add them into the baseline table. This would facilitate proving the methods effectiveness in the field.
- The paper could be benefit from the differentiating them from the unbounded unlearning paper (teacher-student forget optimization), especially how it applies their trick into the federated decentralized domain with active unlearning.
- There is a lack of the ablation study, specifically in the combination of the four components (forget set divergence maximization, retained-data divergence minimization, supervised retained data optimization and post training recovery).
- The paper could benefit for scaling up their experiments using foundation models and scale up the clients. Overall, I think the active federated unlearning is an important field and this paper contributes to it. I am happy to raise the score to be positive if the above can be adequately addressed.
Questions for authors
Questions are generally the weakness above.
- Could the paper reproduce more active FU baselines like you reviewed in the appendix?
- Could the paper differentiate them from the unbounded unlearning paper?
- Could the author provide a comprehensive ablation study?
- Could the author scale up the FU with modern models and datasets like FU papers published in this year?
3 comments
Thank the Author for the Response
Thank the Author for the response. I have increased my score to weak accept.
Thank you
We sincerely thank the reviewer for reading our response and for raising the score to weak accept. Your suggestions, especially the component ablation and the clarification of the relation to SCRUB, have clearly improved the paper. Thank you again for your time and constructive feedback.
Part 1/1
We thank the reviewer for the positive assessment of the setting and the clear list of requests.
Q3: Comprehensive ablation of the four components. Table 13 (Appendix I.1; also shown in our response to Dgg5) ablates each component and adds two controls.
The controls show that neither half works alone. The max step alone collapses the model (10.3%), which is the CIFAR-10 counterpart of the toy collapse in Fig. 2. Plain fine-tuning is competitive at client level, where the departing client just stops contributing (78.3%, ASR 3.1%). But it cannot remove data-level forget samples that stay on active clients (ASR 68.5%). Those require the max step with a learning rate scaled to the forget set; the paper's main data-level results show it does: on CIFAR-100 with 6–10% of each client's data deleted, f-FUM lowers backdoor ASR from 74–81% before unlearning to 0.25–0.69% (Table 1).
Q2: Difference from the unbounded-unlearning paper (SCRUB). The teacher–student max–min objective comes from SCRUB, as stated in the introduction, and the revised conclusion now names SCRUB explicitly. Our contribution is making it work when forget and retained data stay on clients and the server only averages. This involves (1) two separately weighted aggregation phases per cycle, in which a departing client participates only in the max phase (Theorem 1: an inexact step on one contrastive objective); (2) a fixed global teacher; (3) post-training after aggregation, which is necessary (22.5% without it); and (4) a bounded forget-side f-divergence that prevents the collapse caused by averaging max updates (JS up to +24 points over KL where KL collapses). Please see also our answer to MQWK Q1.
Q1: More active FU baselines. The paper compares against NoT, Halimi and MoDE in every setting. These are the active methods we found that fit the aggregator-only protocol: the server only averages full local models and holds no data, generator, auxiliary model or per-client state. The other active methods we review need server-side computation. Goldfish and Jin et al. need server-side data. Wu et al. needs server-side distillation. FFMU uses server-side operator-based aggregation. The recently suggested FedOSD (AAAI 2025) solves an SVD-based projection over individual client gradients on the server. FedCARE (IJCAI 2026) trains a data-free generator on the server and filters head updates against stored heads (details in our response to MQWK). Comparing against them would mix two system models, so we instead make this distinction explicit in Sec. 2 and App. A.
Q4: Scale (modern models, more clients). We agree that scale matters, and we want to be transparent about what we could and could not run. Our experiments use a single H100 (App. D.7), on which each CIFAR-10 retraining reference alone takes ≈2.3 h. Within the rebuttal period we gave priority to the ablation, retraining and learning-rate studies that all three reviewers asked for. A foundation-model or many-client study was not feasible with our compute, and we do not want to report a rushed, under-tuned version of it. On scalability by design: f-FUM adds no server-side state and no per-client computation on the server. The server performs only the sample-weighted averaging of FedAvg, and each client's cost depends only on its own forget and retained sets, not on N. Its cost therefore grows with the number of clients exactly as standard federated training does. Nothing in the method is specific to small models either. It only needs predictive distributions, so it applies unchanged to pretrained backbones and to adapter-based fine-tuning.