acceptodds
Under review as a conference paper at ICLR 2027

Proactive Unlearning Readiness: A Two-Phase Framework for Efficient Federated Unlearning

Abstract

Federated learning (FL) is widely used in privacy-sensitive applications, including medical imaging, recommendation, and personalized modeling. Once deployed, these systems may receive deletion requests that ask the model to forget the influence of specific clients, classes, or samples; in FL, this requirement is typically handled through federated unlearning (FU). Existing FU methods are mostly reactive and often depend on partial retraining or expensive parameter corrections after a request arrives. We propose Efficient and Reversible Proactive Federated Unlearning (ERPFU), a proactive two-phase framework that addresses this bottleneck from both training and unlearning. During training, weight orthogonality regularization (WOR) reduces interference among learned filters and prepares the model for localized future updates. During unlearning, a sparse classifier adapter (SCA) freezes the pretrained backbone and overwrites target knowledge through a small set of classifier parameters, reducing cost while keeping the update path controlled. ERPFU is intended as an efficient approximation to retraining-based unlearning, not as a claim of complete encoder scrubbing. Experiments on FashionMNIST, CIFAR-10, and CIFAR-100, covering client-, class-, and sample-level forgetting, show that ERPFU stays close to retraining in retention while using much less computation and communication. Additional analyses on non-IID partitions, orthogonality strength, and sparsity show that the method remains stable in heterogeneous FL settings.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
federated unlearning, sparse adaptation, federated learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_d3P2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ERPFU combines weight-orthogonality regularization during federated training with retained-data optimization of a fixed, magnitude-selected classifier residual after a deletion request. The encoder and base classifier remain frozen. A conditional analysis separates optimization error, excluded correction coordinates, and the discrepancy between frozen-feature adaptation and full retraining. The reported experiments compare retained and deletion-associated accuracies across several datasets and request types. The paper limits its claims to functional behavior and retained utility, and explicitly does not certify privacy erasure.

Critical strengths
  1. The restricted optimization problem is specified clearly. The paper distinguishes active coordinates from actual dense storage, explains that 90% of the classifier remains trainable, and states the effect of copying base weights into the residual. This makes the computational restriction and its limitations easier to assess.

  2. The analysis separates approximation errors that are often conflated. In particular, optimizing the classifier more accurately cannot eliminate a mismatch caused by the frozen encoder. The covariance and aggregation calculations also explain why orthogonal local weights alone do not imply globally decorrelated representations. These are useful qualifications to the proposed preparation strategy.

Critical weaknesses
  1. The provenance of the central utility results needs clarification. Appendix F, p. 16, says that complete run configurations are unavailable and exact correspondence between the formal SCA specification and every historical run remains unverified. This does not establish that the runs used a different algorithm, but the paper does not identify which records were checked. In particular, can the four Table 1 ERPFU results be traced to runs with a frozen encoder, fixed classifier support, and the stated initialization and evaluation state? These records underpin the main claim of retained accuracy within 1.85 points of retraining. Reusing disclosed benchmark values and lacking matched seeds are not themselves errors; the unresolved issue is whether the central comparison evaluates the defined method under comparable conditions.

  2. The empirical contribution beyond classifier-only adaptation remains modestly supported. Appendix E varies the active fraction from 0.80 to 0.95, but provides no dense-head, random-support, or no-WOR control. The paper correctly acknowledges that the sweep confounds support with copy initialization and does not establish a causal benefit from WOR or magnitude selection. This does not invalidate its narrower feasibility claim. It does leave unclear what the proposed preparation and support rule add to an ordinary retained-data head refit, which limits the strength of the method’s empirical contribution.

Questions for authors
  1. Can the authors map the four main IID ERPFU records to their code versions and configurations, identify any differences from Section 3.3, and confirm comparable partitions and evaluation protocols with retraining? If those records cannot be recovered, a traceable run of the specified method in a representative setting would help establish which conclusions remain supported.

  2. Is there existing evidence comparing the proposed design with dense classifier adaptation under a common preparation and initialization protocol? Such a comparison would help assess the contribution of fixing a magnitude-selected support. A claim that WOR or magnitude selection improves performance would require its corresponding control; the narrower utility-feasibility claim does not require a privacy guarantee or validation of the logit bound.

Reviewer_PwYqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ERPFU, a federated unlearning framework that combines weight orthogonality regularization during federated training with sparse classifier adaptation during unlearning. Upon a deletion request, the encoder and base classifier are frozen, and only a fixed subset of classifier residual parameters are optimized on retained data. A theoretical analysis decomposes the discrepancy from full retraining into optimization, support restriction, and reference approximation errors.

Critical strengths

Federated unlearning is a timely and important problem. In privacy-sensitive domains such as healthcare, participating institutions may need to withdraw training data to comply with regulations, making efficient and verifiable data removal a practical necessity rather than a theoretical convenience. Addressing the computational cost of retraining upon deletion requests is a meaningful contribution to this growing area.

Critical weaknesses

1.The primary goal of federated unlearning is to remove the influence of deleted data, not merely preserve predictive utility relative to retraining. ERPFU freezes the encoder throughout unlearning, so representations learned from the deleted client’s data remain unchanged and may still encode information associated with that data. The experiments only evaluate predictive accuracy on retained and deleted distributions, which is insufficient to establish whether the requested information has been forgotten. 2.WOR is imposed during initial federated training to prepare the model for future unlearning, but orthogonality constraints may degrade original learning capacity, particularly under heterogeneous client distributions. The paper does not establish whether this preparation introduces a performance trade-off before any deletion request occurs. 3.PFU-SKLA is among the most closely related methods, yet the paper relies on previously published results rather than re-evaluating it under matched configurations, raising fairness concerns. FOUL, a relevant recent proactive federated unlearning method discussed in the related work, is absent from experiments. Given that ERPFU essentially performs classifier adaptation on retained data, a standard classifier fine-tuning baseline is also needed to establish the benefits of the proposed design. More comprehensive comparisons under consistent partitions, budgets, and evaluation protocols would strengthen the evidence.

Questions for authors

see weakness

Reviewer_4No6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a method for FUL method that investigates whether adaptation can be restricted to the final classifier to reduce computation and enable clearer theoretical analysis. The method has two phases. During learning, it applies weight orthogonality regularization, aiming to reduce interference between representations and facilitate later unlearning. After a deletion request, it freezes the encoder and original classifier and trains a sparse classifier adapter on retained data, updating only a fixed subset of classifier residuals.

Critical strengths

The paper investigate an practical question: can the unlearning focus on the final classification layer instead of requiring multiple adapter layer across the network. It provide interesting analysis if fixed classifier can reach retraining performance.

Critical weaknesses
  1. The contribution of the proposed works compared to existing work may be not considerable enough. Specifically, the authors have acknowledge in section 2 and 3 that “FUSED freezes the base model and learns sparse adapters in selected layers” [1]. It also mentions “PFU-SKLA combines orthogonal preparation with sensitivity-guided critical-layer selection and sparse adaptation” [2]. Although the specific regularizers and selection rules differ, the broader combination is already established. Can the authors clarify what additional capability the fixed classifier support provides beyond simplifying these methods?

  2. The experimental scope is limited on FashionMNIST, CIFAR10 and CIFAR100. The paper also mentions “These settings evaluate targeted corrective adaptation and do not cover arbitrary benign-sample deletion.” Can the authors evaluate benign-data deletion and clearer domain differences, using benchmarks such as PACS, TerraIncognita, or VLCS, to test whether keeping the encoder fixed remains effective? Also, the experiments in the paper now only learn the client set C ∈ {1, 3, 5, 10}, which may be too less. The author should consider doing experiments on more clients, and more diverse data heterogeneity to tell if the method are robust despite using less adapter.

  3. The reported metrics do not directly test whether the requested information has been forgotten. Section 5.1 mentions “RA and FA are accuracies on independent test partitions associated with retained and deleted data, respectively. They are not training-set membership measurements.” The authors should run a membership inference attack on deleted training examples and compare its results with retraining.

  4. The evaluation only shows the final results, but does not explain what lead to that unlearning effectiveness. This even acknowledge by the paper in section 5.5 “It does not isolate the benefit of magnitude selection or the causal effect of WOR.”. The authors should compare the complete method with three controlled variants: removing orthogonality regularization, updating the entire classifier, and selecting the same number of classifier coordinates randomly.

  5. Also, additional analysis on comparison between restricted adapter version and the related works that have adapter across the network should be included. Without it, reader are hard to understand why only learn and do federated unlearning on final classifier, which normally mean much less capacity, can bring good RA and FA.

  6. One of the benefit of working on only final classifier layer is the computation cost, but it is not clearly discussed in the paper. Adding it should make the paper more informative.

  7. Minor: The abstract writing is very hard to understand. I see that it is not matched with the abstract posted on openreview. Please check can revise it.

[1] Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter - CVPR 2025.

[2] Proactive Federated Unlearning: Sensitivity-Guided Sparse Adaptation on Key Layers - IJCAI 2026.

[3] Computation and Communication Efficient Federated Unlearning via On-server Gradient Conflict Mitigation and Expression - CVPR 2026.

Questions for authors

Please refers to weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.