acceptodds
Under review as a conference paper at ICLR 2027

TARGETED FORGETTING: A FRAMEWORK FOR CLIENT- CLASS CENTRIC KNOWLEDGE REMOVAL IN FEDERATED LEARNING

Abstract

Federated unlearning (FU) aims to remove the influence of specific clients or classes from a trained global model upon request, without accessing raw data or degrading overall performance. Existing FU methods often struggle to fully eliminate targeted information in a single unlearning round, particularly under non-IID data distributions, and may adversely affect non-target performance. In this work, we propose a hybrid federated unlearning framework that supports client-level, class-level, and client-class unlearning through parameter-level updates. It combines server-side gradient residual correction (proxy filtering) with client-side perturbation-based pruning (weight negation) to precisely remove targeted influences without requiring access to local training data. This design enables efficient, data-independent unlearning and generalizes across IID, non-IID, and label-shifted settings. Experiments on MNIST, FMNIST, and CIFAR10 demonstrate that our framework achieves effective single-client, multi-client, single-class, and multi-class unlearning with minimal impact on non-target accuracy, while maintaining scalability and robustness across federated rounds.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Unlearning, Client-Class Unlearning, Privacy Preservation, Non-IID Data, Membership Inference Attack

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2zugOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a federated unlearning framework that handles three kinds of requests. A request can remove a whole client, a whole class, or the data that specific clients contributed to specific classes. The last case, client-class unlearning, is the main new setting. The method has two parts. The first part is weight negation. For each layer, it takes the mean gradient of the target data as a direction. It then scales the weight component along that direction by 1 − 2λ_l, where λ_l depends on layer sensitivity and selectivity. The second part, which the paper calls gradient residual correction or proxy filtering, replays stored client updates from a checkpoint. Each update is weighted by exp(−γι), where ι is the inner product between the update and the gradient of the target loss. For client unlearning, the target clients' updates are dropped during replay and the model is then calibrated on the remaining clients. For client-class unlearning, the gradients are computed on D_{k,c} and only the target clients' updates are filtered. Appendix C gives four theorems on smoothness, utility, a forgetting margin and membership inference. The experiments use MNIST, FMNIST and CIFAR10 with 5 or 10 clients under IID, non-IID and label-flip settings. Class unlearning is compared with FUSED, NoT, and FedAF using error rates, confusion matrices, F1 scores, run time and MIA metrics. Client and client-class results are shown for the proposed method only.

Critical strengths
  1. Client-class unlearning is a useful problem. A realistic case is a client that has wrong labels for one or two classes. Removing the whole client throws away its clean data, and removing the whole class throws away what other clients contributed. The paper handles all three request types in one pipeline, and in the MNIST label-flip experiment in Fig. 5 the flipped class recovers after the request. This is the main reason the paper interests me.

  2. On MNIST and FMNIST class unlearning, the method reaches 100% Error_f, and its Error_r after unlearning is 0.25% and 5.44%, lower than all three baselines in Table 1. Its MIA accuracy is close to 50% in Table 3. The paper also gives pseudo-code for all three modes and a full hyperparameter table, and it openly says that the method is not best on every metric.

Critical weaknesses
  1. The method is not consistent as written, and the theory does not support it. The two phases are not connected in the pseudo-code. Algorithm 1 computes the negated model W^(0) in step 18, but replay starts from the checkpoint W_τ in step 9 and the algorithm returns the replay output. W^(0) is never used again. Algorithms 1.2 and 1.3 have the same problem. Appendix C writes the method as U = C(R(N(W))), yet the replay operator R in Eq. 29 starts from the checkpoint and not from N(W). So it is unclear what the experiments actually ran.

  2. The paper says its unlearning needs no access to local training data. The abstract calls it data-independent and Appendix B calls negation data-free. Yet Eqs. 4, 10, 16 and 20 all need gradients on target samples, and Eq. 10 needs them again at every replay step. Phase 1 is called client-side, but Fig. 6 and line 903 have the server run the weight negation step. The paper never says who computes these gradients or on which data.

  3. The theory relies on assumptions that do not fit the setting, and several steps do not follow. Theorem 2 assumes that every per-sample loss is strongly convex in W and that the original model is already close to the retrained one. Neither is shown for the neural networks in the experiments. The proof then treats L_r as Lipschitz because it is smooth, which does not follow. It also says that a larger β brings L_r as close as we like to L_r(W_r), but Eq. 43 keeps the fixed ε_replay + ε_train term. Theorem 3 assumes |1 − 2λ_l| ≥ κ, which does not make the coefficient negative. Setting λ_l = 0 meets this condition and leaves the model unchanged. Theorem 3 is also about the intermediate W^(0) and not about the model that is returned. In Theorem 4, Eq. 61 bounds the distance to the original model. This can at most show that the attack advantage after unlearning is close to the advantage before it, not that it is small. The step from Eq. 61 to the exp(−γῑ) + C2/η bound in Eq. 56 is not derived.

  4. The experiments do not show that the unlearning works as claimed. There is no retrained model to compare against, although Eq. 2 defines the goal relative to W_r and Appendix E calls W_r the gold standard. There is also no ablation of negation, replay and calibration from one common starting model. Given W1.1, this ablation is needed to see which part does the forgetting.

  5. The FedAF baseline is cited as Yang et al. 2024, but that reference is a time-series forecasting model, not an unlearning method. FedRecon is a paper on missing-modality reconstruction and is cited as unlearning work. In Fig. 2, classes 8 and 9 are already poorly recognized before client unlearning, which conflicts with the text around line 401. The algorithms contain leftover LaTeX color names such as lightyellow and lightblue. Appendix C contains the placeholder phrase "the reference example", and "Eq. equation" appears throughout.

Questions for authors
  1. Who computes the gradients in Eqs. 4, 10, 16 and 20, and on which data? How does the server get target gradients at each replay step without holding target data?

  2. Which checkpoint τ was used for t* = 25 and t* = 40, and was the replay window empty? For client unlearning, how is the influence of the target clients before τ removed?

  3. Can you add results for a model retrained from scratch, covering Error_f, Error_r, Error_t, MIA and time for all three request types? Can you also add an ablation of negation, replay and calibration from the same starting model?

  4. Did all methods start from the same model? Why does FUSED barely change on CIFAR10, why is FedAF missing there, and which FedAF implementation did you use, given that the cited paper is about forecasting?

5 comments
AuthorsOpenReview ↗

Thank you for the careful review and for recognizing client-class unlearning as a useful problem, along with the unified handling of all three request types. Your comments helped us clarify how the stages of our method connect, who computes what during unlearning, and how our method compares with retraining and with baselines started from the same model.

Q1: Our procedure needs no gradients on forget data, and the server never needs target gradients during replay. This is a key privacy property: a client can delete its forget samples as soon as it submits a request. Eqs. (4), (10), (16), and (20) give the general gradient-based form of the two stages; in our experiments, both stages are computed without target gradients.

Weight negation (Eqs. 4, 20): the server flips the sign of the selected layers of the global model, as shown in Fig. 6. This uses no data at all.

Replay weights (Eq. 10, 16): the server weights each client’s stored update by that client’s share of target labels, computed from per-class label histograms reported during training. The weights are fixed before replay begins, so no gradient is required at any replay step.

The only information the server needs beyond standard federated updates is these label counts. We will present this gradient-free form as the main formulation in Section 3 and describe negation as server-side throughout.

Q2: The replay window is not empty. We use the last checkpoint strictly before the request: τ=20 for t^⋆=25 and τ=35 for t^⋆=40, as in Algorithm 1.1 and Appendix E, so replay covers five rounds. We will correct the “≤” in Section 3 to “<”.

Rounds before the checkpoint are already part of it, so the choice of τ trades cost against fidelity. Starting replay from round 0 removes the target clients’ influence from every round: the history is rebuilt from the initial model, which no client has influenced, with the target clients excluded at every round and each remaining client’s stored update recalibrated by a short local training run, following FedEraser (Liu et al., 2021). On a converged CIFAR-10 model, this matches retraining from scratch:

MethodForget ErrorRetained ErrorMIA AUC
Before unlearning6.98%6.71%0.591
Ours (replay from round 0)19.71%6.32%0.501
Retrain from scratch20.26%7.47%0.494

These results use the fully federated procedure, in which the target clients take no part in unlearning; in our earlier client-unlearning experiments, the calibration step was simulated centrally over all clients’ data.

Q3: All configurations start from the same converged model and are compared with a model retrained from scratch on the retained data for the same number of rounds. For CIFAR-10 class unlearning (class 3):

MethodForget Error (%)Retained Error (%)Test Error (%)MIA AUCTime
Before unlearning16.824.8618.470.648—
Retrain from scratch100.004.3823.400.49510,991
Calibration only100.003.8823.040.5381,721
Negation + calibration100.009.1424.290.499890
Replay only100.004.4423.410.4927,889
Replay + calibration100.004.1123.250.4978,769
Replay + negation + calibration100.009.8023.950.4888,767

The ablation shows which part does the forgetting. Calibration alone removes the class from the predictions, but the attack still ranks members above non-members (MIA AUC 0.538 vs. 0.495 for retraining), even though its advantage at the default threshold is zero. Replay removes this remaining signal, and replay with calibration matches retraining on every metric. Negation with calibration reaches complete forgetting at about one twelfth of retraining time, at a cost of about 5 points in Error_r. Our framework therefore offers two operating points: replay with calibration for the closest match to retraining, and negation with calibration for complete forgetting at low cost. Client unlearning follows the same pattern. Replay from round 0 raises the forgotten client’s Error_f from 6.98% to 19.71% (retraining: 20.26%), keeps Error_r at 6.32% (7.47%), and lowers MIA AUC from 0.591 to 0.501 (0.494), at about 80% of retraining time (8,691 s vs. 10,724 s).

Client-class unlearning. Two of ten clients hold half of all class-3 training images, relabeled as class 5, while the other clients keep class 3 correctly labeled. Before unlearning, the model predicts class 5 on 76.0% of the flipped images, and the corruption spreads to all clients (class-3 test error 89.6%):

MethodAgreement with Flipped Label (%)Class-3 Test Error (%)Retained Error (%)MIA AUCTime
Before unlearning75.9689.611.510.582—
Retrain from scratch33.9270.913.070.5268,464
Calibration only43.4065.69.820.5611,591
Negation + calibration43.5273.816.790.543863
Replay only33.6074.213.900.5217,827
Replay + calibration38.1674.312.980.5228,531
Replay + negation + calibration43.4081.918.880.5268,715

Replay removes the flipped contribution to the level of retraining: replay alone reaches agreement 33.6% (retraining: 33.9%) and MIA AUC 0.521 (0.526), and replay with calibration reaches 38.2% and 0.522 while keeping retained error at 12.98% (13.07%). Calibration alone removes most of the flip at about one fifth of the cost but leaves more membership signal (AUC 0.561), and adding negation increases retained error, as in class unlearning. Class 3 recovers only partly, even for retraining, because the benign clients hold only 2,500 class-3 images.

Q4: In the submitted experiments, each baseline was trained in its own federation. We now run every method from the same converged model, with the same evaluation code, and compare each with retraining from scratch. On CIFAR-10 class unlearning (class 3):

MethodForget Error (%)Retained Error (%)Test Error (%)MIA AUCTime
Before unlearning16.824.8618.470.648—
Retrain from scratch100.004.3823.400.49610,991
FUSED47.163.4519.160.6502,375
FedAF100.0086.3087.780.508126
Class-discriminative pruning100.006.1324.400.543368
Replay + calibration (ours)100.004.1123.250.4988,769
Negation + calibration (ours)100.009.1424.290.500890

FUSED. Under this protocol, with its trained adapters applied before every metric, FUSED does change the model: it removes about half of the target class (Error_f 47.16%) while preserving retained accuracy, but leaves membership detectable (MIA AUC 0.650).

FedAF. refers to Li et al., “Class-wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory Generation” (arXiv:2307.03363); we have corrected the citation. It was missing on CIFAR-10 because our implementation initially covered only the MNIST-scale model. As no official code is available, we implemented it following the paper (teacher-generated memories and the sketched EWC preserver), with its specified schedule and hyperparameters, and it is now included on CIFAR-10. At the learning rate its authors specify for CIFAR-10, it forgets the class but loses most retained accuracy on our converged model.

We also added class-discriminative pruning (Wang et al., 2022), a strong class-level method that forgets fully at low cost; it is designed for class removal only, whereas our framework handles client, class, and client-class requests within one pipeline. Among all methods, replay with calibration is the closest to retraining overall.

Reviewer_5px1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a hybrid federated unlearning framework that supports client-level, class-level, and client-class-level removal. For class and client-class targets, it first applies layer-wise weight negation along the normalized gradient direction of the target data, with an adaptive strength computed from layer sensitivity and selectivity. It then replays stored client updates from the most recent checkpoint, reweighting each update by an exponential function of its inner product with the target-data gradient. Client-level unlearning reduces to discarding the target clients' updates during replay, followed by calibration on the remaining clients. Appendix C states differentiability, utility, forgetting-margin, and membership-inference bounds. Experiments on MNIST, FMNIST, and CIFAR-10 with five or ten clients under IID, non-IID, and label-flipped partitions compare class-level unlearning with FUSED, NoT, and FedAF, and report error rates, confusion matrices, membership-inference metrics, and runtime.

Critical strengths
  1. The client-class setting addresses a practical gap. Removing only the contribution of specific clients to specific classes, for example classes whose labels were flipped by malicious clients, while keeping the same classes learned from benign clients, is not directly covered by existing client-level or class-level methods. The label-flipping experiments illustrate this use case (Figures 5 and 10).
  2. The framework is unified and the procedure is described in detail. All three granularities share the same structure of weight negation followed by filtered replay, and Algorithms 1.1-1.3, together with the sequence diagram in Figure 6, make the workflow easy to follow.
Critical weaknesses
  1. The comparison with recent baselines is insufficient. Class-level results include only FUSED, NoT, and FedAF, and FedAF is missing on CIFAR-10 (Table 1). Client-level unlearning has no baseline, although FedEraser and FedRecover, both cited in the paper, and the method of [3] are applicable. Class-discriminative pruning [2], which also removes classes by editing class-specific parameters, is neither discussed nor compared. FedAF is cited as a time-series forecasting Transformer (Yang et al., 2024) rather than the class-wise unlearning method of [1]. On CIFAR-10, FUSED is essentially unchanged after unlearning (82.34% accuracy before and after), which suggests an implementation problem.
  2. The replay weighting appears to have the wrong sign. Eqs. (10)-(11) define ι=⟨Δ,∇ℓ⟩\iota=\langle\Delta,\nabla\ell\rangle and w=exp⁡(−γι)w=\exp(-\gamma\iota). An update that helps learn the target class reduces the target loss, so to first order ι<0\iota<0 and w>1w>1: such updates are amplified rather than suppressed, contrary to the claim on page 4. The proof of Theorem 4 simply assumes ι≥0\iota\ge0 (before Eq. 60). Unless the implementation uses −ι-\iota or ∣ι∣|\iota|, the server-side correction for class and client-class unlearning works against its stated purpose.
  3. With the stated settings, replay removes little or nothing. Section 3 defines τ=max⁡{t′:t′ mod s=0, t′≤t⋆}\tau=\max\{t': t' \bmod s=0,\ t'\le t^\star\}, and Appendix E uses s=5s=5 with t⋆∈{25,40}t^\star\in\{25,40\}, so τ=t⋆\tau=t^\star and the replay window [τ,t⋆−1][\tau,t^\star-1] is empty. Algorithm 1.1 instead uses τ<t⋆\tau<t^\star, which gives only five rounds. In both cases, the target's contributions from all earlier rounds remain in the checkpoint WτW_\tau. Client-level unlearning then amounts to a few rounds of continued training without the target client, which does not remove that client's influence.
  4. The experiments do not support the claim of precise forgetting without collateral damage. The retrained model, defined as the gold standard in Eq. (2) and Appendix E, is never reported. On CIFAR-10, retained-set error rises from 10.6% to 18.92% (Table 1). Counting the confusion matrices in Figure 3, correct predictions on non-target classes fall from 8080 to 7298, with class 4 dropping from 817 to 577, which contradicts the claim of minimal spillover. In Table 2, CIFAR-10 retained and test errors increase, contrary to the text. The MIA tables report negative values for an advantage defined as an absolute value (Eq. 53) and AUC values such as 0.56%.
  5. Parts of the theory are incorrect or do not match the method. In Theorem 3, the condition ∣1−2λl∣≥κ|1-2\lambda_l|\ge\kappa is satisfied by λl=0\lambda_l=0, which leaves the model unchanged, so the negative-margin conclusion cannot follow; the condition should be 1−2λl≤−κ1-2\lambda_l\le-\kappa. Theorem 2 assumes a strongly convex loss and ∥Wt⋆−Wr∥≤εtrain\|W_{t^\star}-W_r\|\le\varepsilon_{\mathrm{train}}, i.e., that the original model is already close to the retrained one. The proof of Theorem 4 derives a drift that grows with γ\gamma but concludes a bound that decreases in γ\gamma. The calibration objective in Eq. (33) appears only in the appendix and is not part of the class-level procedure in Section 3.
  6. The claim of data independence is unclear. The abstract states that unlearning does not require access to local training data, but Eq. (4) needs gradients on the target data, and Eq. (10) needs target-data gradients at every replayed intermediate model W~t′\tilde W_{t'}. The paper does not specify who computes them, how often the target clients must participate, or the communication and storage costs, given that all per-round client updates are stored. This also conflicts with the motivation that clients may be unavailable or unwilling to participate.
Questions for authors
  1. Does the implementation use −ι-\iota, ∣ι∣|\iota|, or a clipped score in Eq. (11)? If not, how is the amplification of updates that help learn the target class avoided?
  2. With s=5s=5 and t⋆∈{25,40}t^\star\in\{25,40\}, is the replay window empty? How are the target's contributions before the checkpoint removed, and could you report results when replay starts from round 0?
  3. Could you report the retrained model, client-level baselines such as FedEraser and FedRecover, and more recent class-level methods on all three datasets?
  4. Who computes the target-data gradients in Eqs. (4) and (10), and in how many rounds must the target clients participate?
1 comment
AuthorsOpenReview ↗

Thank you for the in-depth review and for recognizing that client-class unlearning addresses a practical gap not covered by existing methods. Your questions pointed to the places where our description needed to be more explicit. Below, we clarify the replay weights, the checkpoint choice, and the role of each party during unlearning, and we add comparisons with retraining and with client-level and class-level baselines, all started from the same model.

Q1: Our replay weight does not depend on the gradient inner product, so its sign cannot flip. Each client’s replayed update is weighted by w_k=1-a p_k, where p_k∈[0,1] is the fraction of client k’s data in the forget set and a∈{0.2,0.5}. Clients with p_k>0.1 are excluded entirely. Excluded clients receive weight 0, and every remaining client’s weight lies in [0.95, 1], so no update is amplified and clients holding more target data are always reduced more. The weight also needs only per-client label counts, so no target-data gradient is computed during replay. We will update the exponential form in Eqs. (10)–(11) with this weight in the final version.

Q2: The replay window is not empty. We use the last checkpoint strictly before the request, τ=20 for t^=25 and τ=35 for t^=40, as in Algorithm 1.1 and Appendix E, so replay covers five rounds. We will correct the “≤” in Section 3 to “<”.

Rounds before the checkpoint are already part of it, so the choice of τ trades cost against fidelity. Starting replay from round 0 replays every earlier round without the forget data: the history is rebuilt from the initial model, which no client has influenced, and each stored update is recalibrated with a short local training run on retained data, following FedEraser (Liu et al., 2021). On a converged CIFAR-10 model, it matches retraining from scratch:

ModeForget errorRetained errorMIA AUC
Client (ours / retrain)19.71% / 20.26%6.32% / 7.47%0.501 / 0.494
Class (ours / retrain)100% / 100%4.11% / 4.38%0.498 / 0.496

Replay from round 0 takes about 80% of retraining time; cheaper checkpoint settings reduce this cost at the expense of a larger gap to retraining, and we can report this trade-off in the final version.

Q3: We compare all methods against a model retrained from scratch, with every method starting from the same converged model and evaluated with the same code. For CIFAR-10 class unlearning (class 3):

MethodForget Error (%)Retained Error (%)Test Error (%)MIA AUCTime
Before unlearning16.824.8618.470.648—
Retrain from scratch100.004.3823.400.49610,991
FUSED47.163.4519.160.6502,375
FedAF100.0086.3087.780.508126
Class-discriminative pruning100.006.1324.400.543368
Replay + calibration (ours)100.004.1123.250.4988,769
Negation + calibration (ours)100.009.1424.290.500890

Replay with calibration is the closest to retraining overall, matching it on forgetting, retained accuracy, and membership inference. FUSED removes only about half of the target class and leaves membership detectable (MIA AUC 0.650). FedAF, which refers to Li et al. (arXiv:2307.03363), forgets the class but loses most retained accuracy at the learning rate its authors specify for CIFAR-10. Class-discriminative pruning (Wang et al., 2022), which we have added, is a strong class-level method that forgets fully at low cost; it is, however, designed for class removal only, whereas our framework handles client, class, and client-class requests within a single pipeline.

For client unlearning, our replay from the initial model follows FedEraser (Liu et al., 2021) and matches retraining on CIFAR-10 (forget error 19.71% vs. 20.26%; MIA AUC 0.501 vs. 0.494), as shown in Q2. Results on MNIST and FMNIST, together with FedRecover, can be includ in the final version.

Q4: A key property of our procedure is that it requires no gradients on forget data, so a client can delete its forget samples as soon as it submits a request. Eqs. (4) and (10) give the general gradient-based form of the two stages; in our experiments, both are computed without target gradients. Weight negation is a server-side sign flip of the selected layers, which needs no data; the layers themselves are selected from stored updates and label counts. Replay weights are computed by the server from each client’s share of target labels, using per-class label histograms reported during training, so no target gradient is needed at any replay step. We will present this gradient-free form as the main formulation in Section 3.

The target clients’ participation depends on the request. In client unlearning, they take part in no rounds. In class and client-class unlearning, they participate like any other client, training only on their retained data, in each replayed round and in the calibration rounds (10 in our experiments). The client-unlearning results in Q2 use this fully federated procedure; in our earlier client-unlearning experiments, the calibration step was simulated centrally over all clients’ data. We will update storage and communication costs for every method in the final version.

Reviewer_ViqTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a hybrid federated unlearning framework supporting client-level, class-level, and client-class-specific knowledge removal. The method combines client-side weight negation using target-gradient projections with server-side proxy filtering of historical client updates. The authors provide theoretical claims concerning operator smoothness, utility preservation, targeted forgetting, and membership-inference resilience.

Experiments on MNIST, Fashion-MNIST, and CIFAR-10 evaluate forgetting performance under IID, non-IID, and label-flipped settings, using classification metrics, confusion matrices, membership-inference attacks, and execution time.

Critical strengths
  1. Relevant and fine-grained unlearning objective.

Supporting client-class-specific removal while retaining the same class information contributed by other clients addresses an important problem in heterogeneous federated learning. The unified treatment of multiple unlearning granularities could be valuable if its correctness is established.

  1. Evaluation across several unlearning settings.

The experiments consider different datasets, data heterogeneity regimes, and unlearning scenarios. The combination of forgotten-set performance, retained-set performance, and membership-inference measurements is a useful starting point for evaluating unlearning trade-offs.

Critical weaknesses
  1. Fundamental inconsistencies in the algorithm and theoretical guarantees.

Algorithm 1 computes weight-negated parameters but does not clearly incorporate them into the subsequently replayed and returned model. This appears inconsistent with the theoretical composition of negation, replay, and calibration in Eq. (25).

Furthermore, the exponential proxy-filtering weight in Eqs. (10)–(12) can exceed one for ordinary gradient-descent updates aligned with the target objective, potentially amplifying rather than suppressing their contribution.

The multi-target operator in Eq. (21) is not generally an orthogonal projector without additional orthogonality assumptions or orthonormalization. Several theoretical derivations also contain unsupported steps, including the use of smoothness as a Lipschitz bound, the forgetting-margin condition in Theorem 3, and the membership-inference bound in Theorem 4.

These issues affect the validity of the central methodological claims.

  1. Insufficient evidence of genuine unlearning and privacy preservation.

Although the paper defines a retrained model as the unlearning reference, the experiments do not systematically compare the proposed method against retraining without the forgotten data.

The observed suppression of target-class predictions is not sufficient to establish removal of a specific client's learned influence. On CIFAR-10, the method also incurs substantial degradation in overall accuracy and retained-data performance.

The MIA evaluation requires further clarification. In particular, the reported FMNIST MIA AUC of 26.39% can become 73.61% under score reversal, suggesting that the available measurements do not establish resistance to adaptive membership inference.

Component ablations, matched initialization for baseline comparisons, and statistical uncertainty estimates are also needed.

Questions for authors

Q1 — Algorithmic implementation: How is the output of weight negation incorporated into the final replayed model in Algorithm 1? Can the authors reconcile the checkpoint-selection rule with the experimental unlearning rounds and explain how historical contributions preceding the checkpoint are removed? What prevents exponential filtering weights from exceeding one?

Q2 — Theoretical validity: Can the authors provide valid orthogonal projection operators for multiple target gradients and correct the unsupported steps in Theorems 1–4? Which guarantees remain valid under assumptions appropriate for the neural networks actually evaluated?

Q3 — Unlearning verification: Can the authors provide retrain-from-scratch reference comparisons, ablations isolating negation and proxy filtering, and clarified MIA evaluations under balanced and adaptive attacks? How do the authors reconcile the reported CIFAR-10 retained-data degradation with the claimed selective preservation of non-target knowledge?

3 comments
AuthorsOpenReview ↗

Thank you for the careful review and for recognizing the value of client-class unlearning. Your feedback helped us see where we need to clarify how our model is obtained in Algorithm 1, provide stronger justification for our theoretical claims, and present clearer evidence on forgetting, retained accuracy, and membership inference.

Q1: Weight negation selects the layers most associated with the target (the three layers with the largest stored updates from clients holding target data, combined with a fixed per-layer prior) and flips their sign. What carries into the final model is this layer selection. Replay and calibration both use it: updates to the selected layers are attenuated with their own strength during replay, and these layers get their own learning rates during calibration. Since replay starts from the stored checkpoint, the sign-flipped values themselves are not kept, and the returned model is W^un=C(R(W_τ)) with both stages guided by the selected layers. When negation is instead applied after replay, its effect is kept in the returned model; we evaluate both orderings in the ablation in Q3. We will state the composition explicitly in Algorithm 1 and Eq. (25).

We use the last checkpoint strictly before the request, i.e., τ = 20 for t^* = 25 and τ = 35 for t^* = 40, so replay covers the five rounds in between. Rounds before τ are already part of the checkpoint and are not replayed, so the choice of τ is a trade-off between cost and fidelity. Setting τ = 0 starts replay from the initial model W_0, which no client has influenced, and replays every earlier round without the forget data, recalibrating each stored update with a short local training run on retained data. On a converged CIFAR-10 model, this full replay closely matches retraining from scratch:

Unlearning ModeForget ErrorRetained ErrorMIA AUC
Client (Ours / Retrain)19.71% / 20.26%6.32% / 7.47%0.501 / 0.494
Class (Ours / Retrain)100% / 100%4.11% / 4.38%0.498 / 0.496

Each client's replayed update is weighted by w_k=1-a p_k where p_k∈ [0,1] is the fraction of client k's data in the forget set and a ∈ {0.2, 0.5}; clients with p_k > 0.1 are excluded entirely. Since both a and p_k lie between 0 and 1, every weight satisfies 0 < w_k≤ 1. Excluded clients receive weight 0, and every remaining client’s weight lies in [0.95, 1], so no update is amplified. We will replace the exponential form in Eqs. (10)–(12) with this weight in the final version.

Q2: We provide a valid projector for any number of targets, restate each theorem under assumptions for which every step follows, and state precisely which results apply to the CNNs we evaluate.

Summing u ̂u ̂^⊤ gives a projector only when the target directions are orthogonal, which target gradients generally are not. We therefore orthonormalize first. Stacking the target gradients as G=[g_1,…,g_m ] and taking an orthonormal basis Q of their span (by QR or SVD, discarding near-zero singular values), we use P=QQ^⊤,  P^⊥=I-P.

Since Q^⊤ Q=I, we have P^⊤=P and P^2=QQ^⊤ QQ^⊤=P, so P is an orthogonal projector for any m, and P^⊥ leaves every direction orthogonal to all targets unchanged. Our weight negation applies a whole-layer sign flip and does not depend on this operator; we will present Eq. (21) in this form for completeness.

Corrected statements of Theorems 1–4.

Theorem 1: Differentiability of the composed operator follows from composing differentiable maps when the activations are smooth. The manifold statement would additionally require local invertibility, which we do not need for any later result, so we can remove it.

Theorem 2: The step that treated smoothness as a Lipschitz bound is replaced by the smoothness inequality itself. Since W_r minimizes the retained loss, ∇L_r (W_r )=0, and L-smoothness gives directly L_r (W^un )-L_r (W_r )≤L/2 ‖W^un-W_r ‖^2, This requires neither a Lipschitz bound nor strong convexity. The remaining error terms do not vanish as β grows, so the statement no longer claims the gap can be made arbitrarily small.

Theorem 3: A negative margin requires the target component to change sign, so the condition becomes 1-2λ_l≤-κ, i.e., λ_l≥(1+κ)/2. With this condition, the margin argument holds for the final linear layer, where the logit is linear in the weights. Our experiments negate the first convolutional layer, so Theorem 3 characterizes the negation operator rather than the configurations we evaluate.

Theorem 4: We bound the attack advantage relative to the retrained model, which is what unlearning should achieve. The attack declares “member” when its score is at least 0.5. Requiring the two models’ attack scores to match on every input is too strong for deep networks, so we allow a small fraction of exceptions. If the attack scores of the unlearned and retrained models differ by more than δ on at most a fraction ε of inputs, and the retrained model’s scores place at most mass 2bδ within δ of the threshold, then the “member” rate on each population changes by at most 2bδ+ε. Adding the member and non-member populations gives
|Adv(W^un )-Adv(W_r )|≤4b δ+2ε The unlearned model therefore leaks no more than 4bδ+2ε beyond a model that never saw the forgotten data.

The bound on replay weights holds exactly for our networks, since it follows from the definition w=1-a p_k and does not depend on the architecture. Theorem 4 needs no assumption on the network itself: every quantity in it can be measured directly from the attack scores of the unlearned and retrained models. Because the theorem concerns the advantage at a fixed threshold, while AUC summarizes how the attack ranks examples across all thresholds, we evaluate it with the advantage itself. On CIFAR-10 class unlearning, the attack’s advantage at its threshold drops from 0.177 before unlearning to 0 after replay with calibration, matching retraining, and the bound evaluates to 0.10 (δ = 0.024, b = 0, ε = 0.05). The bound is informative whenever the unlearned and retrained scores agree near the threshold, and becomes loose for models whose scores diverge from retraining, so it complements rather than replaces the empirical comparison. Theorem 2 holds only locally: it needs the loss to be smooth near W_r, which ReLU networks satisfy only piecewise, and its constant L cannot be computed for our models, so we treat it as guidance rather than a quantitative guarantee. Theorem 3 holds for the final linear layer under λ_l≥(1+κ)/2, but it describes the negation step itself; it does not establish forgetting in the model returned after calibration. Theorem 1 holds for smooth activations, and none of our other claims depend on it. Forgetting in the returned model is therefore established empirically, by comparison with retraining from scratch. On converged CIFAR-10, the attack clearly detects membership in the model before unlearning (advantage 0.134 in client unlearning and 0.177 in class unlearning), after replay, its AUC matches that of the retrained model (0.501 vs. 0.494 in client unlearning; 0.498 vs. 0.496 in class unlearning with calibration). Full results are in Q3. We do not claim certified unlearning.

Q3: All methods start from the same converged model and are compared with a model retrained from scratch on the retained data for the same number of rounds. All rows below Retrain are configurations of our framework. Results for CIFAR-10 class unlearning (class 3):

MethodForget Error (%)Retained Error (%)Test Error (%)MIA AUCTime
Before unlearning16.824.8618.470.648—
Retrain from scratch100.004.3823.400.49510,991
Calibration only100.003.8823.040.5381,721
Negation + calibration100.009.1424.290.499890
Replay only100.004.4423.410.4927,889
Replay + calibration100.004.1123.250.4978,769
Replay + negation + calibration100.009.8023.950.4888,767

Ablation. Replay with calibration matches retraining on every metric. Calibration alone removes the class from the predictions, but the attack still ranks members above non-members (AUC 0.538 vs. 0.496 for retraining), even though its advantage at the default threshold is zero; replay removes this remaining signal. Negation with calibration reaches complete forgetting at about one twelfth of retraining time, at a cost of about 5 points in retained error. Our framework therefore offers two operating points: replay with calibration for the closest match to retraining, and negation with calibration for complete forgetting at low cost. Client unlearning shows the same pattern: replay raises the forgotten client’s error from 6.98% to 19.71% (retraining: 20.26%) and lowers MIA AUC from 0.591 to 0.501 (retraining: 0.494).

Client-class unlearning. Two of ten clients hold half of all class-3 training images, relabeled as class 5, while the other clients keep class 3 correctly labeled. Before unlearning, the model predicts class 5 on 76.0% of the flipped images, and the corruption spreads to all clients (class-3 test error 89.6%):

MethodAgreement with Flipped Label (%)Class-3 Test Error (%)Retained Error (%)MIA AUCTime
Before unlearning75.9689.611.510.582—
Retrain from scratch33.9270.913.070.5268,464
Calibration only43.4065.69.820.5611,591
Negation + calibration43.5273.816.790.543863
Replay only33.6074.213.900.5217,827
Replay + calibration38.1674.312.980.5228,531
Replay + negation + calibration43.4081.918.880.5268,715

Replay removes the flipped contribution to the level of retraining: replay alone reaches agreement 33.6% (retraining: 33.9%) and MIA AUC 0.521 (0.526), and replay with calibration reaches 38.2% and 0.522 while keeping retained error at 12.98% (13.07%). Calibration alone removes most of the flip at about one fifth of the cost but leaves more membership signal (AUC 0.561), and adding negation increases retained error, as in class unlearning. Class 3 recovers only partly, even for retraining, because the benign clients hold only 2,500 class-3 images.

MIA evaluation. The attack is a logistic regression on loss-based features, refit on each model’s own outputs using retained training samples (members) versus held-out test samples (non-members), disjoint from the evaluation samples; in class unlearning, both calibration sets exclude the target class. It is evaluated on balanced member and non-member sets of up to 2,000 each (1,000 for CIFAR-10 class and client-class unlearning, limited by the class-3 test set). We report AUC with |AUC-0.5| and the advantage, so inverted scores cannot appear as resistance. A positive control on deliberately overfit CIFAR-10 samples gives AUC 0.754, and before unlearning the attack detects membership (advantage 0.134 for client and 0.177 for class unlearning). To account for an adaptive attacker, we also report the best advantage over all thresholds. Under this attacker, replay with calibration leaks about as much as retraining (0.031 vs. 0.024 in class unlearning; 0.043 vs. 0.050 in client-class unlearning), while calibration alone leaks about two to three times more (0.078 and 0.107). Shadow-model likelihood-ratio attacks will be included in the final version.

Retained-data preservation. Replay with calibration preserves non-target knowledge: retained error is 4.11% and test error 23.25%, versus 4.38% and 23.40% for retraining. The roughly 5-point loss in retained error on CIFAR-10 arises only when negation is included, since it perturbs a layer shared by all classes; this is the cost of the low-cost operating point. We will state the preservation claim for replay with calibration and report this trade-off explicitly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.