- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies decentralized client unlearning when a different, retained client disconnects during unlearning. It combines trajectory-based Taylor–L-BFGS reconstruction with distributed ghost replicas to approximate the unavailable client’s updates and mixing operations. The analysis bounds deviation from counterfactual retraining, while experiments evaluate utility, backdoor removal, and consensus.
Critical strengths
- The paper identifies a meaningful distinction between forgetting a client and losing access to a retained client. Retraining only over active clients would remove information that should be preserved.
- Ghost replicas reconstruct both local updates and mixing participation. The no-ghost ablation shows a clear benefit when the disconnected client holds unique-label informatio
Critical weaknesses
- The tracking theorem assumes bounded gradient-reconstruction errors without establishing their size for the proposed frozen L-BFGS mechanism. The gradient-magnitude test does not ensure approximation accuracy, and increasing gossip cannot correct this error. Thus, the theorem does not establish when the method closely approximates retraining.
- The forgetting evaluation relies mainly on backdoor ASR. Low ASR demonstrates removal of the injected behavior, but does not establish removal of the forgotten client’s broader influence. Accuracy and consensus also provide limited evidence of closeness to the oracle model.
Questions for authors
- Can you quantify the ghost-gradient reconstruction error after disconnection, either theoretically or through measurements against an accessible oracle?
- Does the method match oracle retraining in prediction discrepancy and membership-inference results on the forgotten client’s clean data, beyond backdoor ASR?
Thank you for the constructive comments. We address both concerns below, supported by additional experiments included in the revised appendix E.2.
W1/Q1 – Ghost-gradient reconstruction accuracy. Our theoretical analysis was designed to characterize how gradient-reconstruction and ghost-consensus errors affect the deviation from counterfactual retraining. Accordingly, Theorem 1 provides a conditional tracking bound, without assuming a specific accuracy for the L-BFGS approximation. We agree that the gradient-magnitude safeguard does not guarantee reconstruction accuracy. To further substantiate the effectiveness of the proposed mechanism, we measured the cosine similarity between reconstructed ghost gradients and exact gradients evaluated at the same iterates, obtaining an average of 0.714 (range: 0.577–0.806). We also measured the final relative model-weight error with respect to oracle retraining:
Thus, ghost reconstruction reduces the deviation from oracle retraining by 43.2% compared with the No-Ghost ablation. These results provide empirical support for the reconstruction mechanism, while the theoretical guarantee remains conditional on the approximation errors.
W2/Q2 – Unlearning effectiveness and closeness to oracle retraining. Our original evaluation used backdoor removal to assess forgetting, since the injected behavior provides a measurable signature of the forgotten client's influence. We agree that this alone does not fully characterize unlearning effectiveness. We therefore extended our evaluation to include membership inference attacks based on LiRA (Carlini et al. 2022) on the forgotten client's clean data (see Appendix E.2.3):
Our method reduces the attack AUC to approximately 0.5, closely matching oracle retraining. Furthermore, to complement the original accuracy and consensus results, we evaluated prediction similarity to the oracle:
These results demonstrate close agreement with oracle predictions, particularly on the disconnected client's exclusive-label data, where preserving its contribution is essential. Together with the original backdoor-removal experiments, they provide stronger empirical evidence of effective forgetting while preserving the contribution of the disconnected retained client.
References:
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. Membership inference attacks from first principles. In IEEE Symposium on Security and Privacy (SP), pp. 1897–1914, 2022.