Generalization Guided Machine Unlearning

Deep machine unlearning has become increasingly critical and urgent across various applications. Existing unlearning methods typically modify the model’s weights to scrub the information associated with the forget set. However, these methods often fail to fulfill the effective and efficient unlearning goal and also expose serious privacy leakage risks for data to be forgotten. Rethinking the machine unlearning problem through the lens of model generalization, we provide a fundamental understanding of the essentials of machine unlearning and validate the privacy risks of deep models trained under various generalization-improving policies. Building upon the intrinsic insights and gained key observations, we propose generalization-guided unlearning methods, GENUN, consisting of GENUN-M by leveraging a high-bias checkpoint model and GENUN-F by employing targeted rewinding and unlearning via soft targets derived from contrastive learning. Extensive empirical evaluations demonstrate that GENUN algorithms consistently achieve near-ideal forget quality and utility while maintaining significant speed-ups. They outperform various state-of-the-art baselines and can protect the privacy of the forget data effectively.