DIMU: Importance-Guided Feature Masking with Self-Distillation for Unlearning

Machine unlearning seeks to suppress or remove the influence of designated training data while preserving utility on retained data. We propose Distillation- Regularized Importance-Masked Unlearning (DIMU), an approximate unlearn- ing framework that combines importance-guided feature-channel masking with self-knowledge distillation. DIMU suppresses forget-sensitive channels in the effective penultimate representation exposed to the classifier and adapts only the classifier head while keeping the feature extractor frozen. Self-knowledge distillation regularizes retained predictions using cached pre-unlearning logits. We evaluate class-level unlearning on CIFAR-10/100 across convolutional and transformer-based architectures, random sample-level unlearning on CIFAR-10, and large-scale class-level unlearning on ImageNet-1K using ResNet-50. DIMU achieves zero forgotten-class test accuracy in the primary single-class benchmarks while preserving strong retained utility, and shows favorable forgetting–retention trade-offs under sample-level deletion. It also exhibits competitive empirical membership-distinguishability behavior and a 24.89× speedup over full retraining on ImageNet-1K.