FedCoDS: Federated LLM Unlearning by Coupling Knowledge Distillation with Likelihood Suppression

Federated large language model (LLM) unlearning aims to remove targeted training data without retraining the model from scratch or sharing clients' raw data. However, existing likelihood-based unlearning objectives face two key challenges in federated settings. When the forget set is held by a single client, the requesting client has access to only limited local retain data, making it difficult to preserve knowledge learned from other clients. When the forget set is distributed across multiple clients, heterogeneous local forget data can produce inconsistent forgetting directions that are weakened during federated aggregation. To address these challenges, we propose FedCoDS, a federated LLM unlearning method that couples token-wise likelihood suppression with knowledge distillation from a shared frozen pretrained model. The suppression term directly reduces the likelihood of targeted tokens, while knowledge distillation provides a common distributional reference that constrains model changes and promotes cross-client consistency. FedCoDS combines their complementary strengths to achieve a better balance between effective forgetting and knowledge retention. Experiments on TOFU and MUSE under both single- and multi-client unlearning settings show that FedCoDS consistently improves the forgetting–retention performance over SOTA baselines. Further analysis shows that FedCoDS improves cross-client forget-gradient alignment and produces forget-answer likelihoods closer to those of the retrained model.