- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies federated multimodal unlearning, where specific clients, classes, or samples must be removed from a trained model without centralizing the original data. The authors identify three factors that make unlearning difficult: interactions between visual and textual representations, overlap between forget and retain gradient directions, and the reintroduction of forgotten information during subsequent federated training. To address these issues, the authors propose EASE, which combines BKE, GSD, and PFL. Together, these components remove unwanted knowledge from both modality branches, distinguish forget-related directions from those needed to preserve useful knowledge, and constrain subsequent updates to reduce the risk of information being restored. EASE is evaluated on Flickr30K, MS COCO, and TextCaps using three backbone architectures across client-, class-, and sample-level unlearning tasks. Compared with nine baselines and a retain-only retraining reference, the method shows a promising balance between forgetting and retention while reducing communication costs relative to full retraining. The evaluation also includes ablations, membership inference tests, and analyses of model behavior during continued training.
Critical strengths
-
The method is well motivated. The paper links three challenges in federated multimodal unlearning to the design of BKE, GSD, and PFL: cross-modal information leakage, overlap between forget and retain gradient directions, and the recovery of forgotten information during continued training. The ablation results help support this design. In particular, removing GSD leads to a substantial drop in retain utility, while removing BKE or PFL weakens the framework in the ways these components are intended to address.
-
The experiments cover multiple aspects of unlearning. EASE is tested on three datasets, three backbones, and three unlearning scenarios, with nine baselines and a retraining reference for comparison. The evaluation goes beyond retrieval performance to include low-FPR membership inference, subspace alignment, ablations, continued-training tests, and communication costs. This makes it easier to assess not only how well EASE forgets the target data, but also how much useful knowledge it preserves and how it performs in practice.
Critical weaknesses
-
The theoretical guarantees do not directly establish the empirical unlearning objective. Appendix C provides useful results on parameter projection and retain-subspace preservation. However, Theorems C.8 and C.9 establish geometric properties of the projected parameters rather than showing that EASE behaves similarly to retain-only retraining. Proposition C.12(iii) also relies on Assumption A3. Although the paper motivates this assumption through bilateral excision and reports alignment-residual results, it remains unclear how well the assumption holds across different experimental settings. A clearer explanation of which properties are theoretically guaranteed and which are supported mainly by empirical results would strengthen the paper.
-
The scalability of storing client update histories needs further clarification. For client-level unlearning, GSD relies on stored client update histories. While these updates are already sent to the server during FedAvg, keeping and processing them still adds storage and computational costs. Appendix B.8 explores ways to reduce the amount of stored history, but it is still unclear how these costs grow as the number of clients and training rounds increases, especially when multiple unlearning requests need to be handled over time. Further discussion of these costs would help clarify how practical EASE is for long-running federated learning systems.
Questions for authors
-
How well do the history reduction strategies scale in practice? Appendix B.8 shows that truncating or compressing client histories can reduce storage costs. It would be helpful to understand how these trade-offs change with more clients, longer training runs, and repeated unlearning requests.
-
How should the MIA results be interpreted? The MIA scores in Table 1 are well below 50% for both EASE and Retrain, while Appendix B.4 treats scores closer to 50% as better. Could the authors clarify how the attack labels are defined and explain how these results relate to the low-FPR LiRA results?
-
Were the baselines evaluated under comparable settings? Some baselines are adapted from other unlearning settings. Did all methods have comparable access to client updates and training histories, and were their computational costs measured under similar conditions?