acceptodds
Under review as a conference paper at ICLR 2027

EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure

Abstract

Federated Multimodal Unlearning (FMU) eliminates the influence of a compromised client, a target class, or specified samples from a jointly trained multimodal model without accessing raw data. FMU faces significant challenges due to the entanglement of modalities and client gradient subspaces. Current methods cannot break the cross-modal reconstruction pipeline caused by bilinear coupling, and they do not separate unlearn-exclusive update directions from those shared with remaining clients. To address these, we first introduce an Anchor Principle to formalize bilinear cross-modal coupling, principal-angle subspace entanglement, and ongoing federated updates. We then design EASE, an Entanglement-Aware Subspace Excision framework. Within this framework, both the visual and language branches are updated simultaneously to prevent either branch from independently reconstructing the cross-modal alignment. Cosine–Sine decomposition is then employed to filter update directions unique to the unlearning process while preserving those shared among the remaining clients. In addition, a direction-selective unlearn lock is applied to prevent subsequent training from reverting toward the removed directions, thereby reinforcing the unlearning effect. Across three datasets, three backbone architectures, and three unlearning scenarios, EASE most consistently narrows the deviation from the retrain reference on both forget and retain metrics among nine baselines. For example, on Flickr30K with CLIP-B/32 under client unlearning, EASE matches the retrain reference within R@1 points on the forget set while staying within R@1 points on the retain set.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
federated unlearning, multimodal learning, machine unlearning, vision-language models, subspace excision, cross-modal alignment

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_AhVPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies federated multimodal unlearning, where specific clients, classes, or samples must be removed from a trained model without centralizing the original data. The authors identify three factors that make unlearning difficult: interactions between visual and textual representations, overlap between forget and retain gradient directions, and the reintroduction of forgotten information during subsequent federated training. To address these issues, the authors propose EASE, which combines BKE, GSD, and PFL. Together, these components remove unwanted knowledge from both modality branches, distinguish forget-related directions from those needed to preserve useful knowledge, and constrain subsequent updates to reduce the risk of information being restored. EASE is evaluated on Flickr30K, MS COCO, and TextCaps using three backbone architectures across client-, class-, and sample-level unlearning tasks. Compared with nine baselines and a retain-only retraining reference, the method shows a promising balance between forgetting and retention while reducing communication costs relative to full retraining. The evaluation also includes ablations, membership inference tests, and analyses of model behavior during continued training.

Critical strengths
  1. The method is well motivated. The paper links three challenges in federated multimodal unlearning to the design of BKE, GSD, and PFL: cross-modal information leakage, overlap between forget and retain gradient directions, and the recovery of forgotten information during continued training. The ablation results help support this design. In particular, removing GSD leads to a substantial drop in retain utility, while removing BKE or PFL weakens the framework in the ways these components are intended to address.

  2. The experiments cover multiple aspects of unlearning. EASE is tested on three datasets, three backbones, and three unlearning scenarios, with nine baselines and a retraining reference for comparison. The evaluation goes beyond retrieval performance to include low-FPR membership inference, subspace alignment, ablations, continued-training tests, and communication costs. This makes it easier to assess not only how well EASE forgets the target data, but also how much useful knowledge it preserves and how it performs in practice.

Critical weaknesses
  1. The theoretical guarantees do not directly establish the empirical unlearning objective. Appendix C provides useful results on parameter projection and retain-subspace preservation. However, Theorems C.8 and C.9 establish geometric properties of the projected parameters rather than showing that EASE behaves similarly to retain-only retraining. Proposition C.12(iii) also relies on Assumption A3. Although the paper motivates this assumption through bilateral excision and reports alignment-residual results, it remains unclear how well the assumption holds across different experimental settings. A clearer explanation of which properties are theoretically guaranteed and which are supported mainly by empirical results would strengthen the paper.

  2. The scalability of storing client update histories needs further clarification. For client-level unlearning, GSD relies on stored client update histories. While these updates are already sent to the server during FedAvg, keeping and processing them still adds storage and computational costs. Appendix B.8 explores ways to reduce the amount of stored history, but it is still unclear how these costs grow as the number of clients and training rounds increases, especially when multiple unlearning requests need to be handled over time. Further discussion of these costs would help clarify how practical EASE is for long-running federated learning systems.

Questions for authors
  1. How well do the history reduction strategies scale in practice? Appendix B.8 shows that truncating or compressing client histories can reduce storage costs. It would be helpful to understand how these trade-offs change with more clients, longer training runs, and repeated unlearning requests.

  2. How should the MIA results be interpreted? The MIA scores in Table 1 are well below 50% for both EASE and Retrain, while Appendix B.4 treats scores closer to 50% as better. Could the authors clarify how the attack labels are defined and explain how these results relate to the low-FPR LiRA results?

  3. Were the baselines evaluated under comparable settings? Some baselines are adapted from other unlearning settings. Did all methods have comparable access to client updates and training histories, and were their computational costs measured under similar conditions?

Reviewer_wmQrOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

this submission proposes a federated multimodal unlearning framework. It combines bilateral (image & text) removal, a decomposition that separates forget-only directions from those shared with retain, and a client side projection approach ("Forget Lock") to prevent re-learning.

Critical strengths

Federated unlearning received less attention in unlearning research.

The three-anchor framing sounds exciting at the beginning but then it becomes largely a relabeling of existing techniques.

Critical weaknesses

Novelty is incremental: the components (SVD, orthogonal projection, quadratic regularizer) are standard continual learning tools, and the "three-anchor" framing is largely a relabeling of them.

Assumption (A3) that bilateral "excision" produces near codirectional embedding shifts needs proving.

retention integrity

it's unclear how the proposed solution (the generic principal angle) affect retrieval utility.

The baselines are mostly single-modality methods, and miss the most relevant prior work. The paper claims the bilateral coupling (modality anchor) is new and single modality unlearning cannot address it, but MultiDelete (ECCV 2024, https://arxiv.org/abs/2311.12047) already targets cross-modal association breaking in multimodal unlearning. It also doesn’t compare against any multimodal or CLIP-specific unlearner (MultiDelete, Safe-CLIP), or other approaches cited in the paepr like Cross-Modal Unlearning (Li et al. 2026) and Unlearning in Cross-Modal Retrieval (Lu et al. 2026).

Most baselines under-erase by 60--95 points, which could suggest poorly tuned or mis-adapted baselines needed for fair comparison.

If I understood correctly, the model needs cached per-client update histories (or request-based gradients uploaded by clients), which is a strong access assumption compared to methods that use only the final model.

The efficiency comparison treats communication cost without accounting for the above point.


post rebuttal: scores updated given convincing author responses to most of my comments. I still think the work is an incremental effort. I also think federated unlearning has received less attention compared to broader unlearning research.

Questions for authors

see above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.