acceptodds
Under review as a conference paper at ICLR 2027

Knowledge Regrows Across Modalities: Hidden Failure Mode in MLLM Unlearning

Abstract

Multimodal Large Language Models (MLLMs) have achieved remarkable success in integrating information from various modalities, yet they raise growing concerns about privacy leakage. Machine unlearning is considered a remedy for privacy leakage, but existing multimodal unlearning approaches directly adopt techniques developed for unimodal models. This paper uncovers a hidden failure mode in MLLM unlearning, which we term *Cross-modal Knowledge Regrowth*: textual prompts can restore forgotten visual knowledge after unlearning, even in the absence of images. We identify that this phenomenon is attributed to the reactivation of vision-specific parameters by text-only prompts. Based on these findings, we show that selectively removing these vulnerable weights effectively suppresses cross-modal knowledge regrowth. Extensive experiments demonstrate that our approach reliably preserves unlearning robustness while maintaining model utility. Code is provided in the supplementary material for reproducibility.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
MLLM unlearning, Knowledge regrowth

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_95MEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates cross-modal knowledge regeneration after unlearning in multimodal large language models (MLLMs): after a category of visual knowledge has been removed through unlearning, further fine-tuning on text-only data can restore the “forgotten” visual knowledge without exposure to any images. The authors demonstrate this phenomenon on MLLMU-Bench, UMU-Bench, and CLEAR using four unlearning baselines (GA, GD, KL, and NPO), and propose KoReP, a method based on modality-importance pruning.

Critical strengths

1.A drop in performance immediately after unlearning does not establish that the relevant knowledge has been reliably removed. Examining whether subsequent training can recover the forgotten information is more informative than static evaluation alone.

2.The method is straightforward to implement, and KoReP can serve as a post-processing step for existing unlearning methods. The reported results show that moderate pruning reduces recovery while largely preserving performance on the retain set.

  1. Beyond reporting parameter changes, the authors also suppress recovery by freezing vision-related parameters. This intervention provides stronger evidence than parameter correlations alone.
Critical weaknesses
  1. The information overlap between the relearning data and the forgotten knowledge requires more rigorous clarification. The paper emphasizes that relearning uses no images, but this does not mean that the forgotten information is not reintroduced. If the text data directly contain the personal attributes or facts needed for evaluation, the recovery may constitute cross-modal relearning rather than solely the reactivation of latent knowledge.

  2. A key implementation question concerns training after pruning. The paper sets selected weights to zero and further claims that this blocks text gradients from entering the relevant visual subspace. However, setting weights to zero does not, by itself, prevent subsequent gradient updates.

  3. The defense evaluation is insufficient to support strong claims of robustness. The reported relearning experiments primarily use a single epoch. Limited recovery under a short training budget may indicate that KoReP delays recovery rather than eliminates the capacity for recovery.

4.The evaluation omits some recent models, such as Qwen-VL 3.5, as well as reasoning models such as R1-OneVision.

Questions for authors

See weakness.

Reviewer_Yo2KOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates cross-modal knowledge regrowth in MLLM unlearning, where text-only fine-tuning can restore previously suppressed knowledge under multimodal evaluation. It analyzes modality-dependent parameter importance and proposes KoReP, a pruning method that removes weights identified as vision-specific after unlearning. Experiments across multiple benchmarks and unlearning methods show reduced knowledge recovery while preserving competitive retain-set performance.

Critical strengths
  1. The paper examines whether forgetting remains effective after subsequent fine-tuning, extending evaluation beyond immediate post-unlearning performance. Its focus on cross-modal recovery highlights a potential vulnerability that conventional text-only or static evaluations may overlook.

  2. Modality-specific importance analysis, gradient measurements, and parameter-freezing experiments provide complementary evidence about the pathways involved in cross-modal recovery. These analyses connect the observed behavior to identifiable parameter groups and motivate the proposed pruning strategy.

  3. Experiments cover multiple datasets, model backbones, unlearning algorithms, and classification, generation, and cloze tasks. Comparisons with random pruning and alternative importance criteria, together with analyses of pruning strength and forget-set size, help characterize both the effectiveness and trade-offs of targeted pruning.

Critical weaknesses
  1. Text-only relearning uses target-related personal facts, whereas vision-only relearning uses general observable attributes such as hair color and gender. The two settings therefore differ in both modality and supervision content. This weakens the attribution of asymmetric recovery to cross-modal interactions. Matched experiments should control factual content, data size, and training budget while varying the input modality.

  2. The paper does not establish why vision-specific weights must be removed rather than frozen or excluded from subsequent gradient updates. Comparisons with parameter freezing, gradient masking, and updates restricted to text-specific weights would clarify whether pruning itself is necessary. The threat model should also specify whether restrictions on subsequent training can be enforced.

  3. KoReP selects weights using importance scores computed on forget-set samples, without explicitly measuring their importance to retained capabilities. Low textual importance does not imply low importance to normal visual understanding. The paper should clarify what the retain evaluations cover and separately assess non-sensitive visual attributes, such as clothing color and scene content, before and after pruning and relearning.

  4. The experiments omit relevant methods such as MMUnlearner [1] and ASRU [2]. Evaluating whether these methods exhibit cross-modal knowledge regrowth, and whether KoReP provides additional protection, would strengthen the claimed generality of both the identified failure mode and the proposed defense.

[1] MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models.

[2] ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models.

Questions for authors

See weaknesses. I would be happy to raise my score if the authors adequately address these concerns.

Reviewer_mZPoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies cross-modal knowledge regrowth in MLLMs, showing that fine-tuning on related text data can restore visual knowledge after unlearning. It links this asymmetric recovery to vision-specific parameters that remain responsive to textual updates. Based on this finding, it proposes KoReP, a selective pruning method to suppress regrowth. Experiments across two model backbones, three benchmarks, and multiple unlearning methods evaluate its effectiveness and the trade-off between regrowth suppression and model utility.

Critical strengths
  1. This paper analyzes the limitations of existing MLLM unlearning methods from the perspective of relearning, revealing that multimodal performance can recover after text-only fine-tuning.
  2. The proposed KoReP method effectively suppresses cross-modal knowledge regrowth through targeted pruning.
Critical weaknesses
  1. The two relearning settings are not directly comparable. Text-only relearning reintroduces identity-related facts from the forget set, whereas vision-only relearning concerns visible attributes that the base model may already recognize. The observed asymmetry may therefore arise from differences in supervision rather than modality itself.

  2. Vanilla accuracy on the forget set is relatively low, while the regrowth metric only compares performance before and after relearning. It remains unclear whether the improvement reflects the recovery of forgotten knowledge or the acquisition of new knowledge from the forget set that Vanilla had not previously learned.

Questions for authors
  1. The experiments are limited to LLaVA-1.5-7B and Qwen2-VL-7B. They are based on pre-trained LLMs and then undergo visual alignment and multimodal training. Newer models like Qwen3.5 use early-fusion vision-language pretraining. These training differences may affect how visual and textual knowledge is distributed across modality-specific and shared model parameters. The generalizability of the findings and KoReP to these models remains unverified.
  2. The asymmetry and mechanistic analyses focus on methods adapted from unimodal unlearning. The modality-aware method MANU is evaluated only on MLLMU-Bench in Table 3, without corresponding asymmetry or mechanistic analyses, or results on UMU-Bench and CLEAR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.