- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates cross-modal knowledge regeneration after unlearning in multimodal large language models (MLLMs): after a category of visual knowledge has been removed through unlearning, further fine-tuning on text-only data can restore the “forgotten” visual knowledge without exposure to any images. The authors demonstrate this phenomenon on MLLMU-Bench, UMU-Bench, and CLEAR using four unlearning baselines (GA, GD, KL, and NPO), and propose KoReP, a method based on modality-importance pruning.
Critical strengths
1.A drop in performance immediately after unlearning does not establish that the relevant knowledge has been reliably removed. Examining whether subsequent training can recover the forgotten information is more informative than static evaluation alone.
2.The method is straightforward to implement, and KoReP can serve as a post-processing step for existing unlearning methods. The reported results show that moderate pruning reduces recovery while largely preserving performance on the retain set.
- Beyond reporting parameter changes, the authors also suppress recovery by freezing vision-related parameters. This intervention provides stronger evidence than parameter correlations alone.
Critical weaknesses
-
The information overlap between the relearning data and the forgotten knowledge requires more rigorous clarification. The paper emphasizes that relearning uses no images, but this does not mean that the forgotten information is not reintroduced. If the text data directly contain the personal attributes or facts needed for evaluation, the recovery may constitute cross-modal relearning rather than solely the reactivation of latent knowledge.
-
A key implementation question concerns training after pruning. The paper sets selected weights to zero and further claims that this blocks text gradients from entering the relevant visual subspace. However, setting weights to zero does not, by itself, prevent subsequent gradient updates.
-
The defense evaluation is insufficient to support strong claims of robustness. The reported relearning experiments primarily use a single epoch. Limited recovery under a short training budget may indicate that KoReP delays recovery rather than eliminates the capacity for recovery.
4.The evaluation omits some recent models, such as Qwen-VL 3.5, as well as reasoning models such as R1-OneVision.
Questions for authors
See weakness.