- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes MT-NPO for unlearning in multimodal large language models. It addresses two limitations of existing sequence-level methods: first, different tokens have varying impacts on subsequent generation but are optimized uniformly; second, the same knowledge may be forgotten to different degrees under textual and visual conditions. MT-NPO extends NPO to the token level by introducing FutureKL to estimate each token’s influence on future generation and assign corresponding unlearning weights. It further adaptively balances the unlearning and retention objectives based on the remaining knowledge accessibility and utility degradation of each modality. Experiments across multiple datasets and models demonstrate that MT-NPO achieves strong overall performance.
Critical strengths
-
The paper identifies two important problems in existing multimodal unlearning methods: Sequence Coupling and Modality Imbalance. The motivation is clear and directly addresses practical limitations of current approaches.
-
MT-NPO extends NPO from the sequence level to the token level. It uses FutureKL to estimate how much each token affects future generation, and adaptively balances the unlearning and retention objectives across different modalities, leading to better unlearning performance.
Critical weaknesses
-
The gradient-flow design of the method is not explained clearly enough. Both the FutureKL-derived token weights and the modality-adaptive weights depend on the current model outputs, but it is unclear whether gradients are propagated through these weights during backpropagation.
-
The description of MT-NPO is mainly built around the setting where the same knowledge is associated with multiple modality-conditioned QA pairs or views, and the unlearning strength is adjusted according to the remaining accessibility across these views. However, for datasets such as CLEAR, the paper does not clearly explain how a knowledge target, multiple QA pairs, and different modality views are organized and aligned in practice.
Questions for authors
-
Could the authors clarify the gradient flow in MT-NPO? In particular, are the FutureKL-derived token weights and the modality-adaptive coefficients treated with stop-gradient during optimization, and which quantities are actually differentiated when updating the model?
-
How is MT-NPO instantiated on CLEAR? The original CLEAR dataset does not seem to natively support this task setting.
-
The naming and intuition behind FutureKL are somewhat unclear. In practice, the paper computes a signed suffix score along a fixed target sequence, rather than a KL divergence estimator in the strict mathematical sense. As a result, the term “FutureKL” may be conceptually misleading and could cause confusion regarding its relationship to standard KL divergence.