acceptodds
Under review as a conference paper at ICLR 2027

MT-NPO: Modality-Aware Token-Level Negative Preference Optimization for MLLM Unlearning

Abstract

Multimodal large language model (MLLM) unlearning must suppress target knowledge while preserving retained utility, and this challenge is further complicated by uneven forgetting across modality-conditioned views. These difficulties expose two limitations that existing objectives do not explicitly address. At the token level, sequence-level optimization overlooks the unequal influence of individual answer tokens on future generation. At the modality level, a target may be suppressed under one view yet remain accessible through another. We identify these issues as sequence coupling and modality imbalance, and introduce MT-NPO, a modality-aware token-level unlearning framework that coordinates optimization at both levels. MT-NPO uses bounded FutureKL-guided suffix scores to focus unlearning pressure on influential tokens, while modality-specific unlearning and retention objectives adaptively allocate optimization according to residual target accessibility and utility degradation. Across multiple benchmarks and MLLM backbones, MT-NPO consistently improves the unlearning–retention trade-off and cross-modal balance over representative baselines, highlighting the importance of jointly coordinating unlearning across tokens and modality-conditioned views. Our code is available at https://anonymous.4open.science/r/MTNPO-DDD1/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multimodal Unlearning, Multimodal Large Language Model, Modality Alignment

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_PUtbOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes MT-NPO for unlearning in multimodal large language models. It addresses two limitations of existing sequence-level methods: first, different tokens have varying impacts on subsequent generation but are optimized uniformly; second, the same knowledge may be forgotten to different degrees under textual and visual conditions. MT-NPO extends NPO to the token level by introducing FutureKL to estimate each token’s influence on future generation and assign corresponding unlearning weights. It further adaptively balances the unlearning and retention objectives based on the remaining knowledge accessibility and utility degradation of each modality. Experiments across multiple datasets and models demonstrate that MT-NPO achieves strong overall performance.

Critical strengths
  1. The paper identifies two important problems in existing multimodal unlearning methods: Sequence Coupling and Modality Imbalance. The motivation is clear and directly addresses practical limitations of current approaches.

  2. MT-NPO extends NPO from the sequence level to the token level. It uses FutureKL to estimate how much each token affects future generation, and adaptively balances the unlearning and retention objectives across different modalities, leading to better unlearning performance.

Critical weaknesses
  1. The gradient-flow design of the method is not explained clearly enough. Both the FutureKL-derived token weights and the modality-adaptive weights depend on the current model outputs, but it is unclear whether gradients are propagated through these weights during backpropagation.

  2. The description of MT-NPO is mainly built around the setting where the same knowledge is associated with multiple modality-conditioned QA pairs or views, and the unlearning strength is adjusted according to the remaining accessibility across these views. However, for datasets such as CLEAR, the paper does not clearly explain how a knowledge target, multiple QA pairs, and different modality views are organized and aligned in practice.

Questions for authors
  1. Could the authors clarify the gradient flow in MT-NPO? In particular, are the FutureKL-derived token weights and the modality-adaptive coefficients treated with stop-gradient during optimization, and which quantities are actually differentiated when updating the model?

  2. How is MT-NPO instantiated on CLEAR? The original CLEAR dataset does not seem to natively support this task setting.

  3. The naming and intuition behind FutureKL are somewhat unclear. In practice, the paper computes a signed suffix score along a fixed target sequence, rather than a KL divergence estimator in the strict mathematical sense. As a result, the term “FutureKL” may be conceptually misleading and could cause confusion regarding its relationship to standard KL divergence.

Reviewer_LM7NOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies unlearning in multimodal large language models. It identifies two problems with sequence-level NPO: “sequence coupling” and “modality imbalance.” To address the first, MT-NPO moves the NPO kernel function inside the token summation and, drawing on FIPO’s FutureKL, reweights tokens using a sum of clipped and discounted suffix log-ratios. To address the second, it computes token-level losses separately for each view, then combines the views using softmax weights based on the remaining forgetting loss and the retention loss. Experiments show that, compared with GA, GD, NPO, PO, and MANU, MT-NPO achieves the best UC/UT trade-off and the lowest score on the newly proposed MIC.

Critical strengths
  1. The problem setting has practical significance. The paper clearly distinguishes knowledge access under text and image conditions and evaluates it jointly across classification, cloze, and generation tasks, avoiding judgments about unlearning effectiveness based on only one input condition.
  2. The method is defined relatively clearly, with explicit masking, clipping, normalization, and stop-gradient handling.
  3. Under the reported comparison settings, the overall improvement is substantial.
Critical weaknesses
  1. Under the settings of this paper, the FutureKL weights have very limited expressive power, so it remains unclear what drives the performance improvement of Future-only. When epsilon_minus = epsilon_plus = 0.2, every weight falls within [0.8, 1.2]. However, Table 3 shows a substantial improvement in retention performance on LLaVA from Token-NPO to Future-only (from 0.3545 to 0.4672).

  2. The baseline set and evaluation scope are limited relative to the claims. The comparison covers GA, GD, NPO, PO, and MANU, but omits token-wise https://arxiv.org/abs/2502.19301, https://arxiv.org/abs/2410.07163, and the cited MLLM-specific methods such as https://arxiv.org/abs/2502.11051

Questions for authors

See Weakness above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.