acceptodds
Under review as a conference paper at ICLR 2027

Linear Token Unlearning: A Zero-Dependency Framework for Multimodal LLM Unlearning

Abstract

Machine unlearning aims to eliminate the influence of designated training data from a model to preserve user privacy. For multi-modal large language models (MLLMs), this requires removing cross-modal and uni-modal knowledge; however, most existing MLLM unlearning frameworks only address the cross-modal case. Furthermore, these methods rely on external resources such as retain sets, auxiliary models, or human annotations to guide the unlearning process, significantly limiting their practical applicability. To address these shortcomings, we propose Linear Token Unlearning (LTU), a fast, resource-free token-level optimization framework capable of removing both cross-modal and uni-modal knowledge. LTU employs a first-order linear approximation to identify the tokens causally linked to the memorization of a given sequence in an efficient single-backward pass while avoiding the perturbation bias of prior brute-force scoring; it performs targeted token-level optimization on these tokens to remove the cross-modal and uni-modal knowledge. Experimental results demonstrate that LTU achieves up to a 7 improvement in forget quality over existing MLLM unlearning baselines while maintaining comparable model utility. To the best of our knowledge, LTU is the first MLLM unlearning framework that removes both cross-modal and uni-modal knowledge without reliance on any auxiliary resources.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Vision-Language Model, Unlearning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1KBDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes LTU, a zero-dependency framework for removing both cross-modal and uni-modal knowledge from MLLMs. LTU uses linear-delta to efficiently identify critical tokens and optimize model unlearning. Experimental results show superior performance over existing methods.

Critical strengths
  1. The paper addresses a critical gap in MLLM unlearning by removing both cross-modal and uni-modal knowledge without relying on external resources like retain sets or auxiliary models, which is a significant innovation.
  2. The linear-delta method introduces a first-order approximation to reduce computational overhead and perturbation bias compared to brute-force scoring, enhancing efficiency and accuracy.
  3. Extensive experiments on CLEAR and MLLMU benchmarks demonstrate LTU’s superior performance in forget quality and model utility, with robustness against relearning and quantization attacks.
  4. The framework is scalable to larger models, as shown in the experiments, and the theoretical analysis supports the validity of the linear-delta approach.
Critical weaknesses
  1. The paper lacks detailed discussion on the limitations of linear-delta in scenarios with highly non-convex loss landscapes, as highlighted in Appendix G. The empirical Fisher approximation used in second-order methods might introduce noise, but this is not fully addressed.
  2. The experimental results for uni-modal unlearning (e.g., Table 13 and Table 15) show lower forget quality compared to cross-modal results, suggesting potential weaknesses in handling uni-modal data. The reasons for this discrepancy are not thoroughly analyzed.
  3. The theoretical proofs in Appendix F and G assume local smoothness and convexity, which may not hold in practice. While the paper cites these assumptions, it does not validate them empirically or discuss their implications for real-world applications.
Questions for authors
  1. The paper does not adequately discuss linear-delta’s limitations under highly non-convex loss landscapes and fails to fully examine noise from the empirical Fisher approximation adopted by second-order methods.
  2. The uni-modal unlearning results are worse than cross-modal ones, yet the paper has no in-depth analysis explaining this performance gap.
  3. The theoretical proofs rely on local smoothness and convexity assumptions that may not hold in practice, with no empirical validation or discussion of real-world impacts.
2 comments

We appreciate the reviewer for thoughtful comments. We would like to offer clarifications for each point.

Weakness 1 and Question 1

Linear-delta predicts the effect of replacing a token from the gradient at the original token, so its error grows with the curvature of the loss and with the distance to the replacement. We conduct an experiment to show that curvature is less affected by the choice of the replacement. LTU uses a set of different tokens for replacements (‘#’ or ‘[UNK]’) and chose them randomly for swapping the tokens. If the loss curvature is highly non-linear then each replacement would result in a different output distribution and selected tokens will vary. We ran an experiment to verify if the same set of tokens are selected when different replacements are used. We score the vision and text tokens with Linear-delta and brute-force when the prefix was swapped with ‘[UNK]’ exclusively. Then we report the overlap of top-20 tokens.

Vision token linear-delta overlapVision token delta-score overlapText token linear-delta overlapText token delta-score overlap
0.916 ±\pm 0.0220.922 ±\pm 0.0250.957 ±\pm 0.02740.954 ±\pm 0.019

The result shows that the loss curvature, although non-convex, has minor contribution to error. While this cannot show the entire loss landscape, the result demonstrates that the loss is more linear to the perturbation and hence offers a solid ground for approximation.


Weakness 2 and Question 2

LTU’s uni-modal forget quality is not systematically lower than its cross-modal forget quality. It is lower in 5 of 12 settings and higher in 7 settings. Refer to the table below. In Table 15 (MLLMU 10%), uni-modal forget quality is more than twice the cross-modal value for both backbones (LLaVA 0.357 vs. 0.165; Qwen 0.428 vs. 0.134). Where it is lower, the gap is mostly small (e.g., MLLMU 1% with Qwen: 0.987 vs. 0.998). Moreover, the gaps are also small in terms of the underlying test statistic. Forget quality is the p-value of a KS test, which takes discrete and steeply changing values: on CLEAR 1%, LLaVA’s cross-modal 0.874 and uni-modal 0.766 correspond to KS distances of 0.114 and 0.125 from the retrained model, which is a difference of about one sample. In all settings, LTU’s uni-modal forget quality remains the highest among the compared methods.

BenchmarkLLaVA cross / uniQwen cross / uni
CLEAR 1%0.874 / 0.7660.699 / 0.766
CLEAR 5%0.238 / 0.1610.120 / 0.142
CLEAR 10%0.121 / 0.1100.108 / 0.125
MLLMU 1%0.422 / 0.3780.998 / 0.987
MLLMU 5%0.273 / 0.2840.346 / 0.711
MLLMU 10%0.165 / 0.3570.134 / 0.428

Weakness 3 and Question 3

We first clarify the assumptions.

AssumptionWhere
Local smoothnessAppendix F
ConvexityAppendix G

Appendix F relies only on local smoothness along the perturbation path of each token (Assumption 1); it does not assume convexity. Convexity enters only through the empirical-Fisher approximation in Appendix G, which we discuss under weakness 1.

Assumption 1 requires that, as a token moves along the path to its replacement, the gradient changes at a bounded rate β\beta. Remark 1 left β\beta unestimated. We now estimate it directly. For each tested token we move it 25%, 50%, 75% and 100% of the way to the # replacement, we recompute the gradient, and take the largest change in the gradient per unit distance.

Median β\beta for vision tokensMedian β\beta for text tokens
0.1300.070

The median over tokens is β\beta = 0.130 for visual tokens and β\beta = 0.070 for text tokens. Assumption 1 therefore holds with finite, measured constants along the actual perturbation paths. The values also show that per unit distance, the loss curves less for text tokens than for visual tokens, which is consistent with linear-delta being less reliable for text tokens (Weakness 1). The result shows that LTU produces small β\betas, which means smaller error bounds.

Dear Reviewer 1KBD,

Thank you very much for your time and helpful comments. We hope our response addresses your concerns. As we are approaching the rebuttal deadline, please let us know if you have any further questions or concerns.

Sincerely, Authors of Linear Token Unlearning.

Reviewer_Eg19OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Linear Token Unlearning (LTU), a resource-free method for unlearning both cross-modal and uni-modal knowledge from multimodal LLMs. LTU computes a first-order gradient-based “linear-delta” score for each prefix token, selects the top-kk tokens associated with memorization, applies gradient ascent to those targets, and uses KL matching on non-target tokens to preserve utility. Unlike several prior MLLM methods, it does not require a retain set, auxiliary model, or manual annotations.

Critical strengths
  1. The problem setting and resource profile are practical. LTU addresses both visual-textual and text-only knowledge using only the forget set. Avoiding retain data and auxiliary models is consequential for privacy-sensitive deployments.
  2. The empirical tradeoff in the reported setting is strong. On the 1% tasks, LTU often approaches the retrained reference in forget quality while preserving utility, and the experiments include runtime, 13B scalability, relearning, and quantization analyses.
Critical weaknesses
  1. The core method is under-specified and insufficiently validated. The sign convention for the delta score is inconsistent between the main text and appendix. The optimization objective is unclear for image embeddings, and gradient orthogonalization is not defined algorithmically. More importantly, the paper does not report direct approximation error, rank correlation, or top-kk overlap between linear-delta and exact brute-force scores. The theoretical bound depends on unestimated smoothness quantities and does not establish that the approximation is accurate in the actual large perturbation regime.
  2. Utility is not evaluated on a truly held-out test set. The paper evaluates Model Utility on the retain set, which is excluded from the unlearning update but was still used to train the original model. Thus, it tests preservation of behavior on previously seen data rather than generalization to an independent test distribution. The paper does not provide broad, untouched downstream evaluations of general multimodal ability.
  3. The evaluation focuses on LLaVA-1.5 and Qwen2-VL-2B, so it remains unclear how well LTU transfers to newer architectures.
Questions for authors
  1. How were kk, the learning rate and the number of epochs chosen for each setting? Did selection use forget quality against the retrained model, or anything derived from the retain set? How were the baselines tuned, and with what budget?
  2. Why is LLaVA’s original-model uni-modal utility identical across the 1%, 5% and 10% splits when its cross-modal utility is not? Is the same original model used for all splits?
  3. What exactly is the gradient orthogonalization? Please ablate it, the KL term, and stepwise vs. joint updates.
3 comments

We appreciate the reviewer for thoughtful comments. We would like to offer clarifications for each point.

Weakness 1

The sign convention

We admit that there are typos in Equation 1 and 4. We fixed the typo and updated the manuscript.

Optimization objective for image embeddings

We admit the description is insufficient. Unlike text tokens, vision tokens do not have explicit xtx_t, and thus p(xtxˉ<t)p(x_t \bar x_{<t}) cannot be computed. For vision tokens, we applied cross-entropy to the vision token representation of the original model (before unlearning). This deviates the model from how it understands the image. Thus an accurate equation should be pθ(xtoxˉ<t)p_\theta(x^o_t \bar x_{<t}) where xtox^o_t is the tt-th vision token representation from the original model θo\theta_o.

Rank correlation and Top-k overlap

We conducted an additional experiment to compare the tokens scored from DTO and LTU, and measured the top-20% rank correlation of selected tokens of LTU against those selected by DTO over 20 samples. Spearman shows the rank correlation over all prefix tokens, and Overlap takes the top 20% of tokens by brute force and the top 20% by linear-delta, and computes the share of brute force’s top 20% that linear-delta also picks.

Vision token SpearmanVision token overlapText token SpearmanText token overlap
0.580±0.0130.5890.024±0.1480.357

The result shows that vision tokens are strongly aligned, yet text tokens are less aligned. We hypothesize that approximation errors are different for each modality. Vision token is inherently continuous while text token is discrete. Thus swapping text produces a much larger suffix loss difference, which is less accurately approximated through LTU.


Weakness 2

Model Utility is not computed on the retain set alone: as stated in Sec. 4, it aggregates the retain set with held-out real-face and real-world data that were never used in fine-tuning or unlearning. The benchmark strictly evaluates the unlearned models’ performance on retain set and other held-out datasets to assess the model performance, in-domain held-out performance and general vision language modeling tasks. The model utility from the benchmarks is actually an aggregated score across all tasks.

CLEAR [1] uses real faces to test in-domain held-out model performance, and a real world dataset to assess general vision-language modeling capability. Their real face dataset is MillionCelebs dataset [2], which consists of celebrity face-name pairs. Their real-world dataset is a set of visual question answering dataset from X.ai [3]. Both of them are not part of the retain set and never included in fine tuning. Similarly, MLLMU-bench[4] uses Real-Celebrity dataset to test in-domain held-out model performance, and uses MMMU [5] to assess reasoning ability.

Thus LTU’s strong model utility (Fig. 1, Fig. 2, Fig. 5) shows that LTU preserves held-out performance and general vision language tasks.


Weakness 3

LTU assumes a standard large vision language model that has a vision encoder, a projector and a decoder LLM. Since newer models are also in the same architecture, there is nothing that immediately blocks running LTU on the newer models. Qwen3-VL uses DeepStack that operates similarly to skip connections from vision encoder into the layers of LLM. The pipeline transfers additional visual information to support visual reasoning. We are conducting experiments using Qwen3-VL and we will follow up and report the result once we have them.


[1] Dontsov, Alexey, et al. "Clear: Character unlearning in textual and visual modalities." Findings of the Association for Computational Linguistics: ACL 2025. 2025.

[2] Yaobin Zhang, Weihong Deng, Mei Wang, Jiani Hu, Xian Li, Dongyue Zhao, and Dongchao Wen. Global-local gcn: Large-scale label noise cleansing for face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7731–7740, 2020.

[3] x.ai. 2024. Grok-1.5 vision preview

[4] Liu, Zheyuan, et al. "Protecting privacy in multimodal large language models with mllmu-bench." Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

[5] Yue, Xiang, et al. "Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024.

We appreciate the reviewer for thoughtful questions. We would like to offer clarifications for each point.

Question 1

We used the grid search over the hyperparameters. The grid is the one in our hyperparameter study (Figure 3 and 4). We selected the configuration with the best forget quality and model utility. We used the same grid search budget for the baselines as well. Epoch was fixed to 8. The retain set and the re-trained model are never used throughout the unlearning process of LTU, including hyperparameter search.


Question 2

The same original model θo\theta_o is used throughout all splits. The size of the retain set changes with respect to the forget ratio. Since model utility is a harmonic mean over the scores of retain, held-out and real-world datasets, the size of retain set could affect both uni-modal and cross-modal model utility. We conjecture that Llava has weak performance on one dataset for evaluating model utility, and that weak score lowered the overall model utility.


Question 3

Please refer to rebuttal revision for the gradient orthogonalization. We conducted an experiment ablating KL and joint update. Here are the results.

ModelCross-modal MUCross-modal FQUni-modal MUUni-modal FQ
Full0.40150.69890.39560.7659
No KL0.0000.0070.0000.006
Joint Update0.21220.45720.20850.5142

KL is an essential part of the optimization. Without KL, the model quickly loses coherency (general vision-language modeling capability) and it results in low forget quality and low model utility.

Joint update reduces both model utility and forget quality. We conjecture that combining all gradient signals blurs the optimization direction that results in sub-optimal performance.

Dear Reviewer Eg19,

Thank you very much for your time and helpful comments. We hope our response addresses your concerns. As we are approaching the rebuttal deadline, please let us know if you have any further questions or concerns.

Sincerely, Authors of Linear Token Unlearning.

Reviewer_8cvzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Linear Token Unlearning (LTU), a framework for removing both cross-modal and uni-modal knowledge from multimodal large language models (MLLMs) using only the forget set and the original model. Building on DTO's token-selection paradigm, LTU replaces the brute-force per-token perturbation scoring with a first-order Taylor approximation of the suffix loss change (linear-delta), reducing token identification from hundreds of forward passes to a single backward pass. The selected target tokens undergo gradient ascent for forgetting, while KL minimization on non-target tokens within the same forget sequence preserves model utility.

Critical strengths

Meaningful Problem Setting. The paper addresses a practically important scenario where retain data are unavailable and both cross-modal associations and text-only knowledge must be removed. The comparison table (Table 1) clearly positions LTU against prior work, and the zero-dependency claim is well-motivated by privacy regulations restricting retain-set reuse.

Clear Computational Benefit. The single-backward-pass scoring is a genuine efficiency improvement over DTO's O(q) forward passes, which becomes critical when the prefix contains hundreds of visual tokens (e.g., 576 for LLaVA-1.5). The runtime measurements (Table 8) and the 13B scalability experiment (Table 9) provide useful evidence that the cost does not grow with the number of scored tokens.

Reasonable Experimental Coverage. The experiments span two benchmarks, three model scales, multiple forgetting proportions, and include robustness evaluations (relearning and quantization attacks) plus a token-scoring ablation (Figure 5).

Critical weaknesses

Limited Technical Contribution Beyond DTO. This is the main weakness. DTO already provides the full pipeline of identifying influential prefix tokens, applying targeted unlearning, and constraining non-target behavior via KL regularization. LTU's principal change is replacing the finite loss difference with a standard gradient-based approximation, L(e+δ)−L(e)≈∇L⋅δL(e+\delta)-L(e) \approx \nabla L \cdot \delta. While extending this to multimodal inputs is useful given the number of visual tokens, the presentation does not sufficiently separate computational savings from changes in token-selection behavior and downstream improvements. Claims of more accurate causal identification require stronger evidence than improved aggregate metrics alone.

Theoretical Analysis Adds Limited Insight. Appendix F bounds the Taylor remainder and derives ranking-stability conditions. These are standard local-smoothness and margin arguments. The analysis does not establish that the scores identify causal memorization mechanisms, nor that first-order scoring eliminates the bias associated with the replacement embedding choice—the score still depends on the perturbation direction δ. Preserving a ranking under small approximation error does not establish that the ranking itself is appropriate for selective unlearning.

Core Objective Not Well Defined. There are several inconsistencies that prevent confident reconstruction. (1) Equation (1) defines the score as perturbed minus original log-probability, yet the text describes an increase in NLL—these have opposite signs. (2) Equation (4) applies a positive gradient update to target-token log-probability, which as written increases likelihood rather than decreasing it. (3) Most critically, cross-modal target tokens are continuous visual embeddings, yet Eq. (4) writes pθ(xt∣x<t)p_\theta(x_t \mid x_{<t}) as if they belong to the text vocabulary. It is unclear how this probability is defined for visual tokens in a model with a text-only output head. Does the implementation optimize a masked textual loss, a representation-level objective, or some other surrogate? This requires clarification beyond a sign fix.

Questions for authors

Q1. Please provide the exact implemented losses and update directions using a consistent NLL or log-probability convention throughout, including pseudocode.

Q2. How is the forgetting loss defined for selected continuous visual tokens? Please specify the supervision signal, token indexing, masking rules, and how gradients are routed from the text loss back to specific vision-token positions.

Q3. What evidence distinguishes improved causal identification from a computationally cheaper heuristic that happens to yield a better downstream trade-off? Figure 5 compares scoring methods on aggregate metrics but does not directly verify whether selected tokens correspond to memorization circuits (e.g., via probing or causal intervention).

Q4. How sensitive are the selected tokens and final results to the choice of replacement embedding? Are the smoothness assumptions underlying the ranking-stability analysis empirically satisfied?

3 comments

We appreciate the reviewer for thoughtful comments. We would like to offer clarifications for each point.

Weakness 1 & Question 3

LTU adds what token-level unlearning needs to work in large vision-language models; (1) a cross-modal formulation that scores vision tokens against textual answer, with a forgetting loss defined for continuous vision tokens. LTU defines prefix and suffix exclusively, while the size of the prefix and suffix is a tunable hyperparameters in DTO; (2) Linear-delta, which scores all 576 vision tokens in one backward pass, instead of 576 forward passes; and (3) stepwise cross- and uni-modal unlearning, making LTU, to our knowledge, the first LVLM unlearning method to remove both kinds of knowledge without additional resources.

Both LTU and DTO share the same target quantity: the change of suffix NLL depending on the ablation of the prefix token. Delta-score from DTO estimates it by finite difference. Linear-delta from LTU estimates it by the first-order term and estimates all prefix tokens in one backward pass. If linear-delta faithfully estimates the target quantity, it should also choose tokens that are chosen by delta-score (DTO). We conduct an additional experiment to compare tokens selected by linear-delta and delta-score. We report the token overlap of the top 20% of the tokens.

Vision token overlapText token overlap
0.5890.357

The result shows that two scores moderately agree, meaning that both LTU and DTO estimates for the same quantity. Simultaneously, LTU does not reproduce DTO's selection (about 41% of the vision targets differ). The disagreement between them comes from bias and errors of each of them. DTO suffers the model bias: the score is affected by the model’s reaction to an irregular input, while LTU suffers approximation error from discarding the higher order terms (Appendix F). And the difference helps: with the unlearning objective fixed, LTU's tokens give a better model utility / forget quality trade-off than DTO's (Fig. 5). We conjecture that LTU chooses better tokens because the second order term of linear-delta contains the model bias that DTO suffers, and hence removing it helped (Line 950 - 954).

We agree that Fig. 5 shows LTU selects better unlearning targets, not that it identifies memorization circuits; we will revise our wording accordingly (e.g., "causally linked" → "most influential on the suffix loss")


Weakness 2 and Question 4

We agree: Appendix F only shows that linear-delta and brute force select the same clearly high-scored tokens, reject the same clearly low-scored ones, and can differ near the cut-off; we will be careful to claim it as a guarantee that the score identifies memorization mechanisms.

We used a set of different tokens for replacements (‘#’ or ‘[UNK]’) and chose them randomly for swapping the tokens. To answer the reviewer’s question, we ran an experiment to verify if the same set of tokens are selected when different replacements are used. We score the vision and text tokens with Linear-delta and delta-score when the prefix was swapped with ‘#’ and ‘[UNK]’ exclusively. Then we report the overlap of top-20 tokens.

Vision token linear-delta overlapVision token delta-score overlapText token linear-delta overlapText token delta-score overlap
0.916 ±\pm 0.0220.922 ±\pm 0.0250.957 ±\pm 0.02740.954 ±\pm 0.019

The result suggests that linear-delta is less affected by the choice of replacement. More than 90% of the tokens are equally scored in top-20% regardless of which replacement was used.


Weakness 3, Question 1 and 2.

We admit that signs are incorrect for Eq. 1 and Eq. 4. Both signs before ∑\sum in Eq. 1 have to be flipped. Similarly, the first ∑\sum term at Eq. 4 has to be flipped to negative. This is fixed in the rebuttal revision.

We admit the description is insufficient. Unlike text tokens, vision tokens do not have explicit xtx_t, and thus p(xtxˉ<t)p(x_t \bar x_{<t}) cannot be computed. For vision tokens, we applied cross-entropy to the vision token representation of the original model (before unlearning). This deviates the model from how it understands the image. Thus an accurate equation should be pθ(xtoxˉ<t)p_\theta(x^o_t \bar x_{<t}) where xtox^o_t is the tt-th vision token representation from the original model θo\theta_o.

Dear Reviewer 8cvz,

Thank you very much for your time and helpful comments. We hope our response addresses your concerns. As we are approaching the rebuttal deadline, please let us know if you have any further questions or concerns.

Sincerely, Authors of Linear Token Unlearning.

Reviewer_8cvzOpenReview ↗

Response to Authors

Thank you for the clarifications and additional experiments. I appreciate the sign corrections and the proposed revision of the claims about causal identification.

My main concern is that the forgetting loss for continuous vision tokens remains insufficiently defined. Please clarify how the predicted and target distributions are constructed, ideally with a short pseudocode example. The additional analysis also leaves my concern about the limited methodological contribution beyond DTO largely unchanged.

I therefore maintain my rating of 2.

Comments on the paper

1 comment

We thank the AC and reviewers (8cvz, Eg19, 1KBD). We updated the rebuttal revision in blue. The reviewers agree that LTU addresses an important, practical problem, it has clear efficiency gain (one backward pass vs. O(q) forward passes) and it has strong, broad experiments.

The concerns of the reviewers and our responses are summarized below.

Concern (Reviewer)Response
Sign errors in Eq. 1 and 4 (8cvz, Eg19)Typos, now fixed in the revision. Equation 1 and 4 are updated.
Forgetting loss for continuous vision tokens is unclear (8cvz, Eg19)Clarified: cross-entropy against the original model's vision-token representation
Sensitivity to the replacement embedding (8cvz, 1KBD)New experiment: over 91% of the top-ranked tokens are the same across replacement choices, for both vision and text tokens.
Gradient orthogonalization undefined (Eg19)Clarified in the revision. LTU never uses it.
Utility not evaluated on held-out data (Eg19)Already in the paper (Sec. 4): Model Utility includes held-out real-face data and general VQA/MMMU data, none of which were used in fine-tuning or unlearning.
Limited contribution beyond DTO (8cvz)LTU adds (i) cross-modal scoring and a forgetting loss for vision tokens, (ii) linear-delta: 1 backward pass instead of 576 forward passes, and (iii) stepwise cross- and uni-modal unlearning. New analysis: LTU and DTO estimate the same quantity (58.9% top-20% vision-token overlap) but select different tokens. With the objective held fixed, LTU's tokens give a better trade-off (Fig. 5).
Approximation accuracy vs. brute-force not reported (Eg19)New experiment: Spearman correlation and top-k overlap between linear-delta and brute-force scores.
Ablations of the KL term and stepwise updates (Eg19)New experiment: without KL the model collapses (MU ≈ 0). Joint updates lower cross-modal FQ from 0.699 to 0.457. Both components are essential.

We are happy to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.