acceptodds
Under review as a conference paper at ICLR 2027

Reasoning-Aware Cross-Modal Association Unlearning in Multimodal Large Reasoning Models

Abstract

Multimodal large reasoning models (MLRMs) integrate visual inputs with textual and parametric knowledge, allowing sensitive information to be accessed through cross-modal associations during reasoning. This raises privacy concerns when a visual input should no longer provide access to such information. Machine unlearning offers a way to remove this influence without retraining the model from scratch. However, existing multimodal unlearning methods mainly target samples, concepts, or knowledge content, without explicitly separating the knowledge from the cross-modal association through which it is accessed. In reasoning models, this limitation is further exposed: target information may disappear from the final answer while remaining in intermediate reasoning, or simply be replaced by incorrect substitute knowledge. We therefore study cross-modal association unlearning, where the forget target is a designated visual-to-knowledge association: the model should suppress observable target-bearing behavior under the designated visual context while preserving access to the same knowledge in permitted contexts. We propose Visual-Keyed Counterfactual Rebinding (VKCR), a parameter-efficient framework for cross-modal association editing. VKCR first localizes the edit to a compact association-conditioned parameter subspace. It then reduces support for the designated target at task outputs and counterfactually rebinds the affected reasoning behavior by discouraging target-bearing or unsupported continuations while favoring valid non-sensitive alternatives. We further introduce CLEAR-R and MLLMU-R to evaluate target leakage, hallucinated reassociation, reasoning preservation, and modality-specific utility. Experiments show that VKCR substantially reduces image-conditioned target leakage and hallucinated reassociation, including across heterogeneous task formulations, while preserving non-sensitive visual reasoning and permitted text-only knowledge.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine unlearning, Multimodal Large Reasoning Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zt7qOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Visual-Keyed Counterfactual Rebinding (VKCR) for cross-modal association unlearning. It specifically aims to suppress a designated image-conditioned association to target knowledge while preserving permissible text-only access and non-sensitive reasoning. Experiments are performed on two small MLLMs (Qwen2.5-VL-3B-Instruct and LLaVA-1.5-7B-HF), one reasoning MLLM (R1-Onevision-7B-RL) on two benchmarks (CLEAR-R and MLLMU-R).

Critical strengths

The experimental results show substantial improvements under the proposed evaluation protocol compared to previous multimodal unlearning baselines.

Critical weaknesses
  1. In my view, the core setting is simply a combination of modality-selective forgetting from MMUnlearner and reasoning-trace unlearning from R²MU. Therefore, a basic baseline combining MMUnlearner's update constraints with a multimodal adaptation of R²MU's reasoning objective should be included.
    Without such a comparison, it is hard to evaluate the contribution of the proposed method.

  2. Tables 1 and 3 use F-RSP to support reasoning preservation. However, according to Appendix B.5, it allows high-overlap contradictions to pass. I am concerned about whether such a metric can truly reflect differences between the methods.

Questions for authors
  1. How can F-RSH ensure that it reliably distinguishes fabricated details from diverse details (e.g., correct details but absent from the reference annotation)?

  2. In line 93-94, it is said that "Effective unlearning must therefore suppress the influence of the designated association while preserving non-sensitive relations of the same visual subject". What are non-sensitive relations? How to evaluate the preservation of non-sensitive relations.

7 comments
AuthorsOpenReview ↗

Response to Reviewer zt7q (Part 1)

We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. We appreciate the recognition of the effectiveness of VKCR and its substantial improvements over existing multimodal unlearning baselines under our evaluation protocol. We address the reviewer's questions and concerns point by point below.

W1. Comparison with MMUnlearner + R²MU

To directly address this concern, we implement a combined baseline integrating MMUnlearner's selective update constraints with R²MU's reasoning-unlearning objective.

Table R1. Combined-baseline results (F5 generation).

DatasetMethodF-RL ↓F-RSL ↓F-RSH ↓F-RSP ↑R-RL ↑
CLEAR-RMMUnlearner + R²MU26.7422.9545.9287.3125.96
VKCR26.697.3918.3692.4127.27
MLLMU-RMMUnlearner + R²MU35.7662.1437.2881.3244.08
VKCR31.2535.2821.2594.8159.37

VKCR consistently outperforms the combined baseline across both benchmarks. On MLLMU-R, it achieves lower forget ROUGE-L, F-RSL and F-RSH, alongside higher F-RSP and retain ROUGE-L, demonstrating stronger association forgetting, reduced hallucinated reassociation, and better retained utility. Similar improvements are observed on CLEAR-R. Importantly, VKCR goes beyond combining selective parameter updates with reasoning suppression: it identifies realized target-bearing or unsupported failures, suppresses the sensitive values, and counterfactually rebinds the affected reasoning branch toward evidence-grounded alternatives. This failure-conditioned correction distinguishes VKCR from the hybrid baseline, enabling stronger association forgetting while better controlling hallucinated reassociation and preserving useful reasoning.

W2. High-Overlap Contradictions in F-RSP

Metric design and targeted validation. F-RSP is designed to assess whether annotated non-sensitive reasoning steps remain preserved after unlearning. As detailed in Appendix B.5, it combines deterministic lexical matching with LLM-based semantic evaluation for steps not resolved by lexical matching, capturing both lexical and semantic preservation. To further validate the reliability of this evaluation procedure, we conduct a targeted human audit of lexical-pass steps and F-RSH-flagged claims, directly examining whether automatically accepted reasoning steps are semantically preserved and whether detected hallucinations are genuinely unsupported.

Table R2. Targeted CLEAR-R human validation (step/claim-level audit).

MethodLexical-pass steps confirmed semantically preserved ↑F-RSH flagged claims confirmed unsupported ↑
Vanilla24/25 (96.0%)14/15 (93.3%)
MMUnlearner24/25 (96.0%)14/15 (93.3%)
R²MU23/25 (92.0%)13/15 (86.7%)
VKCR24/25 (96.0%)14/15 (93.3%)
Overall95/100 (95.0%)55/60 (91.7%)

Reliability and cross-metric validation. Human annotators confirm 95/100 sampled lexical-pass steps, with comparable confirmation rates across methods (92-96%). Although semantic matching is not error-free, all methods are evaluated using the same protocol, and the similar confirmation rates provide no clear indication of method-specific lexical bias or an artificial advantage for VKCR in this audit. Separately, 55/60 F-RSH-flagged claims are confirmed as unsupported, supporting the complementary detection of hallucinated reassociation. We therefore interpret F-RSP jointly with F-RSL and F-RSH, rather than treating lexical preservation as sufficient evidence of successful unlearning. Retained utility is separately evaluated through R-RL and R-RSP.

Qualitative corroboration and limitations. Appendix C.2 (Table 11) further presents actual reasoning traces and final answers across methods, illustrating failures that aggregate preservation scores alone cannot distinguish: KL-Min retains the forgotten identity, several baselines introduce unsupported identity substitutions, whereas VKCR avoids identity claims while preserving grounded visual descriptions. These observations are consistent with the quantitative results and support the interpretation that VKCR preserves useful reasoning without merely substituting unsupported associations.

AuthorsOpenReview ↗

Response to Reviewer zt7q (Part 2)

Q1. Correct Additional Details versus Hallucinations

Evidence-grounded hallucination assessment. F-RSH does not equate reference mismatch with hallucination. As specified in Appendix B.5, the semantic judge is explicitly instructed not to flag reasoning merely because it differs from the reference answer. Instead, F-RSH targets unsupported identity substitutions, fabricated attributes, and other claims lacking support from the available evidence. Thus, a valid additional observation is not inherently a hallucination simply because it is absent from the reference annotation.

Human validation and limitations. To assess potential false positives, we manually audit F-RSH-flagged claims against the permitted source evidence. As shown in Table R2, human annotators confirm 55/60 (91.7%) flagged claims as unsupported. This supports the reliability of positive hallucination flags. We clarify this distinction and limitation in Appendix B.5.

Q2. Meaning and Preservation of Non-Sensitive Relations

Under the original identity-level task, non-sensitive relations primarily mean grounded, non-identifying visual attributes and scene relationships (e.g., clothing, objects, and spatial context). The stricter case of other permissible factual relations of the same identity is tested separately in the new paired relation experiment (Table R3): forbidden-relation recovery decreases by 35.60 points, versus an 11.83-point decrease for retained relations, thereby supporting preferential relation-level suppression.

Table R3. Paired relation-level evaluation (CLEAR-R F5 %).

Relation subsetMethodRSLRSHRSP
Forget 50%Vanilla51.4336.8291.78
NPO40.6250.4890.37
R²MU26.7448.9184.36
VKCR15.8318.9491.12
Retain 50%Vanilla49.1736.1492.38
NPO43.5843.7290.66
R²MU33.2143.8685.17
VKCR37.3435.8790.36

Dear Reviewer zt7q,

As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal. We hope our additional experiments and clarifications have addressed your concerns. We would greatly appreciate any further feedback and would be happy to clarify any remaining questions.

Thank you again for your valuable feedback, time, and consideration of our work.

Best regards,

The Authors

Reviewer_RXqROpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to suppress selected image-conditioned knowledge access while retaining allowed textual knowledge and non-sensitive visual reasoning. VKCR combines low-rank editing, target and invalid-value suppression, counterfactual continuations, protected updates, and residual correction. CLEAR-R and MLLMU-R add reasoning annotations and distinguish target leakage, hallucinated reassociation, and preservation. Experiments examine forgetting ratios, text access, on-key/off-key images, perturbations, and component ablations. The stated objective is behavioral suppression; the paper does not claim certified removal of a neural pathway.

Critical strengths
  1. Separating original-target leakage from hallucinated reassociation is a useful contribution. In the 5% CLEAR-R generation setting, F-RSL drops from 50.28% to 7.39%, a 42.89-point reduction. F-RSH also falls from 36.47% to 18.36% in this setting (Table 1). This supports the view that improvement is not simply obtained by replacing the original target with another unsupported identity.

  2. The text-access and on-key/off-key controls make the evaluation more informative. For CLEAR-R, the leakage reduction is 42.89 points on the target image, compared with 3.57 on shuffled images and 2.33 on blank images (Table 2). This supports concentration around the targeted visual access condition, although it does not establish selectivity between knowledge relations for the same subject.

Critical weaknesses
  1. The main experiments show that VKCR reduces leakage from image-conditioned queries, but I am less sure that they establish relation-level selectivity. The text-access and on-key/off-key controls are useful, but they do not directly test different knowledge relations for the same visual subject. I would find the central claim more convincing with a paired test that forgets one relation while preserving another allowed relation.

  2. The utility cost on MLLMU-R still gives me some pause. In the 5% forgetting generation setting with LLaVA-1.5-7B, F-RSL falls from 98.94% to 35.28%, while retain ROUGE-L drops from 68.86 to 59.37 (Table 1). That is a substantial leakage reduction, with a noticeable 9.49-point cost to retained performance. I would like a clearer picture of how this balance changes with edit strength, and whether less aggressive settings retain most of the benefit.

Questions for authors
  1. Appendix D.3 uses identity-level forget memberships, whereas the introduction discusses preserving other allowed relations for the same visual subject. Is there an existing test that forgets one relation while retaining another permitted knowledge relation, or should the demonstrated scope remain identity-level?

  2. Beyond the formulas and settings already given, could you supply the actual module/basis choices, numerical tolerances, loss configuration, and correction budget? Please also give the optimization, validation-selection, and final-test memberships used by this implementation.

  3. How well does the complete metric cascade agree with blinded human judgments, including lexical-pass steps? For the 94.81% F-RSP result, please report the numerical eligible-sample count and sensitivity to the retained-step coverage threshold. Per-seed leakage and retain-utility results would help interpret the 35.28% leakage and 9.49-point ROUGE-L loss.

I will further consider my score in light of the other reviewers’ comments and whether the authors’ response adequately addresses my concerns.

9 comments
AuthorsOpenReview ↗

Response to Reviewer RXqR (Part 1)

We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. We appreciate the recognition of our distinction between target leakage and hallucinated reassociation, as well as the effectiveness of our text-access and on-key/off-key controls. We address the reviewer's concerns and questions point by point below.

W1 / Q1. Same-Identity Relation-Level Selectivity

To evaluate within-identity relation-level selectivity, we designate 50% of each selected subject's relations for forgetting and 50% as permitted relations on CLEAR-R F5. We additionally compare VKCR with NPO (preference-based forgetting) and R²MU (reasoning-aware unlearning).

Table R1. Paired relation-level evaluation (CLEAR-R F5 %).

Relation subsetMethodRSLRSHRSP
Forget 50%Vanilla51.4336.8291.78
NPO40.6250.4890.37
R²MU26.7448.9184.36
VKCR15.8318.9491.12
Retain 50%Vanilla49.1736.1492.38
NPO43.5843.7290.66
R²MU33.2143.8685.17
VKCR37.3435.8790.36

VKCR achieves the lowest targeted-relation leakage (15.83%) and reasoning hallucination rate (18.94%), while preserving the most non-sensitive reasoning among the evaluated unlearning methods on the forget subset. Targeted-relation recovery decreases by 35.60 percentage points, approximately three times the 11.83-point reduction on permitted relations of the same identity. Although this indicates some collateral interference, VKCR preserves permitted-relation recovery better than R²MU (37.34% vs. 33.21%) while achieving substantially stronger targeted forgetting (15.83% vs. 26.74%). These results support VKCR's finer-grained association selectivity and a more favorable forgetting–preservation trade-off.

W2. Forgetting-Utility Trade-off and Edit Strength

VKCR achieves stronger reasoning leakage suppression than existing baselines, although this comes with some retained-utility degradation due to shared model representations. To directly address the reviewer's concern about the 9.49-point retain ROUGE-L reduction, we evaluate the forgetting-utility trade-off across different edit strengths on MLLMU-R F5 generation.

Table R2. MLLMU-R F5 generation edit-strength sensitivity.

ConfigurationF-RSL ↓R-RL ↑Leakage reduction (pp)Retain drop (pp)
Vanilla98.9468.86——
Stage-A at scale 0.7063.7667.1535.181.71
Stage-A at scale 0.8051.9365.5247.013.34
Stage-A at scale 0.8546.8764.3152.074.55
Stage-A at scale 0.9044.3263.6354.625.23
Stage-A at scale 1.0039.8757.9259.0710.94

The results demonstrate a controllable forgetting-utility trade-off. At scale 0.85, the Stage-A edit achieves a 52.07-point leakage reduction with a 4.55-point retention drop, illustrating the trade-off available through less aggressive initial edits. We select scale 0.90 based on validation constraints balancing leakage, hallucination, and retention. Although shared representations can introduce some interference with retained knowledge, these results demonstrate that VKCR can adjust forgetting strength to accommodate different preservation requirements.

Q2. Implementation Details and Data-Split Reproducibility

We have expanded Appendix A.3 and B.2 to detail the editable modules, basis construction, loss coefficients, optimization schedules, numerical tolerances, and correction criteria. CLEAR-R employs contrast-derived attention-output editing, whereas MLLMU-R uses causal localization with L12.k, L12.v, L14.k, L20.o, L20.down, and L31.down, totaling 503,808 trainable parameters across classification, generation, and cloze. Both use rank-8 updates without test-time intervention, and residual correction is limited to two rounds under leakage, hallucination, and retention constraints.

Optimization, validation-selection, and final-test memberships. We follow the official identity-level forget/retain partitions across all forgetting ratios. For MLLMU-R F5, 25 of the 500 fictitious identities are designated for forgetting, while the remaining 475 belong to the retain partition. Unlearning optimization uses the designated forget records together with a matched subset of 25 retain identities for preservation. Edit-strength selection and residual-correction acceptance use separate SFT-derived calibration records associated with the corresponding forget and retain identities, without reusing optimization or final-evaluation records. Final benchmark evaluation measures forgetting and retained utility on the designated forget and retain records from the full split. Appendix D.3 provides the corresponding data memberships and evaluation protocol.

AuthorsOpenReview ↗

Response to Reviewer RXqR (Part 2)

Q3. Human Agreement, F-RSP Eligibility, and Seed Stability

Human agreement and evaluation reliability. Our human audit shows strong agreement with the complete evaluation pipeline, with Cohen's κ of 0.939 / 0.896 / 0.895 for F-RSL / F-RSH / F-RSP, respectively. A separate manual audit confirms 95 of 100 sampled lexical-pass retained steps as valid matches, providing complementary evidence for the reliability of the lexical matching stage.

Table R3. CLEAR-R automatic (A) versus human (H) evaluation.

MethodF-RSL A/H ↓F-RSH A/H ↓F-RSP A/H ↑R-RSP A/H ↑
Vanilla0.6 / 0.60.4 / 0.50.9 / 0.90.8 / 0.7
MMUnlearner0.2 / 0.30.5 / 0.60.9 / 0.90.7 / 0.7
R²MU0.2 / 0.20.5 / 0.50.8 / 0.70.6 / 0.5
VKCR0.1 / 0.10.1 / 0.10.9 / 0.90.9 / 0.9
Cohen's κ0.9390.8960.8950.875

Table R4. Targeted CLEAR-R human validation.

MethodLexical-pass steps confirmed semantically preserved ↑F-RSH flagged claims confirmed unsupported ↑
Vanilla24/25 (96.0%)14/15 (93.3%)
MMUnlearner24/25 (96.0%)14/15 (93.3%)
R²MU23/25 (92.0%)13/15 (86.7%)
VKCR24/25 (96.0%)14/15 (93.3%)
Overall95/100 (95.0%)55/60 (91.7%)

F-RSP eligibility and threshold sensitivity. Eligibility is determined by the presence of at least one evaluable non-sensitive reasoning step in the reference, independently of the model-generated response and coverage threshold. Eligible evaluations with no preserved reasoning steps are counted as failures. Token-level F1 ≥ 0.60 identifies lexical matches, while unmatched steps undergo semantic evaluation. We assess threshold sensitivity by varying the required preservation coverage while keeping the eligible evaluation population and step-level judgments fixed.

Table R5. F-RSP coverage-threshold sensitivity on MLLMU-R F5.

Coverage thresholdEligible evaluationsPreserved evaluations
0.25150148
0.50 (default)150142
0.75150135

The fixed eligibility denominator ensures that increasing the coverage threshold does not selectively exclude unsuccessful generations. Notably, 135 of 150 evaluations still satisfy the stricter 0.75 threshold, indicating that a substantial majority preserve at least 75% of their annotated non-sensitive reasoning steps. Together with the human-agreement and lexical-matching audits, this provides complementary evidence for the reliability of F-RSP.

Table R6. MLLMU-R F5 generation results across three seeds.

SeedF-RSL (%) ↓Retain ROUGE-L ↑
034.0860.18
135.2359.43
236.5358.50
Mean35.2859.37

The results exhibit limited seed-to-seed variation in the MLLMU-R F5 generation setting. We have expanded Appendix C to clarify the human-audit protocol, reference-based F-RSP eligibility, lexical and semantic matching, coverage-threshold sensitivity, evaluation units, and seed-level aggregation.

Dear Reviewer RXqR,

As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal. We hope our additional experiments and clarifications have addressed your concerns. We would greatly appreciate any further feedback and would be happy to clarify any remaining questions.

Thank you again for your valuable feedback, time, and consideration of our work.

Best regards,

The Authors

Reviewer_YBzaOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies cross-modal association unlearning in multimodal reasoning models. The setting asks a model to stop recovering a piece of target knowledge from a designated image–query context, both in its final answer and in its generated reasoning. The same knowledge should remain reachable through text-only queries, non-sensitive reasoning about the image should be preserved, and the model should not fill the gap with a fabricated substitute, a failure mode the authors call hallucinated re-association.

The proposed method, VKCR, decides where to edit and how to edit. A small low-rank update is restricted to parameters associated with the designated visual condition, identified either from contrasts between target-bearing and non-sensitive hidden states or from interventions that swap the image for a matched control. The method then rolls out the model, locates the earliest point where target-bearing or unsupported content appears, and trains three terms at that point: it lowers the support for the annotated target, lowers the likelihood of the concrete value the model actually produced, and shifts the continuation toward a valid branch that keeps only grounded content. A preservation constraint, a search over update strength, and a bounded residual-correction stage complete the pipeline.

The authors also introduce two reasoning-annotated benchmarks, CLEAR-R and MLLMU-R, together with four reasoning-level metrics scored partly by an LLM judge. Experiments on two instruction-tuned backbones and one reasoning backbone compare VKCR with seven unlearning baselines. They report lower reasoning leakage and lower hallucinated re-association, supported by selectivity, perturbation, cross-task and ablation analyses.

Critical strengths

S1. Reasonable motivation

The paper separates the knowledge itself from the visual route through which it is accessed. This matches realistic requests in which a photo should no longer reveal a person's profile, while the textual facts remain legitimately available. Measuring re-association separately from leakage prevents a method from appearing successful by replacing one identity with another. The qualitative comparison shows that most baselines exhibit exactly this behavior. This framing affects how multimodal unlearning should be evaluated in general.

S2. Failure-targeted rebinding objective

Anchoring the edit at the earliest offending reasoning prefix, and shifting the continuation toward a grounded alternative, targets the substitution behavior that gradient-ascent and preference-based objectives induce. The ablation without reassociation control and the on-key/off-key control are the appropriate kinds of evidence for an association-level claim.

Critical weaknesses

W1. Insufficient evidence of association removal

The paper defines association removal behaviorally, as the target no longer being recoverable from outputs or reasoning under the designated context, including under input perturbations. The experiments do not yet establish this. The robustness study applies a query paraphrase and mild Gaussian noise, under which leakage decreases, and it reports no vanilla results under the same perturbations. No stronger probe is evaluated, such as brief relearning, other images of the same identity, or adversarial prompts. Classification on MLLMU-R also retains a forget accuracy of about 75% on a four-option task. The measurement of leakage itself relies on a GPT-5.4-mini judge whose agreement with human raters is not reported. In the off-key control, the vanilla model registers 15.5% leakage with a blank image, although the CLEAR-R query carries no identifying cue, which is consistent with a false-positive floor. Because F-RSH is a binary indicator of any unsupported concrete claim, shorter and more generic outputs are also favored by construction. Every comparison in the paper is expressed through these measurements, so it remains unclear how much of the reported improvement reflects removal of the association rather than surface suppression, judge behavior, or output length.

W2. Unjustified suppression of realized outputs

The realized-value term lowers the likelihood of whatever concrete value the model produced after the offending prefix, and the rule that decides whether such a value is "reliably resolved" and "unsupported" is not specified. Some of these values may be reasonable inferences from visible content, or common attribute values shared by retained identities. Because the parameters are shared, directly suppressing them may affect behavior beyond the designated association. The reported preservation measures are limited to the same benchmarks, namely retain-set ROUGE, retain-set reasoning and text-only access. The celebrity split is described but not reported, and no general multimodal or reasoning benchmark is evaluated, while retain ROUGE on MLLMU-R already decreases from 68.86 to 59.37. This term is the component that distinguishes VKCR from target-only suppression, so whether its cost stays within the designated association bears directly on the method's contribution.

Questions for authors

Clarification requests

  1. Reliability of the leakage measurement. The vanilla model shows 15–22% leakage with shuffled and blank images, although the CLEAR-R query is constant and carries no identifying cue. Where does this leakage come from, and should F-RSL be read relative to this level? What is the agreement between the GPT-5.4-mini judge and human raters on F-RSL and F-RSH?
  2. Construction of the training signals. Neither the main text nor the appendix specifies how the realized value ziz_i and its reliability indicator ρi\rho_i are determined, or how the valid continuation τ+\tau^+ is generated. How are they obtained? How often is the extracted value a reasonable inference from the image, or a value shared with retained identities? Does τ+\tau^+ reuse the sanitized answers or retained steps that also define the evaluation metrics?
  3. Contribution of residual correction. What does the residual-correction stage contribute on its own, in terms of accepted rounds and metrics before and after? Does the "w/o On-Policy Refinement" ablation refer to this stage or to the rollout-based supervision?

Suggestions

  • The missing discussion of reasoning-guided unlearning such as Targeted Reasoning Unlearning[1], which is closely related to the proposed rebinding objective, could be added to help position the contribution.
  • Results on a general multimodal or reasoning benchmark, or on the celebrity split, might help clarify whether the realized-value suppression affects general capability.
  • One way to probe whether the association is removed could be brief relearning on held-out facts, or evaluation with other images of the same identity.
  • A small human study, or reporting output length alongside F-RSH, might help clarify the reliability of the reasoning-level metrics.

[1]: Explainable LLM Unlearning through Reasoning

I would be happy to raise the score if the author could provide more evidence to answer my question.

5 comments
AuthorsOpenReview ↗

Response to Reviewer YBza (Part 1)

We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. We appreciate the recognition of our motivation to distinguish underlying knowledge from its visual access pathway, the importance of evaluating hallucinated reassociation separately from target leakage, and the value of our failure-targeted rebinding objective. We address the reviewer's questions and concerns point by point below.

W1a. Behavioral Association Suppression and Robustness

As explicitly discussed in Section 5.8, our notion of association unlearning is behavioral rather than mechanistic. To strengthen the evidence, we extend our evaluation with matched Vanilla controls and two stronger recovery probes.

Table R1. MLLMU-R F5 generation: reasoning-sensitive leakage (F-RSL, % ↓).

Input conditionVanillaVKCR
Clean98.9435.28
Query paraphrase93.1034.12
Visual noise97.6032.83
Joint perturbation91.5030.64
Adversarial prompt87.1037.80
Alternative same-identity image86.2029.70

All conditions use matched Vanilla/VKCR evaluations under the same F-RSL evaluation procedure. For adversarial prompting, the Vanilla and VKCR checkpoints receive the same original image and question, together with an additional instruction encouraging specific identity-related inferences. For the alternative-image probe, we construct identity-preserving variants of the original portraits by modifying non-identifying visual details. The robustness probes and their evaluation settings are described in Appendix C.3. Matched Vanilla controls and stronger recovery probes confirm that VKCR substantially suppresses target-sensitive reasoning leakage across all six conditions, including adversarial prompts and identity-preserving image variants, supporting effective and robust behavioral association unlearning.

Regarding the reviewer's concern about classification, VKCR consistently improves forgetting compared with all baselines across classification, generation, and cloze tasks, particularly in the challenging classification setting. As shown in Fig. 3 in manuscript, VKCR achieves the lowest classification F-RSL and forget accuracy among all compared methods. Although classification remains challenging across baselines and the residual accuracy exceeds the 25% chance level, these reductions demonstrate effective suppression at both the answer and reasoning levels.

W1b / Q1. Leakage Measurement, Off-Key Controls, and Output Length

We performed a blinded CLEAR-R audit over 10 forget and 10 retain outputs per method.

Table R2. CLEAR-R automatic (A) versus human (H) evaluation.

MethodF-RSL A/H↓F-RSH A/H↓F-RSP A/H↑R-RSP A/H↑
Vanilla0.6/0.60.4/0.50.9/0.90.8/0.7
MMUnlearner0.2/0.30.5/0.60.9/0.90.7/0.7
R²MU0.2/0.20.5/0.50.8/0.70.6/0.5
VKCR0.1/0.10.1/0.10.9/0.90.9/0.9
Cohen's κ0.9390.8960.8950.875

Cohen's κ\kappa measures agreement on paired binary judgments. For R-RSP, human annotations replace only the AI judge's semantic-validity decisions, leaving ROUGE-L unchanged. The consistently high agreement, together with closely matched automatic and human scores, supports the reliability of our reasoning-aware evaluation metrics.

Off-key controls. Shuffled and blank images remove the designated visual key, leaving no valid image-grounded target identity to recover. To assess whether VKCR inadvertently suppresses identity-related information outside the target context, we deliberately adopt a broader detection criterion: any identity-related content in the generated output is counted as a positive detection, regardless of whether it matches the forgotten target. Thus, Vanilla's 15.52% blank-image score reflects this conservative off-key detection criterion rather than genuine target-specific leakage. Under this criterion, VKCR produces only small changes on shuffled and blank images. This contrast supports the context selectivity of VKCR's behavioral edit rather than indiscriminate suppression of identity-related outputs.

Output informativeness and F-RSH. To assess whether the reduction in F-RSH reflects shorter or more generic outputs rather than fewer unsupported claims, we compare output length and visual reasoning content before and after unlearning (Table R3). VKCR reduces F-RSH by 49.7%, while CoT length, image-attributed reasoning steps, and image-evidence tags decrease by only 5.3%, 1.9%, and 4.3%, respectively. This indicates that the improvement is not primarily driven by reduced output informativeness. The human audits in Tables R2 further support the reliability of the evaluation.

Table R3. CLEAR-R output informativeness.

MeasureVanillaVKCRRelative change
F-RSH (%) ↓, main Table 136.4718.36−49.7%
Mean CoT words121.44114.98−5.3%
Image-attributed reasoning steps3.643.57−1.9%
Image-evidence tags4.854.64−4.3%
AuthorsOpenReview ↗

Response to Reviewer YBza (Part 2)

W2 / Q2. Realized-Value Suppression and Training Signals

Training supervision and reliability. As detailed in Appendix A.2, VKCR identifies invalid reasoning continuations τi−\tau_i^- from on-policy model rollouts and extracts their realized values ziz_i, with eligibility ρi=1\rho_i=1 requiring reliable span resolution, semantic-type consistency, and a prohibited or unsupported claim. GPT-5.4-mini-assisted verification excludes 3/34 CLEAR-R and 12/162 MLLMU-R preliminary candidates, yielding final eligibility rates of 64.58% and 66.67%. Separately, counterfactual continuations τi+\tau_i^+ are newly constructed from the original benchmark evidence and on-policy reasoning context, rather than copied from the sanitized answers or retained steps used for evaluation. Candidates are checked for target leakage, unsupported claims, evidence consistency, and preservation of non-sensitive reasoning, achieving cumulative acceptance rates of 92.16% and 92.00% after at one revision.

Contribution of realized-value suppression. Compared with the variant without realized-value suppression, full VKCR reduces F-RSL from 20.80 to 7.39 and F-RSH from 31.40 to 18.36, while improving F-RSP from 86.70 to 92.41. These results demonstrate that suppressing realized failures improves association forgetting with non-sensitive reasoning preservation.

Table R4. CLEAR-R F5 training-signal ablation.

Training signalF-RSL ↓F-RSH ↓F-RSP ↑
Sanitized-answer SFT31.5029.8068.20
Trajectory imitation only24.3026.1073.50
VKCR without realized-value suppression20.8031.4086.70
Full VKCR7.3918.3692.41

Preservation and limitations. As discussed in Section 5.8, the retain-utility degradation partly reflects the coupling between forget and retain knowledge in shared model representations. Moreover, the higher retention scores of some baselines, such as MANU, R-MUSE and MMUnlearner, should be considered alongside their weaker forgetting performance. R²MU achieves aggressive answer-level forgetting but substantially degrades retain ROUGE-L (19.36), whereas VKCR preserves much higher retain utility (59.37) while achieving lower reasoning leakage. On the disjoint celebrity set, VKCR also maintains classification and generation performance close to Vanilla, suggesting that broader, previously acquired knowledge remains largely accessible. Together, these results indicate that VKCR offers a more favorable forgetting–utility trade-off, although interference with closely coupled retained knowledge remains a limitation as we declared in Section 5.8.

Table R5. Celebrity-set utility on MLLMU-R F5 (Appendix C). We additionally evaluate 153 celebrity identities to assess collateral effects beyond the original identity splits.

MethodCls ↑Gen ↑Cloze ↑
Vanilla61.537.34.6
Grad. Diff.60.740.22.9
KL-Min59.333.91.6
NPO59.734.02.9
MANU63.038.24.2
R²MU47.115.43.9
R-MUSE61.337.23.3
MMUnlearner62.339.45.2
VKCR61.137.14.1

Q3. Contribution of Residual Correction

Stage-A performs the initial association edit using supervision derived from model rollouts. Stage-B then re-evaluates the edited model and corrects remaining leakage or unsupported reassociation. Thus, the w/o On-Policy Refinement ablation in Table 4 removes rollout-derived supervision rather than Stage-B alone, since Stage-A also relies on this supervision.

Table R6. Stage-wise performance on MLLMU-R F5 (generation).

StageF-RL ↓F-RSL ↓F-RSH ↓F-RSP ↑R-RL ↑
Stage-A only35.6144.3229.6794.9863.63
Stage-A + Stage-B31.2535.2821.2594.8159.37
Change−4.36−9.04−8.42−0.17−4.26

In the reported correction run, one proposal was attempted and accepted within a maximum budget of two rounds. Adding Stage-B reduces F-RSL by 9.04 points and F-RSH by 8.42 points, with only a 0.17-point decrease in F-RSP, although retain ROUGE-L decreases by 4.26 points. These results isolate the incremental contribution of residual correction, distinct from the broader on-policy refinement ablation.

AuthorsOpenReview ↗

Response to Reviewer YBza (Part 3)

Response to Suggestion

We thank the reviewer for these suggestions, which we address point by point below.

  1. In the revised manuscript, we expand the discussion of Targeted Reasoning Unlearning (TRU), clarifying VKCR's distinct focus on image-conditioned association suppression and failure-localized rebinding. Both TRU and VKCR leverage reasoning-guided supervision for selective unlearning. While TRU primarily focuses on controlling the forgetting scope and maintaining coherent post-unlearning responses in LLM, VKCR addresses image-conditioned knowledge associations in multimodal reasoning models. Specifically, VKCR identifies the model's realized offending reasoning continuations and performs evidence-grounded counterfactual branch rebinding. Moreover, VKCR explicitly targets hallucinated reassociation, where suppressed knowledge is replaced by unsupported concrete facts, through realized-value suppression.

  2. We add a celebrity-set utility evaluation on MLLMU-R (Table R5), where VKCR remains close to Vanilla in classification and generation, providing additional evidence of preservation beyond the original identity splits.

  3. VKCR is designed to suppress designated visual-to-knowledge associations behaviorally, rather than to permanently erase knowledge or prevent subsequent relearning. To further examine behavioral robustness, we consider brief relearning on held-out, target-related facts. Starting from each unlearned checkpoint, we perform 50 supervised fine-tuning steps using 50 previously unseen image-QA examples associated with the designated forget identities.

Table R7. Brief relearning stress test on MLLMU-R F5 (F-RSL, % ↓).

MethodBefore ↓After ↓Increase (pp) ↓
Grad. Diff.66.6182.4615.85
NPO86.9494.167.22
R²MU54.1870.2516.07
MMUnlearner88.2793.144.87
VKCR (Ours)35.2841.235.95

Relearning increases leakage for all evaluated methods. Although VKCR does not achieve the smallest leakage increase, it maintains the lowest absolute post-attack F-RSL, substantially below R²MU and the other baselines. This suggests that VKCR's behavioral suppression remains comparatively effective after brief re-exposure.

  1. We include adversarial and alternative-image recovery tests (Table R1), blinded human validation (Table R2), and output-length analysis (Table R3) as we illustrated in previous response.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.