- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Visual-Keyed Counterfactual Rebinding (VKCR) for cross-modal association unlearning. It specifically aims to suppress a designated image-conditioned association to target knowledge while preserving permissible text-only access and non-sensitive reasoning. Experiments are performed on two small MLLMs (Qwen2.5-VL-3B-Instruct and LLaVA-1.5-7B-HF), one reasoning MLLM (R1-Onevision-7B-RL) on two benchmarks (CLEAR-R and MLLMU-R).
Critical strengths
The experimental results show substantial improvements under the proposed evaluation protocol compared to previous multimodal unlearning baselines.
Critical weaknesses
-
In my view, the core setting is simply a combination of modality-selective forgetting from MMUnlearner and reasoning-trace unlearning from R²MU. Therefore, a basic baseline combining MMUnlearner's update constraints with a multimodal adaptation of R²MU's reasoning objective should be included.
Without such a comparison, it is hard to evaluate the contribution of the proposed method. -
Tables 1 and 3 use F-RSP to support reasoning preservation. However, according to Appendix B.5, it allows high-overlap contradictions to pass. I am concerned about whether such a metric can truly reflect differences between the methods.
Questions for authors
-
How can F-RSH ensure that it reliably distinguishes fabricated details from diverse details (e.g., correct details but absent from the reference annotation)?
-
In line 93-94, it is said that "Effective unlearning must therefore suppress the influence of the designated association while preserving non-sensitive relations of the same visual subject". What are non-sensitive relations? How to evaluate the preservation of non-sensitive relations.
7 comments
Response to Reviewer zt7q (Part 2)
Q1. Correct Additional Details versus Hallucinations
Evidence-grounded hallucination assessment. F-RSH does not equate reference mismatch with hallucination. As specified in Appendix B.5, the semantic judge is explicitly instructed not to flag reasoning merely because it differs from the reference answer. Instead, F-RSH targets unsupported identity substitutions, fabricated attributes, and other claims lacking support from the available evidence. Thus, a valid additional observation is not inherently a hallucination simply because it is absent from the reference annotation.
Human validation and limitations. To assess potential false positives, we manually audit F-RSH-flagged claims against the permitted source evidence. As shown in Table R2, human annotators confirm 55/60 (91.7%) flagged claims as unsupported. This supports the reliability of positive hallucination flags. We clarify this distinction and limitation in Appendix B.5.
Q2. Meaning and Preservation of Non-Sensitive Relations
Under the original identity-level task, non-sensitive relations primarily mean grounded, non-identifying visual attributes and scene relationships (e.g., clothing, objects, and spatial context). The stricter case of other permissible factual relations of the same identity is tested separately in the new paired relation experiment (Table R3): forbidden-relation recovery decreases by 35.60 points, versus an 11.83-point decrease for retained relations, thereby supporting preferential relation-level suppression.
Table R3. Paired relation-level evaluation (CLEAR-R F5 %).
| Relation subset | Method | RSL | RSH | RSP |
|---|---|---|---|---|
| Forget 50% | Vanilla | 51.43 | 36.82 | 91.78 |
| NPO | 40.62 | 50.48 | 90.37 | |
| R²MU | 26.74 | 48.91 | 84.36 | |
| VKCR | 15.83 | 18.94 | 91.12 | |
| Retain 50% | Vanilla | 49.17 | 36.14 | 92.38 |
| NPO | 43.58 | 43.72 | 90.66 | |
| R²MU | 33.21 | 43.86 | 85.17 | |
| VKCR | 37.34 | 35.87 | 90.36 |
Dear Reviewer zt7q,
As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal. We hope our additional experiments and clarifications have addressed your concerns. We would greatly appreciate any further feedback and would be happy to clarify any remaining questions.
Thank you again for your valuable feedback, time, and consideration of our work.
Best regards,
The Authors
Response to Reviewer zt7q (Part 1)
We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. We appreciate the recognition of the effectiveness of VKCR and its substantial improvements over existing multimodal unlearning baselines under our evaluation protocol. We address the reviewer's questions and concerns point by point below.
W1. Comparison with MMUnlearner + R²MU
To directly address this concern, we implement a combined baseline integrating MMUnlearner's selective update constraints with R²MU's reasoning-unlearning objective.
Table R1. Combined-baseline results (F5 generation).
VKCR consistently outperforms the combined baseline across both benchmarks. On MLLMU-R, it achieves lower forget ROUGE-L, F-RSL and F-RSH, alongside higher F-RSP and retain ROUGE-L, demonstrating stronger association forgetting, reduced hallucinated reassociation, and better retained utility. Similar improvements are observed on CLEAR-R. Importantly, VKCR goes beyond combining selective parameter updates with reasoning suppression: it identifies realized target-bearing or unsupported failures, suppresses the sensitive values, and counterfactually rebinds the affected reasoning branch toward evidence-grounded alternatives. This failure-conditioned correction distinguishes VKCR from the hybrid baseline, enabling stronger association forgetting while better controlling hallucinated reassociation and preserving useful reasoning.
W2. High-Overlap Contradictions in F-RSP
Metric design and targeted validation. F-RSP is designed to assess whether annotated non-sensitive reasoning steps remain preserved after unlearning. As detailed in Appendix B.5, it combines deterministic lexical matching with LLM-based semantic evaluation for steps not resolved by lexical matching, capturing both lexical and semantic preservation. To further validate the reliability of this evaluation procedure, we conduct a targeted human audit of lexical-pass steps and F-RSH-flagged claims, directly examining whether automatically accepted reasoning steps are semantically preserved and whether detected hallucinations are genuinely unsupported.
Table R2. Targeted CLEAR-R human validation (step/claim-level audit).
Reliability and cross-metric validation. Human annotators confirm 95/100 sampled lexical-pass steps, with comparable confirmation rates across methods (92-96%). Although semantic matching is not error-free, all methods are evaluated using the same protocol, and the similar confirmation rates provide no clear indication of method-specific lexical bias or an artificial advantage for VKCR in this audit. Separately, 55/60 F-RSH-flagged claims are confirmed as unsupported, supporting the complementary detection of hallucinated reassociation. We therefore interpret F-RSP jointly with F-RSL and F-RSH, rather than treating lexical preservation as sufficient evidence of successful unlearning. Retained utility is separately evaluated through R-RL and R-RSP.
Qualitative corroboration and limitations. Appendix C.2 (Table 11) further presents actual reasoning traces and final answers across methods, illustrating failures that aggregate preservation scores alone cannot distinguish: KL-Min retains the forgotten identity, several baselines introduce unsupported identity substitutions, whereas VKCR avoids identity claims while preserving grounded visual descriptions. These observations are consistent with the quantitative results and support the interpretation that VKCR preserves useful reasoning without merely substituting unsupported associations.