- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper introduces PRMU, a benchmark for person-centric multimodal knowledge unlearning without access to the original training corpus. It contains 1,080 public figures and 91.9K textual and visual probes, evaluating target forgetting, preservation of knowledge about related and unrelated people, and general multimodal utility. The authors also propose Similarity-Gated Projection Editing (SGPE), combining model-generated proxy supervision, protected projection editing, and input-dependent activation.
Critical strengths
The paper tackles a timely and challenging problem: removing person-related knowledge from multimodal models without access to the original training data.
Critical weaknesses
-
The proposed method SGPE leaves the original checkpoint unchanged and applies an external weight edit through a similarity gate. The results show that it suppresses target answers, but do not establish whether the knowledge remains accessible when the gate fails to activate. Incorrect factual substitutions also count as successful forgetting, making it difficult to distinguish knowledge removal from changes in output behavior.
-
The paper does not test whether using only some of a person’s facts during unlearning also removes the remaining facts (selective forgetting). For example, if the method receives 50% of a person’s information, can it forget the other 50%? Without this experiment, comprehensive person-level forgetting is not established.
-
Unlearning also harms knowledge about other people. For example, Qwen3-VL-8B batch unlearning, visual QA scores fall from .596 to .346 for related people and from .629 to .352 for unrelated people, even though MMBench accuracy remains unchanged.
Questions for authors
-
What operational criterion distinguishes successful knowledge removal from gated response suppression in this work? Do you have evidence that target information remains inaccessible through inputs that fail to activate the similarity gate?
-
Evidence needed to support the stated scope: are evaluation facts explicitly separated from proxy supervision and gate calibration at the underlying-fact level? If so, please report forgetting separately for supplied and withheld facts. A controlled experiment using only 50% (or other probability) of each person’s facts would affect my assessment of comprehensive person-level forgetting.
-
How should the locality claim be reconciled with the substantial neighbor and unrelated-person degradation in Table 2, and for Table 6 where removing protected projection improves neighbor visual QA from .346 to .473?