PPU-Bench: Real-World Multimodal Benchmark for Personalized Partial Unlearning

Multimodal Large Language Models (MLLMs) may memorize person-related information that remains accessible through both visual and textual inputs. However, existing MLLM unlearning benchmarks often rely on synthetic knowledge injection or complete subject-level deletion, overlooking real-world knowledge and fine-grained deletion targets within the same subject. We introduce PPU-Bench, a real-world, knowledge-injection-free benchmark for personalized partial unlearning in MLLMs. PPU-Bench contains over 24K multimodal and unimodal samples derived from pre-existing knowledge of 500 public figures under three progressively fine-grained settings: Complete, Selective, and Personalized Unlearning. It evaluates six unlearning methods across four MLLM backbones using paired QA/VQA probes, multiple evaluation formats, general-utility tests, and robustness attacks. Our experiments reveal that Complete Unlearning often suppresses visual identity associations while leaving the underlying factual knowledge accessible through textual queries. Selective Unlearning better captures category-level factual forgetting but introduces pronounced modality and forget–retain trade-offs, whereas Personalized Unlearning exposes substantial difficulties in controlling subject-specific factual boundaries. Robustness analysis under cross-image and prompt-based attacks further reveals setting-dependent vulnerabilities.