MUREN-Bench: Evaluating Locality in Real-World Multimodal Unlearning

Machine unlearning requires a trained model to remove designated knowledge while leaving the rest of its capability intact, a property known as locality. For multimodal large language models, real-world requests such as right-to-be-forgotten claims, copyright takedowns, and trademark disputes make this task concrete: the model must stop recognizing one specific real-world entity and answering questions about it, while preserving the knowledge surrounding that entity. Existing multimodal unlearning benchmarks fall short of this setting in two ways: they typically construct forget targets by injecting fictitious entities through fine-tuning, so the model lacks organic knowledge of the target and its real-world surroundings; and they measure locality with a single averaged retain score over entities far from the target, so failures to preserve closely related knowledge remain invisible. To address both problems, we introduce MUREN-Bench (Multimodal Unlearning of Real-world Entity Neighborhoods), a benchmark built entirely from real-world entities across six categories: artifacts, biological entities, celebrities, IP characters, landmarks, and trademarks. The benchmark contains 90 groups, each pairing one forget target with a hierarchical retain set ordered by relatedness to the target: a distant anchor, an intermediate neighbor, and the closest neighbor. This structure turns locality from a single averaged score into a profile over proximity to the target. Every entity is probed through image-only, image-text, and text-only routes, yielding 360 entities and 57,240 probes in total. Experiments with seven unlearning methods on the LLaVA-1.5-7B and Qwen3-VL-8B backbones, compared against the original models, reveal a shared limitation: methods that suppress the target often fail to preserve nearby entities, and none achieves reliable locality within the target's neighborhood.