Black-Box Backdoor Detection in Multimodal Large Language Models via Counterfactual Response Persistence

Multimodal large language models (MLLMs) are vulnerable to backdoor attacks, where hidden triggers in images, text, or both can induce attacker-specified behaviours. Detecting such backdoors is particularly challenging in the black-box setting, where defenders can access only model inputs and generated responses. Existing defences are often designed for unimodal models or rely on model internals, while directly perturbing a modality in MLLMs can simultaneously alter task-relevant semantics, making response changes difficult to interpret. We uncover a previously overlooked phenomenon, termed the **modality shortcut**: backdoor-induced responses exhibit abnormally high persistence under cross-modal interventions, unlike benign responses. Building on this observation, we propose ${\mathbf{B}}^3{\mathbf{M}}^2$, a training-free black-box detector for backdoor activation in MLLMs. ${\mathbf{B}}^3{\mathbf{M}}^2$ independently replaces each input modality with clean references and measures how strongly the original response persists after each intervention. Inputs exhibiting abnormally high persistence in either modality are identified as backdoored. ${\mathbf{B}}^3{\mathbf{M}}^2$ requires no access to model parameters, logits, or internal activations, and assumes no knowledge of the trigger, attack mechanism, or target response. Under this strictly black-box setting, ${\mathbf{B}}^3{\mathbf{M}}^2$ achieves at least 99.5% average TPR across all model–dataset settings at a 1% FPR operating point across nine attacks on LLaVA-1.5-7B and Qwen2.5-VL-7B over VQA-E and COCO, while remaining effective against adaptive attacks. Notably, it uses only five clean reference pairs and eleven queries per input, highlighting its effectiveness and practicality for black-box backdoor detection.