- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper addresses the issue of MLLM backdoor detection being bypassed by adaptive attacks. The authors propose BAWD - by marking the poisoned training set and the trusted benign reference set separately, training a lightweight classifier to learn the distribution differences between the two, this score is theoretically consistent with the post- poisoning posterior sorting, and then using the maximum score within the fixed ranking window as the cliff threshold. Experiments show that even if the attacker directly targets the deployed detector, the re-fitted BAWD can still maintain an F1 score of over 99.5%, while the baseline detector completely fails.
Critical strengths
-
The idea of leveraging the fact that the reference set is drawn from the same distribution as the clean data, in order to separate clean from poisoned samples within the audited mixture, is clever and well-motivated.
-
The paper conducts comprehensive and extensive experiments.
-
The method is simple in design and has a low deployment barrier.
Critical weaknesses
-
The paper provides no real, data-driven examples showing that the attacker's processing makes the features hard to distinguish, and that the features become separable again after the defender refits a new witness on the attacked model. Only the manually simulated scatter plots in Figure 1 illustrate this phenomenon.
-
The filtering ratio is hard-constrained within [3%, 35%], so the method cannot handle cases that fall outside this range. What is the justification for choosing 3% and 35% as the two endpoints?
-
In practical use, it is difficult to simultaneously obtain the mixed (audited) dataset and a "sufficiently clean and identically distributed" reference set.
-
The method description in Figure 2 does not correspond one-to-one to the method description in the main text, which increases the reading burden. The six steps shown in the figure should be described separately and aligned with the text.
-
The paper excessively uses \vspace to compress the page count; for example, there is no vertical spacing between each subsection and its surrounding content.
-
It is unclear how the proposed method is tied to the multimodal setting. The core mechanism (learning a source-discrimination score between the audited mixture and a clean reference) appears modality-agnostic and would seemingly apply unchanged to unimodal models. The paper would benefit from clarifying what specific challenges the MLLM setting introduces, and what in the method is designed to address them.
Questions for authors
N/A
2 comments
Response
Thank you for the author's reply. This part has answered my confusion, but I will keep my score.
Response to Reviewer T95p
We thank Reviewer T95p for the constructive feedback and presentation suggestions. All revisions are highlighted in blue in the updated manuscript, and we respond to each point below.
W1. Real Data-Driven Demonstration of Frozen Failure and Refitted Recovery
To complement the schematic scatter plots in Fig. 1 with real data, we added Fig. 5 in App. M (referenced in Sec. 4.4) using actual cached representations from the Chain-L round-2 checkpoint with MLLM weights held fixed (99.8% ASR). On these identical features, the attacked frozen detector ranks poisoned samples below benign ones (0.2020 AUROC, 0.0% F1), whereas refitting BAWD-L restores clean separation (0.9976 AUROC, 99.5% F1), illustrated both via measured score distributions and shared-coordinate t-SNE projections of all 6,218 audited examples.
W2. Justification and Sensitivity of the
[3%, 35%]Candidate WindowThe
[3%, 35%]topgapendpoints act as a broad engineering range prior within which the sharpest adjacent score drop determines the cutoff: the 3% floor prevents single-outlier cutoffs in the extreme tail, and the 35% ceiling avoids flagging bulk benign data, without fixing the flagged fraction at the true 10% rate. At 10% poisoning, BAWD-L's selected candidate set and F1 are completely invariant under[1%, 20%]and[1%, 50%]across all 11 main settings (App. F.4, Table 16). Below 3% poisoning, the 3% floor mechanically caps precision, but applying a pre-specified[0.5%, 35%]window to the exact same fitted scores without retraining restores high F1 at 2% poisoning (App. K, Table 32):[3%, 35%])[0.5%, 35%])Practitioners anticipating lower contamination can therefore simply lower the window floor without retraining.
W3. Practical Availability of a Suitable Reference Set
While cross-task references indeed degrade detection (App. H.4), the reference need only be same-task and mostly clean rather than large, perfectly pure, or drawn from the audited split. When a modest same-task set is retained before adding third-party fine-tuning data, 400 examples already yield 94.4% to 97.9% F1 (Fig. 4a), an ID-disjoint held-out pool yields 99.0% to 99.8% F1 (Sec. 4.6), and 1% to 2% poison contamination still preserves 94.9% to 98.6% F1 on detector-aware attacks while reference contamination stays below the audit rate (Sec. 3.6; Fig. 4b). Furthermore, when no external clean reference exists at all, our new label-free behavioral selection step in App. L (Table 33; Sec. 4.6) builds a same-task pseudo-reference directly from the poisoned audit pool across five draws:
W4 & W5. Alignment between Figure 2 and Main Text, and Heading Spacing
We rewrote the Sec. 3 overview to align step-by-step with the six stages of redrawn Fig. 2, progressing from data sources (Step 1, Sec. 3.1), frozen MLLM extraction and shared standardization/PCA (Steps 2 and 3, Sec. 3.3), and source-classifier fitting (Step 4, Sec. 3.4) to OOB aggregation and
topgapcandidate selection (Steps 5 and 6, Sec. 3.5). We also removed all negative\vspacecommands around section and subsection headings to restore the standard ICLR template spacing within the 10-page limit.W6. Connection to the Multimodal MLLM Setting
While source discrimination itself is a modality-agnostic statistical principle, our audit interface is designed around three challenges specific to generative MLLMs. Because multimodal triggers may reside in the image, the text, or solely their cross-modal conjunction (as in Dual-Key and Adaptive Dual-Key, where BAWD-L achieves 99.5% and 100.0% F1 in Table 1), extraction targets the final prompt token after joint image-text fusion. Stopping before response tokens prevents the detector from directly using repeated target answer text. Class-free source supervision handles free-form MLLM responses without requiring a fixed output taxonomy and also applies to valid-option targets such as B (App. F.3).
We hope these revisions and experiments address all of your concerns, and we would appreciate your considering an updated score.