- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes an evidence-routing perspective for detecting diverse backdoor activations in LVLMs. Instead of relying on the appearance or location of backdoor triggers, the method localizes a decision moment during autoregressive generation, constructs an evidence support graph using attention and counterfactual masking, and measures deviations from clean evidence-routing behavior estimated from a calibration set. Experiments across multiple LVLMs, datasets, and backdoor configurations show consistently strong detection performance.
Critical strengths
-
The paper provides an interesting unified perspective on diverse LVLM backdoors. Modeling backdoor activation through abnormal evidence routing, rather than relying on specific trigger appearances or locations, is well motivated and potentially applicable to heterogeneous trigger mechanisms.
-
The empirical evaluation is relatively comprehensive and shows consistently strong performance. The method is evaluated across multiple LVLM backbones, datasets, and substantially different backdoor configurations, including multimodal, instance-adaptive, and stealthy visual triggers.
Critical weaknesses
-
The overall methodological presentation could be improved. The paper lacks a framework overview illustrating the complete pipeline. In particular, Figure 1 directly presents the visualization of decision-moment localization rather than an overview of the proposed method. Given that the method consists of several sequential components, including decision-moment localization, evidence support graph construction, counterfactual verification, and calibration-based detection, an overview figure would substantially improve readability.
-
The extremely high detection performance raises questions about generalization to genuinely unseen backdoor mechanisms. The reported results are nearly perfect across the evaluated settings, while the considered backdoor configurations are known and instantiated during evaluation. It would be important to clarify whether the proposed evidence-routing signature remains equally effective for unseen trigger mechanisms or attack families that are not represented in the current experimental design.
-
The discussion of related internal-signal-based detection approaches appears incomplete. The paper primarily positions the proposed approach against trigger-inversion and attention-based backdoor detection, with limited discussion of methods that detect abnormal LVLM behavior using hidden representations, embeddings, or uncertainty signals. Relevant examples include evidential uncertainty-based LVLM misbehavior detection [1], TruthPrInt [2], and UniProbe [3]. Discussing these approaches would better clarify how evidence routing differs from existing internal-representation-based detection paradigms.
[1] Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification.
[2] TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination via Latent Truthful-Guided Pre-Intervention.
[3] UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations.
Questions for authors
See the weakness.
1. Overview figure
We thank the reviewer for this valuable suggestion and agree that an overview would improve readability. We will add an overview figure that illustrates the complete pipeline of our framework.
2. Generalization to unseen backdoor mechanisms
Thank you for this question. Our framework does not learn what any particular backdoor looks like. It characterizes how evidence is routed to the output, using only clean calibration data. Every backdoor in our evaluation is unseen by our framework.
We respectfully note that evaluating on an attack is different from training on it. Any study must run attacks to test them, and our framework never sees them beforehand. The near-perfect results reflect the large gap between backdoored and clean evidence routes, not familiarity with the attacks.
This is what makes the signature general. The four configurations differ in modality, trigger form, and implantation stage: multimodal instruction tuning (VL-Trojan), cross-modal invisible triggers (BadCM), instance-adaptive triggers (IAG), and encoder-level stealthy triggers (BadVision). They all show the same abnormal evidence routing.
3.Related internal-signal-based methods
Thank you for pointing out these works. We agree that discussing them makes our positioning more complete. They address hallucination and general misbehavior rather than backdoors, and they examine whether an output is abnormal. We will add these to the related work. Our work instead examines how an output is reached. We causally verify, via counterfactual interventions, which evidence supports the output, and compare the route with clean inputs that produce the same output, using clean data only.