- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes VARDE, an inference-time backdoor (trigger) detector for image classifiers that is interpretable by construction. At every layer of the network, a BatchTopK sparse autoencoder (SAE) maps activations to a sparse set of human-interpretable "concepts". An input is scored by the squared Mahalanobis distance between its (globally pooled) concept vector and the class-conditional concept distribution estimated from a clean holdout set, using a low-rank PPCA covariance estimator. Per-layer scores are converted into empirical p-values on a disjoint calibration split and combined via an empirical variant of Brown's method (moment-matched scaled chi-squared) to handle inter-layer dependence, yielding a per-class calibrated threshold at FPR α = 0.05. The central technical contribution is exact additive decomposability: because the test statistic is built from Mahalanobis distances — a quadratic form in the concept vector — it splits identically into per-layer and per-concept contributions, and at convolutional layers further into exact per-position contributions, producing "concept maps" that localize which input regions drove the verdict. The explanation is not a post-hoc surrogate; it is the detector. All of this incurs no forward or backward passes beyond the one needed for the prediction. Validation. Four datasets (CIFAR-10, GTSRB, Tiny ImageNet, CelebA) × two architectures (ResNet18, VGG16 — chosen to contrast sequential vs. residual topologies) × five attacks (BadNets, Blended, IAD, WaNet, physical eyeglasses), against the two SOTA single-pass inference-time detectors Beatrix and Orion, over five seeds. VARDE is competitive or better on patch/blended/physical attacks and clearly strongest on input-aware (IAD) attacks (mean TPR@5% 85.1 vs. Beatrix 35.8, Orion 56.6), but degrades sharply on the imperceptible warping attack WaNet under ResNet18 (9.2% TPR on CIFAR-10 vs. Orion's 85.5%). A blur-perturbation study shows concept maps localize triggers more faithfully than Grad-CAM and a random baseline. A controlled Mann–Whitney analysis (Section 4.5) tests whether the top attributed concepts actually correlate with the trigger, using a random-concept control and a core-clean control: the correlation holds for BadNets/Blended/physical triggers, is ambiguous for IAD (the clean control separates too), and fails for WaNet in the settings where detection is weakest. An appendix ablation (Table 4) shows that scoring raw activations instead of SAE concepts costs 0.19 TPR on IAD and 0.08 on WaNet, while patch triggers are unaffected.
Critical strengths
1 — Interpretability by construction with exact (non-approximate) attribution. The verdict, its per-layer/per-concept breakdown, and its spatial concept map are all identities of the detector's own statistic, computed in the same forward pass as the prediction. This eliminates the fidelity gap that fundamentally limits Grad-CAM-based explanations of detector decisions (which attribute the classifier's prediction, not the detector's verdict, via a backward pass). This is consequential: it makes detector verdicts auditable and contestable in a way that no prior inference-time backdoor defense supports, and the same machinery transfers conceptually to other anomaly-detection settings. The strongest empirical support is the blur-perturbation study (Fig. 2/10): concept-map-guided perturbation recovers the true label from smaller image fractions than Grad-CAM across attacks.
2 — Unusually rigorous, self-critical evaluation methodology. The paper does three things defense papers routinely skip: (i) a controlled Mann–Whitney analysis asking whether the attributed concepts correlate with the trigger, against both a random-concept control and a core-clean control — and reports honestly when the answer is negative (IAD, WaNet); (ii) a raw-activation vs. SAE-concept ablation isolating the SAE's contribution (helps precisely for dynamic/imperceptible triggers, unnecessary for patch triggers); (iii) prominent reporting of failure modes (WaNet on ResNet18) rather than burying them in averages. The baseline treatment (Beatrix, Orion re-run under matched clean-data budgets and FPR calibration) is fair and transparent about comparability limits. This level of experimental hygiene materially raises my confidence in the reported conclusions.
Critical weaknesses
1 — The trigger-alignment analysis does not test all signals the detector uses. Section 3.5 permits a positive anomaly contribution from a decrease in concept activation, whereas Section 4.5’s one-sided statistic counts only increases. Figure 3 therefore cannot support its stated interpretation without a signed or attribution-aware analysis. The choice of clean comparison inputs also needs clarification to rule out source-versus-target class differences.
2 — Detection performance is highly setting-dependent. VARDE trails Orion on average for BadNets, WaNet, and the physical attack (Table 1). On WaNet it obtains 9.2% versus Orion’s 85.5% TPR for ResNet/CIFAR-10; the weakness is not confined to ResNet, since VGG/Tiny ImageNet yields 3.2% versus 26.1% (Table 5). This limits the breadth of the “competitive” claim.
Questions for authors
- Can you repeat Section 4.5 with an analysis that captures both increased and decreased concept activation, preferably comparing each triggered image with its untriggered counterpart? Which clean population is used in the current Mann–Whitney comparisons?
- What explains the WaNet failures across architectures, especially on Tiny ImageNet? Do any individual layers retain discriminative signal that the combined score loses?
- What are VARDE’s inference latency, memory use, and SAE fitting costs relative to Beatrix and Orion? These measurements matter to the single-pass efficiency claim.
- Under what attacker-knowledge assumptions is the method intended to operate? If the attacker knows the detector, how does detection change when trigger generation also minimizes VARDE’s score?
2 comments
Response (2/2)
On latency, memory and fitting cost
The new Appendix D reports the cost for all settings and all three detectors. We report operation counts (FLOPs) instead of wall-clock latency, because our three implementations are not equally optimized. Per query, VARDE adds 43% to 59% of the victim's forward pass, nearly all of it for the SAE encoders, while Beatrix adds at most 5% and Orion less than 1%. VARDE stores 18 to 155 MB, against 4 to 421 MB for Beatrix and 0.2 to 22 MB for Orion. Fitting VARDE is a one-off cost of 2.7% to 47% of training the victim, dominated by SAE training. VARDE thus needs no second forward or backward pass through the model, at a cost of about 1.5 forward passes per query, but it is not the cheapest of the three, which Section 5 now states.
On the threat model and adaptive attacks
Section 3.1 now states the attacker's capabilities and knowledge. The attacker controls the training data and procedure, but must deliver a standard model, so the backdoor cannot be placed in the architecture or the forward-pass code. In the main experiments, the attacker does not know the defence, as in the evaluations of Beatrix and Orion.
We added an adaptive attack (new Appendix B). The attacker knows VARDE in full but has no access to the defender's SAEs, class profiles or clean data. It trains the trigger jointly with the model and penalizes the distance between triggered inputs and the clean target class under a surrogate of VARDE built from its own data, so the trigger preserves the target class's concept distribution. The attack succeeds on 95% to 100% of triggered inputs, with clean accuracy close to that of the other attacks (Table 2). With calibration by true label, since the jointly learned trigger misroutes clean inputs as WaNet does, VARDE detects 53.4% of triggered inputs at 5% FPR, comparable to Orion's 55.0%, while Beatrix detects 6.0%, so the attack partly evades VARDE, as it does Orion. As a control, we train the same attack without the penalty (, new Table 3): the penalty substantially lowers VARDE's detection (TPR at 5% FPR), while Orion's changes much less except on VGG16 TinyImageNet, where it lowers both.
| Arch. | Dataset | VARDE () | VARDE () | Orion () | Orion () |
|---|---|---|---|---|---|
| ResNet18 | CIFAR-10 | 0.66±0.15 | 0.29±0.09 | 0.36±0.13 | 0.47±0.10 |
| GTSRB | 1.00±0.01 | 0.76±0.06 | 0.99±0.02 | 0.87±0.14 | |
| TinyImageNet | 0.86±0.12 | 0.59±0.33 | 0.63±0.19 | 0.46±0.24 | |
| VGG16 | CIFAR-10 | 0.94±0.03 | 0.15±0.02 | 0.26±0.10 | 0.19±0.09 |
| GTSRB | 1.00±0.00 | 0.85±0.19 | 0.99±0.01 | 0.95±0.07 | |
| TinyImageNet | 0.95±0.07 | 0.57±0.17 | 0.76±0.10 | 0.37±0.13 |
Response (1/2)
We thank the reviewer for the thoughtful feedback and valuable suggestions. We have carefully considered each point and will make corresponding revisions to the manuscript (changes are in blue).
On the two-sided trigger-alignment analysis and the clean population
We repeated Section 4.5 with a two-sided score, which now replaces the one-sided score in Section 4.5 and Figure 3. The clean population is the same poisoned images without the trigger, so a difference between source and target classes cannot produce the separation. Section 4.5 now states this. The conclusions are unchanged. For BadNets, Blended and the physical trigger, the attributed concepts separate triggered from clean inputs and the clean control stays well below. For IAD, the clean control separates as well, so the attributed concepts are already present in the clean image. For WaNet, where the one-sided score showed no separation, the two-sided score shows none either, so the one-sided score did not hide a suppression effect.
On the WaNet failures and individual layers
Most of the gap comes from how VARDE calibrates its per-class thresholds (new Appendix C). Each class's threshold is calibrated on clean inputs grouped by predicted class. The WaNet models misclassify many clean inputs into the target class: in the settings where VARDE fails, 9 to 55 times as many as into an average other class, against about 3 times where VARDE detects WaNet (Figure 7). These inputs enter the calibration set of the target class and raise its threshold, and since VARDE scores triggered inputs like them, triggered inputs are not flagged. Orion uses a single threshold for all classes and is not affected.
To test this explanation, we group the calibration inputs by true label and change nothing else.
WaNet TPR at 5% FPR, calibration grouped by predicted (Pred.) or true (True) label.
Detection rises where VARDE failed, least on ResNet18 CIFAR-10, where triggered inputs also lie close to the target-class profile (Appendix C), which we state as a remaining limitation. On TinyImageNet, the models' high clean error puts misclassified inputs into every calibration subset, not only that of the target class.
Individual layers do retain signal that the combined score loses. In all four failing settings, some layers, mostly deep ones, separate triggered WaNet inputs from clean ones better than the combined score (Figure 9), for example with an AUROC of 0.97 against 0.89 on ResNet18 GTSRB. No layer subset is best in every setting (Table 4). Choosing a subset from WaNet results would tune the defence on the attack it is evaluated against, so the main results keep all layers.