acceptodds
Under review as a conference paper at ICLR 2027

Interpretable backdoor detection using sparse concept representations

Abstract

Inference-time backdoor detection methods aim to identify model inputs that contain a hidden trigger. However, few explain why an input is flagged as poisoned. Without such explanations, the detector's decisions are difficult to audit, contest, and learn from. To address this, we propose VARDE (Verdict Attribution via Representation DEcomposition), a backdoor detection method that uses human-interpretable concepts. VARDE transforms the activations into such concepts using a sparse autoencoder at every layer. The verdict is based on the Mahalanobis distance between present concepts and the estimated class-conditional distribution of concepts. The distance decomposes additively over concepts, providing an exact value of how much a particular concept at a specific layer contributed to the verdict. For CNNs, this decomposition maps onto the model's input space; each concept's contribution splits exactly across spatial locations, yielding a concept map that localises it in the input. VARDE is computationally efficient, as the single forward pass producing the model prediction also yields the verdict and its explanation. We show for a variety of image classification datasets how VARDE explains its verdict while being competitive with state-of-the-art inference-time detectors. We also compare VARDE against Grad-CAM, commonly used in backdoor detection. VARDE's concept maps are more faithful than Grad-CAM's saliency maps, consistently concentrating more strongly on the trigger region.

Primary area:
interpretability and explainable AI
Keywords:
XAI, interpretability, concepts, sparse autoencoder, backdoor, defence

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kZovOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes VARDE, an inference-time backdoor (trigger) detector for image classifiers that is interpretable by construction. At every layer of the network, a BatchTopK sparse autoencoder (SAE) maps activations to a sparse set of human-interpretable "concepts". An input is scored by the squared Mahalanobis distance between its (globally pooled) concept vector and the class-conditional concept distribution estimated from a clean holdout set, using a low-rank PPCA covariance estimator. Per-layer scores are converted into empirical p-values on a disjoint calibration split and combined via an empirical variant of Brown's method (moment-matched scaled chi-squared) to handle inter-layer dependence, yielding a per-class calibrated threshold at FPR α = 0.05. The central technical contribution is exact additive decomposability: because the test statistic is built from Mahalanobis distances — a quadratic form in the concept vector — it splits identically into per-layer and per-concept contributions, and at convolutional layers further into exact per-position contributions, producing "concept maps" that localize which input regions drove the verdict. The explanation is not a post-hoc surrogate; it is the detector. All of this incurs no forward or backward passes beyond the one needed for the prediction. Validation. Four datasets (CIFAR-10, GTSRB, Tiny ImageNet, CelebA) × two architectures (ResNet18, VGG16 — chosen to contrast sequential vs. residual topologies) × five attacks (BadNets, Blended, IAD, WaNet, physical eyeglasses), against the two SOTA single-pass inference-time detectors Beatrix and Orion, over five seeds. VARDE is competitive or better on patch/blended/physical attacks and clearly strongest on input-aware (IAD) attacks (mean TPR@5% 85.1 vs. Beatrix 35.8, Orion 56.6), but degrades sharply on the imperceptible warping attack WaNet under ResNet18 (9.2% TPR on CIFAR-10 vs. Orion's 85.5%). A blur-perturbation study shows concept maps localize triggers more faithfully than Grad-CAM and a random baseline. A controlled Mann–Whitney analysis (Section 4.5) tests whether the top attributed concepts actually correlate with the trigger, using a random-concept control and a core-clean control: the correlation holds for BadNets/Blended/physical triggers, is ambiguous for IAD (the clean control separates too), and fails for WaNet in the settings where detection is weakest. An appendix ablation (Table 4) shows that scoring raw activations instead of SAE concepts costs 0.19 TPR on IAD and 0.08 on WaNet, while patch triggers are unaffected.

Critical strengths

1 — Interpretability by construction with exact (non-approximate) attribution. The verdict, its per-layer/per-concept breakdown, and its spatial concept map are all identities of the detector's own statistic, computed in the same forward pass as the prediction. This eliminates the fidelity gap that fundamentally limits Grad-CAM-based explanations of detector decisions (which attribute the classifier's prediction, not the detector's verdict, via a backward pass). This is consequential: it makes detector verdicts auditable and contestable in a way that no prior inference-time backdoor defense supports, and the same machinery transfers conceptually to other anomaly-detection settings. The strongest empirical support is the blur-perturbation study (Fig. 2/10): concept-map-guided perturbation recovers the true label from smaller image fractions than Grad-CAM across attacks.

2 — Unusually rigorous, self-critical evaluation methodology. The paper does three things defense papers routinely skip: (i) a controlled Mann–Whitney analysis asking whether the attributed concepts correlate with the trigger, against both a random-concept control and a core-clean control — and reports honestly when the answer is negative (IAD, WaNet); (ii) a raw-activation vs. SAE-concept ablation isolating the SAE's contribution (helps precisely for dynamic/imperceptible triggers, unnecessary for patch triggers); (iii) prominent reporting of failure modes (WaNet on ResNet18) rather than burying them in averages. The baseline treatment (Beatrix, Orion re-run under matched clean-data budgets and FPR calibration) is fair and transparent about comparability limits. This level of experimental hygiene materially raises my confidence in the reported conclusions.

Critical weaknesses

1 — The trigger-alignment analysis does not test all signals the detector uses. Section 3.5 permits a positive anomaly contribution from a decrease in concept activation, whereas Section 4.5’s one-sided statistic counts only increases. Figure 3 therefore cannot support its stated interpretation without a signed or attribution-aware analysis. The choice of clean comparison inputs also needs clarification to rule out source-versus-target class differences.

2 — Detection performance is highly setting-dependent. VARDE trails Orion on average for BadNets, WaNet, and the physical attack (Table 1). On WaNet it obtains 9.2% versus Orion’s 85.5% TPR for ResNet/CIFAR-10; the weakness is not confined to ResNet, since VGG/Tiny ImageNet yields 3.2% versus 26.1% (Table 5). This limits the breadth of the “competitive” claim.

Questions for authors
  1. Can you repeat Section 4.5 with an analysis that captures both increased and decreased concept activation, preferably comparing each triggered image with its untriggered counterpart? Which clean population is used in the current Mann–Whitney comparisons?
  2. What explains the WaNet failures across architectures, especially on Tiny ImageNet? Do any individual layers retain discriminative signal that the combined score loses?
  3. What are VARDE’s inference latency, memory use, and SAE fitting costs relative to Beatrix and Orion? These measurements matter to the single-pass efficiency claim.
  4. Under what attacker-knowledge assumptions is the method intended to operate? If the attacker knows the detector, how does detection change when trigger generation also minimizes VARDE’s score?
2 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for the thoughtful feedback and valuable suggestions. We have carefully considered each point and will make corresponding revisions to the manuscript (changes are in blue).

On the two-sided trigger-alignment analysis and the clean population

We repeated Section 4.5 with a two-sided score, which now replaces the one-sided score in Section 4.5 and Figure 3. The clean population is the same poisoned images without the trigger, so a difference between source and target classes cannot produce the separation. Section 4.5 now states this. The conclusions are unchanged. For BadNets, Blended and the physical trigger, the attributed concepts separate triggered from clean inputs and the clean control stays well below. For IAD, the clean control separates as well, so the attributed concepts are already present in the clean image. For WaNet, where the one-sided score showed no separation, the two-sided score shows none either, so the one-sided score did not hide a suppression effect.

On the WaNet failures and individual layers

Most of the gap comes from how VARDE calibrates its per-class thresholds (new Appendix C). Each class's threshold is calibrated on clean inputs grouped by predicted class. The WaNet models misclassify many clean inputs into the target class: in the settings where VARDE fails, 9 to 55 times as many as into an average other class, against about 3 times where VARDE detects WaNet (Figure 7). These inputs enter the calibration set of the target class and raise its threshold, and since VARDE scores triggered inputs like them, triggered inputs are not flagged. Orion uses a single threshold for all classes and is not affected.

To test this explanation, we group the calibration inputs by true label and change nothing else.

Arch.DatasetAll (Pred.)All (True)Last third (Pred.)Last third (True)Orion
ResNet18CIFAR-100.13±0.030.22±0.030.14±0.020.24±0.040.86±0.02
GTSRB0.38±0.080.80±0.060.59±0.180.98±0.011.00±0.00
TinyImageNet0.02±0.010.21±0.060.02±0.000.09±0.070.19±0.02
VGG16CIFAR-100.94±0.040.94±0.050.85±0.130.83±0.150.80±0.10
GTSRB1.00±0.001.00±0.001.00±0.001.00±0.000.99±0.01
TinyImageNet0.04±0.000.35±0.090.05±0.020.24±0.070.26±0.03

WaNet TPR at 5% FPR, calibration grouped by predicted (Pred.) or true (True) label.

Detection rises where VARDE failed, least on ResNet18 CIFAR-10, where triggered inputs also lie close to the target-class profile (Appendix C), which we state as a remaining limitation. On TinyImageNet, the models' high clean error puts misclassified inputs into every calibration subset, not only that of the target class.

Individual layers do retain signal that the combined score loses. In all four failing settings, some layers, mostly deep ones, separate triggered WaNet inputs from clean ones better than the combined score (Figure 9), for example with an AUROC of 0.97 against 0.89 on ResNet18 GTSRB. No layer subset is best in every setting (Table 4). Choosing a subset from WaNet results would tune the defence on the attack it is evaluated against, so the main results keep all layers.

AuthorsOpenReview ↗

Response (2/2)

On latency, memory and fitting cost

The new Appendix D reports the cost for all settings and all three detectors. We report operation counts (FLOPs) instead of wall-clock latency, because our three implementations are not equally optimized. Per query, VARDE adds 43% to 59% of the victim's forward pass, nearly all of it for the SAE encoders, while Beatrix adds at most 5% and Orion less than 1%. VARDE stores 18 to 155 MB, against 4 to 421 MB for Beatrix and 0.2 to 22 MB for Orion. Fitting VARDE is a one-off cost of 2.7% to 47% of training the victim, dominated by SAE training. VARDE thus needs no second forward or backward pass through the model, at a cost of about 1.5 forward passes per query, but it is not the cheapest of the three, which Section 5 now states.

On the threat model and adaptive attacks

Section 3.1 now states the attacker's capabilities and knowledge. The attacker controls the training data and procedure, but must deliver a standard model, so the backdoor cannot be placed in the architecture or the forward-pass code. In the main experiments, the attacker does not know the defence, as in the evaluations of Beatrix and Orion.

We added an adaptive attack (new Appendix B). The attacker knows VARDE in full but has no access to the defender's SAEs, class profiles or clean data. It trains the trigger jointly with the model and penalizes the distance between triggered inputs and the clean target class under a surrogate of VARDE built from its own data, so the trigger preserves the target class's concept distribution. The attack succeeds on 95% to 100% of triggered inputs, with clean accuracy close to that of the other attacks (Table 2). With calibration by true label, since the jointly learned trigger misroutes clean inputs as WaNet does, VARDE detects 53.4% of triggered inputs at 5% FPR, comparable to Orion's 55.0%, while Beatrix detects 6.0%, so the attack partly evades VARDE, as it does Orion. As a control, we train the same attack without the penalty (λ=0\lambda = 0, new Table 3): the penalty substantially lowers VARDE's detection (TPR at 5% FPR), while Orion's changes much less except on VGG16 TinyImageNet, where it lowers both.

Arch.DatasetVARDE (λ=0\lambda=0)VARDE (λ=1\lambda=1)Orion (λ=0\lambda=0)Orion (λ=1\lambda=1)
ResNet18CIFAR-100.66±0.150.29±0.090.36±0.130.47±0.10
GTSRB1.00±0.010.76±0.060.99±0.020.87±0.14
TinyImageNet0.86±0.120.59±0.330.63±0.190.46±0.24
VGG16CIFAR-100.94±0.030.15±0.020.26±0.100.19±0.09
GTSRB1.00±0.000.85±0.190.99±0.010.95±0.07
TinyImageNet0.95±0.070.57±0.170.76±0.100.37±0.13
Reviewer_65wPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces VARDE, an interpretable, inference-stage backdoor detection method designed to clarify why an input sample is classified as backdoored. The method employs sparse autoencoders (SAEs) to transform activation values ​​from the model’s intermediate layers into sparse conceptual representations, and it identifies suspicious inputs by aggregating Mahalanobis distances across these layers. By decomposing detection statistics into the contributions of individual concepts, VARDE provides concept-level explanations and spatial attribution maps without requiring additional forward or backward passes. Experiments conducted across various datasets, model architectures, and backdoor attack scenarios demonstrate that VARDE’s detection performance is highly competitive with existing methods.

Critical strengths
  • The paper addresses an important limitation of existing backdoor detectors, which typically provide anomaly scores without explaining why an input is flagged as poisoned.

  • The presentation is easy to follow.

Critical weaknesses
  • Although the paper claims to provide human-interpretable and concept-level explanations, the semantic meaning of SAE features is mainly inferred through visual inspection of top-activating examples. There is no rigorous quantitative evaluation of concept interpretability or monosemanticity.

  • While VARDE performs well on several attacks, its performance is considerably worse than Orion on WaNet in a few cases in Table 1.

  • Additional computational cost of training a separate SAE for each layer and estimating class-conditional distributions using clean reference data.

Questions for authors

Although the proposed method provides precise, concept-level attributions for detection scores, it remains unclear whether these explanations faithfully reflect the underlying backdoor mechanism. In contrast, trigger-inversion methods, such as Neural Cleanse (NC), offer a more direct and actionable form of interpretability by recovering the input-space perturbations (i.e., triggers) that induce the attacker’s target predictions. Such reconstructed triggers can be empirically validated via attack success rates, thereby establishing a causal link between the explanation (the reconstructed trigger) and the backdoor behavior. Conversely, VARDE primarily identifies concepts and image regions that contribute to anomaly scores. These elements may not necessarily be the critical features that actually trigger the backdoor. While the two approaches operate under different settings and computational constraints, the paper could be further strengthened by discussing this distinction and incorporating an intervention-based evaluation (demonstrating a causal relationship between the identified concepts and the backdoor behavior), rather than merely a correlation with anomalous representations.

3 comments

We thank the reviewer for the detailed feedback. We have addressed the raised concerns and strengthened the manuscript accordingly, incorporating the reviewer's suggestions (changes marked in blue).

On faithfulness, trigger inversion and concept interpretability

We agree that the explanation should be validated causally, as a reconstructed trigger is validated by its attack success rate. We added an intervention experiment (new Section 4.6). We write the concepts that VARDE ranks highest into clean inputs of other classes, in concept space, and count how many of these inputs the model then predicts as the target class. For BadNets, Blended and IAD, most inputs flip when well below 1% of the concepts are used. Random concepts need roughly a hundred times as many, and concepts typical of the target class roughly ten times as many. The attributed concepts are therefore sufficient to trigger the backdoor, which is the causal link the reviewer asks for. The experiment also evaluates the concepts empirically, by what they do in the model rather than by what their top-activating examples look like. We do not score interpretability or monosemanticity with a human study or an automated metric, which Section 5 now states as a limitation. Section 2 now also contrasts VARDE with trigger inversion (Wang et al., 2019): a reconstructed trigger explains the model, whereas VARDE explains the verdict on an individual input.

On VARDE's performance on WaNet

Most of the gap comes from how VARDE calibrates its per-class thresholds (new Appendix C). The WaNet models misclassify many clean inputs into the target class, and since thresholds are calibrated on clean inputs grouped by predicted class, these raise the target class's threshold and triggered inputs go unflagged. Calibrating by true label instead, with nothing else changed, confirms this:

Arch.DatasetVARDE (Pred.)VARDE (True)Orion
ResNet18CIFAR-100.13±0.030.22±0.030.86±0.02
GTSRB0.38±0.080.80±0.061.00±0.00
TinyImageNet0.02±0.010.21±0.060.19±0.02
VGG16CIFAR-100.94±0.040.94±0.050.80±0.10
GTSRB1.00±0.001.00±0.000.99±0.01
TinyImageNet0.04±0.000.35±0.090.26±0.03

WaNet TPR at 5% FPR, calibration grouped by predicted (Pred.) or true (True) label.

Detection rises where VARDE failed, least on ResNet18 CIFAR-10, where triggered inputs also lie close to the target-class profile (Appendix C), which we state as a remaining limitation.

On computational cost

The new Appendix D quantifies the cost. Fitting is dominated by SAE training, a one-off cost of 2.7% (ResNet18 TinyImageNet) to 47% (ResNet18 CIFAR-10) of training the victim model, and estimating the class profiles and thresholds is negligible next to it. All of the detectors use the same clean data. Per query, VARDE adds about half a forward pass, nearly all of it for the SAE encoders, while Beatrix adds at most 5% and Orion less than 1%. VARDE thus costjs about 1.5 forward passes per query, but it is not the cheapest of the three, which Section 5 now states. The explanation itself adds only a few percent of a forward pass.

Hi,

Thanks for your rebuttal and additional intervention experiments. I am still concerned about whether the resulting spatial maps can faithfully localize the actual backdoor triggers. VARDE derives its concept maps from the spatial contributions of SAE features to the class-conditional anomaly score. However, the spatial locations contributing most to the representation deviation do not necessarily correspond to the actual support of the injected trigger. This distinction is particularly important for recent imperceptible backdoor attacks such as Narcissus [1] and Grond [2], which employ small-magnitude perturbations distributed across the entire input image rather than localized patches. Also, clean-trained models could be misled (just like backdoor behaviours) by naturally occurring features [3]. In these cases, the trigger does not have a well-defined local region that can be directly extracted and transplanted.

Therefore, I'd like to keep my current rating.

[1]Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information. CCS 2023.

[2] Towards Backdoor Stealthiness in Model Parameter Space. CCS 2025.

[3] Rethinking Backdoor Attacks. ICML 2023.

We thank the reviewer for the continued engagement. We agree that the claim about the spatial maps should be stated more precisely, and we will revise the manuscript accordingly. However, we would like to argue that the experiments already address the remaining concerns regarding full-image triggers, as we attempt to clarify in the following. We would also like to point out that the concern regarding naturally occurring features is shared by all defence methods.

The concept map shows where the model reads the trigger, which is not necessarily where the trigger is written in the input. For local triggers such as BadNets, the two coincide: blurring less than 10% of the image, guided by the map, returns half of the predictions to their original class (see Figure 2 in the manuscript). For global triggers, which perturb the whole image, the two need not coincide. For Blended, the contribution concentrates in a subset of the image, and blurring that subset alone removes the backdoor, so the model does not use the full injected pattern (Section 4.3). Here, the map departs from the support of the injected trigger and correctly identifies the part of the trigger the model reads. For WaNet, whose imperceptible warp covers the whole image, random perturbations are as effective as concept-guided perturbations (Section 4.4), as expected when the model reads the trigger across the whole input. We therefore already evaluate full-image triggers, which is precisely what Narcissus [1] and Grond [2] use.

Importantly, our method's detection does not rely on the concept map, which follows from the same decomposition and improves its interpretability, but is not needed for the verdict. The intervention experiment (Section 4.6) extracts the attributed concepts and transplants them into clean inputs of other classes in concept space, which reproduces the backdoor behaviour. The concepts are not tied to a location in the input, so the test requires no local trigger region and applies directly to triggers distributed over the whole image. This goes beyond input-space explanations, such as Grad-CAM saliency or the triggers reconstructed by Neural Cleanse.

Regarding naturally occurring features, this is among the attack surfaces we investigate. Specifically, our physical attack uses eyeglasses on CelebA, a naturally occurring feature, as the trigger. Eyeglasses are not an unusual feature, but one unusual for the predicted class (Section 4.3). Khaddaj et al. [3] show that, without structural assumptions, backdoor triggers cannot in general be distinguished from natural features, so every detector must necessarily rest on some assumption. VARDE's assumption is that a backdoor induces an anomalous class-conditional deviation in SAE concept space, and a natural shortcut with the same signature would also be flagged. For an explanation method, we consider this correct behaviour, because the maps report what the model uses, not the intent behind it. For detection, it is a limitation that VARDE shares with every method, as shown in [3].

For clarification purposes, we will rephrase the abstract, the title of Section 4.4 and the conclusion to state that the concept maps localise the representational evidence for the anomaly, not necessarily the trigger itself. Section 4.4 and the future work in Section 5 already note this. In Section 5, we will also state that VARDE does not claim to separate malicious from natural features, and that our attack set does not include clean-label attacks such as Narcissus [1] and Grond [2].

We respect the decision to keep the rating, and we thank the reviewer for the points that make the paper's claims more precise.

[1] Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information. CCS 2023.

[2] Towards Backdoor Stealthiness in Model Parameter Space. CCS 2025.

[3] Rethinking Backdoor Attacks. ICML 2023.

Reviewer_XPewOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VARDE, an inference-time backdoor detector that uses sparse autoencoders to identify suspicious concepts in a model's internal representations. The method explains its detection decisions through concept-level contributions and spatial maps while achieving competitive detection performance against existing defenses.

Critical strengths
  • The method directly explains which concepts contribute to a detection decision, making its explanations more closely tied to the detector than traditional post-hoc methods.

  • The experiments cover several backdoor attacks across different datasets and architectures, with particularly strong detection results against input-aware attacks.

Critical weaknesses
  • The detection performance is highly inconsistent across architectures and attacks. For example, VARDE detects only 9.2 percent of WaNet attacks on CIFAR-10 with ResNet18, while Orion detects 85.5 percent, raising serious concerns about its reliability as a security defense.

  • The paper claims that its explanations are interpretable, but the experiments mainly show that certain concepts correlate with poisoned inputs or highlight suspicious regions. These results do not establish that the identified concepts actually represent the backdoor mechanism rather than unrelated features that happen to correlate with the trigger.

  • The evaluation considers only fixed attacks without testing whether an attacker can deliberately evade the proposed detector. Since the method relies on deviations from clean concept distributions, an adaptive attacker could potentially design triggers that preserve these distributions while still causing misclassification.

Questions for authors
  • Can the authors explain why VARDE performs substantially worse than Orion against WaNet on ResNet18 and provide evidence identifying the cause?

  • Can the authors provide controlled experiments showing that the highest-ranked concepts causally contribute to backdoor activation rather than merely correlate with poisoned inputs?

  • Can the authors evaluate whether an adaptive attacker who knows VARDE's detection mechanism can create effective backdoors that evade its concept-based anomaly detection?

1 comment

We thank the reviewer for the constructive feedback. We have addressed the raised concerns and will revise the manuscript accordingly (changes marked in blue).

On why VARDE performs worse than Orion on WaNet

Most of the gap comes from how VARDE calibrates its per-class thresholds (new Appendix C). Each class's threshold is calibrated on clean inputs grouped by predicted class. The WaNet models misclassify many clean inputs into the target class: in the settings where VARDE fails, 9 to 55 times as many as into an average other class, against about 3 times where VARDE detects WaNet (Figure 7). These inputs enter the calibration set of the target class and raise its threshold, and since VARDE scores triggered inputs like them, triggered inputs are not flagged. Orion uses a single threshold for all classes and is not affected.

To test this explanation, we group the calibration inputs by true label and change nothing else.

Arch.DatasetVARDE (Pred.)VARDE (True)Orion
ResNet18CIFAR-100.13±0.030.22±0.030.86±0.02
GTSRB0.38±0.080.80±0.061.00±0.00
TinyImageNet0.02±0.010.21±0.060.19±0.02
VGG16CIFAR-100.94±0.040.94±0.050.80±0.10
GTSRB1.00±0.001.00±0.000.99±0.01
TinyImageNet0.04±0.000.35±0.090.26±0.03

WaNet TPR at 5% FPR, calibration grouped by predicted (Pred.) or true (True) label.

Detection rises where VARDE failed, least on ResNet18 CIFAR-10, where triggered inputs also lie close to the target-class profile (Appendix C), a remaining limitation.

Part of the remaining gap comes from combining all layers. Some deep layers separate triggered WaNet inputs from clean ones better than the combined score (Figure 9), and combining only the last third of the layers raises detection on ResNet18 GTSRB from 37.5% to 58.5%. No layer subset is best in every setting, so the main results keep all layers (Table 4).

On causal evidence for the attributed concepts

We added an intervention experiment (new Section 4.6). We write the concepts that VARDE ranks highest into clean inputs of other classes, in concept space, and count how many of these inputs the model then predicts as the target class. For BadNets, Blended and IAD, most inputs flip when well below 1% of the concepts are used. Random concepts need roughly a hundred times as many, and concepts typical of the target class roughly ten times as many (Figure 4). The attributed concepts are therefore sufficient to trigger the backdoor, and not merely because they belong to the target class.

On adaptive attacks

We added an adaptive attack (new Appendix B). The attacker knows VARDE in full but has no access to the defender's SAEs, class profiles or clean data. It trains the trigger jointly with the model and penalizes the distance between triggered inputs and the clean target class under a surrogate of VARDE built from its own data, so the trigger preserves the target class's concept distribution. The attack succeeds on 95% to 100% of triggered inputs, with clean accuracy close to that of the other attacks (Table 2). With calibration by true label, since the jointly learned trigger misroutes clean inputs as WaNet does, VARDE detects 53.4% of triggered inputs at 5% FPR, comparable to Orion's 55.0%, while Beatrix detects 6.0%, so the attack partly evades VARDE and Orion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: correction to Tables 1 and 8

Correction to Tables 1 and 8 (all reviewers)

While re-running experiments for this response, we found an error in the VARDE columns of Tables 1 and 8 (revised manuscript). Each VARDE entry averaged the method with its ablation variants on the same victim models (Tables 6 and 7). The corrected entries report the method alone. Since the reported estimator is the strongest of the variants, most entries rise. The correction does not resolve any of the reviewers' concerns. The WaNet results, cited in the reviews, change from 9.2% to 13.1% (ResNet18 CIFAR-10) and from 3.2% to 3.5% (VGG16 TinyImageNet), still below Orion. We address these concerns in the responses. The table gives every VARDE entry of Table 8 as published → corrected (mean ± standard deviation over the five victim models).

Arch.DatasetAttackTPR at 5% FPR (%)AUROC
ResNet18CIFAR-10BadNets100.0±0.0 → 100.0±0.01.000±0.001 → 0.999±0.001
Blended96.8±1.6 → 97.0±0.90.984±0.004 → 0.983±0.005
IAD97.8±7.5 → 99.8±0.50.994±0.012 → 0.997±0.003
WaNet9.2±3.1 → 13.1±2.70.578±0.060 → 0.660±0.017
Clean5.5±0.8 → 5.8±0.90.498±0.014 → 0.499±0.015
GTSRBBadNets99.7±1.0 → 99.9±0.10.996±0.005 → 0.997±0.002
Blended100.0±0.0 → 100.0±0.00.998±0.002 → 0.999±0.001
IAD84.8±30.4 → 84.8±34.00.949±0.100 → 0.954±0.100
WaNet20.4±8.5 → 37.5±8.00.734±0.074 → 0.887±0.024
Clean5.0±1.4 → 4.9±1.50.509±0.011 → 0.508±0.012
TinyImageNetBadNets35.7±20.5 → 53.2±11.90.899±0.030 → 0.919±0.022
Blended27.2±13.1 → 32.7±12.70.866±0.048 → 0.869±0.043
IAD53.6±42.4 → 93.4±6.60.907±0.092 → 0.987±0.009
WaNet2.6±0.8 → 2.2±0.80.439±0.022 → 0.421±0.015
Clean4.9±0.6 → 4.7±0.40.493±0.007 → 0.493±0.007
CelebAPhysical96.8±2.1 → 97.9±1.10.987±0.004 → 0.989±0.003
VGG16CIFAR-10BadNets100.0±0.0 → 100.0±0.00.998±0.002 → 0.995±0.002
Blended97.6±2.1 → 98.4±1.00.987±0.005 → 0.988±0.004
IAD99.9±0.5 → 99.8±0.40.997±0.003 → 0.997±0.003
WaNet85.0±13.9 → 93.9±4.40.967±0.032 → 0.985±0.010
Clean5.4±0.7 → 5.6±0.60.506±0.017 → 0.509±0.015
GTSRBBadNets100.0±0.0 → 100.0±0.00.998±0.003 → 0.998±0.001
Blended100.0±0.0 → 100.0±0.00.999±0.001 → 1.000±0.000
IAD90.6±15.4 → 91.5±14.40.983±0.025 → 0.984±0.023
WaNet98.9±2.9 → 100.0±0.00.994±0.006 → 0.997±0.002
Clean5.1±0.7 → 4.9±0.70.506±0.009 → 0.505±0.009
TinyImageNetBadNets65.9±14.4 → 74.3±13.20.952±0.018 → 0.961±0.019
Blended31.8±18.5 → 47.6±11.30.901±0.033 → 0.927±0.017
IAD84.1±19.8 → 98.9±2.20.975±0.026 → 0.995±0.007
WaNet3.2±0.6 → 3.5±0.40.545±0.051 → 0.565±0.054
Clean4.7±0.5 → 4.9±0.60.499±0.007 → 0.502±0.006
CelebAPhysical93.8±2.3 → 93.9±2.80.981±0.002 → 0.982±0.003

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.