acceptodds
Under review as a conference paper at ICLR 2027

BackX: A Backdoor-Based XAI Benchmark for High-Fidelity Attribution Evaluation

Abstract

Attribution methods compute importance scores for input features to explain model predictions. However, assessing the faithfulness of these methods remains challenging due to the absence of attribution ground truth to model predictions. In this work, we first identify a set of fidelity criteria that reliable benchmarks for attribution methods are expected to fulfill, thereby facilitating a systematic assessment of attribution benchmarks. Next, we introduce a Backdoor-based eXplainable AI benchmark (BackX) that adheres to the desired fidelity criteria. We theoretically establish the superiority of our approach over the existing benchmarks for well-founded attribution evaluation. With extensive analysis, we further establish a standardized evaluation setup that mitigates confounding factors such as post-processing techniques and explained predictions, thereby ensuring a fair and consistent benchmarking. This setup is employed for a comprehensive comparison of existing methods using BackX. Finally, our analysis also offers insights into defending against neural Trojans by utilizing the attributions.

Primary area:
datasets and benchmarks
Keywords:
Explainable AI, Attribution Methods, Benchmarking, Evaluation Fidelity, Backdoor Attacks

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_DXysOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BackX evaluates attribution methods by planting a backdoor and treating the trigger as a known cause. Trigger recall measures how much of the trigger the top-ranked attribution pixels cover. A second test replaces those pixels with pixels from the clean image and checks whether the attack success rate falls. The paper states four fidelity criteria, scores prior benchmarks on them, and gives BackX 3.5 against 2.5 for the existing neural-Trojan benchmark and 3.0 for Null Feature / CLEVR-XAI. It then compares nine attribution methods on CIFAR-10, GTSRB, and ImageNet under Blend and ISSBA, and argues for one standardized setup of post-processing and output type.

Critical strengths

The paper is easy to follow. Figure 2 states the five-step pipeline, and the criteria in Section 2 give the reader a checklist before the experiments. The setup study is also informative: absolute value versus signed scores, and logits versus probabilities, change results for the same method. The visibility result is concrete. Making the trigger more visible does not improve trigger recall, and the poisoning rate has little effect.

Critical weaknesses

The benchmark’s central move is already published. Lin, Lee, and Celik, “What do you see? Evaluation of explainable artificial intelligence interpretability through neural backdoors” (KDD 2021), use a neural Trojan as attribution ground truth. BackX’s own Table 1 gives that line 2.5 and itself 3.5. The gap is produced by the authors’ four criteria. On input-distribution invariance, BackX marks itself only partial: recovering an image by swapping in clean pixels still changes the input. The explained model is the Trojaned model. A public preprint of this work has been available since May 2024 (arXiv:2405.02344, v2 in October 2025) and still leads with the claim of being the first fair setup. Relative to KDD 2021, the new pieces are the checklist, a watermark Blend trigger, and a sweep of setup choices. That is not enough for a new benchmark at this venue.

The experiments do not lock the setup that the third contribution depends on. The body explains probabilities with CAM methods and logits with gradient and integration methods, and takes absolute values for gradient methods. The appendix instead explains probabilities with integration methods. The visualization captions use logits for CAM and integration methods, and absolute probabilities for gradient methods. ISSBA on ImageNet is dropped because random attributions can already recover the attack, so a drop in attack success rate on that setting does not show that the map found the trigger. There is no table that ranks the same methods under BackX and under Lin et al. (2021). Gradient and integration methods beat CAM, and which of those two wins depends on the dataset. That is a useful sensitivity study of known explainers on Trojaned models.

Questions for authors

Please point to one protocol sentence that matches the body, the appendix, and the figure captions at the same time. On the methods you share with Lin et al. (2021), which ranking changes under BackX, and which of their ground-truth failures does BackX correct? I would need both answers in the main text before this was more than an extension of that KDD paper.

4 comments
AuthorsOpenReview ↗

Response to Reviewer DXys (Part 1/2)

We sincerely thank the reviewer for the close reading and constructive feedback. We address each concern in turn below and hope our responses clarify our work and adequately address the reviewer's questions and concerns. The manuscript has been revised to include the supporting experimental results and explanations, with updates marked in blue.

W1. The contribution of BackX beyond NeuronTrojan.

BackX contributes a fidelity framework supported by theoretical guarantees and controlled validation, alongside standardized benchmarking, multimodal coverage, and new attribution insights. Specifically, BackX’s contributions fall into two main areas. 1. Fidelity framework and theoretically grounded benchmark design. BackX introduces four fidelity criteria and connects benchmark design to explicit theoretical guarantees. Its analysis bounds input shift and the effect of mask leakage, while watermark training, clean recovery of selected pixels, and sensitive metrics address the corresponding sources of evaluation error. 2. Standardized benchmarking, multimodal coverage, and attribution insights. BackX standardizes attribution post-processing and explained outputs, reducing confounding effects in method comparisons. The benchmark covers 15 attribution methods and 5 attack families, with evaluation spanning both vision and language. These systematic comparisons reveal how spatial resolution, smoothing, and reference and path construction affect attribution performance across evaluation settings. BackX thus provides a consistent basis for method comparison and broader insights into attribution behavior across attacks and modalities.

We further validated our design using a CNN with one convolutional layer and predefined weights, making pixel contributions to the target logit analytically computable. Oracle attribution masks with equal pixel budgets remove 20%, 50%, and 80% of the total target logit contribution. As shown in the table below, NeuronTrojan assigns identical IoU and original class recovery rate (RR) scores, whereas BackX logit and probability reductions distinguish all three levels correctly. This controlled comparison provides direct evidence of improved metric fidelity against analytically known contributions. These results show that improved fidelity comes from the evaluation design of BackX, rather than through the backdoor mechanism alone. We have added this experiment and its setup details to Section 4 of the revised manuscript.

Table R1. Evaluation scores for oracle explanations with known contributions.

Evaluation metric20% contribution removed50% contribution removed80% contribution removed
Ground truth (%)20.00050.00080.000
NeuronTrojan IoU (%)25.00025.00025.000
NeuronTrojan RR (%)100.000100.000100.000
BackX logit reduction (%)20.00050.00080.000
BackX probability reduction (%)37.95681.74597.039
AuthorsOpenReview ↗

Response to Reviewer DXys (Part 2/2)

W2&Q1. Evaluation consistency and comparison with NeuronTrojan.

We thank the reviewer for pointing out these typos, which we have corrected. The main text, appendices, and figure captions now consistently describe the unified attribution setup used throughout our experiments. Table R2 provides details of the explained output and post-processing used for each method.

Table R2. Unified attribution settings.

Attribution methodsExplained outputPost-processing
GCAM, FGOutput probabilityOriginal values
Grad, GGCAM, SGOutput logitAbsolute values
IG, IG-Uni., AGI, LPIOutput logitOriginal values

To further isolate the contribution of the individual designs in BackX, we compared all nine attribution methods under BackX and individual component ablations, alongside NeuronTrojan. Each ablation replaces one BackX component with the corresponding NeuronTrojan design: fully visible triggers during training (α=1), class recovery rate (RR) scoring, or bounding-box recovery (BBox). As shown in Table R3, BBox recovery causes the largest individual ranking shift (τ=0.167), crediting explanations for trigger pixels outside their selections. RR scoring also changes the ordering (τ=0.778), while training with α=1 yields τ=0.833. NeuronTrojan yields τ=0.056 across nine methods and τ=0.048 for the seven shared methods, excluding Guided Grad-CAM and LPI. These comparisons highlight the contribution of BackX to benchmark fidelity. This ablation analysis and its interpretation have been incorporated into Section 8 of the revised manuscript.

Table R3. Component ablation analysis of BackX and NeuronTrojan. Parentheses show changes relative to BackX.

Attribution methodBackXα=1RR scoringBBox recoveryNeuronTrojan
Grad-CAM9 (-0)9 (-0)9 (-0)2 (-7)1 (-8)
FullGrad7 (-0)8 (+1)8 (+1)6 (-1)6 (-1)
Guided Grad-CAM3 (-0)2 (-1)4 (+1)1 (-2)2 (-1)
AGI6 (-0)6 (-0)6 (-0)5 (-1)4 (-2)
SmoothGrad1 (-0)1 (-0)1 (-0)3 (+2)3 (+2)
LPI5 (-0)4 (-1)2 (-3)4 (-1)5 (-0)
Integrated Gradients8 (-0)7 (-1)7 (-1)9 (+1)9 (+1)
IG-Uniform2 (-0)3 (+1)3 (+1)7 (+5)7 (+5)
Input Gradients4 (-0)5 (+1)5 (+1)8 (+4)8 (+4)
Kendall’s τ vs BackX1.0000.8330.7780.1670.056
AuthorsOpenReview ↗

Looking forward to further feedback

We thank the reviewer again for the careful review and constructive feedback. With the discussion period nearing its end, we would be grateful if the reviewer could take a look at our responses and let us know whether the concerns have been addressed. This would allow us to address any remaining concerns before the discussion closes.

For convenience, we briefly summarize the main additions already reported in our responses and incorporated into the revised manuscript.

  • Controlled fidelity validation (W1). We conducted an additional controlled oracle experiment to directly validate BackX’s fidelity improvements over NeuronTrojan. With analytically known feature contributions, BackX correctly distinguishes explanations with different true contributions that NeuronTrojan’s metrics score identically.
    Manuscript updates: The new experiment and its analysis are now included in Section 4 (Figure 3).

  • Head-to-head comparison and component ablations (W2 & Q1). We conducted additional head-to-head comparisons and component ablations across nine attribution methods. The results reveal substantial ranking discrepancies between NeuronTrojan and BackX and expose evaluation biases that undermine the reliability of NeuronTrojan’s method comparisons. BackX addresses these fidelity limitations to provide more reliable attribution benchmarking.
    Manuscript updates: The additional comparisons and ablation analyses are now included in Section 8 (Figure 10).

  • Unified evaluation protocol (W2 & Q1). We corrected the typos concerning explained outputs and attribution post-processing.
    Manuscript updates: The main text, appendices, and figure captions now consistently describe these unified settings.

Our earlier responses and the revised manuscript provide the supporting results and detailed explanations. We hope these additions and clarifications address the main concerns, and we would be very grateful if the reviewer would consider updating the rating in light of these additions.

Reviewer_DXysOpenReview ↗

Reply

Thanks, But I will remain my score.

Reviewer_sMnCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes BackX, a backdoor-based framework for evaluating feature attribution methods, guided by four key criteria: functional mapping invariance, input distribution invariance, attribution verifiability, and metric sensitivity. The framework evaluates explanations of a backdoored model’s predictions by replacing highly attributed features in poisoned inputs with their clean counterparts. Attribution quality is then assessed using attack success rate, trigger recall, and fractional logit change. Experiments analyze how attribution post-processing and the choice of model output to explain affect attribution performance, and compare attribution methods across both visual and textual domains.

Critical strengths

The paper provides a principled framework for attribution evaluation. The paper organizes benchmark design around four explicit criteria: functional mapping invariance, input distribution invariance, attribution verifiability, and metric sensitivity. These criteria provide a useful framework for identifying limitations of existing benchmarks and motivating the design of BackX.

It also provides useful empirical analysis of evaluation choices. The paper investigates how attribution post-processing and the choice of explained output affect evaluation results. The experiments also cover multiple attribution families and both visual and textual domains, which provides practical insights into their strengths and limitations.

Critical weaknesses

The choice of backdoor attack may confound the assessment of attribution quality. ASR not only depends on whether the attribution identifies relevant features, but also on how easily the backdoor attack can be neutralized. Consequently, ASR differences across attack settings cannot be attributed solely to explanation quality. Although comparisons under a fixed attack setting remain meaningful, the resulting rankings may reflect capabilities specific to that attack rather than general attribution fidelity. The paper should clarify this distinction and examine how consistently its conclusions hold across different attack mechanisms.

Also, the effect of attack choice on the full benchmark is not fully explored. The main text evaluates Blend and ISSBA, and Appendix L.1 extends the comparison to additional attacks from a backdoor-defense perspective. However, the main text provides uneven metric coverage across attacks: ASR and fractional logit change are reported for both Blend and ISSBA, whereas trigger recall is reported only for Blend. The extended analysis only reports ASR at a fixed recovery rate, without the trigger recall or fractional logit change results. It therefore remains unclear how attack choice affects other metrics and whether they produce consistent rankings of attribution methods. A more systematic comparison across attack methods would strengthen the benchmark's conclusions.

Questions for authors
  1. How consistent are the rankings of attribution methods across different attacks? How do the proposed metrics, attack success rate, trigger recall, and fractional logit change agree in their rankings under each attack?
  2. Given the differences observed across attacks, how should attack methods be selected or combined?
3 comments
AuthorsOpenReview ↗

Response to Reviewer sMnC (Part 1/2)

We sincerely appreciate the reviewer's thoughtful assessment and constructive suggestions. We respond to each point below and hope our responses adequately address the reviewer's questions and concerns. The additional experiments and clarifications have been incorporated into the revised manuscript, with changes highlighted in blue.

W1. Attack difficulty and attribution quality.

We thank the reviewer for highlighting the role of attack difficulty. Like other attribution benchmarks, BackX offers a necessary check on attribution faithfulness rather than a universal guarantee. Failure to explain even one verified feature driving the model prediction provides concrete evidence against general attribution faithfulness. BackX strengthens this check through controllable ground truth and tighter fidelity guarantees. Regardless of attack difficulty, a faithful explanation must capture the verified effect of the trigger on the prediction. To broaden coverage while retaining verifiable targets, BackX evaluates diverse triggers and attack mechanisms through multiple metrics across vision and language. This broadens the scope of the necessary test while preserving the fidelity of its attribution targets.

W2&Q1. Ranking consistency across attacks and evaluation metrics.

To assess how attack choice affects method rankings and their consistency across metrics, we compared all nine methods under Blend and ISSBA on CIFAR-10. As shown in the table below, SmoothGrad combines strong trigger localization with effective target suppression and clean-class recovery under Blend. Guided Grad-CAM substantially outperforms Grad-CAM, demonstrating the benefit of refining activation maps with input gradients. The performance gap between IG and IG-Uniform shows that reference selection substantially affects integration-based attribution.

Table R1. Blend metric values and rankings in parentheses.

MethodASR ↓TR ↑Target-class frac. logit change ↓Clean-class frac. logit change ↑
Grad-CAM0.246 (9)0.562 (5)0.193 (9)0.510 (5)
FullGrad0.064 (7)0.567 (4)0.070 (7)0.512 (4)
Input Gradients0.022 (3)0.519 (6)0.038 (4)0.515 (3)
Guided Grad-CAM0.016 (1)0.592 (3)0.021 (3)0.550 (2)
SmoothGrad0.021 (2)0.799 (1)0.001 (1)0.615 (1)
Integrated Gradients0.181 (8)0.368 (9)0.111 (8)0.406 (9)
IG-Uniform0.023 (4)0.458 (7)0.012 (2)0.466 (7)
AGI0.039 (5)0.669 (2)0.061 (6)0.505 (6)
LPI0.063 (6)0.370 (8)0.061 (5)0.433 (8)

To examine whether the advantages observed under Blend persist across trigger mechanisms, we compared the same nine methods under ISSBA. As shown in the table below, LPI and IG outperform SmoothGrad across all ISSBA metrics, reversing their relative performance under Blend. This contrast reveals distinct strengths of reference-path integration and gradient smoothing across trigger mechanisms. The substantial differences among LPI, IG, and AGI further highlight the importance of reference and path construction within the integration family. These results reinforce the paper’s conclusion that detailed input gradients improve coarse CAM attribution. BackX further reveals how smoothing, reference selection, and integration paths respond to different trigger mechanisms, capturing both consistent methodological advantages and attack-dependent strengths. This cross-attack and cross-metric analysis has been incorporated into Appendix J.4 of the revised manuscript.

Table R2. ISSBA metric values and rankings in parentheses.

MethodASR ↓Target-class frac. logit change ↓Clean-class frac. logit change ↑
Grad-CAM0.896 (8)0.275 (8)0.511 (8)
FullGrad0.650 (7)0.240 (7)0.538 (7)
Input Gradients0.391 (4)0.182 (4)0.573 (4)
Guided Grad-CAM0.315 (2)0.171 (2)0.600 (2)
SmoothGrad0.550 (5)0.227 (6)0.555 (5)
Integrated Gradients0.380 (3)0.180 (3)0.573 (3)
IG-Uniform0.578 (6)0.216 (5)0.542 (6)
AGI0.897 (9)0.303 (9)0.462 (9)
LPI0.169 (1)0.137 (1)0.627 (1)
AuthorsOpenReview ↗

Response to Reviewer sMnC (Part 2/2)

Q2. Choosing attacks and combined evaluation results.

We select complementary attacks to reveal the strengths of attribution methods under different trigger mechanisms. Blend covers shared, localized triggers, while ISSBA covers distributed, input-specific triggers. Matched clean references support controlled feature recovery under both attacks, broadening coverage while preserving attribution verifiability.

We interpret the results jointly across attacks, retaining separate rankings for each attack and metric. As shown in the table below, Guided Grad-CAM performs consistently well across both attacks, while SmoothGrad and LPI demonstrate complementary strengths under Blend and ISSBA. Different attribution methods therefore have complementary strengths across trigger mechanisms. Selecting complementary attacks and interpreting their results jointly allows BackX to identify both consistently effective methods and strengths specific to each mechanism. We have included this cross-attack analysis and its interpretation in Appendix J.4 of the revised manuscript.

Table R3. Attribution method rankings across attacks.

Attribution methodBlend rankISSBA rank
Grad-CAM98
FullGrad77
Input Gradients44
Guided Grad-CAM32
SmoothGrad16
Integrated Gradients83
IG-Uniform25
AGI69
LPI51
AuthorsOpenReview ↗

Looking forward to further feedback

We thank the reviewer again for the careful review and constructive feedback. With the discussion period nearing its end, we would be grateful if the reviewer could take a look at our responses and let us know whether the concerns have been addressed. This would allow us to address any remaining concerns before the discussion closes.

For convenience, we briefly summarize the main additions already reported in our responses and incorporated into the revised manuscript.

  • Attack difficulty and evaluation scope (W1). We clarified that BackX provides a necessary test of attribution faithfulness through verifiable targets and tighter fidelity guarantees. Explanations must capture the features driving a prediction. Methods are compared within fixed attack settings, while complementary attacks broaden coverage.
    Manuscript updates: Our scope is further clarified in Appendix M.

  • Cross-attack and cross-metric evaluation (W2 & Q1). We jointly analyzed existing Blend and ISSBA results for all nine methods across metrics. Refining coarse activation maps with input gradients provides consistent benefits. Smoothing and path integration show attack-dependent strengths. Localization and prediction-based metrics provide complementary assessments.
    Manuscript updates: The additional analysis has been added to Appendix J.4 (Figure 21).

  • Attack selection and joint interpretation (Q2). We explained the complementary roles of Blend’s localized triggers and ISSBA’s distributed, input-specific triggers. We interpret their results jointly while retaining separate rankings for each attack and metric, identifying both consistently effective methods and mechanism-specific strengths.
    Manuscript updates: The attack-selection rationale has been included in Appendix J.4.

Our earlier responses and the revised manuscript provide the supporting results and detailed explanations. We hope these additions and clarifications address the main concerns, and we would be very grateful if the reviewer would consider updating the rating in light of these additions.

Reviewer_27NzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BackX, a backdoor-based benchmark for evaluating attribution (XAI) methods.

Critical strengths
  1. A clear criterion-based framing of the benchmark problem, with a practically useful standardized-setup analysis. Distinguishing functional-mapping vs. input-distribution shifts vs. verifiability vs. metric sensitivity is a useful organizing lens, and the empirical demonstration that abs-vs-original and logit-vs-prob materially reorder method rankings (Figs. 3–4) addresses a real confound that prior benchmarking papers often leave uncontrolled.
  2. Broad, multi-domain, multi-attack empirical coverage. CIFAR-10/GTSRB/ImageNet vision plus BERT text; visible Blend and invisible input-specific ISSBA triggers; multiple architectures/depths and ~15 attribution methods; extensive appendices (Figs. 13–29, Tables 10–12) including token-level trigger-dominance verification (Table 11) that directly addresses the salience-diffusion concern for text.
Critical weaknesses
  1. The core novelty delta versus NeuronTrojan (Lin et al., KDD 2021) is not experimentally isolated. BackX's entire premise ("triggers as ground truth") is already what NeuronTrojan did; the claimed improvements — watermark alignment v^(α)=αv+(1−α)x⊙S*, trigger-only-region focus, the TR overlap metric, invisibility — are never ablated against NeuronTrojan's exact protocol. Table 12's BackX-vs-NeuronTrojan Kendall τ=0.048 disagreement is presented as validation that BackX is the better benchmark, but it could equally reflect trigger size/position/visibility differences. Without a head-to-head ablation, the incremental contribution is asserted rather than demonstrated.
  2. The flagship "attribution verifiability" guarantee rests on an unmeasured quantity. Prop. 3.2 shows leakage equals I(x̂;S), yet the paper never estimates this mutual information (or a proxy, e.g., a leakage classifier), and the claim that watermark replacement makes leaked class information "negligible" (Sec. 3.3) is asserted, not tested. This is the paper's central differentiator over prior trigger-based benchmarks, and it currently rests on an assumption. Compounding this, main benchmark curves (Figs. 5–8) report no standard errors, seeds, or random trigger-position/shape variation.
Questions for authors
  1. NeuronTrojan head-to-head. Can you run the same nine methods under NeuronTrojan's exact trigger protocol alongside BackX, and separately ablate each BackX design choice (watermark α, trigger-only region, TR metric) to show which one causes the τ=0.048 ranking shift? Without this, the incremental contribution over Lin et al. is unproven.
  2. Empirical leakage. Can you measure I(x̂;S) (or a classifier-based proxy) before and after watermark replacement, to substantiate the "negligible leakage" claim on which attribution verifiability depends? As written, Prop. 3.2 is a definition, not an empirical bound.
  3. Scope of the ground truth. Trigger-localization performance for the poisoned target class is evaluated; general attribution faithfulness (true-class attributions, natural images) is out of scope. How do you justify that trigger-localization is a necessary condition for general faithfulness, given that methods can localize a salient patch without being globally faithful?
  4. Statistical stability. How stable are rankings across random trigger positions/shapes and random seeds? Figs. 5–8 report no error bars; the 7-method Kendall τ values in Table 12 are especially small-sample.
3 comments
AuthorsOpenReview ↗

Response to Reviewer 27Nz (Part 1/2)

We sincerely thank the reviewer for the careful evaluation and valuable feedback. We provide our point-by-point responses below and hope they satisfactorily address the concerns raised. The manuscript has been revised to include the supporting experimental results and explanations, with updates marked in blue.

W1&Q1. Improvements over NeuronTrojan and the contribution of each design choice.

To isolate the contribution of the individual designs in BackX, we compared all nine attribution methods under BackX and individual component ablations, alongside NeuronTrojan. Each ablation replaces one BackX component with the corresponding NeuronTrojan design: fully visible triggers during training (α=1), class recovery rate (RR) scoring, or bounding-box (BBox) recovery. As shown below, training with α=1 changes attribution rankings relative to BackX (τ=0.833). BackX uses watermark triggers at visibility 0.5 to encourage robust trigger dependence, whereas this variant trains with fully visible triggers. RR scoring changes the ordering (τ=0.778), while bounding-box recovery produces a larger shift (τ=0.167), promoting Grad-CAM from ninth to second and Guided Grad-CAM from third to first. Bounding-box recovery can therefore credit an explanation for trigger pixels it never selected. NeuronTrojan’s ranking yields τ=0.056 across nine methods and τ=0.048 for the original seven methods, excluding Guided Grad-CAM and LPI. These ablations clarify how the individual designs in BackX contribute to benchmark fidelity. We have added these comparisons and their analysis to Section 8 of the revised manuscript.

Table R1. Component ablation analysis of BackX and NeuronTrojan.

Attribution methodBackXα=1RR scoringBBox recoveryNeuronTrojan
Grad-CAM9 (-0)9 (-0)9 (-0)2 (-7)1 (-8)
FullGrad7 (-0)8 (+1)8 (+1)6 (-1)6 (-1)
GGrad-CAM3 (-0)2 (-1)4 (+1)1 (-2)2 (-1)
AGI6 (-0)6 (-0)6 (-0)5 (-1)4 (-2)
SmoothGrad1 (-0)1 (-0)1 (-0)3 (+2)3 (+2)
LPI5 (-0)4 (-1)2 (-3)4 (-1)5 (-0)
IG8 (-0)7 (-1)7 (-1)9 (+1)9 (+1)
IG-Uniform2 (-0)3 (+1)3 (+1)7 (+5)7 (+5)
Input Gradients4 (-0)5 (+1)5 (+1)8 (+4)8 (+4)
Kendall’s τ vs BackX1.0000.8330.7780.1670.056

To directly validate the fidelity gains of BackX’s evaluation design, we conducted a controlled experiment with analytically known feature contributions. We constructed a CNN with one convolutional layer and predefined weights, making pixel contributions to the target logit analytically computable. We used oracle attribution masks with equal pixel budgets to test whether BackX and NeuronTrojan distinguish differences in attribution quality. As shown in the table below, explanations removing 20%, 50%, and 80% of the known contribution receive identical IoU and original class recovery rate (RR) scores under NeuronTrojan, whereas BackX logit and probability reductions distinguish all three levels in the correct order. Combined with the component ablations, these results show that BackX improves benchmark fidelity by correctly distinguishing explanations with different true contributions. We have added this experiment and its setup details to Section 4 of the revised manuscript.

Table R2. Evaluation scores for oracle explanations with known contributions.

Evaluation metric20% contribution removed50% contribution removed80% contribution removed
Ground truth (%)20.00050.00080.000
NeuronTrojan IoU ↑25.00025.00025.000
NeuronTrojan RR (%) ↑100.000100.000100.000
BackX logit reduction (%) ↑20.00050.00080.000
BackX probability reduction (%) ↑37.95681.74597.039
AuthorsOpenReview ↗

Response to Reviewer 27Nz (Part 2/2)

W2&Q2. Empirical evidence for negligible information leakage.

To empirically examine the leakage assumption in Proposition 3.2, we measured the target-class prediction probability after trigger replacement on CIFAR-10. We compared clean replacement with zero replacement and Gaussian blur using the same images and recovery masks. As shown in the table below, clean replacement yields mean target-class probabilities of 0.023 at 90% replacement and 0.001 at complete replacement, consistently below both alternatives. These results support negligible target-class information leakage after complete clean replacement. We have added these prediction results to Section 4 of the revised manuscript.

Table R3. Target-class prediction probability after trigger replacement.

Replacement method90% replaced100% replaced
Replacement (ours)0.023 ± 0.0840.001 ± 0.004
Zero replacement0.148 ± 0.1410.089 ± 0.072
Gaussian blur0.477 ± 0.3290.134 ± 0.220

To further test whether recovery leaves detectable traces of the mask, we trained a separate MLP classifier for each replacement method to distinguish recovered images from their clean counterparts. As shown in the table below, classification accuracy falls to chance level at 50.0% after complete clean replacement, compared with 81.0% for zero replacement and 82.0% for Gaussian blur. Combined with the prediction results above, these findings provide empirical support for negligible information leakage, consistent with the assumption in Proposition 3.2.

Table R4. Classification accuracy for distinguishing recovered images from clean images.

Replacement method90% replaced100% replaced
Replacement (ours)62.0%50.0%
Zero replacement83.0%81.0%
Gaussian blur82.0%82.0%

Q3. The necessary test of attribution faithfulness.

We thank the reviewer for raising this point. Like all existing attribution benchmarks, BackX can provide only a necessary test, not a sufficient guarantee of general faithfulness. Our aim is to improve the fidelity of this test through controlled and verifiable attribution targets. An explanation that fails to reflect a trigger’s verified effect on the model prediction provides a counterexample to general faithfulness. Faithfully explaining a verified feature driving model predictions is a necessary requirement for general attribution faithfulness. We therefore evaluate diverse trigger patterns and attack mechanisms across vision and language, broadening coverage while preserving controlled attribution targets. BackX thus combines stronger fidelity guarantees with broader evaluation coverage, providing a more reliable test of attribution faithfulness.

Q4. Ranking stability and uncertainty in the seven-method correlations.

To quantify ranking stability and score variation, we evaluated all nine attribution methods on CIFAR-10 across five independent training seeds, random trigger positions, and three trigger patterns with different shapes. As shown in the table below, all nine methods preserve their baseline rankings under clean-class fractional logit change. Scores vary only slightly across seeds, trigger positions, and trigger patterns. These results demonstrate the stability of our cross-benchmark comparisons across different training seeds and trigger configurations. We have included this stability analysis in Appendix J.4 of the revised manuscript.

Table R5. Clean-class fractional logit change (mean ± SD) and ranking changes across training seeds, trigger positions, and trigger patterns.

MethodAcross 5 seeds ↑ΔRankRandom trigger positions ↑ΔRank3 trigger patterns ↑ΔRank
SmoothGrad0.616 ± 0.00600.615 ± 0.01200.614 ± 0.0090
GGrad-CAM0.550 ± 0.00700.551 ± 0.01000.548 ± 0.0140
Input Gradients0.516 ± 0.00700.517 ± 0.01400.514 ± 0.0160
FullGrad0.513 ± 0.00600.514 ± 0.01600.511 ± 0.0180
Grad-CAM0.510 ± 0.00800.511 ± 0.01500.509 ± 0.0170
AGI0.505 ± 0.01000.506 ± 0.01500.504 ± 0.0130
IG-Uniform0.465 ± 0.00800.467 ± 0.01100.464 ± 0.0230
LPI0.431 ± 0.00900.433 ± 0.01100.431 ± 0.0220
IG0.405 ± 0.00500.406 ± 0.01400.408 ± 0.0180
AuthorsOpenReview ↗

Looking forward to further feedback

We thank the reviewer again for the careful review and constructive feedback. With the discussion period nearing its end, we would be grateful if the reviewer could take a look at our responses and let us know whether the concerns have been addressed. This would allow us to address any remaining concerns before the discussion closes.

For convenience, we briefly summarize the main additions already reported in our responses and incorporated into the revised manuscript.

  • Improvements over NeuronTrojan (W1 & Q1). We conducted two sets of additional experiments to evaluate BackX’s improvements over NeuronTrojan. First, comparisons with NeuronTrojan and component ablations across nine attribution methods isolate how watermark blending, recovery regions, and scoring affect rankings. Second, a controlled oracle experiment shows that BackX correctly distinguishes explanations with different true contributions that NeuronTrojan’s metrics score identically.
    Manuscript updates: The new results and analyses have been added in Sections 4 and 8 (Figures 3 and 10).

  • Empirical leakage evaluation (W2 & Q2). We added target-class prediction measurements and classifier-based leakage tests against zero replacement and Gaussian blur. Clean replacement yields substantially lower residual target-class probabilities and makes recovered images less distinguishable from their clean counterparts.\ Manuscript updates: The new results of the leakage analysis are included in Section 4 (Table 3).

  • Scope of attribution ground truth (Q3). We clarified that BackX provides a necessary test of attribution faithfulness through verifiable targets and tighter fidelity guarantees. Failure to capture the features driving a prediction, including planted triggers, provides evidence against attribution reliability. While passing alone does not establish general faithfulness, diverse triggers, attacks, and modalities broaden coverage.
    Manuscript updates: Our scope is further clarified in Appendix M.

  • Ranking stability (W2 & Q4). We added experiments across independent training seeds, random trigger positions, and different trigger shapes. All nine methods preserve their rankings under fractional logit change, with small score variations across the tested configurations.
    Manuscript updates: The new results and corresponding analysis are included in Appendix J.4 (Figure 20).

Our earlier responses and the revised manuscript provide the supporting results and detailed explanations. We hope these additions and clarifications address the main concerns, and we would be very grateful if the reviewer would consider updating the rating in light of these additions.

Reviewer_V7FqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper proposes BackX, a benchmark for attribution methods built on backdoored models. The planted trigger is treated as the attribution ground truth. Attributions are scored by restoring the top-ranked features from the matched clean input and measuring residual ASR, trigger recall and logit recovery (Section 3). The authors also list four fidelity criteria (Table 1), give two propositions on fidelity (Section 3.3), fix post-processing and output choices (Section 4), compare about fifteen attribution methods on vision and text, and try attribution-guided recovery as a backdoor defense (Section 6).

Critical strengths
  1. Restoring features from the exact clean counterpart is a nice design. It avoids the out-of-distribution problem of zero or blur baselines, and splitting trigger recall from prediction recovery separates "where" from "how much it matters".

  2. The coverage is wide: several attack families, many attribution methods, image and text. Appendix K.3 has a useful text control that compares deleting the trigger with deleting the strongest same-length non-trigger span.

Critical weaknesses
  1. The core claim that the trigger is the ground truth isn't really shown. Section 3.1 says the model focuses "solely" on the trigger, but Appendix E mainly supports this with high ASR. High ASR only means the trigger can flip the label. Background features can still move the target score, and some trigger pixels may be redundant. Table 11 shows trigger dominance in text (0.88 to 0.92), not exclusivity, and there is no such control for images. This matters a lot, because Appendix D separates BackX from Lin et al. (2021) exactly on this point.

  2. The theory doesn't back the fidelity ratings in Table 1. Proposition 3.1 only bounds the paired input distance, which isn't the same as distribution invariance. Proposition 3.2 assumes the mask leaks little information (I(x̂; S) ≤ ε), and nothing in the paper checks this. Table 1 rates BackX "Strong" on attribution verifiability (and Lin et al. only "Weak"), which gives it the top score of 3.5, but that rating rests on exactly these unchecked assumptions.

  3. The validation is somewhat circular. The settings in Section 4 are picked using BackX results (Figures 3 and 4) and then used to rank methods. In Table 12, agreement with DiFull, DiffID and DiffROAR is presented as support, even though Table 1 rates these benchmarks as falling short, and agreement with NeuronTrojans is near zero (τ = 0.048). I don't see results across independent training seeds or held-out trigger configurations, and Appendix M says stability across trigger variations wasn't measured.

Questions for authors
  1. For the image models, can you show partial trigger removal, trigger-only inputs and matched background interventions, like the text control in Table 11? (suggestions for future work)

  2. What evidence supports the leakage assumption in Proposition 3.2? (clarification requests)

  3. Why should agreement with benchmarks you consider flawed count as evidence of fidelity? And do the rankings hold across seeds and new triggers? (clarification requests)

6 comments
AuthorsOpenReview ↗

Response to Reviewer V7Fq (Part 1/2)

We sincerely thank the reviewer for the thoughtful comments and constructive suggestions. We address each concern below and hope our responses help clarify and strengthen our work. The additional experiments and clarifications have been incorporated into the revised manuscript, with changes highlighted in blue.

W1&Q1. Trigger dependence and the validity of attribution ground truth.

To test whether prediction changes are driven by the trigger rather than background features, we evaluated target-class predictions across different natural and pixel-shuffled backgrounds on CIFAR-10 using ResNet-18. As shown in the table below, the full trigger maintains a mean target-class prediction probability of 1.000 on natural backgrounds and 0.993 on pixel-shuffled backgrounds. These predictions remain strong despite the disruption of background structure, indicating that the trigger does not rely on the original spatial arrangement of background features. Partial trigger removal also effectively reduces the target-class prediction probability, while complete removal returns it to the no-trigger baseline. These results support the validity of the trigger as a controlled attribution target in both image and text models. We have incorporated these additional tests into the empirical fidelity validation in Section 4 of the revised manuscript.

Table R1. Target-class predictions under background changes and partial trigger removal.

ExperimentTarget probabilityTarget-class prediction rate
Natural backgrounds (no trigger)0.001 ± 0.0050.000
Trigger on natural backgrounds1.000 ± 0.0001.000
Shuffled backgrounds (no trigger)0.002 ± 0.0120.000
Trigger on shuffled backgrounds0.993 ± 0.0431.000
30% trigger removal0.639 ± 0.3380.644
60% trigger removal0.121 ± 0.1270.104
100% trigger removal0.001 ± 0.0050.000

W2&Q2. Theoretical guarantees and leakage evidence.

Proposition 3.1 bounds input shift by restoring clean pixels, leaving only the remaining trigger perturbation. For our localized watermark, the spatially limited perturbation and blending keep this bound small. We claim bounded shift without claiming to eliminate it, consistent with the “partial” rating in Table 1. We have further clarified this scope in the revised manuscript.

To empirically examine the leakage assumption in Proposition 3.2, we measured the target-class prediction probability after trigger replacement on CIFAR-10. As shown below, our replacement method reduces the mean probability to 0.023 at 90% replacement and 0.001 at complete replacement, consistently below zero replacement and Gaussian blur. These results provide empirical support for negligible target-class information leakage in our benchmark. We have added these results to Section 4 of the revised manuscript to substantiate the leakage analysis.

Table R2. Target-class prediction probability after trigger replacement.

Replacement method90% replaced100% replaced
Replacement (ours)0.023 ± 0.0840.001 ± 0.004
Zero replacement0.148 ± 0.1410.089 ± 0.072
Gaussian blur0.477 ± 0.3290.134 ± 0.220

To test whether recovery leaves detectable traces of the mask, we trained a separate MLP classifier for each replacement method to distinguish recovered images from their clean counterparts. At complete replacement, classification accuracy remains at chance level for our method of 50.0%, while remaining at 81.0% for zero replacement and 82.0% for Gaussian blur. Together, these results provide empirical evidence consistent with the assumption of negligible mutual information between the recovery mask and the recovered image in Proposition 3.2.

Table R3. Classification accuracy for distinguishing recovered images from clean images.

Replacement method90% replaced100% replaced
Clean replacement (ours)62.0%50.0%
Zero replacement83.0%81.0%
Gaussian blur82.0%82.0%
AuthorsOpenReview ↗

Response to Reviewer V7Fq (Part 2/2)

W3&Q3. Validation circularity and ranking stability.

To directly validate the fidelity gains of BackX’s evaluation design, we conducted a controlled experiment with analytically known feature contributions. We constructed a CNN with one convolutional layer and predefined weights, making pixel contributions to the target logit analytically computable. We used oracle attribution masks with equal pixel budgets to test whether BackX and NeuronTrojan distinguish differences in attribution quality. As shown in the table below, explanations removing 20%, 50%, and 80% of the known contribution receive identical IoU and original class recovery rate (RR) scores under NeuronTrojan, whereas BackX logit and probability reductions distinguish all three levels in the correct order. These results show that BackX improves benchmark fidelity by correctly distinguishing explanations with different true contributions. We have added this experiment and its setup details to Section 4 of the revised manuscript.

Table R4. Evaluation scores for oracle explanations with known contributions.

Evaluation metric20% contribution removed50% contribution removed80% contribution removed
Ground truth (%)20.00050.00080.000
NeuronTrojan IoU (%) ↑25.00025.00025.000
NeuronTrojan RR (%) ↑100.000100.000100.000
BackX logit reduction (%) ↑20.00050.00080.000
BackX prob. reduction (%) ↑37.95681.74597.039

To separate validation from setup selection, we tested additional models trained with five independent seeds and three held-out trigger patterns under a fixed evaluation protocol. As shown below, all nine methods preserve their baseline rankings, with small variations in clean-class fractional logit change. These results show that the rankings hold beyond the models and trigger configuration used to select the evaluation settings. The ranking stability across seeds and trigger patterns demonstrates that our cross-benchmark correlations are robust to these variations. We have included the benchmarking stability analysis in Appendix J.4 of the revised manuscript.

Table R5. Fractional logit change (mean ± SD) and rank changes across training seeds and held-out trigger patterns.

MethodAcross 5 seeds ↑ΔRankAcross 3 trigger patterns ↑ΔRank
SmoothGrad0.616 ± 0.00600.614 ± 0.0090
GGCAM0.550 ± 0.00700.548 ± 0.0140
Input Gradients0.516 ± 0.00700.514 ± 0.0160
FullGrad0.513 ± 0.00600.511 ± 0.0180
GCAM0.510 ± 0.00800.509 ± 0.0170
AGI0.505 ± 0.01000.504 ± 0.0130
IG-Uniform0.465 ± 0.00800.464 ± 0.0230
LPI0.431 ± 0.00900.431 ± 0.0220
IG0.405 ± 0.00500.408 ± 0.0180

Thanks for the detailed response and the new experiments. The background-shuffle and partial-removal results help with my first concern, and the seed/trigger-pattern table and the ablation against NeuronTrojan are useful additions.

I’m still not fully convinced on two points. For leakage, at 100% replacement the recovered image is just the clean image, so 0.001 target probability and 50% detector accuracy are expected by construction. The informative case is partial replacement, and there the detector is at 62%, above chance, so some mask leakage is detectable in the regime the benchmark actually uses. For ranking stability, the gaps between neighbouring methods (e.g. 0.516 / 0.513 / 0.510) are smaller than the reported SDs (up to 0.018), yet ΔRank is 0 everywhere. Is ΔRank computed on the mean, or does the ranking hold in every individual seed and pattern? Also, my question about using agreement with DiFull/DiffID/DiffROAR as evidence of fidelity wasn’t really addressed.

Given the new controls, I’ve raised my Soundness score from 1 to 2, but I’ll keep my rating at 2 for now.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Discussion

We sincerely thank the reviewers for their constructive feedback and the Area Chair for their time and consideration. As the discussion period concludes, we would like to summarize the main concerns raised and how we have addressed them in our revised manuscript.

1. Theoretical foundations and empirical validation of benchmark fidelity (addressing concerns from V7Fq, 27Nz, and DXys). We conducted controlled oracle experiments and component ablations to directly validate the fidelity improvements of BackX. The results reveal substantial evaluation biases in NeuronTrojan, which can credit explanations for features they never identify and fail to distinguish explanations with different true contributions. BackX addresses these failures through its theoretically grounded recovery and scoring designs, demonstrating substantial improvements in benchmark fidelity over existing backdoor-based evaluation.

  • Manuscript updates: New validation results and comparative analyses are included in Sections 4 and 8 (Figures 3 and 10).

2. Empirical evidence of negligible information leakage (addressing concerns from V7Fq and 27Nz). We conducted additional experiments under background variations, partial trigger removal, and different recovery strategies. The results validate the trigger as the dominant attribution target and demonstrate that our framework substantially reduces information leakage compared with conventional replacement strategies, even under partial recovery. These findings further substantiate the attribution verifiability of BackX.

  • Manuscript updates: New empirical validation results are included in Section 4 (Tables 2 and 3).

3. Validated benchmarking stability across independent settings (addressing concerns from V7Fq and 27Nz). Additional experiments across independent training seeds, random trigger positions, and different trigger patterns show that all nine attribution methods preserve their rankings under clean-class fractional logit change, with small score variations. These results validate the stability of our benchmarking beyond the original evaluation settings.

  • Manuscript updates: New stability results and analyses are included in Appendix J.4 (Figure 20).

4. Systematic attribution benchmarking and insights across vision and language (addressing concerns from 27Nz and sMnC). BackX provides a necessary test of attribution faithfulness through controlled and verifiable attribution targets across vision and language. Our additional analyses across attacks and metrics reveal both consistent methodological advantages and attack-specific strengths, providing insights that guide attribution method selection and development.

  • Manuscript updates: New comparative analyses and discussions are included in Appendices J.4 and M (Figure 21).

Following our rebuttal, reviewer V7Fq increased their soundness score and explicitly acknowledged that their specific questions had been addressed.

We believe that the additional experiments and clarifications address the key concerns raised during review and further substantiate the contributions of BackX. BackX goes well beyond using neural backdoors for evaluation, advancing both the foundations and practice of attribution benchmarking through systematic fidelity criteria, theoretically grounded and empirically validated designs, and standardized evaluation across vision and language.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.