acceptodds
Under review as a conference paper at ICLR 2027

Shared-Response Agreement Fabrication: A Model-Side Attack on Multi-Explainer Visual Explanations

Abstract

Post-hoc visual explanations are often checked with multiple explainers. When several explainers highlight the same object region, this agreement is usually treated as evidence that the model relies on object semantics. We show that this interpretation can fail. We first place seven common explainers on a shared spatial partition and prove that their block-level heatmaps are driven by the same first-order deletion response, up to explainer-specific display maps and residual terms. Thus, multi-explainer agreement can reflect a shared local response rather than independent evidence. Based on this structure, we propose Shared-response Agreement Fabrication (SAF), a model-side attack that fabricates object-centered multi-explainer agreement. SAF trains the classifier so that a patch encodes the class decision, while a flat-germ objective suppresses the shared response on the patch-affected region and an object-region objective keeps it high on the original object region. After release, the classifier predicts from the patch, but the considered explainers still highlight the original object region. We provide a sufficient condition for this fabricated agreement and show that it does not imply the largest finite deletion effect. Experiments on standard image classification benchmarks show that SAF achieves higher attack success across datasets, model architectures, and explanation methods than representative explanation attacks.

Primary area:
interpretability and explainable AI
Keywords:
Explainable AI, Visual Explanations, Multi-Explainer Agreement, Explanation Attacks, Model-Side Attacks, Shortcut Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_cawuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper challenges the view that common agreement between visual explainers indicates essential spatial areas driving predictions. It reformulates several XAI approaches with an uniform equation, and then presents an adversarial attack method for multiple explainers. In particular, the method trains a classifier where the prediction is related to an arbitrarily selected patch but visual explainers attend to common objects. Experimental results on image classification highlight the effects of the method across datasets and baselines.

Critical strengths

(1) It is an interesting view to derive an adversarial algorithm that simultaneously attacks multiple XAI approaches.

(2) The paper identifies a critical issue about the causality between visual explanation and model predictions.

(3) The paper carries out extensive experiments under diverse settings, and provides detailed ablation study for a comprehensive evaluation.

Critical weaknesses

(1) While aiming to achieve an integral formula for attacking multiple visual explainers, Equation 8 reads like a coarse decomposition with specific terms for different methods.

(2) The experimental results are difficult to read. For instance, while the ASR and TopK-BBox are discussed in the appendix, there is no definition of AGG-Mean, Opt-Agg, etc. Therefore, it is unclear about the advantages in each setup.

(3) The paper frames itself as the first attacking method for multiple explainers, but in the experiment section CEAA is cited as a method achieving a similar goal.

(4) Despite starting with an interesting observation on a common pitfall of visual explainer, the paper falls short of explaining the reasoning behind the failure.

(5) I feel the presentation of the main paper needs considerable improvement. It abruptly ends with some results, without summary or comparison with prior studies (e.g., what are the key distinctions in details). A lot of important information, e.g., definition of tasks and key experimental results are kept in the appendix.

Questions for authors

What insights can the proposed attacking method provide in the improvement of XAI methods?

Reviewer_wz3mOpenReview ↗
Rating
2
Confidence
1
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper questions whether agreement among visual explanation methods establishes that a classifier uses the intended object evidence. It expresses seven explainers through a common block-level first-order deletion response, with explainer-specific transformations and residuals. Its Shared-Response Agreement Fabrication (SAF) attack trains a classifier to predict a class encoded by a patch while encouraging explanations to remain concentrated on the original object. A flat-germ loss suppresses patch-region responses, and an object-region loss preserves object-centered responses. Experiments cover three image datasets and three CNN architectures, with explanation, aggregation, detection, and patch-intervention evaluations.

Critical strengths
  • A significant and clearly motivated auditing problem. Agreement among explainers can be mistaken for independent corroboration. Constructing a model whose predictions follow a patch while its explanations remain plausible is a useful stress test of that assumption.
  • Broad and promising empirical evidence. The study covers seven explainers from several families and reports substantial gains over the selected attacks. On CUB with ResNet-50, SAF reports ASRs of 83.5%-91.2% across individual explainers (Table 2), alongside a small clean-accuracy drop.
  • Mechanism checks beyond visual examples. The removal ablations, patch-swap and deletion interventions, shuffled-response control, and residual-certificate measurements help test different parts of the proposed explanation. In particular, Figure 3c provides evidence that the prediction cue differs from the highlighted region.
Critical weaknesses
  1. The theoretical decomposition needs a correction and a clearer connection to the tested networks. In Appendix D.5, Eq. (95) sets the coefficient to zero when the block response is zero, but Eq. (96) then assumes that the corresponding masked sum is also zero. Signed contributions can cancel before masking without cancelling afterward: contributions +1 and -1 give a zero block response, whereas mask weights 1 and 0 give a masked sum of 1. Thus Eq. (96) needs an additional condition or a residual term, which may affect the subsequent decomposition and certificate. This case is relevant because SAF deliberately suppresses block responses. Additionally, the IG derivation assumes twice-differentiable behavior along a path (Appendix D.2); please explain how activation-boundary crossings in the tested ReLU networks are handled. These concerns qualify the theoretical claims without, by themselves, refuting the empirical attack results.

  2. The central simultaneous-agreement claim needs a direct joint evaluation. Eq. (5) requires every explainer to satisfy its threshold on the same sample. Table 2 reports marginal ASRs for individual explainers, while the aggregation experiments measure a combined heatmap. Neither directly reports how often the same fixed classifier fools all seven explainers on the same image. Similarly, the average certificate rate in Appendix F.3 is computed over sample-explainer pairs rather than the intersection across explainers. Please report the joint all-seven success rate, ideally together with the number of explainers fooled per sample and uncertainty across repeated runs. This would directly test the distinctive claim of the paper using the existing evaluation outputs.

  3. Some implementation details needed to reproduce the shared-response objective remain underspecified. Algorithm 1 introduces the quadrature nodes and weights, and an expanded patch-affected block set, without a concrete numerical specification in Appendix F. Please provide the common partition resolution, the rule for expanding the patch-affected set, and the quadrature configuration in one place.

Questions for authors
  1. How should Eqs. (95)-(96) handle zero block responses caused by cancellation? Does the corrected residual change the reported decomposition errors or certificate rates?
  2. How are pathwise Hessian residuals evaluated when the image path crosses ReLU activation boundaries? Are smooth activations, explicit boundary terms, or another justified construction used?
  3. For one fixed SAF classifier, what fraction of test images simultaneously achieves the target prediction and the object-mass threshold for all seven explainers? Can you report this joint rate with repeated-run uncertainty?
Reviewer_ZJJQOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies and questions the idea that "if multiple post-hoc visual explainers, when applied to the same classifier, highlight the same object, then that is evidence the classifier relies on object semantics". The authors argue that this idea is incorrect.

The first thing the paper does to back up this claim is try to characterize the shared structure behind multi-explainer agreement. They do so by taking a first-order view of multi-explainer agreement and analyzing the explanations of different methods by putting them on a common spatial partition. There, the authors "derive a decomposition in which the same block-level first-order deletion response forms the shared term, while the display map and residual are explainer-specific." The claim is that the decomposition allows them to show that agreement can occur without there being evidence that the classifier relies on object semantics.

The second contribution is that the authors use this finding to propose a model-side attack that acts on multiple explainers. The method is called SAF. The authors argue that SAF has three distinct advantages compared to previous attack methods. The authors provide experimental evidence to support their claims.

Critical strengths

** Strength 1: A way to analyze multiple post-hoc visual explainers**

I think the idea/method to place the explanations of multiple explainers on the same playing field using a regular spatial partition is interesting and could be very relevant. In general, it is very tough to compare or analyze various methods. Thus, having a principled way to do that is useful (particularly because it does not seem like the partition scheme/idea is biasing or favoring one particular explainer).

** Strength 2: Experiments** The experiments seem to be comprehensive and illustrate the advantages of SAF.

Critical weaknesses

** Weakness 1: Lack of clarity/care **

The major weakness of this paper that prevents from recommending acceptance is the overall lack of clarity.

Abstract: An abstract should distill the key contributions of a paper without becoming overly technical. Basically after sentence 4, the abstract it very difficult to understand. For example, the reader does not know what a "spatial" partition is, what "residuals" are, or a "flat-germ" objective is.

Introduction: When transitioning to the introduction (and throughout the entire paper) citations are never in brackets, even though in many instances they should be. The authors also introduce "attack" at the end of paragraph one without ever explaining what that is. Moreover, the paragraph on redundant systems comes out of nowhere. I am not really sure why that is relevant or necessary to define. The introduction then gets overly technical, describing block-level responses, spatial partitions, residuals. I am not saying I do not understand what these are (later in the paper they are explained to varying levels of detail) but in an introduction readers can get lost very quickly.

Section 2 and 3: Easy to understand.

Section 4: Theorem 1 seems to be one of the main contributions. While authors do briefly touch on what the required conditions are, I believe they should explain the regularity conditions in more depth so that readers can get a better sense of the result. Also, Figure 2 is difficult to read and not well presented visually.

Section 5: Based off the result in section 4, the design choices in section 5 seem reasonable. However, the authors present theorem 2 yet provide no discussion about what the result is actually telling us. It's seems to be a very comprehensive result, yet it is not clear what its actually telling us.

Section 6: Experiments seem comprehensive.

There is no conclusion/discussion that wraps up the story.

Questions for authors

I have two main questions for the authors.

  1. Could they explain more the reasoning behind their partition scheme in section 2? Why is this method approriate.

  2. What are the implications of Theorem 2. What is it telling us?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.