- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper challenges the view that common agreement between visual explainers indicates essential spatial areas driving predictions. It reformulates several XAI approaches with an uniform equation, and then presents an adversarial attack method for multiple explainers. In particular, the method trains a classifier where the prediction is related to an arbitrarily selected patch but visual explainers attend to common objects. Experimental results on image classification highlight the effects of the method across datasets and baselines.
Critical strengths
(1) It is an interesting view to derive an adversarial algorithm that simultaneously attacks multiple XAI approaches.
(2) The paper identifies a critical issue about the causality between visual explanation and model predictions.
(3) The paper carries out extensive experiments under diverse settings, and provides detailed ablation study for a comprehensive evaluation.
Critical weaknesses
(1) While aiming to achieve an integral formula for attacking multiple visual explainers, Equation 8 reads like a coarse decomposition with specific terms for different methods.
(2) The experimental results are difficult to read. For instance, while the ASR and TopK-BBox are discussed in the appendix, there is no definition of AGG-Mean, Opt-Agg, etc. Therefore, it is unclear about the advantages in each setup.
(3) The paper frames itself as the first attacking method for multiple explainers, but in the experiment section CEAA is cited as a method achieving a similar goal.
(4) Despite starting with an interesting observation on a common pitfall of visual explainer, the paper falls short of explaining the reasoning behind the failure.
(5) I feel the presentation of the main paper needs considerable improvement. It abruptly ends with some results, without summary or comparison with prior studies (e.g., what are the key distinctions in details). A lot of important information, e.g., definition of tasks and key experimental results are kept in the appendix.
Questions for authors
What insights can the proposed attacking method provide in the improvement of XAI methods?