acceptodds
Withdrawn from ICLR 2027

Where to Place Local Perturbations? Breaking Image Classifiers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The placement of local disturbances is a critical factor in robustness testing for vision models. A straightforward strategy is random placement, while attribution methods can guide perturbations toward regions considered relevant for the model prediction. We study which placement strategies induce the strongest degradation under a fixed perturbation budget. Focusing on image classification as a controlled first setting, we compare attribution-based guidance with matched random baselines across datasets, architectures, perturbation operators, and spatial resolutions. We find that no guidance strategy is uniformly strongest. Object-aware random masking can approach or exceed attribution-based guidance in several ImageNet-S50 settings, whereas SHAP induces stronger degradation in other configurations, particularly on Oxford Flowers. Our results show that effective perturbation placement is strongly configuration-dependent and provide a basis for future robustness testing of more complex vision tasks.

Primary area:
interpretability and explainable AI
Keywords:
Guided degradation, model robustness, localized perturbations

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_49UjOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how the placement of local perturbations affects image classifiers under matched masking budgets. It compares attribution-guided and random masking across datasets, architectures, patch sizes, and perturbation operators, using confidence degradation curves and complementary summary metrics. The results show that no strategy is uniformly strongest: object-aware random masking can match or exceed attribution-based guidance in some settings, while SHAP induces stronger degradation in others.

Critical strengths

The paper compares attribution-guided and random masking across multiple datasets, architectures, and perturbation settings. Including object-aware random baselines provides a useful reference for assessing the added value of attribution-based guidance.

Critical weaknesses
  1. Inconsistency between the metric definition and its interpretation. In Eq. (7), the point of the steepest confidence drop is defined using arg⁡max⁡bdD^(b)db\arg\max_b \frac{d\hat D(b)}{db}. However, since DD represents confidence, the steepest drop should correspond to the minimum derivative rather than the maximum. Moreover, the paper seems to mix up the location and magnitude of the confidence drop. It is unclear whether this is a typo or an implementation issue.

  2. Numerical inconsistencies across tables. In Tables 2/9, the relative AUC for ResNet50 with MDP-32 and Random Object First is 0.00077, while Table 13 reports 0.0077 for the same setting, a tenfold difference. A similar issue appears for ViT. Additionally, some CP0.5CP_{0.5} and ∇B\nabla B values in Table 8 appear to be swapped compared with the main results. These inconsistencies raise concerns about the reliability of the reported results.

  3. Questionable interpretation of the metrics. The paper generally assumes that lower AUCbandAUC_{\text{band}} indicates stronger degradation. However, this metric measures the variation across samples, and a lower value does not necessarily imply stronger degradation. For example, it could indicate either consistently low or consistently high confidence across samples. This interpretation needs further clarification, especially since the proposed diagnostic metrics are a key contribution of the paper.

Questions for authors

Seen above, this paper requires substantial rewriting, along with more rigorous and carefully conducted experiments.

Reviewer_Y2z3OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to select image regions for progressive occlusion under a fixed perturbation budget, and compares the effects of attribution methods such as SHAP, Grad-CAM, and Integrated Gradients against random occlusion strategies on classification confidence. The authors find that no single strategy dominates across all datasets and perturbation settings, and that object-aware random masking can even outperform attribution methods in some scenarios.

Critical strengths
  1. Table 1 shows R_obj far below R_all under absolute blindness yet different guidance orderings across the two axes—identifying a real confound in standard deletion evaluation and providing a reusable diagnostic.
  2. Multi-descriptor trajectory analysis exposes failure modes hidden by single-score metrics.
Critical weaknesses
  1. The framework is largely a repackaging of standard deletion/insertion evaluation: the authors themselves note that standard deletion is a special case (absolute blindness + fixed operator + AUC). The claimed increments—object-relative blindness normalization, trajectory descriptors (∇B, CP, AUC_band), and treating attribution guidance as just one perturbation schedule—are individually incremental, and object-relative blindness relies on privileged foreground annotations. The paper disclaims "ranking attribution methods," yet its main tables still present guidance rankings, and the central conclusion ("everything depends on configuration") is close to unfalsifiable. It needs explicit testable hypotheses, interaction-effect analyses (e.g., variance decomposition), and significance testing rather than point-estimate orderings.

  2. The definition of (7) is problematic on two counts. (i) Direction is wrong: degradation curves are decreasing, so the steepest decrease corresponds to the minimum (most negative) derivative, not the maximum. (ii) Using ∇B for a scalar argmax location is misleading. Differentiability of the interpolated \hat{D}_j, tie-breaking at nodes, interpolation scheme, and sensitivity to grid size K are all unspecified.

  3. The paper explicitly frames its metrics as "observational descriptors…rather than calibrated robustness measures" (§3.1, §6) and admits no external validation against ImageNet-C/AutoAttack. Yet the abstract, conclusion, and Appendix F repeatedly interpret the same metrics in robustness language. Since masking itself introduces distribution shift and CP/∇B depend on untested τ, K, and interpolation choices, readers may wrongly infer that higher AUC/CP means genuine robustness.

Questions for authors

The paper concludes that degradation behavior depends on the dataset, blindness axis, metric, patch size, and operator. Can the authors report a variance decomposition or mixed-effects analysis (guidance × dataset × axis × patch size) showing which factors and interactions are statistically significant, with effect sizes?

Can the authors report paired significance tests (with multiple-comparison correction) for the key claims—e.g., SHAP > R_obj on Oxford Flowers, R_obj ≈ SHAP on ImageNet-S50? If most pairwise differences are within noise, many of the paper's comparative statements need to be withdrawn.

Reviewer_omcMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

Adversarial perturbations and image disturbances can degrade model performance. Studying their effects is important for understanding model robustness. This paper investigates how the spatial placement of local perturbations affects image classifier predictions. It compares several xAI-guided perturbation strategies with random and object-aware random perturbations under fixed budgets. The results show that no single strategy consistently induces the strongest degradation. Object-aware random perturbations can outperform xAI-guided methods in some settings, while SHAP performs better in others.

Critical strengths

The paper does provide an extensive set of experiments and comparisons regarding xAI-guided perturbation methods. I want to acknowledge the experimental breadth. However, I do not see novelty and insights from the breadth of experiments.

Critical weaknesses
  • The paper's methodology and findings overlap with existing perturbation-based xAI evaluation studies. Several relevant works are missing from the related work [1,2,3] . The paper also cites [4], but does not clearly explain how its findings differ from that work. The presented observations overlap with existing findings. The novelty of the proposed evaluation framework is therefore unclear or limited.

  • The paper evaluates model robustness, or simply measures how quickly predictions deteriorate as important visual information is removed? Removing important image regions naturally reduces the information available for classification. A drop in confidence may therefore reflect the loss of necessary visual information rather than a genuine model vulnerability.

  • Static xAI guidance may not capture changes in feature importance as image regions are progressively removed.

  • The paper focuses on patch-based perturbations guided by xAI methods but does not compare them with conventional adversarial attacks. Adversarial noise is non uniform and spreads across an image unevenly based on such attacks. So, adversarially vulnerable regions may not coincide with regions identified as important by xAI methods. Restricting perturbations to selected patches may overlook other sources of model vulnerability. A comparison with standard adversarial attacks and optimized localized attacks would help establish whether the proposed guidance strategies are effective for worst-case robustness testing, rather than simply measuring sensitivity to local information removal.

  1. Evaluating feature attribution methods in the image domain, Machine Learning (2024), Springer Nature
  2. Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond, JMLR 2023
  3. Reliable Evaluation of Attribution Maps in CNNs: A Perturbation-Based Approach, IJCV 2025
  4. On the Evaluation Consistency of Attribution-Based Explanations, ECCV 2024
Questions for authors
  • Please look into Weaknesses. Please clarify the paper's contribution as compared to the missing related work and also to the cited but gap in positioning as compared to [4].

  • The results show that object-aware random masking can outperform xAI-guided perturbations in some settings. Can the authors provide a deeper analysis of why this happens and how the behavior changes with patch size, architecture, and perturbation type?

  • The results suggest that object localization alone can sometimes be more effective than xAI-based guidance for degrading model confidence. How much of this advantage comes from access to ground-truth object regions, rather than the perturbation strategy itself? Can the authors separate these effects through controlled comparisons?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.