acceptodds
Withdrawn from ICLR 2027

RIGHT PREDICTIONS, MISLEADING EXPLANATIONS: ON THE VULNERABILITY OF VISION–LANGUAGE MODEL EXPLANATIONS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Explanation mechanisms are increasingly used to support transparency and trust in vision–language models (VLMs), particularly in settings where model decisions require human oversight. However, the robustness of these explanations remains insufficiently understood. In this work, we investigate whether explanation heatmaps in VLMs, particularly CLIP-based models, remain reliable under adversarial conditions. We show that explanation maps can be systematically manipulated while preserving the model’s original prediction, revealing a disconnect between predictive behavior and the evidence the explanation presents. To study this vulnerability, we introduce X-Shift, a white-box attack that perturbs the input image to redirect the explanation heatmap of the predicted class onto the region that the explainer associates with a second, attacker-chosen label, without altering the predicted output. Unlike conventional adversarial attacks that aim to induce misclassification, X-Shift specifically targets the integrity of the explanation process itself. The attack operates without modifying model parameters. We evaluate the proposed approach on MS-COCO, Flickr30k and ImageNet-1k with three CLIP backbones, demonstrating that the explanation is relocated onto the target region under small perturbations on image–backbone pairs while the prediction is preserved, with every result compared against random noise of the same budget. Our findings highlight a fundamental limitation of current explanation mechanisms in VLMs and raise concerns about their use as reliable indicators of model trustworthiness in high-impact applications.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
VLMs, Explainable AI (XAI), Adversarial Attacks, Explanation Manipulation, Trustworthy AI.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qx6fOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the vulnerability of VLM explanations to adversarial perturbations. It proposes X-Shift, a white-box attack that redirects the explanation heatmap of a predicted class toward a target region while preserving the original prediction. The method combines normalization-invariant explanation objectives with prediction-preservation losses. Experiments on three datasets and three CLIP backbones show explanation redirection on 45 selected image-backbone pairs.

Critical strengths
  1. The paper is generally well written, organized, and easy to follow. The problem formulation is clear, and the proposed attack objectives are consistent with the goal of manipulating explanations while preserving predictions.

  2. The results demonstrate explanation redirection while maintaining the original predictions on the evaluated samples.

Critical weaknesses
  1. Several existing studies have already demonstrated that model explanations can be manipulated while preserving predictions, although most do not focus on VLM. The paper discusses these studies in Section 2 but does not include direct experimental comparisons with them.

  2. The current baselines do not adequately establish the advantage of X-Shift over existing explanation-targeted attacks. Given the white-box setting, demonstrating successful explanation manipulation is less surprising without stronger competing methods. This also makes it difficult to assess the methodological novelty beyond adapting existing ideas to CLIP-based explanations.

  3. The motivation for Section 3.4 and Proposition 1 is not entirely convincing. The proposition characterizes gradient directions in explanation-map space, but it remains unclear how this property translates into more effective optimization of input perturbations. In particular, the theoretical result does not directly establish the practical advantage of the proposed attack.

  4. In Table 3, removing the contrast loss improves IoU→t from 0.86 to 0.92, whereas removing the shape loss substantially reduces it to 0.46. This suggests that the shape loss accounts for most of the redirection effect, while the necessity of the contrast loss remains unclear.

Questions for authors
  1. Can the authors compare X-Shift with existing explanation-targeted attacks under the same white-box setting and perturbation budget?

  2. What is the practical role of Proposition 1 in the attack design? Can the authors further explain how the normalization-invariance property in explanation-map space improves optimization in input space? If available, an ablation that changes only the normalization strategy would be useful.

  3. Given that removing the contrast loss improves the reported IoU→t, what motivates retaining it in the final objective? Could the authors provide further analysis of the interactions between the contrast, shape, and preservation losses?

  4. Why select the 45 selected image-backbone pairs rather than the entire datasets? Also, given the relatively limited transferability reported in Appendix C, could the authors clarify the scope of their generalization claims?

Reviewer_XUXXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a novel attack on CLIP-based models that shifts the explanation area to an incorrect region of the image. Consequently, even when the model's predicted labels remain correct, any subsequent remedy or solution is mistakenly applied to the wrong parts of the input. To achieve this, the authors propose X-Shift, a white-box attack designed to alter the model's attention mapping while preserving all other outputs. The authors demonstrate their approach on three CLIP models, showing successful attacks on the generated explanation outcomes.

Critical strengths
  • The attack setup is highly compelling. By specifically targeting the explanation area while keeping the output predictions intact, the attack can easily mislead users about "where" the underlying errors actually lie, ultimately preventing them from effectively resolving the issues.
Critical weaknesses
  • A major concern is whether CLIP remains a practical choice for generating explanations in this domain. While exploring diverse research directions is valuable, it is important to evaluate more widely adopted models. The paper does not explain why CLIP was selected for the surgery dataset instead of a VLM. If a VLM yields a higher IoU score, CLIP might no longer be used for these tasks which mainly asked for accuracy, which would significantly limit the real-world impact of the proposed attack despite its technical effectiveness.
  • Another critical question is how the adversarial perturbation would actually be deployed in a real-world scenario. Since the attack targets open-source models that users typically host locally or in their own private environments, inserting the perturbation poses a major practical challenge. Without a clear deployment vector, there is a risk that this threat model remains purely theoretical and confined to paper.
Questions for authors
  • Could the authors please clarify why CLIP remains a highly favorable target for this attack compared to more recent VLMs?
  • Similarly, could you elaborate on the practical deployment vectors for this threat model? Specifically, how could an attacker realistically introduce these perturbations to pollute the explanation outcomes for end-users in practice?
Reviewer_WyREOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether explanation heatmaps for CLIP-based vision–language models can be manipulated while preserving their predictions. It proposes X-Shift, a white-box input attack that redirects the predicted class’s explanation toward the clean explanation associated with a second label. The objective combines contrast and spatial-distribution matching on standardized explanation maps with penalties preserving logits, probabilities, image embeddings, and the decision margin. A proposition establishes the explanation objective’s invariance to positive affine transformations of the map.

Experiments cover three CLIP backbones and three datasets, using five screened images per dataset–backbone combination. At an ℓ∞ budget of 4/255, the attack succeeds on all 45 evaluated pairs without changing predictions, with target-map IoU ranging from 0.75 to 0.97 across experimental cells. The paper also reports random-noise controls, perceptual metrics, component ablations, and transfer experiments.

Critical strengths
  1. The attack convincingly separates explanation manipulation from prediction changes on the evaluated examples. The paper checks more than label preservation: it also measures embedding similarity and maximum probability changes, with same-budget random-noise controls. The large explanation shifts alongside small output changes provide persuasive evidence that prediction monitoring can miss manipulation of the attacked explainer.

  2. The objective design is supported by informative ablations. On the MS-COCO subset, replacing the standardized objective with the raw-score objective reduces target-map IoU from 0.86 to 0.45. Removing prediction-preservation terms causes nine prediction flips among fifteen pairs. These comparisons help explain which design choices produce redirection and which preserve predictive behavior.

Critical weaknesses
  1. The evaluation is too limited to establish the breadth suggested by the framing. Each dataset–backbone combination contains only five accepted images. Screening requires a large prediction-probability gap, and target selection favors spatial separation between source and target explanations. Although the selection procedure and rejection counts are disclosed, the resulting 45/45 success rate describes this restricted population. Transfer is also limited: 6/45 cases for patch similarity, 4/45 for attention rollout, 9/45 for ScoreCAM, and 20/85 across backbones. The evidence therefore supports a vulnerability of the specifically attacked model–explainer configuration more strongly than broad vulnerability across VLM explanation mechanisms.

  2. The incremental contribution over prior explanation attacks is not established sufficiently. Targeting explanation maps while approximately preserving model outputs predates this work, as demonstrated by Dombrowski et al. (2019). Applying this idea to CLIP and accounting for map normalization are potentially useful contributions. However, the empirical comparisons include random noise, prediction-targeted PGD, and a raw-score objective, rather than a directly adapted prior explanation-matching attack under matched constraints. Moreover, removing the contrast term improves target IoU to 0.92, indicating that the spatial matching term carries the main effect. A stronger targeted baseline is necessary to determine how much the proposed formulation adds.

Questions for authors
  1. Assessment-critical robustness check: How sensitive are success and redirection magnitude to the confidence-screening threshold and automatic target-selection rule? Can results on the already screened image pool clarify performance outside the five accepted examples per cell?

  2. Assessment-critical baseline: How does X-Shift compare with an adapted prior targeted explanation-matching objective using the same explainer, perturbation budget, optimization steps, preservation penalties, and appropriately normalized maps?

  3. Interpretation clarification: On how many evaluated examples is the original prediction verified as correct, and on how many does the target label correspond to an object actually present? Since targets can be absent on ImageNet, distinguishing semantic redirection from matching another label’s background activation would clarify the claims about misleading explanations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.