acceptodds
Under review as a conference paper at ICLR 2027

Beyond Image-Wide Color Shifts: Scene-Compatible Semantic-Region Triggers for Backdoor Attacks

Abstract

Image-wide color space backdoor attacks can be visually subtle, but applying a single transformation to semantically distinct regions may produce chromatic changes that conflict with object boundaries and local illumination. We argue that stealth depends not only on perturbation magnitude, but also on where the transformation is applied and how its strength adapts to local content. Guided by this insight, we propose a scene-compatible semantic backdoor framework that decouples semantic carrier selection from spatially adaptive perturbation control. A stability-aware selection strategy identifies complete regions whose appearance changes are plausible within the surrounding scene, while texture-adaptive modulation assigns bounded, input-dependent color shifts according to local visual tolerance. Across four dataset–architecture settings, our method consistently achieves attack success rates above 98% while preserving clean accuracy and low perceptual distortion. Evaluations under representative defenses further suggest that semantically grounded, sample-dependent triggers challenge assumptions based on fixed or spatially regular trigger patterns.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Backdoor attacks, semantic carrier selection, scene-compatible triggers, color space perturbations, visual stealthiness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Sx66OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper identifies scene-level artifacts in image-wide color backdoors as a mismatch between shared image transformations and object-level scene structure, and formulate a scene compatible semantic trigger paradigm that determines where a local appearance change is contextually permitted. Guided by this insight, the authors propose a scene-compatible semantic backdoor framework that decouples semantic carrier selection from spatially adaptive perturbation control.

Critical strengths

Based on the visualizations, the method indeed shows a clear advantage under illumination changes.

Critical weaknesses

Only three defense methods are compared, all from before 2019; the defense-evasion experiments are therefore not convincing.

The research approach is problematic: the datasets used for experimental validation are simple image classification tasks (with simple ResNet-like models), while the proposed method relies on a more complex multimodal model.

Questions for authors

1.The defense comparison includes only three methods, all from before 2019; consequently, the defense-evasion experiments provide limited support for the claimed robustness. To substantiate the evaluation, comparisons with more recent and diverse defense baselines are necessary.

2.Stealthiness is also closely related to the poisoning rate. The main experiments adopt a 10% poisoning rate, whereas Narcissus can achieve an attack with only 0.05% (invisible semantic triggers). Therefore, the claimed contribution to stealthiness is not sufficiently convincing.

3.The statement “We argue that stealth depends not only on perturbation magnitude, but also on where the transformation is applied and how its strength adapts to local content” is questionable. Under this principle, would common unavoidable noise be misclassified as backdoor triggers? This concern warrants clarification.

4.The research design raises methodological concerns: the experimental validation relies on simple image classification tasks (with simple ResNet-like models), whereas the proposed method is based on a more complex multimodal model. This mismatch raises the question of whether the evaluation is appropriately aligned with the method’s intended setting.

5.Semantic, sample-dependent triggers have already been proposed, such as SIBA and Narcissus. In light of these prior works, what are the core innovations and advantages of the present paper?

6.Victims do not have access to clean images. Therefore, provided that poisoned images do not appear abnormal, they are difficult to detect. The core of backdoor stealthiness lies in indistinguishability under both machine-based numerical analysis and human visual inspection, rather than in excessive reliance on absolute similarity to the original image. This paper further improves stealthiness with respect to brightness, but the brightness difference from the original image is difficult for victims to observe directly. Thus, the practical value of this improvement remains unclear.

7.The writing and framing of contributions require improvement: the authors claim three contributions in the Introduction, but experimental results should not be presented as independent innovations or contributions.

8.The theoretical and efficiency analyses are also incomplete. Neither a comparison of computational cost nor a theoretical foundation for the proposed assumptions has been fully analyzed.

Reviewer_CjA4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a scene-compatible semantic backdoor attack that aims to solve the visual inconsistencies introduced by image-wide color transformations. It decouples semantic carrier selection from spatially adaptive perturbation control and utilizes the existing language and visual models to identify the regions whose changes are plausible within the surroundings. It then applies texture-adaptive, input-dependent perturbations in Oklab color space to improve visual stealthiness while maintaining attack effectiveness. However, the proposed method seems to be a simple combination and utilization of existing technologies. Meanwhile, for experiments, important comparisons and ablations are missing, and the defense evaluation has several shortcomings.

Critical strengths
  1. The motivation is clear, and the reported attack success rates are high.

  2. The method is evaluated on both CNNs and a ViT.

  3. The appendix provides prompts and implementation details.

Critical weaknesses
  1. The technical contributions of this paper are unclear and limited. The method combines existing region proposal, segmentation, prompt-based ranking, and learned color perturbations. The related work does not adequately explain its differences from semantic backdoors, input-dependent backdoors, or region-based adversarial attacks.

  2. The claim is not verified and supported by its experiments. PSNR, SSIM, LPIPS, and DreamSim measure similarity, not whether a color change fits an object’s material or local lighting.

  3. The comparisons and ablations are insufficient. Relevant backdoor attacks such as LIRA and Input-Aware are missing, and full baseline comparisons cover only Imagenette/ResNet18.

Questions for authors
  1. Can the author clearly claim the technical contributions of the paper?
Reviewer_mpRAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a scene-compatible semantic backdoor that selects carrier regions based on scene semantics and illumination compatibility, and applies texture-adaptive perturbations according to local visual characteristics. The proposed method achieves high attack success rates while maintaining low perceptual distortion across diverse datasets and model architectures. Its effectiveness is further evaluated through comparisons with existing backdoor attacks and representative defense methods.

Critical strengths
  • This paper proposes a scene-compatible semantic backdoor by combining carrier selection based on scene semantics and illumination conditions with texture-adaptive perturbation control. This approach enables the trigger to better align with the surrounding scene.
  • The proposed method achieves high attack success rates while maintaining low perceptual distortion across diverse datasets and model architectures. The experiments provide empirical evidence of both attack effectiveness and low perceptual distortion.
Critical weaknesses
  • The paper proposes a carrier selection strategy using Qwen and MiniGPT-4, but the reliability of the selected carriers is not sufficiently validated. In particular, there is limited evidence regarding selection consistency across different prompt formulations and the semantic and illumination compatibility of the selected regions, as assessed through human evaluation or other independent assessments.
  • Although the paper evaluates several representative backdoor defenses, its effectiveness against adaptive defenses tailored to the proposed attack remains unexplored. Furthermore, the lack of robustness evaluation under image transformations and preprocessing limits the assessment of whether the attack remains effective in practical settings.
Questions for authors

The following questions are related to the critical weaknesses identified above.

  • The rationale for selecting Qwen and MiniGPT-4 and the extent to which carrier selection depends on the choice and performance of these models are not clearly established. Could the paper examine whether similar results can be obtained with other VLMs and how different prompt formulations affect selection consistency and effectiveness? Additionally, could the semantic and illumination compatibility of the selected carriers be independently assessed through human evaluation or other methods?
  • The robustness of the proposed attack against adaptive defenses tailored to its characteristics, as well as against image transformations and preprocessing operations, remains insufficiently evaluated. Could the paper provide additional experimental results showing whether high attack success rates are maintained under these conditions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.