acceptodds
Under review as a conference paper at ICLR 2027

Exploring Full-View RGB-T Vehicle Adversarial Robustness in the Physical World

Abstract

Multimodal RGB-Thermal (RGB-T) vehicle detectors are widely assumed to be robust owing to their complementary dual-modality design. However, their security under adversarial attacks remains insufficiently explored, particularly across the full viewpoint hemisphere. In this paper, we present the first full-angle physical adversarial attack against RGB-T vehicle detectors, covering azimuth angles of – and elevation angles of –. We first conduct a comprehensive full-angle vulnerability analysis, revealing that existing attack methods suffer significant performance degradation under this challenging setting. To address the identified challenges of modality imbalance, angular imbalance, and digital-to-physical transfer gap, we propose three key techniques: (1) an adversarial shape-texture co-optimization (STCO) method that balances attack effectiveness across both modalities; (2) an angle-adaptive attack strategy (AAAS) that allocates more optimization effort to hard-to-attack viewpoints; and (3) a cross-modal foreground and background Expectation over Transformation (CFBE) method that enhances physical-world transferability. We further propose an angle-adaptive adversarial training method as a defense baseline. Extensive experiments in both digital and physical worlds, demonstrate the effectiveness of our attack against the state-of-the-art RGB-T detector and its strong transferability to unseen detectors.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Physical adversarial attack, adversarial example, Trustworthy AI, Multimodal AI, RGB-T

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UEt3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

To address the identified challenges of modality imbalance, angular imbalance, and digital-to-physical transfer gap, this paper propose three key techniques: (1) an adversarial shape-texture co-optimization (STCO) method that balances attack effectiveness across both modalities; (2) an angle-adaptive attack strategy (AAAS) that allocates more optimization effort to hard-to-attack viewpoints; and (3) a cross-modal foreground and background Expectation over Transformation (CFBE) method that enhances physical-world transferability. Extensive experiments in both digital and physical worlds, demonstrate the effectiveness of our attack against the state-of-the- art RGB-T detector and its strong transferability to unseen detectors.

Critical strengths
  1. The authors conducted extensive experiments, effectively demonstrating the validity of the proposed scheme.
  2. The full-view attack proposed by the authors is a significant issue.
Critical weaknesses
  1. This framework primarily combines various techniques explored in previous research on adversarial attacks. Although integrating these components into RGB-T attacks is reasonable, the paper does not sufficiently demonstrate the novelty of their approach.
  2. The paper does not clearly establish whether the baseline attacks are evaluated under comparable conditions in terms of the number of stickers, surface coverage, and full-view optimization budget. This makes it difficult to determine whether the reported improvements arise from the proposed techniques or differences in experimental configurations.
  3. The physical experiments are conducted on 1:24 scale model vehicles under controlled conditions. It remains unclear whether the proposed adversarial patterns can maintain their effectiveness on full-size vehicles and across different physical environments.
  4. The paper primarily reports average ASR across different viewing angles, without providing worst-viewpoint or lower-percentile ASR.
Questions for authors
  1. Can the authors provide comparisons in which the baseline attacks are optimized using the same number of stickers, surface coverage, and full-view optimization budget as the proposed method?
  2. How well do the attack patterns generalize to full-size vehicles and different physical environments?
  3. Can the authors report worst-viewpoint or lower-percentile ASR to demonstrate whether the proposed method consistently improves attack effectiveness at difficult viewing angles?
Reviewer_UpQxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the full-view physical adversarial robustness of RGB-T vehicle detectors. The authors introduce an evaluation protocol covering 360 degrees of azimuth and 0–90 degrees of elevation, and identify limitations of existing attacks under large viewpoint variations. To address these challenges, they propose shape-texture co-optimization (STCO), an angle-adaptive attack strategy (AAAS), and cross-modal foreground-background expectation over transformation (CFBE). An angle-adaptive adversarial training method is also introduced as a defense baseline.

Critical strengths

Earlier RGB-T attacks targeted narrow viewpoints. This paper sweep a far wider angular range, which surfaces detector weaknesses that aggregate scores alone would hide a sharper stress test for multimodal perception.

This paper add three modules, each fixing one failure: imbalanced modalities, skewed viewpoints, and the digital-to-physical gap. Ablations show they help for different reasons, and physical attacks beat existing baselines by a clear margin.

Critical weaknesses

Physical tests use only 1:24-scale car models and six representative viewpoint groups, not the full azimuth-elevation grid swept in simulation. Miniature cars also differ from real ones in thermal signature, imaging scale, and material response. The results show multi-view effectiveness on scaled models but not full-view attacks on real vehicles. This undercuts the paper's central claim.

Equation (1) defines the attack as evasion at every viewpoint, yet results are reported as average ASR. A mean of 82.4% says nothing about per-viewpoint success; side-view and overhead ASRs in the physical tests are only 51.0% and 37.0%. To back the full-view claim, the paper should report worst-viewpoint performance, angular coverage at fixed thresholds, and failure distributions.

Questions for authors

see weakness

Reviewer_8guUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a physically-realizable adversarial attack on RGB-T vehicle detection models. It builds upon a prior work on thermal detection adversarial attacks that optimizes the shape of a 3D mesh and then casts its shadow on a car with the shadow being the adversarial patch.

Critical strengths

The topic is of great interest. This work extends the prior method to the multi-modal RGB-T domain and additionally manipulates a vehicle appearance and colours (outside the thermal shadow patch). This way both modalities are targeted.

It also focus on the problem of multi-viewing angle attacks; i.e. attacks where a car is viewed from various angles (along sides and top primarily). The paper shows how prior works struggle to successfully attack detection models under most possible viewing angles and only succeed within a small window. They propose training augmentation that scales view point sampling the inverse of the attack success rate.

Critical weaknesses

It is not convinced that the paper (e.g., Fig. 1) explains the core motivation behind the paper accurately. There seems to be confounding factors and an inexplicable failure of attack methods where they should normally work. Specifically I have two main concerns:

  1. Confounding factors: If clean car detection already fails at high elevation (fig 3c) how can we make a claim on the attack success of any adversarial method. What if they’re not successfully attacking the models, it’s just that the models are failing to detect by default (Fig 3c shows high elevation clean car detection being really bad).
  2. I would expect prior methods that didn’t care about view point angles and just train no typical low elevation settings to show high ASR at least on some angles along the azimuth but generally at low elevation. But we only see high ASR in fig1 near the top. My impression is that all prior methods including the proposed method (under single-patch) fail at the typical attack view point / angle (low elevation) and succeed only at high elevation where we know even a clean car is hardly detected.

The comparison methods seem limited. Not sure whether the most related SOTAs were compared here.

Also, there are writing and reference issues: E.g., The cfbe acronym is not used within the section that’s describing it; EOT acronym not explained or expanded first time. E.g., It doesn’t seem that the ref mentioned in related works (Wei et al. 2023b) is describing top-down views as mentioned. E.g., It’s unclear what YOLOv9 is doing in the paper? E.g., I’m not sure how the clean vehicle is performing with a higher ASR than a multi-modal adversarial attack baseline CDUPatch. Page 7 line 364 E.g., From Figure 3 and Table 1, it seems digital performs really well on the side profile of a car (side (θ ≈90◦±30◦)) but physical evaluation shows the side being much worse?

Questions for authors
  1. Better explain and justify the motivation and novelty.
  2. Compare with more SOTAs or justify the currently investigated comparison methods in the tables are SOTAs and sufficient for comparison.
  3. Provide more technical details as mentioned above, and explain the result discrepancy in Fig. 3 and Table 1.
  4. Check and confirm the references to make sure the cited contents are consistent with the ref(s).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.