acceptodds
Under review as a conference paper at ICLR 2027

Still Camouflage, Moving Illusion: View-Induced Trajectory Manipulation in Autonomous Driving

Abstract

Existing physical adversarial attacks on vision-based autonomous driving induce time-evolving perception errors, including biased object tracking or trajectory prediction, through (i) a sophisticated physical patch that induces detection-box drift when entering the viewing range, or (ii) dynamically changing patches that cause different perception errors at different time. In both cases, viewing-angle variation is treated as a challenge, requiring adversarial patches to remain effective across frames under varying views, leading to complex multi-view optimization. In contrast, we show that viewing-angle variation itself can be turned into an attack tool. We design a new attack paradigm where a static, passive adversarial camouflage is mounted on a vehicle whose view-dependent appearance naturally evolves with relative motion, inducing consistent feature drift across frames. This causes the system to infer a physically plausible but incorrect trajectory, such as a false cut-in, which propagates to downstream decision-making and triggers unnecessary braking. Unlike prior approaches that require multi-view robustness or active intervention, our attack emerges from normal driving dynamics and is easy to deploy: a parked vehicle with a natural camouflage can induce hard braking in passing autonomous vehicles. We demonstrate the novel attack on the nuScenes dataset, showing the effectiveness with a system-level success rate of up to 64.4% on held-out test scenes, measured by hard-braking events, and robustness across different scene backgrounds, relative speeds, and perception models. We further validate the attack by case studies in Carla simulation and physical experiments on scaled car models.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Physical Adversarial Attacks, Autonomous Driving, Adversarial Camouflage, Viewing-angle Variation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7dBzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes an attack against autonomous vehicle perception (3D object detection). The attack adds a camouflage (i.e., a large patch) to a vehicle in order to create a displacement of the 3d bbox. The authors worked on getting a more "robust" attack success by optimizing the patch/camouflage to be viewing-angle invariant. It means that the perturbation should remain effective even when the victim vehicle approaches the attacker. Experimental results show some attack success rate from 50% to 87% depending on the scenario. The victim displacement is less than a meter but triggers braking at ~3.5 m/s2 (which is a moderate braking like when you are catching a changing yellow light).

Critical strengths

The paper is well-written and easy to follow. The authors said they used AI to assist, but it felt good to read a paper that isn't straight output of coding agent :-)

The authors have done a lot of work in testing the validity of their camouflage attack and I applaud their thoroughness.

Critical weaknesses

The idea of adding perturbations onto object to fool object detector is not novel. The problem of finding the static camouflage is exactly the same as for generating effective adversarial patches. An attacker must optimize the right perturbation that would resist expectations over transformation (angle, environmental condition, speed, size, color). The difference here is that this is 3D, but 3D adversarial perturbations have been also well-studied (see CVPR publications). Moreover, optimizing for viewing-angle invariance is similar (the same?) to EoT. Though, the authors acknowledged EoT in Section 4.4, and uses similar loss objectives. So novelty-wise, this is weak.

But overall, my issue is that I didn't learn anything from the paper. We already new the limited effects of adversarial patches (and can't explain why they sometimes transfer to other object detectors).

Attacker model and attack parameters: - What is the size of the camouflage (i.e., adversarial patch)? You only said that this is attached to the body frame, but it doesn't say how much area is required to get an effective attack. - Why a white-box access? The transfer attack you suggest is more realistic as we often don't know what OEM's vision model are (nor we can easily get access to it).

Experiments - Why K=3 (number of frames during which the perturbations should be effective)? This sounds too low of a bar. - Results Table 2: could you explain how we get a lower ASR for SUV-R-S BEVDet (compared to BEVDepth) while the d1, d2, d3 are higher (displacement)? - Section 5.2, I don't understand why the "cross-scenario training" is discussed here. Reading it from the start, I didn't see anything about training. So, it looks like I missed something or you forgot to explain properly what this is and how it related to the ASR (or displacement). - The attacks has a similar effect across the scenario. It would be worth adding a couple of sentences about the differences between these scenarios. As is, I don't know the difference and hence can't say how "generalizable" is the attack success. - Section 5.3 "factor effects": I like that you studied the sensitivity to different factors. But the conclusion is underwhelming. It is obvious that the closer the victim (camera) is to the camouflage, the larger it perceives the perturbation and hence it more affected (this is the same things as the size of the adversarial patch). So, as we don't actually learn anything from this experiment, I suggest moving it to the Appendix. - Section 5.4 "cross vehicle type": what is the actual size difference between SUV and Sedan? You need to study the mesh differences to demonstrate that this transferability is meaningful. As is, SUV and Sedan are very much the same shape, so I wouldn't expect much difference in the optimal camouflage, which would explain the "transferability". - Section 5.6: you are showing the 3d bbox displacement, but do you also have the results for the tracker? I am curious if the tracking algorithm thinks the vehicle is exiting the parking space, or if there is no movement "detected" (hence the braking is due to "object close to the victim vehicle"). In Figure 6, we see the red BEV planning, so it seems you have the trajectory prediction. How consistent is this prediction? Any sudden changes would raise concerns of being a noisy measurement/prediction.

  • Editorial comments • Table 1: to me the "viewing-angle dependency" X means "no dependency" but that isn't what is meant here. Might be worth changing to "no viewing-angle dependency". • Figure 2 would benefit to be bigger. • Would be worth explaining the targeted 3D bbox algorithms (maybe just FastBEV++) and how the camouflage affects it.
Questions for authors
  • Section 5.4 "transferability": here again you are facing the exact same results as what was shown by previous studies on adversarial patch. Transferability to multiple detector is not easy, and ensemble learning helps sometimes, but we don't understand fundamentally why. I would hope that you dig deeper in the theoretical reasons of why transferability works or doesn't. At least that would be a significant contribution.
1 comment

Dear Reviewer,

As the interactive rebuttal period is nearing its end, we sincerely hope that our clarifications have addressed your questions, and look forward to your feedback.

We sincerely appreciate your time and consideration :-)

The Authors

Reviewer_VSUhOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that a physical attack on a camera driving stack can use viewpoint change instead of fighting it. A pattern fixed on a vehicle looks different as the ego vehicle moves, and the authors use that change to push the detected 3D box in one direction over a few frames. Tracking and prediction can treat the shift as a cut-in, and a downstream planner brakes. The pattern is static at deployment. There is no display and no per-frame update.

The main numbers are on nuScenes (220 scenarios). The camouflage is rendered onto existing frames. For SUV-R-S and SEDAN-R-S, each with 60 clips, one pattern is trained on four folds and tested on the held-out fold. Hard braking means maximum braking of at least 3.0 m/s² inside a three-frame window. The best cell in Table 2 is 64.4% (sedan, front-right, same direction, BEVDet, Pipeline B), a mean over three seeds. Averaged over the two scenarios and three detectors, ASR is 54.8% on Pipeline A and 60.2% on Pipeline B. ST-P3 is at 32.5% ASR, with mean maximum braking 2.31 m/s² (Appendix H). The other maneuver categories are optimized and scored on the same clips (Table 3, Pipeline A). A CARLA case study and a 1:15 print-and-recapture study are included.

Critical strengths
  • The distinction in Table 1 is real and worth discussing. Prior vehicle camouflage mostly targets hiding or misclassification and treats viewpoint as a nuisance. Tracker-hijacking work often needs a pattern that changes over time. A fixed pattern whose temporal structure comes from ordinary relative motion is a different mechanism. On BEVDet, Table 6 agrees with that part of the story: removing the viewing-angle filter drops final-frame displacement from 0.67 m to 0.48 m and ASR from 54.5%/62.5% to 35.0%/51.7%.
  • The main protocol is a real experiment. Two categories use 5-fold cross-validation, three detectors, two planner stacks, and three seeds. Appendix E shows the headline cell is stable across seeds (64.43 ± 1.96 on Pipeline B). Success is defined at the planner, with box motion, prediction error, and path clearance reported beside it. That is the right level for a system-level claim.
  • The negative transfer result is included, which is the correct result to show. In Table 4 the best cross-detector final-frame displacement is 0.13 m. Appendix F reports that ensemble training adds 0.09 m on average and at most 0.19 m, and that the transferred effect stays well below the white-box displacement.
  • Section 6 states that commercial vehicles were not tested. Section 5.3 presents the factor plots as observational because the nuScenes factors are correlated, and it says those factor effects weaken under cross-validation.
Critical weaknesses
  • The abstract's 64.4% is the best cell in Table 2, not the result. It is sedan / front-right / same direction / BEVDet / Pipeline B. The same table's averages are 54.8% and 60.2%. On ST-P3, Appendix H reports 32.5% ASR and mean maximum braking of 2.31 m/s², below the authors' own 3.0 m/s² line. "Up to 64.4% on held-out test scenes" leads with the best white-box cell.
  • Held-out here means another clip of the same vehicle type, the same side, and the same direction. A new city, sensor, or stack is not part of the split. Table 3 then drops the split. One pattern is optimized on all clips of a category and scored on those clips, and the caption limits the table to Pipeline A. The ten cells contain 19, 13, 14, 8, 9, 6, 4, 13, 6, and 8 clips (100 total). The reported average ASR of 75.6% is a same-set number. It is not a generalization result and should not be pooled with the cross-validation ASR.
  • Box displacement is the component along a direction chosen on the training split and frozen for the matching test split. A larger d3 means the box moved the way the objective asked. That matches the optimization. It is weak evidence of an independent motion illusion. The planner metrics are the separate evidence, and they are the ones that have to carry the system-level claim.
  • The braking rates in Table 2 are white-box, which the threat model allows during optimization. The practical reading of one parked pattern meeting an unknown stack is Table 4, where the best transferred final-frame displacement is 0.13 m. Ensemble training does not close that gap. Cross-vehicle transfer is a different and stronger bbox result: Table 5 reports 0.34 m and 0.35 m final-frame displacement on the other body, with no planner ASR.
  • Appendix G, Table 10, re-scores Jia et al., AttrackZone, and ControlLoc on this protocol: d3 of 0.53 m versus 0.10 m for Jia et al., and ASR of 54.8%/60.2% versus 12.5%/13.3%. The paper states that these methods target 2D detection or tracking. The gap is real under this scoring. It is a weak comparison for a 3D camouflage claim. ControlLoc's Pipeline B ASR is 14.2%, so Jia et al. is not the strongest braking baseline on both pipelines. Table 2 also has no clean-clip rate. The scenes are prefiltered so the target is already ahead of the ego vehicle and relevant to its path. Appendix D makes a pure short-horizon explanation unlikely: at four frames, ASR is 54.9%/59.6%, close to the three-frame result, while the progressive-displacement rate falls to 54.2%.
  • The closed-loop and physical evidence is a case study, and the abstract's parked-vehicle sentence is ahead of it. In CARLA the attack brakes in 3 of 4 scripted scenarios, from 25 km/h to about 11 km/h, and does not brake at the intersection. Clean runs do not brake. Section 5.6 names BEVDet, CenterTrack, HiVT, and PID control. Appendix I describes BEVDet, ego-motion-compensated tracking, constant-velocity prediction, and an ACC planner with PID braking. Those descriptions are not the same stack. Appendix J is not the body camouflage from the method section. Two flat patches are printed, placed on a 1:15 model, and filmed with a phone. Last-frame drift is 5.3 cm on the optimization scene and 2.4 cm on another scene, reported as 0.80 m and 0.36 m after scaling by 15. That study has no planner. On the digital side, Appendix B shows the effect depends on coverage: at 25% of the paintable region, ASR falls from 54.8%/60.2% to 11.7%/19.7%. Whether a driver or a simple consistency check would flag a body-sized pattern is left to a short mitigation paragraph.
Questions for authors
  1. On the same nuScenes clips and planners, what fraction of clean sequences already reach 3.0 m/s² inside the three-frame window? CARLA reports no braking on clean runs. Table 2 does not. Please add that row.
  2. Please keep cross-validation ASR and same-set ASR separate, and give the clip count for each cell. Table 3 is Pipeline A, fit and scored on the same clips. Which number should be compared with prior work?
  3. Table 4's best cross-detector final-frame displacement is 0.13 m, and no planner metric is reported there. Does any black-box pattern still move the planner, or do the braking results require white-box access to the victim detector?
  4. In Appendix J, what is the box motion for a same-size non-adversarial printed cover? Why should 5.3 cm and 2.4 cm on the model, multiplied by 15, be read as 0.80 m and 0.36 m on a real vehicle? Does either printed pattern change a planner output?
  5. Which software stack produced Table 11? Section 5.6 names CenterTrack and HiVT. Appendix I describes ego-motion-compensated tracking, constant-velocity prediction, and an ACC/PID brake.
2 comments

Dear Reviewer,

As the interactive rebuttal period is nearing its end, we sincerely hope that our clarifications have addressed your questions, and look forward to your feedback.

We sincerely appreciate your time and consideration :-)

The Authors

Reviewer_VSUhOpenReview ↗

Official comments

After reviewing all reviewers' comment and authors' reply. I decided to keep my score.

I have read the paper and used AI to generate partial reviews. Frankly, I do not have sufficient expertise to assess this submission.

Reviewer_RDp9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a physical adversarial attack on vision-based autonomous driving systems. The main idea is to apply static camouflage to a vehicle so that it creates fake motion. The fake motion comes from the natural change in viewing angle. As the viewing angle changes, the camouflage shifts the detected 3D bounding box across frames. This shift can mislead tracking, trajectory prediction, and planning. The authors design a framework with three parts: scenario selection, differentiable rendering, and planning-guided optimization. Experiments on nuScenes show an attack success rate of up to 64.4%. The paper also includes CARLA simulations and physical experiments with scaled vehicle models.

Critical strengths
  1. Most adversarial camouflage methods try to stay robust to viewpoint changes. This paper instead uses those changes to create a motion illusion. This offers a fresh perspective on physical adversarial attacks.

  2. The authors study how detection errors propagate through tracking, trajectory prediction, and planning. They show that relatively small detection errors can eventually cause unnecessary braking. This makes the work more meaningful.

  3. The evaluation is broad. The authors test three 3D detectors, two downstream pipelines, and one end-to-end driving model. The ablation studies, CARLA simulations, and physical experiments further support the effectiveness of the attack.

Critical weaknesses
  1. Table 4 shows that camouflage optimized for one detector performs poorly on other detectors. For example, the final displacement drops from 0.67 m on BEVDet to only 0.01 m on BEVDepth. This is a major limitation because attackers may not know which perception model a real autonomous vehicle uses.

  2. The ablation study without VAF is useful, but it also changes the training samples. It is therefore difficult to determine how much of the improvement comes from viewing-angle variation. A direct comparison with conventional static camouflage or a method without temporal optimization would help clarify this contribution.

  3. The authors use AFF and VAF to select suitable attack scenarios, but they do not report how many original scenarios are filtered out. This makes it difficult to assess how often the attack would work under normal driving conditions. In addition, the experiments in Table 3 use the same scenarios for optimization and evaluation. These results therefore do not demonstrate generalization to unseen scenes.

  4. The physical experiments use a 1:15-scale vehicle model and an iPhone camera. The CARLA evaluation also covers only four driving scenarios. Although the results are encouraging, it remains unclear whether the camouflage would work reliably on real vehicles under different camera settings, viewing distances, and lighting conditions.

Questions for authors
  1. Table 4 shows that camouflage optimized for one detector barely works on other detectors. For example, the final displacement drops from 0.67 m on BEVDet to only 0.01 m on BEVDepth. This is a major limitation because attackers may not know which perception model a real autonomous vehicle uses.

  2. The ablation without VAF is useful, but it also changes which samples are used for training. It is therefore hard to tell how much of the improvement comes from viewing-angle variation. A more direct comparison would help clarify the contribution. Examples include conventional static camouflage or a method without temporal optimization.

  3. The authors use AFF and VAF to select suitable attack scenarios. However, it is unclear how many original scenarios are filtered out. This makes it hard to judge how often the attack would work in normal driving. In addition, the scenario-specific experiments in Table 3 use the same scenarios for optimization and evaluation. These results therefore do not show generalization to unseen scenes.

  4. The physical experiments use a 1:15-scale vehicle model and an iPhone camera. The CARLA evaluation covers only four driving scenarios. These results are encouraging. However, it remains unclear whether the same camouflage would work reliably on real vehicles. Real conditions involve different cameras, distances, and lighting.

1 comment

Dear Reviewer,

As the interactive rebuttal period is nearing its end, we sincerely hope that our clarifications have addressed your questions, and look forward to your feedback.

We sincerely appreciate your time and consideration :-)

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.