- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces Test-time Adversarial Takeover (TAKO), a framework that learns a vocabulary of universal adversarial patches offline via differentiable diffusion inference to induce directional action biases in frozen robotic diffusion policies. At test time, a human operator switches among these patches in the camera stream to steer the robot in closed loop across 2D manipulation, aerial delivery, and ground navigation tasks (spanning ResNet/Transformer backbones and DDPM/DDIM/flow-matching samplers), achieving 100% takeover success while target-policy matching (TPM) baselines fail completely.
Critical strengths
-
Novel composable takeover formulation
-
Broad cross-architecture and real-robot validation
-
good writing
Critical weaknesses
-
Table 1 claims "physical realizability", yet the real-world navigation experiment digitally overlays patches onto the camera stream via software rather than placing physical printed patches in the environment. If an attacker already possesses man-in-the-middle access to modify the live digital sensor stream in real time, hijacking the robot via visual patches is less direct than simply spoofing downstream action commands, weakening the practical justification of the threat model.
-
The quantitative evaluation only compares TAKO against a self-constructed, completely failing TPM baseline (0% ASR) without adapting existing diffusion-policy attacks (e.g., DP-Attacker) for targeted steering comparisons.
Questions for authors
-
Given that patches are digitally injected into the camera stream in RealNav, could you justify claiming "Physical realizability" in Table 1 and clarify a realistic security scenario where an attacker can arbitrarily overwrite live camera frames in real time but cannot directly alter the policy's action outputs or control topics?
-
Which specific inference sampler was used in Table 4 to achieve 40/40 online takeover success, and how does human control perform when the offline per-step directional success rate is relatively low (e.g., 65.8% for SimNav DDPM in Table 3)? Additionally, can stronger baselines (such as adapting DP-Attacker with a directional objective) be evaluated?