- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TrajPoison, a clean-label, trigger-free availability poisoning attack on VLA training data. It perturbs the images of a subset of demonstrations, focusing on high-curvature time windows of the trajectory, using action-weighted PGD with cross-frame momentum. With 10% poisoned demonstrations and ε = 4/255, success rates on RoboTwin drop by about half on Easy and to near zero on Hard for RDT, π0 and OpenVLA.
Critical strengths
- Availability poisoning of robot demonstration data is a relevant and unexplored threat, and the idea of targeting action-critical moments of a trajectory is intuitive.
- The evaluation covers three architectures, cross-model transfer, a real-robot setting, and several input-space defenses, with component ablations.
Critical weaknesses
- The effect is strong but is not well explained or controlled. Perturbing about 2% of the frames (10% of trajectories × ~20% of timesteps) with ε = 4/255 halves π0 Easy SR (84.0 → 47.5), and even 2% of trajectories gives 60.2. The paper does not explain why this works beyond one sentence about "adversarially displaced visual features", and the mechanism analysis (attention, teacher-forced, closed-loop) describes how failures appear rather than why the poison works. There is no random-noise control with the same ε and ratio, and even the adapted backdoor baselines already reduce π0 Easy SR to about 57–61, which suggests the setup may be sensitive to any perturbation. Unlearnable Examples and adversarial poisoning, the closest prior work, are not compared.
- The method design is not well justified and partly inconsistent. For single-frame surrogates (e.g., π0, OpenVLA), the gradient for δ_t is q_t times the gradient of d_t, and q_t is cancelled by the L1 normalization and sign in Eq. 3, so the frame weighting has no effect; it is unclear what the "w/o action weighting" ablation removes. The cross-frame momentum makes perturbations in a window temporally consistent, but the paper states the attack does not need a history-based victim, and it is not explained why temporal consistency should matter for a single-frame policy. Eq. 3 mixes iteration and time indices, and the number of iterations and window length are not given. Figure 2 also shows a temporal encoder and a "noise detection" step (detected → poisoning blocked), which do not appear in the method or experiments.
- The evaluation is incomplete and partly inconsistent. In Table 4, the frame-wise and uniform variants have NAE equal to or below the clean model (0.042 / 0.037 vs. 0.042) while SR drops, which is hard to interpret. Only test-time defenses are evaluated; training-time defenses that are standard for availability poisoning (adversarial training, strong augmentation, early stopping, poison detection) are not. The real-robot clean success rates are low, which makes the relative drop less informative.
Questions for authors
- What is the effect of random noise with the same ε and poisoning ratio?
- How does TrajPoison compare with Unlearnable Examples and adversarial poisoning under the same setting? Why can poisoning about 2% of frames override the remaining clean data?
- Given that q_t cancels under the normalized sign update, what exactly does the "w/o action weighting" ablation remove? Why does temporal consistency of perturbations matter for single-frame policies?
- What does the "noise detection" step in Figure 2 refer to? Please also give the number of iterations and the window length.
3 comments
Response to Reviewer jAJg (2/2): Temporal consistency, defenses, and evaluation metrics
Q3 (continued): Temporal consistency
After selecting the action-critical window centers, each center defines a temporal window:
corresponding to 11 frames before overlap merging.
Regarding temporal consistency, we clarify that cross-frame coordination does not require the victim policy to jointly process multiple frames or employ an explicit temporal encoder.
For a single-frame policy, each observation is processed independently at inference time:
However, the model parameters are shared across all timesteps. During fine-tuning, observations from neighboring timesteps contribute to updates of the same model parameters.
If perturbations are optimized independently for each frame, their induced visual biases may be inconsistent across neighboring observations, resulting in less coordinated training signals.
TrajPoison addresses this issue through cross-frame momentum during perturbation construction. By coordinating perturbation updates across adjacent observations, it encourages more consistent visual biases across temporally neighboring training samples. These observations subsequently contribute to the optimization of shared policy parameters, even though they are processed independently during inference.
Therefore, the effectiveness of cross-frame coordination in single-frame policies arises from the interaction between temporally neighboring training observations and shared model parameters, rather than from an architectural requirement to process video sequences jointly.
Our ablation results support this design: removing cross-frame momentum increases RDT Easy SR from 36.1% to 48.6% and decreases NAE from 0.245 to 0.156.
Q4. Noise detection, optimization iterations, and window length
We clarify that the noise-detection discussion concerns whether perturbations introduced by TrajPoison can be identified by conventional low-level perturbation-screening methods.
We have completed Strong Data Augmentation and Video Re-compression evaluations; Feature-Squeezing-based Poison Detection remains ongoing.
The results of the first two defense categories are summarized below.
Table 8. Evaluation against data-side defenses.
| Model | Defense | Clean SR | Poisoned SR | Clean NAE | Poisoned NAE |
|---|---|---|---|---|---|
| RDT Easy | Two-way MixUp | 67.2 | 41.3 | 0.071 | 0.207 |
| Four-way MixUp | 65.4 | 39.6 | 0.075 | 0.223 | |
| CutMix | 66.3 | 40.6 | 0.073 | 0.215 | |
| Video Re-compression | 60.5 | 42.3 | 0.075 | 0.194 | |
| π0 Easy | Two-way MixUp | 71.2 | 53.7 | 0.080 | 0.165 |
| Four-way MixUp | 69.4 | 51.2 | 0.084 | 0.181 | |
| CutMix | 70.0 | 52.6 | 0.082 | 0.173 | |
| Video Re-compression | 73.5 | 55.5 | 0.070 | 0.164 |
The results show that TrajPoison remains effective under all evaluated data transformations. The clean-to-poisoned SR gap remains substantial across both architectures, indicating that these common training-time transformations do not eliminate the poisoning effect.
The Feature-Squeezing-based poison detection experiment is currently ongoing. Its results will be incorporated into the revised manuscript when available.
Implementation details: Poison optimization uses 20 iterations. The window selection and 11-frame length are specified above; overlapping windows are merged before optimization (Appendix A.3–A.4).
Q5. Relationship between NAE and SR
We thank the reviewer for identifying the apparent inconsistency in the original ablation results.
The frame-wise and uniform-frame variants exhibit global NAE values equal to or below that of the clean policy (0.042 and 0.037, respectively, compared with 0.042), while their SR decreases.
NAE and SR measure different aspects of policy performance. NAE is an averaged continuous measure of action prediction error across timesteps and action dimensions, whereas SR measures successful completion over an entire closed-loop rollout.
Consequently, localized action deviations at critical manipulation stages, such as grasping or object placement, may cause task failure while contributing only marginally to the global average NAE. A relatively small prediction error at a critical control transition can therefore have a much larger effect on task completion than its contribution to the global average error suggests.
We therefore interpret these results as showing that changes in global average action error and task-level success are not strictly correlated. In particular, the frame-wise and uniform-frame variants should not be interpreted as evidence of increased global action prediction error.
In contrast, the full TrajPoison method increases NAE from 0.042 to 0.245 while reducing SR from 70.4% to 36.1%. Removing cross-frame momentum or action weighting reduces this effect, with NAE decreasing to 0.156 and 0.173, respectively.
We will clarify the distinction between global prediction error and task-level execution failures in the revised discussion.
I thank the authors for the rebuttal, which does clarify several points that were originally unclear. However, the many inconsistencies and contradictions in the paper itself remain unresolved. For instance, the noise-detection step in Figure 2, and I hope the authors will further polish the paper itself. I am therefore keeping my original score.
Response to Reviewer jAJg (1/2): Random noise, poisoning mechanism, and action weighting
Q1. Random noise with the same perturbation budget and poisoning ratio
We agree that random noise with the same perturbation budget and poisoning ratio provides an important control for evaluating the contribution of the proposed optimization.
We have conducted matched-budget random-noise experiments on RDT Easy, π0 Easy, and OpenVLA Easy.
Table 7. Comparison between random noise and TrajPoison.
The random-noise baseline uses the same selected demonstrations and frames, poisoning ratio (ρ=10) and perturbation budget (ϵ=4/255), while replacing the optimized perturbations with random noise.
Random noise leaves RDT and π0 largely unchanged and causes only a modest OpenVLA SR decrease (64.1% to 60.4%), unlike TrajPoison.
Overall, random noise does not reproduce the substantial degradation caused by TrajPoison. These results suggest that the attack effectiveness does not arise simply from introducing bounded pixel perturbations, but from the targeted optimization of action-critical observations and their temporal coordination.
The slight improvement observed on RDT may be consistent with a weak regularization effect of small random perturbations, although we do not interpret it as evidence of a general robustness improvement.
Q2. Why can poisoning approximately 2% of frames affect the remaining clean data?
The influence of poisoned frames is not proportional to their frequency in the training dataset. In robotic manipulation, task success often depends on a small number of fine-grained control transitions, such as grasping, precise alignment, and object placement. Small action deviations at these stages can cause an entire manipulation episode to fail, even when most other actions remain accurate.
TrajPoison specifically targets these high-impact temporal regions rather than uniformly modifying demonstration frames. We select the top 20% of candidate timesteps ranked by qt as action-critical window centers. Given a poisoning ratio of 10% at the demonstration level, this corresponds to a nominal selection of approximately 2% of temporal positions before temporal-window expansion, assuming comparable trajectory lengths.
Importantly, this approximately 2% figure refers to selected window centers, rather than the final proportion of unique modified frames after expanding and merging temporal windows. The exact modified-frame proportion depends on trajectory lengths and window overlap.
These poisoned observations do not overwrite or replace the remaining clean data. Instead, they contribute to updating the shared parameters of the VLA policy. Their influence is therefore not restricted to the modified frames themselves. By targeting visually sensitive observations associated with important control transitions, TrajPoison can induce a bias in the learned perception-action mapping despite the presence of a larger number of clean observations.
Furthermore, the induced perception-action bias can affect subsequent robot states and observations during deployment. An initial action deviation may alter the execution trajectory, leading to further deviations through closed-loop robot-environment interaction.
Our teacher-forced and closed-loop experiments provide supporting evidence. On RDT Easy, NAE increases from 0.040 to 0.137 under teacher forcing, where predicted actions are not executed. Under closed-loop execution, NAE further increases from 0.042 to 0.245.
These results illustrate how perturbing a small fraction of control-critical observations can influence the shared policy parameters and subsequently affect the entire execution trajectory.
Q3. Clarification of q_t, Ω_t, and temporal consistency
We have revised the formulation to explicitly distinguish timestep selection from action-dimension weighting.
The timestep importance score qt is used to rank candidate timesteps and select the top 20% as action-critical window centers. It therefore determines where perturbations are applied.
For the single-frame surrogate, the scalar qt cancels in the normalized per-frame gradient update and does not affect its direction. The effective action-dimension weighting is provided by the vector Ωt, which reflects the relative importance of different action dimensions in the perturbation objective.
Accordingly, the w/o action weighting ablation replaces Ωt with an all-ones vector while keeping the selected action-critical windows unchanged. This removes dimension-wise action weighting without changing temporal localization.
(Continued in Part 2.)