acceptodds
Under review as a conference paper at ICLR 2027

TrajPoison: Trajectory-Aware Clean-Label Poisoning of Vision-Language-Action Models via Robotic Demonstrations

Abstract

Vision-Language-Action (VLA) models have recently emerged as a powerful paradigm for robotic manipulation by enabling end-to-end policy learning from multimodal observations. However, the increasing reliance on demonstration-driven training, where robotic policies are refined using heterogeneous demonstrations collected from diverse sources, introduces new vulnerabilities into the data supply chain. Existing data poisoning studies mainly focus on static perception models and overlook the unique characteristics of robotic policies, where small deviations in action prediction can accumulate through closed-loop interaction and lead to significant execution failures. In this work, we introduce TrajPoison, a trajectory-aware training-time poisoning framework that reveals the vulnerability of VLA models to compromised robotic demonstrations. Under a clean-label setting, TrajPoison identifies action-critical segments within robotic trajectories and generates visually subtle, gradient-aligned perturbations while preserving language instructions, robot states, action labels, and trajectory semantics. Unlike conventional poisoning approaches that independently corrupt individual samples, TrajPoison exploits the intrinsic structure of robotic trajectories to inject consistent perturbations into the training pipeline, causing the learned policy to acquire biased perception-action mappings after training. We evaluate TrajPoison across multiple VLA architectures, including RDT, , OpenVLA, and real-world robotic platforms. Extensive experiments demonstrate that TrajPoison consistently induces significant task performance degradation and action deviations compared with existing poisoning baselines under identical poisoning budgets. Further analysis shows that the induced policy bias is amplified through closed-loop robot-environment interaction, leading to progressively accumulated execution errors during deployment. These findings expose a previously underexplored vulnerability in VLA training pipelines and highlight the importance of secure robotic data collection, provenance verification, and robustness evaluation for reliable deployment of embodied foundation models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Vision-Language-Action Models, Data Poisoning, Embodied AI Security

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jAJgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TrajPoison, a clean-label, trigger-free availability poisoning attack on VLA training data. It perturbs the images of a subset of demonstrations, focusing on high-curvature time windows of the trajectory, using action-weighted PGD with cross-frame momentum. With 10% poisoned demonstrations and ε = 4/255, success rates on RoboTwin drop by about half on Easy and to near zero on Hard for RDT, π0 and OpenVLA.

Critical strengths
  1. Availability poisoning of robot demonstration data is a relevant and unexplored threat, and the idea of targeting action-critical moments of a trajectory is intuitive.
  2. The evaluation covers three architectures, cross-model transfer, a real-robot setting, and several input-space defenses, with component ablations.
Critical weaknesses
  1. The effect is strong but is not well explained or controlled. Perturbing about 2% of the frames (10% of trajectories × ~20% of timesteps) with ε = 4/255 halves π0 Easy SR (84.0 → 47.5), and even 2% of trajectories gives 60.2. The paper does not explain why this works beyond one sentence about "adversarially displaced visual features", and the mechanism analysis (attention, teacher-forced, closed-loop) describes how failures appear rather than why the poison works. There is no random-noise control with the same ε and ratio, and even the adapted backdoor baselines already reduce π0 Easy SR to about 57–61, which suggests the setup may be sensitive to any perturbation. Unlearnable Examples and adversarial poisoning, the closest prior work, are not compared.
  2. The method design is not well justified and partly inconsistent. For single-frame surrogates (e.g., π0, OpenVLA), the gradient for δ_t is q_t times the gradient of d_t, and q_t is cancelled by the L1 normalization and sign in Eq. 3, so the frame weighting has no effect; it is unclear what the "w/o action weighting" ablation removes. The cross-frame momentum makes perturbations in a window temporally consistent, but the paper states the attack does not need a history-based victim, and it is not explained why temporal consistency should matter for a single-frame policy. Eq. 3 mixes iteration and time indices, and the number of iterations and window length are not given. Figure 2 also shows a temporal encoder and a "noise detection" step (detected → poisoning blocked), which do not appear in the method or experiments.
  3. The evaluation is incomplete and partly inconsistent. In Table 4, the frame-wise and uniform variants have NAE equal to or below the clean model (0.042 / 0.037 vs. 0.042) while SR drops, which is hard to interpret. Only test-time defenses are evaluated; training-time defenses that are standard for availability poisoning (adversarial training, strong augmentation, early stopping, poison detection) are not. The real-robot clean success rates are low, which makes the relative drop less informative.
Questions for authors
  1. What is the effect of random noise with the same ε and poisoning ratio?
  2. How does TrajPoison compare with Unlearnable Examples and adversarial poisoning under the same setting? Why can poisoning about 2% of frames override the remaining clean data?
  3. Given that q_t cancels under the normalized sign update, what exactly does the "w/o action weighting" ablation remove? Why does temporal consistency of perturbations matter for single-frame policies?
  4. What does the "noise detection" step in Figure 2 refer to? Please also give the number of iterations and the window length.
3 comments
AuthorsOpenReview ↗

Response to Reviewer jAJg (1/2): Random noise, poisoning mechanism, and action weighting

Q1. Random noise with the same perturbation budget and poisoning ratio

We agree that random noise with the same perturbation budget and poisoning ratio provides an important control for evaluating the contribution of the proposed optimization.

We have conducted matched-budget random-noise experiments on RDT Easy, π0 Easy, and OpenVLA Easy.

Table 7. Comparison between random noise and TrajPoison.

ModelMethodSR ↓NAE ↑
RDT EasyClean70.40.042
Random Noise71.20.040
TrajPoison36.10.245
π0 EasyClean84.00.065
Random Noise83.50.073
TrajPoison47.50.198
OpenVLA EasyClean64.10.095
Random Noise60.40.096
TrajPoison28.40.284

The random-noise baseline uses the same selected demonstrations and frames, poisoning ratio (ρ=10\rho=10\\%) and perturbation budget (ϵ=4/255\epsilon=4/255), while replacing the optimized perturbations with random noise.

Random noise leaves RDT and π0 largely unchanged and causes only a modest OpenVLA SR decrease (64.1% to 60.4%), unlike TrajPoison.

Overall, random noise does not reproduce the substantial degradation caused by TrajPoison. These results suggest that the attack effectiveness does not arise simply from introducing bounded pixel perturbations, but from the targeted optimization of action-critical observations and their temporal coordination.

The slight improvement observed on RDT may be consistent with a weak regularization effect of small random perturbations, although we do not interpret it as evidence of a general robustness improvement.

Q2. Why can poisoning approximately 2% of frames affect the remaining clean data?

The influence of poisoned frames is not proportional to their frequency in the training dataset. In robotic manipulation, task success often depends on a small number of fine-grained control transitions, such as grasping, precise alignment, and object placement. Small action deviations at these stages can cause an entire manipulation episode to fail, even when most other actions remain accurate.

TrajPoison specifically targets these high-impact temporal regions rather than uniformly modifying demonstration frames. We select the top 20% of candidate timesteps ranked by qtq_t as action-critical window centers. Given a poisoning ratio of 10% at the demonstration level, this corresponds to a nominal selection of approximately 2% of temporal positions before temporal-window expansion, assuming comparable trajectory lengths.

Importantly, this approximately 2% figure refers to selected window centers, rather than the final proportion of unique modified frames after expanding and merging temporal windows. The exact modified-frame proportion depends on trajectory lengths and window overlap.

These poisoned observations do not overwrite or replace the remaining clean data. Instead, they contribute to updating the shared parameters of the VLA policy. Their influence is therefore not restricted to the modified frames themselves. By targeting visually sensitive observations associated with important control transitions, TrajPoison can induce a bias in the learned perception-action mapping despite the presence of a larger number of clean observations.

Furthermore, the induced perception-action bias can affect subsequent robot states and observations during deployment. An initial action deviation may alter the execution trajectory, leading to further deviations through closed-loop robot-environment interaction.

Our teacher-forced and closed-loop experiments provide supporting evidence. On RDT Easy, NAE increases from 0.040 to 0.137 under teacher forcing, where predicted actions are not executed. Under closed-loop execution, NAE further increases from 0.042 to 0.245.

These results illustrate how perturbing a small fraction of control-critical observations can influence the shared policy parameters and subsequently affect the entire execution trajectory.

Q3. Clarification of q_t, Ω_t, and temporal consistency

We have revised the formulation to explicitly distinguish timestep selection from action-dimension weighting.

The timestep importance score qtq_t is used to rank candidate timesteps and select the top 20% as action-critical window centers. It therefore determines where perturbations are applied.

For the single-frame surrogate, the scalar qtq_t cancels in the normalized per-frame gradient update and does not affect its direction. The effective action-dimension weighting is provided by the vector Ωt\Omega_t, which reflects the relative importance of different action dimensions in the perturbation objective.

Accordingly, the w/o action weighting ablation replaces Ωt\Omega_t with an all-ones vector while keeping the selected action-critical windows unchanged. This removes dimension-wise action weighting without changing temporal localization.

(Continued in Part 2.)

AuthorsOpenReview ↗

Response to Reviewer jAJg (2/2): Temporal consistency, defenses, and evaluation metrics

Q3 (continued): Temporal consistency

After selecting the action-critical window centers, each center tt defines a temporal window:

[t−5,t+5],[t-5,t+5],

corresponding to 11 frames before overlap merging.

Regarding temporal consistency, we clarify that cross-frame coordination does not require the victim policy to jointly process multiple frames or employ an explicit temporal encoder.

For a single-frame policy, each observation is processed independently at inference time:

at=fθ(ot).a_t=f_\theta(o_t).

However, the model parameters θ\theta are shared across all timesteps. During fine-tuning, observations from neighboring timesteps contribute to updates of the same model parameters.

If perturbations are optimized independently for each frame, their induced visual biases may be inconsistent across neighboring observations, resulting in less coordinated training signals.

TrajPoison addresses this issue through cross-frame momentum during perturbation construction. By coordinating perturbation updates across adjacent observations, it encourages more consistent visual biases across temporally neighboring training samples. These observations subsequently contribute to the optimization of shared policy parameters, even though they are processed independently during inference.

Therefore, the effectiveness of cross-frame coordination in single-frame policies arises from the interaction between temporally neighboring training observations and shared model parameters, rather than from an architectural requirement to process video sequences jointly.

Our ablation results support this design: removing cross-frame momentum increases RDT Easy SR from 36.1% to 48.6% and decreases NAE from 0.245 to 0.156.

Q4. Noise detection, optimization iterations, and window length

We clarify that the noise-detection discussion concerns whether perturbations introduced by TrajPoison can be identified by conventional low-level perturbation-screening methods.

We have completed Strong Data Augmentation and Video Re-compression evaluations; Feature-Squeezing-based Poison Detection remains ongoing.

The results of the first two defense categories are summarized below.

Table 8. Evaluation against data-side defenses.

ModelDefenseClean SRPoisoned SRClean NAEPoisoned NAE
RDT EasyTwo-way MixUp67.241.30.0710.207
Four-way MixUp65.439.60.0750.223
CutMix66.340.60.0730.215
Video Re-compression60.542.30.0750.194
π0 EasyTwo-way MixUp71.253.70.0800.165
Four-way MixUp69.451.20.0840.181
CutMix70.052.60.0820.173
Video Re-compression73.555.50.0700.164

The results show that TrajPoison remains effective under all evaluated data transformations. The clean-to-poisoned SR gap remains substantial across both architectures, indicating that these common training-time transformations do not eliminate the poisoning effect.

The Feature-Squeezing-based poison detection experiment is currently ongoing. Its results will be incorporated into the revised manuscript when available.

Implementation details: Poison optimization uses 20 iterations. The window selection and 11-frame length are specified above; overlapping windows are merged before optimization (Appendix A.3–A.4).

Q5. Relationship between NAE and SR

We thank the reviewer for identifying the apparent inconsistency in the original ablation results.

The frame-wise and uniform-frame variants exhibit global NAE values equal to or below that of the clean policy (0.042 and 0.037, respectively, compared with 0.042), while their SR decreases.

NAE and SR measure different aspects of policy performance. NAE is an averaged continuous measure of action prediction error across timesteps and action dimensions, whereas SR measures successful completion over an entire closed-loop rollout.

Consequently, localized action deviations at critical manipulation stages, such as grasping or object placement, may cause task failure while contributing only marginally to the global average NAE. A relatively small prediction error at a critical control transition can therefore have a much larger effect on task completion than its contribution to the global average error suggests.

We therefore interpret these results as showing that changes in global average action error and task-level success are not strictly correlated. In particular, the frame-wise and uniform-frame variants should not be interpreted as evidence of increased global action prediction error.

In contrast, the full TrajPoison method increases NAE from 0.042 to 0.245 while reducing SR from 70.4% to 36.1%. Removing cross-frame momentum or action weighting reduces this effect, with NAE decreasing to 0.156 and 0.173, respectively.

We will clarify the distinction between global prediction error and task-level execution failures in the revised discussion.

I thank the authors for the rebuttal, which does clarify several points that were originally unclear. However, the many inconsistencies and contradictions in the paper itself remain unresolved. For instance, the noise-detection step in Figure 2, and I hope the authors will further polish the paper itself. I am therefore keeping my original score.

Reviewer_dHMhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TrajPoison, a trajectory-aware clean-label data poisoning attack against VLA models. The work investigates vulnerabilities arising from the use of externally sourced robotic demonstrations during downstream policy fine-tuning. Unlike conventional poisoning approaches targeting static visual inputs, TrajPoison considers the temporal structure of robotic demonstrations and the potential accumulation of action prediction errors during closed-loop execution. Experiments are conducted on five VLA architectures, including RDT, π0, and OpenVLA in simulation, as well as π0.5 and DM0 on a physical robot. The reported results demonstrate substantial degradation in task success rates and increased action prediction errors compared with several poisoning baselines. Additional experiments examine contamination ratios, component ablations, cross-architecture transferability, and the relationship between one-step prediction bias and closed-loop error accumulation.

Critical strengths
  1. TrajPoison explicitly incorporates the temporal structure of robotic demonstrations. The combination of action-critical segment selection, action-sensitive perturbation objectives, and cross-frame coordination provides a reasonable approach to targeting control-sensitive portions of robotic trajectories.

  2. An important strength is the inclusion of real-world experiments on an Aloha-AgileX platform. The substantial reduction in task success rates for π0.5 and DM0 provides stronger practical evidence than simulation-only evaluations.

Critical weaknesses
  1. The proposed method primarily combines several established ideas: action-curvature-based keyframe selection, gradient-based adversarial perturbation optimization, and momentum-based updates across neighboring frames. Although adapting these techniques to robotic demonstrations is practically meaningful, the underlying algorithmic components are not fundamentally new.

  2. The attacker is assumed to modify externally sourced robotic demonstrations while preserving all nonvisual information and maintaining visually subtle perturbations. However, the main attack construction relies on gradient access to the victim model or a suitable surrogate.

Questions for authors
  1. Could the authors clarify the fundamental methodological novelty of TrajPoison beyond combining action-curvature-based frame selection, gradient-based adversarial perturbations, and temporal momentum? How does the proposed framework differ from existing temporally coordinated poisoning or adversarial optimization methods?

  2. The proposed attack assumes access to gradients from the victim model or a suitable surrogate. Under realistic robotic data-supply-chain conditions, how feasible is this assumption? Could the authors evaluate TrajPoison under more restrictive black-box settings where the attacker has limited knowledge of the victim architecture and fine-tuning procedure?

3 comments
AuthorsOpenReview ↗

Response to Reviewer dHMh: Methodological contribution and transfer setting

Q1. Fundamental methodological novelty

We appreciate the reviewer's request for a clearer explanation of the methodological contribution.

The central contribution of TrajPoison is a control-sensitive poisoning formulation that connects action-critical temporal localization, action-dimension-aware optimization, and cross-frame perturbation coordination through the structure of robotic demonstrations.

Importantly, these components are not simply independent techniques applied sequentially. They jointly determine where perturbations are introduced, which control deviations are emphasized, and how perturbations are coordinated across temporally neighboring observations.

First, the attack target is determined by robot control dynamics rather than image-level importance.

We measure local action curvature using:

\left(|a_{t+1}-2a_t+a_{t-1}|\right), \qquad q_t=\frac{\\|\Omega_t\\|_1}{d_a}.$$ The scalar score $q_t$ ranks candidate timesteps, and the top 20% are selected as action-critical window centers. These regions correspond to temporally sensitive control transitions, such as grasping, alignment, and object placement, where small action deviations can have disproportionate effects on task completion. **Second, the optimization objective is defined in the action space rather than the image classification space.** The dimension-wise vector $\Omega_t$ weights the deviation between predicted and expert actions, emphasizing control dimensions according to their local action importance. We further clarify the distinct roles of $q_t$ and $\Omega_t$. For the single-frame surrogate, the scalar $q_t$ cancels in the normalized per-frame gradient update and therefore does not determine the gradient direction. The effective action-dimension weighting is provided by $\Omega_t$. Thus, $q_t$ is used for action-critical timestep selection, whereas $\Omega_t$ determines the relative importance of action dimensions in the perturbation objective. **Third, perturbations are coordinated across temporally adjacent observations.** Robotic demonstrations are temporally structured sequences rather than collections of independent images. Independently optimizing each frame may introduce inconsistent visual perturbations across neighboring observations. TrajPoison uses cross-frame momentum to coordinate perturbation updates within action-critical windows, producing more consistent perturbations across adjacent observations. Together, these components define a trajectory-level, action-sensitive poisoning objective: - Action dynamics determine **where** to perturb. - Action-dimension weighting determines **which control deviations** to emphasize. - Cross-frame coordination determines **how perturbations are coordinated** across neighboring observations. This coupling is specific to robotic perception-action learning. Unlike conventional image-level poisoning, which primarily optimizes individual samples or image-level prediction errors, TrajPoison targets the visual-to-action mapping at control-critical stages while preserving expert action labels and non-visual demonstration information. The three components are connected through the same underlying robotic trajectory: action dynamics guide temporal localization, the selected observations determine where the action-sensitive objective is applied, and cross-frame coordination shapes the perturbations across these selected observations. Thus, the novelty lies in formulating poisoning around the control dynamics and temporal structure of robotic demonstrations, rather than simply applying existing image-level perturbation techniques to VLA training data. ## Q2. White-box assumptions and cross-architecture transfer We clarify the transfer setting through our surrogate-to-victim experiments, described in Section 4.5 and Appendix F.1. Poisoned demonstrations are generated using OpenVLA as a differentiable surrogate and transferred to RDT and π0 without accessing the target models' weights, gradients, checkpoints, or training losses. **Table 6. Cross-architecture transfer from an OpenVLA poison-generation surrogate (SR, %).** | Training Data | π0 Easy | π0 Hard | RDT Easy | RDT Hard | |---|---:|---:|---:|---:| | Clean | 84.0 | 41.0 | 70.4 | 31.3 | | OpenVLA-generated poison | 58.2 | 14.5 | 48.6 | 11.2 | The results demonstrate target-model gradient-free transfer across different VLA architectures. We distinguish two settings in our evaluation: poison generation using gradients from a public model or suitable surrogate, and transfer to victim models without accessing their gradients. The latter is a surrogate-based, target-gradient-free setting rather than unrestricted black-box poisoning without any suitable surrogate. We will make this distinction explicit in the threat-model description. Unrestricted black-box poisoning remains an important direction for future work.

Thank you for the clarification. However, I still believe that the proposed method primarily adapts existing attack techniques to the embodied AI, rather than introducing a sufficiently novel methodological contribution. Therefore, I will maintain my original score.

Thank you for considering our clarification. We understand that our response has not changed your assessment of the methodological novelty, and we respect your decision to maintain the original score.

We would nevertheless like to emphasize that our contribution is not to introduce new generic attack primitives or optimization algorithms, but to formulate a control-sensitive poisoning approach tailored to the structure of robotic demonstrations. Specifically, TrajPoison uses action curvature to identify control-critical observations, action-dimension weighting to guide perturbation optimization, and cross-frame coordination to account for the temporal structure of demonstrations. These design choices are intended to address the specific characteristics of VLA poisoning rather than simply applying existing image-level attacks to robotic data.

We appreciate your constructive feedback, which has helped us better clarify the scope and positioning of our methodological contributions.

Reviewer_N699OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies a threat for trigger-free availability attack on VLA. An attacker contributes demonstrations whose RGB frames carry bounded perturbations, while instructions, proprioception and action labels stay unchanged, with the goal of degrading the fine-tuned policy on clean inputs. TrajPoison selects action-critical windows by local action curvature, maximizes a curvature-weighted action deviation through a surrogate VLA, and coordinates perturbations across adjacent frames with momentum-style PGD. With 10% of demonstrations poisoned, success drops sharply for RDT, Pi0 and OpenVLA on 12 RoboTwin tasks and for Pi0.5 and DM0 on a physical Aloha-AgileX robot, and TrajPoison is stronger than adapted baselines (BadNets, Gradient Matching, EO66, CSDPC, BadVLA/GoBA). Further experiments cover ablations, OpenVLA-to-RDT/Pi0 transfer, a teacher-forced vs closed-loop analysis of error amplification, and three inference-time defenses.

Critical strengths
  • Results span three simulated architectures with different action heads (diffusion, flow, tokenized), two real-robot VLAs with 30 paired trials per task, cross-architecture transfer, and several baselines under identical budgets and seeds. This makes the core claim, that small clean-label image perturbations in 10% of fine-tuning data can substantially degrade VLA policies, credible and relevant to anyone assembling third-party robot data.
  • Comparing teacher-forced NAE (0.040 to 0.137) with closed-loop NAE (0.042 to 0.245) shows that a modest learned bias roughly doubles under robot–environment feedback. That observation is specific to embodied policies and goes beyond what static-classifier poisoning work shows.
Critical weaknesses
  • The core recipe closely follows "adversarial examples make strong poisons" and unlearnable-examples work, which is not discussed. The VLA-specific additions are curvature-based frame selection and cross-frame momentum. The ablation does show they matter on RDT Easy, but only on that one model/split.
  • The attack assumes white-box gradients of the same public model that the victim fine-tunes, and the authors acknowledge it is detectable by post-training validation. Yet the evaluated defenses are all inference-time (smoothing, median blur, action EMA). Standard data-side countermeasures are not tested; it is hard to judge practical severity.
Questions for authors
  1. Do the victim's official fine-tuning recipes include image augmentation? How does TrajPoison fare under training-time augmentation and video re-compression, which real data pipelines apply by default?
  2. Can you report seed-level variance and confidence intervals so these effects can be judged against noise?
  3. Physical clean-trained success is low (24.7% for Pi0.5, 28.0% for DM0). Why is it so low, and does the attack remain as damaging when the clean policy is strong?
  4. How does the method relate to Fowl et al. (2021) and Huang et al. (2021, unlearnable examples)? What would those methods achieve if applied frame-wise to the same windows?
4 comments
AuthorsOpenReview ↗

Response to Reviewer N699 (1/2): Defenses, statistics, and physical protocol

Q1. Training-time defenses and data augmentation

We appreciate the suggestion to evaluate TrajPoison against training-time defenses.

Our original experiments follow the official RoboTwin LoRA fine-tuning recipes and their corresponding configurations for each victim model. To further investigate the effectiveness of data-side defenses, we have added three types of evaluations:

  • Strong Data Augmentation: We evaluate two-way MixUp, four-way MixUp, and CutMix using their corresponding official configurations.
  • Video Re-compression: We apply video re-compression to clean and poisoned demonstrations following the corresponding official configuration before fine-tuning.
  • Poison Detection: We additionally investigate Feature Squeezing (Xu et al., NDSS 2018) as a representative low-level perturbation-screening method. This experiment is currently ongoing.

The results of the first two defense categories are summarized in Table 1.

Table 1. Evaluation against data-side defenses.

ModelDefenseClean SRPoisoned SRClean NAEPoisoned NAE
RDT EasyTwo-way MixUp67.241.30.0710.207
Four-way MixUp65.439.60.0750.223
CutMix66.340.60.0730.215
Video Re-compression60.542.30.0750.194
π0 EasyTwo-way MixUp71.253.70.0800.165
Four-way MixUp69.451.20.0840.181
CutMix70.052.60.0820.173
Video Re-compression73.555.50.0700.164

The results show that TrajPoison remains effective under all evaluated data transformations. For RDT, the clean-to-poisoned SR gap remains between 18.2 and 25.9 percentage points across the four defenses. For π0, the corresponding gap ranges from 17.4 to 18.2 percentage points.

Moreover, the poisoned policies consistently exhibit higher NAE than their corresponding clean policies under the same defense. These results demonstrate that common training-time data transformations do not eliminate the poisoning effect, although the magnitude of degradation varies across defense configurations.

Q2. Variance and confidence intervals

We agree that reporting statistical uncertainty is important for evaluating the reliability of our results.

We have added seed-level mean and standard deviation, together with 95% confidence intervals, for all simulated VLA architectures and environment splits in Appendix I (Table 17).

Table 2. Seed-level simulation summary (mean ± standard deviation and 95% across-seed t confidence interval).

ModelSplitClean SR (%)Poisoned SR (%)Clean NAEPoisoned NAE
RDTEasy70.4 ± 0.8 [68.4, 72.4]36.1 ± 1.3 [32.9, 39.3]0.042 ± 0.001 [0.040, 0.044]0.245 ± 0.009 [0.223, 0.267]
RDTHard31.3 ± 1.0 [28.8, 33.8]3.7 ± 0.8 [1.7, 5.7]0.154 ± 0.006 [0.139, 0.169]0.388 ± 0.012 [0.358, 0.418]
π0Easy84.0 ± 1.0 [81.5, 86.5]47.5 ± 1.8 [43.0, 52.0]0.065 ± 0.003 [0.058, 0.072]0.198 ± 0.008 [0.178, 0.218]
π0Hard41.0 ± 1.4 [37.5, 44.5]5.4 ± 1.1 [2.7, 8.1]0.112 ± 0.005 [0.100, 0.124]0.412 ± 0.015 [0.375, 0.449]
OpenVLAEasy64.1 ± 1.1 [61.4, 66.8]28.4 ± 1.6 [24.4, 32.4]0.095 ± 0.004 [0.085, 0.105]0.284 ± 0.010 [0.259, 0.309]
OpenVLAHard18.1 ± 0.8 [16.1, 20.1]1.8 ± 0.6 [0.3, 3.3]0.103 ± 0.004 [0.093, 0.113]0.397 ± 0.013 [0.365, 0.429]

All reported 95% confidence intervals are computed across the three independent training seeds using two-sided Student's t intervals, with n=3n=3 and df=2df=2:

xˉ±t0.975,2s3.\bar{x}\pm t_{0.975,2}\frac{s}{\sqrt{3}}.

For the paired SR degradation effect, we calculate the clean-trained minus poisoned-trained SR difference for each task and seed, average the 12 task-level differences within each seed, and then compute the 95% t confidence interval across the three seed-level paired estimates. This preserves the clean–poisoned pairing and equal task weighting.

The results consistently show substantial performance degradation across all three architectures and both environment splits, while the across-seed confidence intervals provide additional evidence of the consistency of our findings.

Q3. Relatively low clean success rates on the physical robot

We acknowledge that the absolute clean success rates in our physical experiments are relatively modest. Our physical evaluation covers 10 RoboChallenge tasks with 30 fully autonomous trials per task and condition. Unlike controlled simulation, physical execution is affected by motor variation, sensing latency, and contact friction, which introduce additional challenges during manipulation.

Clean and poisoned conditions use paired initial states with matched robot initialization, object placement, camera configuration, background, illumination, and task setup. We therefore report both the success counts and the corresponding success rates.

(Continued in Part 2.)

AuthorsOpenReview ↗

Response to Reviewer N699 (2/2): Physical results, related work, and ablations

Q3 (continued)

To further examine whether TrajPoison remains effective against strong clean policies, we also consider the simulation results on Easy tasks.

Table 3. Attack effectiveness under different clean policy performance levels.

ModelClean SRPoisoned SRAbsolute Drop
RDT70.436.134.3
π084.047.536.5
OpenVLA64.128.435.7

Notably, π0 achieves 84.0% SR under clean training, yet its success rate decreases to 47.5% under TrajPoison. RDT also exhibits a substantial decrease from 70.4% to 36.1%. These results demonstrate that the attack is not limited to policies with low initial performance and remains effective against relatively strong clean policies.

For physical evaluation, π0.5 decreases from 74/300 successful trials (24.67%) to 1/300 (0.33%), while DM0 decreases from 84/300 (28.00%) to 13/300 (4.33%). These correspond to relative success-rate reductions of 98.65% and 84.52%, respectively.

Together, the simulation and physical results demonstrate the effectiveness of TrajPoison across policies with different initial performance levels and under real-world execution conditions.

Q4. Relation to Adversarial Examples Make Strong Poisons and Unlearnable Examples

We appreciate the reviewer for identifying the close connections to adversarial-example-based poisoning and unlearnable examples. We have added comparisons with the methods of Fowl et al. and Huang et al. under VLA evaluation settings.

Table 4. Comparison with adversarial-example-based poisoning and unlearnable examples.

ModelMethodSR ↓NAE ↑
RDT EasyAdversarial Examples Make Strong Poisons53.80.093
Unlearnable Examples60.60.074
TrajPoison36.10.245
π0 EasyAdversarial Examples Make Strong Poisons55.40.096
Unlearnable Examples64.50.088
TrajPoison47.50.198

The results show that TrajPoison consistently achieves lower SR and higher NAE than both adapted baselines across RDT and π0.

On RDT Easy, TrajPoison reduces SR to 36.1%, compared with 53.8% and 60.6% for the two baselines, respectively. On π0 Easy, it achieves 47.5% SR, compared with 55.4% and 64.5%.

Adversarial Examples Make Strong Poisons investigates adversarial-example-based data poisoning, whereas Unlearnable Examples constructs perturbations that interfere with the learning of individual training samples. These methods primarily formulate perturbations at the sample or image level.

In contrast, TrajPoison formulates poisoning around the control dynamics of robotic demonstrations. Specifically, it identifies action-critical temporal regions, optimizes perturbations according to action-dimension importance, and coordinates perturbations across neighboring observations.

The resulting objective differs from independently maximizing image-level prediction errors or minimizing the learnability of individual samples. It targets deviations in the visual-to-action mapping at control-critical stages while preserving the original expert action labels and non-visual demonstration information.

Furthermore, our teacher-forced and closed-loop experiments reveal an embodied-specific consequence: an initially induced one-step action bias can be further amplified through robot-environment interaction.

Q5. Component ablations are evaluated only on RDT

We agree that evaluating component contributions on an additional VLA architecture provides a more comprehensive assessment.

We have therefore extended the component ablation to π0 Easy, which employs a different flow-matching action-generation mechanism.

Table 5. Additional component ablations on π0 Easy.

VariantSR ↓NAE ↑
Clean84.00.065
Frame-wise75.60.073
Uniform-frame77.80.075
Random-window80.90.069
w/o cross-frame momentum66.70.098
w/o action weighting59.50.103
Full TrajPoison47.50.198

The full TrajPoison achieves the lowest SR and highest NAE among the evaluated variants. Removing cross-frame momentum increases SR to 66.7% and reduces NAE to 0.098, while removing action-dimension weighting increases SR to 59.5% and reduces NAE to 0.103.

For the w/o action weighting variant, we replace the dimension-wise weighting vector Ωt\Omega_t with an all-ones vector while keeping the selected action-critical windows unchanged. This isolates the contribution of action-dimension weighting from that of temporal window selection.

Similarly, frame-wise, uniform-frame, and random-window variants all exhibit higher SR and lower NAE than the full method.

These results provide additional evidence that action-critical localization, action-dimension weighting, and cross-frame coordination contribute to the effectiveness of TrajPoison beyond the original RDT evaluation.

Thank you for the rebuttal and the new experiments. It resolve my concerns about statistical reliability and about the ablation covering only one model. The new data-side defense results and the Fowl et al. and Huang et al. comparisons are also helpful.

Do the official LoRA fine-tuning recipes already include image augmentation? I think MixUp and CutMix are uncommon in robot learning. Standard augmentations such as random crop and color jitter would be more representative.

AuthorsOpenReview ↗

Clarification on Standard Image Augmentations

Thank you for this helpful question.

Yes. The official LoRA fine-tuning recipes used in our experiments already include model-specific image augmentation pipelines, which remain enabled in both clean and poisoned training. Specifically, the augmentations used for each model are as follows:

ModelStandard training-time augmentations
π0For non-wrist camera images, random cropping retains 95% of the original width and height, followed by resizing to the original resolution and random rotation within ±5°. Color jitter is applied to all camera views, with brightness, contrast, and saturation parameters of 0.3, 0.4, and 0.5, respectively.
RDTThe official fine-tuning script enables image_aug. For 50% of valid images, it randomly selects color jitter, image corruption, or both. Color jitter uses brightness, contrast, saturation, and hue parameters of 0.3, 0.4, 0.5, and 0.03, respectively. Image corruption includes noise and blur operations.
OpenVLAThe official fine-tuning recipe enables image_aug=True, including random resized cropping with a fixed crop area of 90% of the original image and a fixed aspect ratio of 1.0, followed by resizing to the model input resolution. The crop location is randomly sampled. Color jitter uses a brightness parameter of 0.2, contrast and saturation ranges of [0.8, 1.2], and a hue parameter of 0.05.

Clean and poisoned demonstrations are processed using the same model-specific augmentation configurations during fine-tuning. Therefore, these standard augmentations are already incorporated into our original poisoning experiments.

The corresponding results on the Easy split are:

ModelClean SR (%)Poisoned SR (%)
π084.047.5
RDT70.436.1
OpenVLA64.128.4

These results demonstrate that TrajPoison remains effective under the standard image augmentation pipelines used by the respective VLA models.

The MixUp, CutMix, and video re-compression experiments added in our rebuttal are supplementary evaluations of additional data-side transformations beyond those already included in the original training pipelines. We will clarify the model-specific augmentation configurations in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.