acceptodds
Under review as a conference paper at ICLR 2027

Poisoning Robot Demonstrations with Hidden Physical Costs

Abstract

Robot learning pipelines increasingly rely on large-scale demonstration data aggregated from diverse sources, where task success is often the primary criterion for data validation and filtering. However, successful demonstrations can still encode physically costly execution patterns. We show that this creates a safety blind spot, where policies can complete tasks while systematically incurring hidden physical burden. To study this failure mode, we introduce Fatigue Injection, a diagnostic data-poisoning method that injects small, structured perturbations into successful demonstrations during data collection, targeting physical execution costs rather than task behavior. We quantify the resulting burden using cumulative fatigue damage and operating energy, two hardware-aware metrics that expose physical costs hidden behind successful executions. Across simulation and real-world manipulation tasks, policies trained on poisoned data achieve success rates comparable to clean baselines while incurring substantially higher cumulative fatigue damage and energy costs. Notably, this effect persists under partial poisoning and across diverse policy architectures, and cannot be easily mitigated by simple inference-time smoothing or identified by generic data-inspection baselines. Together, these results show that task success alone is an insufficient quality gate for safe robot learning, motivating hardware-aware auditing of execution-level dynamics and physical burden in demonstration data to support safe real-world deployment.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Robot Safety, Imitation Learning, Data Poisoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cdqNOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies a vulnerability in robot imitation learning: demonstrations that successfully complete tasks can still teach policies physically costly execution patterns. It introduces Fatigue Injection, a data-poisoning method that adds small alternating action perturbations during demonstration collection and retains only successful rollouts. Its contributions are this threat model and construction method, together with evaluation using a torque-based fatigue proxy and absolute mechanical work. Experiments on LIBERO-10 and two real-world Franka manipulation tasks, using π0.5, GR00T N1.5, and cross-architecture transfer with Diffusion Policy, show substantially increased physical-cost metrics while maintaining comparable task success at selected perturbation levels.

Critical strengths
  • The paper is clearly written and generally easy to follow. Its illustrations help communicate the attack construction and the distinction between task success and physical execution costs.
  • The perturbation design is simple and heuristic, yet the experiments show that it can induce substantially higher torque-based fatigue scores and mechanical work while preserving comparable task success at selected perturbation levels. Validation in both simulation and real-robot tasks supports the relevance of this failure mode.
Critical weaknesses

My main reservation concerns the practical scope and resource requirements of the malicious-data-provider threat model. The experiments demonstrate a failure mode, but the claim that the attacker assumptions are weak and realistic would benefit from a more concrete account of the resources required.

  • The cost of constructing poisoned demonstrations is insufficiently characterized. The real-robot procedure requires executing perturbed actions during data collection, potentially imposing additional physical burden on the collection hardware for attackers. This does not invalidate the attack, but it raises a practical question about its cost.
  • The threat model assumes a potentially restrictive combination of attacker capabilities. The attacker must have access to a competent source policy and a compatible robot data-collection setup, and must be able to deploy that policy and modify its control commands to generate poisoned demonstrations. This combines the capabilities of a data supplier, a deployment engineer able to operate a robot rollout pipeline, and—when source-policy training or adaptation is required—an ML engineer at a model company or research lab. It is unclear why a malicious data contributor would realistically possess this combination of capabilities. Although using a pretrained policy may remove the need for source-policy training, it does not eliminate the deployment and data-collection requirements. The paper should justify why these combined capabilities constitute a weak and realistic attacker assumption, particularly when data collection, deployment, and model development are handled by separate parties.
  • The practical significance of the attack depends on whether this combined role is plausible. If data collection and policy development are handled by separate parties, a malicious contributor may not possess the policy-deployment capabilities assumed here. Please explain which realistic provider or insider could exercise the required control over both rollout generation and demonstration collection, rather than treating this access as inherent to being a data supplier.
Questions for authors
  • Appendix C explains that directly modifying action labels in existing demonstrations can break consistency between the recorded actions and subsequent observations. Have you compared the proposed online collection procedure with offline action-label perturbation under comparable conditions? Such a comparison would clarify whether transition-consistent collection is empirically necessary for inducing the reported downstream physical costs, rather than only desirable for constructing valid demonstrations.

I would consider raising my score if the authors convincingly justify the assumed combination of attacker capabilities and clarify the necessity of the proposed data-collection procedure.

6 comments
AuthorsOpenReview ↗

Response to Reviewer cdqN (1/3)

We sincerely thank the reviewer for the thoughtful questions, which concern the core assumptions of our work. We address each point below.

[W1] The cost of constructing poisoned dataset

We agree with the reviewer that executing perturbed actions on a real robot during demonstration collection places an additional physical burden on the attacker’s hardware. This burden is part of the cost of constructing a poisoned demonstration dataset, and we have made this point explicit in the revised limitation section.

The attacker and the victim, however, incur this burden over different periods. For the attacker’s robot, the additional burden is limited to the demonstration collection phase. For the victim’s robot, fatigue damage can accumulate in its components each time the deployed policy repeats the perturbed actions. With repeated use over an extended period, the cumulative burden on the victim’s hardware can therefore exceed the burden on the attacker’s hardware during data collection.

Finally, as shown in our response to Q1, comparable physical costs are also induced when the action labels of previously collected demonstrations are modified offline. This option requires no execution of perturbed actions and thus imposes no additional burden on the attacker’s hardware during data poisoning.

AuthorsOpenReview ↗

Response to Reviewer cdqN (2/3)

[W2] Attacker capabilities and scope

We thank the reviewer for raising this important point. Here, we clarify who could realistically possess the required combination of capabilities. We agree that our rollout-based procedure requires the attacker to (i) collect demonstrations, (ii) obtain a source policy, and (iii) deploy it to record poisoned rollouts. We have stated this explicitly in Section 4.1. However, we emphasize that recent advances in robotics have substantially lowered these barriers to performing all three steps, , making our proposed attack increasingly accessible to individual practitioners with commodity hardware and open-source tools. For example:

  • Low-cost hardware: Open-source robots such as SO-ARM [1] provide affordable hardware for teleoperation and demonstration collection.
  • End-to-end tooling: LeRobot [2] supports demonstration recording, policy training, and on-robot rollout recording within one pipeline.
  • Pretrained policies: SmolVLA [3], pretrained on public community data, can be fine-tuned on a single consumer GPU.

These resources allow a single contributor to handle data collection, policy adaptation, and rollout recording. For the insider case, robot learning engineer positions at several companies explicitly cover both demonstration collection and the deployment of trained policies on real robots [4,5]. An engineer in such a role could control both rollout generation and data recording, even if the source policy were developed by another team.

Such contributors can share demonstrations through platforms such as the Hugging Face Hub, and poisoned contributions could influence multiple downstream models if incorporated into their training data. Based on these examples, we define the attacker’s capabilities as demonstration collection, policy training or adaptation when needed, and policy execution for rollout collection, and we consider a threat model combining these roles to be realistic.

[1] https://huggingface.co/docs/lerobot/so101

[2] https://huggingface.co/lerobot

[3] https://huggingface.co/blog/smolvla#community-datasets

[4] https://www.xdof.ai/careers/member-of-technical-staff-robotics-system-integration-7d4bc745-b82f-4f49-8550-c9c07367c799

[5] https://www.proception.ai/careers/a9526836-2c4f-468d-ae43-ad2da7b6c029

AuthorsOpenReview ↗

Response to Reviewer cdqN (3/3)

[Q1] Offline action-label perturbation vs. online collection

We thank the reviewer for suggesting this comparison. We compared offline action-label perturbation against our online collection procedure using π0.5\pi_{0.5} on LIBERO-10 with the perturbation scale fixed at α^=1.0\hat{\alpha} = 1.0.

MethodSR (%)CFD ratio
Offline action-label perturbation85.5±1.885.5 \pm 1.855.06±1.5255.06 \pm 1.52
Online collection (ours)87.1±1.187.1 \pm 1.155.7±0.755.7 \pm 0.7

Note. Both policies were trained for ten epochs. Following the main text, CFD ratios are reported as the mean over the two most vulnerable joints (Joints 3 and 5). Values are reported as mean ±\pm standard deviation across seeds.

As illustrated in the table, we observe that both perturbation methods yield similarly large CFD ratios, suggesting that transition-consistent collection is not strictly necessary for inducing the physical cost itself. Online collection achieves a slightly higher success rate, although the difference is small. We adopt online collection because it keeps the poisoned demonstrations consistent between actions and observations; we discuss this consistency further in Appendix C.

We hope these clarifications address the reviewer’s concerns. We would be grateful for any further questions, and for the reviewer’s consideration of a score revision if the responses are found satisfactory.

Reviewer_KQsMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies an underexplored vulnerability in data-driven robot learning: demonstrations can preserve task success while encoding oscillatory control patterns that substantially increase mechanical fatigue and energy consumption. FATIGUE INJECTION is conceptually simple but effective, and the combination of simulation, real-robot experiments, partial-poisoning studies, architecture transfer, and attempted defenses makes the empirical case considerably stronger than a purely synthetic attack paper. A key concern, however, is that the reported cumulative fatigue damage is a relative torque-based proxy rather than a calibrated prediction of actual hardware lifetime, and stronger attacks become detectably jerky under some simple kinematic inspection metrics. Thus, I find the safety problem highly interesting and the diagnostic contribution convincing, while the magnitude and practical stealth of the real-world hardware-damage claim should be interpreted somewhat more cautiously.

Critical strengths

The paper makes the valuable point that task success and physical robot health can be almost completely decoupled—a successful robot policy can still be systematically harmful to its hardware.

Broad empirical validation: The attack is demonstrated on LIBERO-10 and two real FR3 manipulation tasks, works with as little as 10% poisoned training data, and transfers between π0.5 and Diffusion Policy architectures.

The authors go beyond merely showing an attack. They test data inspection, motion-jitter detection, spectral smoothness, Best-of-N smoothing, and EMA filtering, establishing useful baselines for future defensive work.

Critical weaknesses

“Fatigue damage” is not actual measured hardware degradation and is one(!) hand-designed safety issue but there are many more (see e.g. the paper: Semantically safe robot manipulation: From semantic scene understanding to motion safeguards, RAL 2025). Joint torque is assumed to scale with local material stress and material/geometric constants are canceled when reporting relative damage. This supports comparisons on identical hardware but cannot directly establish component lifetime or actual wear without calibration. Also these type of safety concerns are not new and have been carefully studied in the safe learning literature (e.g., paper: Safe learning in robotics: From learning-based control to safe reinforcement learning).

The strongest attacks are not necessarily very stealthy. Motion-jitter inspection reaches balanced precision around 0.83 at the stronger perturbation setting, suggesting that execution-level auditing can expose the attack when its physical effect is large.

The threat model deserves more scrutiny. The attacker needs to influence action-level behavior during demonstration collection while preserving task completion. Whether this is realistic for the major large-scale robot datasets motivating the paper, and how provenance/signing or robot-side logging changes the threat model, deserves a deeper analysis.

Questions for authors

My main concern is that the literature around safe learning and safety filtering has not been mentioned or compared to at all (this literature provides broader and stronger safety guarantees). Examples include this paper: Safety filtering while training: Improving the performance and sample efficiency of reinforcement learning agents (which studies how chattering can be avoided) but there are many more safety filtering works that can consider any type of constraint.

How does your method compare to those mthods?

5 comments
AuthorsOpenReview ↗

Response to Reviewer KQsM (1/4)

We sincerely thank the reviewer for the detailed comments on the CFD metric and for pointing us to the safe learning literature. We address each point below.

[W1] CFD validity and interpretation

CFD as a relative fatigue metric. We agree with the reviewer's interpretation that CFD does not directly measure hardware degradation or predict component lifetime without hardware-specific calibration.

While CFD does not directly measure hardware degradation in lifetime, we would like to emphasize that it provides a physically grounded measure of relative fatigue accumulation, enabling meaningful comparisons of the physical wear that poisoned policies may cause on identical hardware. Importantly, under repeated operation in the same regime, an nn-fold CFD ratio corresponds to reaching the same modeled fatigue level in approximately 1n\frac{1}{n} as many executions. This relative form is also what makes the metric practical: hardware-specific material and geometric constants are rarely available to practitioners, whereas joint torques are logged on most robots, so relative fatigue can be computed from data that already exists in deployment.

Relation to safe learning literature. We agree that fatigue is an established safety concern. Our contribution is to expose its amplification through data poisoning, rather than introduce fatigue itself. We discuss the related safety literature in Q1 below.

AuthorsOpenReview ↗

Response to Reviewer KQsM (2/4)

[W2] Detectability and the scope of the stealth claim

We thank the reviewer for this point. We would like to clarify that a balanced precision of 0.83 in Table 3 does not necessarily imply the practical utility of a deployable detector, for the following reason.

Notice that the inspection methods in Table 3 assign an inspection score to each demonstration. This requires a detector to decide which demonstrations to reject based on their inspection scores. However, selecting a reliable threshold requires knowing which inspection scores correspond to clean versus poisoned demonstrations. Critically, this is precisely the information that a detector cannot assume access to when auditing an already poisoned dataset. Moreover, the optimal threshold may differ across tasks, further complicating reliable use of execution-level auditing in practice.

To assess this limitation in practice, we conduct an end-to-end evaluation under conditions that reflect those a defender is likely to face, as discussed above. As detailed in Appendix G, each detection threshold is selected independently of test outcomes, using clean demonstrations from the other nine tasks, and the same filtering procedure is applied to clean data as well. Under this protocol, the filters remove only about a third of the poisoned demonstrations at α^=1.0\hat{\alpha} = 1.0, and the physical burden after retraining decreases only in proportion.

These results highlight the gap between promising inspection performance under a controlled evaluation setting and the practical ability to mitigate poisoning through data filtering. In the revised version of the paper, we have revised the claim about defense effectiveness in the abstract, and we will discuss this point further in Section 5.4.

AuthorsOpenReview ↗

Response to Reviewer KQsM (3/4)

[Q1] Comparison with Safety Filters

We thank the reviewer for this question. We compare our work with prior research on safety filtering and discuss how fatigue-aware control relates to our setting. We have also added the works the reviewer pointed out to the Related Work section.

Comparison with prior work on safety filtering. Safety filters correct actions that could violate a specified constraint, such as drone position and speed limits in Safety Filtering While Training [1] or collision avoidance for a robot arm in RAIL [2]. In every demonstration in our poisoned dataset, the robot successfully completes the task. A filter can allow actions that increase joint fatigue if they satisfy the constraints it checks. In Safety Filtering While Training, chattering arises from interactions between the controller and safety filter, whereas the oscillations we study are learned from poisoned demonstrations.

Fatigue-aware control and study scope. Controllers that explicitly account for joint fatigue could reduce the load on the joints, though the trade-off depends on the task. For example, limiting speed and acceleration in a mobile robot [3] increased completion time in high-payload simulations and tracking error in physical experiments on rough terrain. We study a setting in which the deployed controller does not explicitly limit cumulative joint fatigue. Our experiments show that policies trained on poisoned demonstrations can achieve task success rates similar to those of policies trained on clean demonstrations, even when joint torque measurements indicate greater estimated cumulative fatigue.

[1] Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents

[2] RAIL: Reachability-Aided Imitation Learning for Safe Policy Execution

[3] A Comprehensive Study of a Predictive Fatigue Aware Adaptive Control Framework for Mobile Robots Using Low Cost Sensing

Reviewer_aTcCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces alternating perturbations during demonstration collection and retains successful trajectories, increasing downstream physical costs while preserving task success. Simulation, real-robot, and cross-policy experiments use a torque-based fatigue proxy (CFD) and absolute mechanical work. The problem is relevant, but the attack combines simple perturbations, success filtering, and established metrics. Its learning mechanism and resistance to data cleaning remain insufficiently established, limiting methodological novelty and significance.

Critical strengths

The problem this work aims to address is of great significance. That is, with the increasingly widespread application of robots, attacks on robots and their training processes will also pose a major threat.

Critical weaknesses
  1. The occasional success-rate improvement under poisoning is unexplained. Matched recollection controls and statistical uncertainty estimates are needed to determine whether the comparison is confounded by data selection, training conditions, or random variation.
  2. Resilience to rule-based data cleaning remains unestablished. Simple filtering may remove the perturbation pattern, but its effectiveness after policy retraining has not been evaluated.
Questions for authors

(1) Why are zero-mean alternating perturbations not averaged out during training, particularly without explicit phase inputs? Please explain the roles of feedback, history, and action chunks, supported by spectral comparisons or a randomized-phase control.

(2) “10% poisoning” counts tasks, not randomly selected within-task demonstrations, and Figure 5 uses one seed. Please explanation trajectory-level rates, aggregation, and multi-seed variability. Why does poisoned-policy success sometimes exceed clean performance (78% versus 74% for stacking)? To separate perturbation effects from recollection, success filtering, and randomness, compare against unperturbed recollection with identical filtering, matching data sizes, training budgets, and evaluation conditions, and report statistical intervals for differences and performance preservation.

(3) Could rules based on action differences, acceleration, or high-frequency oscillations remove the attack? Table 3 reports motion-jitter balanced precision of 0.83/0.95 at nominal perturbation levels 1.0/1.5. Please evaluate filtering followed by retraining and deployment, reporting retention, clean-data false positives, success, CFD, and mechanical work. Set thresholds independently of test outcomes and apply identical filtering to clean data with matched retained sizes. This would assess whether the attack withstands routine data cleaning and substantiate stealth and defense-resilience claims.

7 comments
AuthorsOpenReview ↗

Response to Reviewer aTcC (1/4)

We thank the reviewer for the careful reading and concrete suggestions. We address the success-rate concerns in Weakness 1 and Question 2 together, and the data-cleaning concerns in Weakness 2 and Question 3 together.

[W1, Q2] Success-rate improvement under poisoning

The improvement appears in both settings of Table 8: on the real robot (+2 pp for PnP Peach, +4 pp for Stack Cube) and in simulation (+0.6 pp at α^=1.0\hat{\alpha} = 1.0). We believe it reflects evaluation variance rather than an effect of poisoning, for the following reasons.

On the real robot, with 50 trials per condition, the 95% confidence intervals of these differences (−7 to +11 pp and −13 to +21 pp) include zero, so the two success rates cannot be distinguished.

In simulation, we report results over three training seeds, with the clean baseline trained under the same recipe and evaluated under the same conditions as the poisoned policies. Per-task differences go in both directions and are not monotone in α^\hat{\alpha}.

α^\hat{\alpha}0.00.51.01.52.02.5
SR (%)86.5±1.086.5 \pm 1.085.2±2.685.2 \pm 2.687.1±1.187.1 \pm 1.186.6±0.886.6 \pm 0.882.3±1.582.3 \pm 1.579.9±3.079.9 \pm 3.0
CFD ratio1.18±0.011.18 \pm 0.018.63±0.258.63 \pm 0.2555.7±0.755.7 \pm 0.7132.8±2.4132.8 \pm 2.4205.7±2.5205.7 \pm 2.5298.5±3.1298.5 \pm 3.1

Note. π0.5\pi_{0.5} on LIBERO-10; α^=0\hat{\alpha} = 0 is the clean baseline. CFD ratios are the mean over Joints 3 and 5, as in the main text. Mean ±\pm standard deviation over three training seeds.

Where the mean SR exceeds the clean baseline, the difference is within one standard deviation. These results are consistent with our attack remaining effective while preserving task success.

AuthorsOpenReview ↗

Response to Reviewer aTcC (2/4)

[Q2] Poisoning rates, aggregation, and multi-seed variability

We thank the reviewer for this careful observation. Following the reviewer’s suggestion, we poisoned a fraction of the trajectories within every task while keeping the dataset size and training budget fixed, and we repeated each setting with three training seeds.

Poisoned trajectories0%10%20%40%60%80%100%
SR (%)86.5±1.086.5 \pm 1.085.3±1.585.3 \pm 1.586.4±4.086.4 \pm 4.086.1±2.386.1 \pm 2.387.2±1.987.2 \pm 1.987.0±2.487.0 \pm 2.487.1±1.187.1 \pm 1.1
CFD ratio1.18±0.011.18 \pm 0.013.85±0.253.85 \pm 0.258.29±0.488.29 \pm 0.4818.98±0.3618.98 \pm 0.3630.76±0.5930.76 \pm 0.5944.14±1.1244.14 \pm 1.1255.7±0.755.7 \pm 0.7

Note. π0.5\pi_{0.5} on LIBERO-10 at α^=1.0\hat{\alpha} = 1.0. ±\pm denotes the standard deviation over three training seeds. 0% is the clean baseline. CFD ratio is the mean over Joints 3 and 5, as in the main text.

The CFD ratio increases monotonically with the fraction of poisoned trajectories, while the success rate remains similar across all fractions, matching the trend in Figure 5 for task-level poisoning. Poisoned trajectories at each fraction are a subset of those at higher fractions, and SR and CFD are aggregated as in the main text.

We have added these results to Appendix F, and will update Section 5.3 and Figure 5 in the final version, together with the remaining poisoning fractions, which are currently running with the same three seeds.

AuthorsOpenReview ↗

Response to Reviewer aTcC (3/4)

[W2, Q3] Data inspection followed by retraining

We agree that detection scores alone do not establish resilience to data cleaning. We therefore evaluated the full pipeline: flagging trajectories, removing them, retraining the policy, and deploying it. We used motion jitter and DemInf; spectral smoothness ranks poisoned demonstrations no better than the random rate, so we did not retrain with it.

Training dataFilterPoisoned Removed (%)SR (%)CFD ratioOperating Energy ratio
Fully poisonedNone088.454.951.13
Fully poisonedMotion jitter2974.261.801.15
Fully poisonedDemInf3883.255.871.13
20% poisonedNone088.67.731.08
20% poisonedMotion jitter3582.86.631.09
20% poisonedDemInf3484.86.311.07
Clean (0% poisoned)Motion jitter–86.01.171.06

Note. π0.5\pi_{0.5} on LIBERO-10 at α^=1.0\hat{\alpha} = 1.0, single training seed. “Poisoned Removed” is the fraction of poisoned trajectories discarded by the filter. Operating Energy ratio corresponds to the mechanical work the reviewer refers to. Each threshold was set independently of test outcomes, using clean data from the other nine tasks, and the same filter was applied to clean data.

We include the 20% setting because, when all data are poisoned, whatever survives the filter is still poisoned. A mixed dataset is needed to see whether filtering actually reduces the poisoned fraction.

In both settings, the filters remove about a third of the poisoned trajectories. At 20% poisoning, the CFD ratio decreases modestly, and in both settings the Operating Energy ratio is essentially unchanged. With motion jitter on fully poisoned data, the CFD ratio even rises. This is because the filter and the CFD ratio do not rank tasks the same way: the filter removes every demonstration of the task with the roughest clean motion, which also has a low CFD ratio, so that task drops out of the average.

These inspection methods remove some poisoned trajectories but do not remove the physical burden. We are extending the evaluation to additional poisoning fractions and three training seeds for the final version.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common Response

We sincerely thank the reviewers for their thoughtful feedback. We summarize the main additions below, with details in our individual responses.

1. Threat model and data collection

  • Attacker capabilities and collection costs (cdqN, KQsM)
  • Offline-versus-online comparison and action–observation consistency (cdqN)

2. Policy performance and perturbation analysis

  • Statistical uncertainty in success rates (aTcC)
  • Trajectory-level partial poisoning across three training seeds (aTcC)
  • Spectral and chunk-phase analyses (aTcC)

3. Data cleaning and defense scope

  • Filtering followed by retraining (aTcC, KQsM)
  • Revised defense-effectiveness claim (aTcC, KQsM)

4. CFD interpretation and safety literature

  • Clarification of CFD interpretation and limitations (KQsM)
  • Discussion of safe learning and safety filtering (KQsM)

The revised text is highlighted in blue\color{blue}{\text{blue}} in the updated manuscript for ease of review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.