LIBERO-Dynamic: Benchmarking Robotic Manipulation in Dynamic Environments

Most simulation benchmarks for robotic manipulation keep target objects static and treat policy inference as instantaneous. These assumptions simplify repeatable evaluation, but they hide how a moving target evolves during computation and how stale an observation may be when its predicted action is executed. We introduce LIBERO-Dynamic, a controlled dynamic extension of all 40 LIBERO tasks. It drives objects through contact friction along three trajectory families and three speed tiers, while continuously advancing the physical world according to calibrated, model-specific inference latency. The same motion mechanism also supports training: we replay verified static demonstrations and compensate robot actions online using the target’s displacement relative to the reference, producing 9,000 quality-filtered dynamic trajectories. We evaluate six pretrained policies on a shared set of 1,800 dynamic conditions per configuration, compare execution strategies and action horizons with the checkpoint fixed, and fine-tune three architectures using dynamic and mixed data. The experiments show that faster motion degrades every evaluated policy, but not uniformly; execution timing and horizon can change the behavior of the same policy; and, in the measured sweep, asynchronous replanning benefits from frequent feedback only after the action queue can be sustained without underflow. Dynamic demonstrations improve each evaluated architecture under at least one training recipe, while the magnitude of improvement varies with temporal conditioning, data composition, and training budget. In a small-scale physical study using policies trained on real-robot dynamic demonstrations, we observe execution-strategy differences consistent with those found in simulation, with task-dependent effects, providing complementary evidence for the corresponding benchmark findings. By making object motion, inference latency, and action execution explicit and controllable within a broad task suite, LIBERO-Dynamic provides a reproducible platform for evaluating and improving robot policies for moving-target manipulation.