RASBench: A Safety Benchmark for Embodied Agents

General-purpose embodied agents are designed to understand user intent and com- plete tasks. These agents rely on functional modules to receive user instructions, observe the environment, sense their proprioceptive state, and execute physical actions. However, in practice, these modules may operate abnormally during task execution, causing robots to deviate from their intended tasks or behave unsafely. Existing evaluations have not adequately characterized how such abnormalities affect agent operation. We introduce RASBench (Robotic Agent Safety Bench- mark), a system-level benchmark for evaluating embodied agents under abnormal operating conditions across diverse application settings, including warehouses, kitchens, and retail environments. RASBench comprises 200 task instances across eight simulation scenes and defines 14 types of safety constraints on robots’ phys- ical actions. It applies controlled interventions to these functional modules to induce abnormal operating conditions and uses execution traces from paired sim- ulation runs with and without intervention to quantify changes in task performance and execution safety. With RASBench, we evaluate several embodied agents on a simulated quadrupedal mobile manipulation platform. Our experiments show that some agents exhibit severe degradation in task performance and execution safety under the tested interventions. These findings highlight the need to account for ex- ecution safety under complex operating conditions when designing and evaluating embodied agents.