PhysSense: Benchmarking Sensor-Grounded Physical Safety Reasoning in Embodied Agents
Abstract
Embodied agents increasingly rely on vision-language and vision-language-action models to reason about and interact with the physical world, yet many safety-critical states cannot be reliably inferred from visual observations alone. Properties such as temperature, contact force, payload weight, electrical state, gas concentration, obstacle clearance, and robot internal conditions often require dedicated physical sensing, creating a fundamental partial-observability challenge for embodied safety. We introduce **PhysSense**, a standards-grounded benchmark for evaluating whether embodied models can identify safety-relevant physical variables, recognize when critical evidence is missing, and make sensor-grounded safety decisions. The benchmark contains 580 instances derived from 200 semantic task template across ten operational physical-risk categories, and formulates safety reasoning as a three-way decision problem: Safe, Unsafe, or Ask, where Ask explicitly represents the need for additional physical sensing rather than generic uncertainty. To further study efficient physical-safety reasoning, we propose **PhysReason**, a training-free Big-Brain/Small-Brain framework in which a stronger model externalizes reusable physical-safety knowledge into an explicit RuleBank offline, while lightweight models retrieve and apply relevant rules during online inference. Across eight closed- and open-source models, PhysReason improves average decision accuracy from 67.74% to 85.19% and increases recall on missing-information cases from 83.96% to 97.36%. Our results reveal substantial limitations in current models' ability to reason about unobserved physical states and suggest that explicit sensor-grounded knowledge can improve embodied safety reasoning without requiring parameter updates or repeated online use of a large foundation model. The source code is available at https://anonymous.4open.science/r/PhysSense-FB77
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.