OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies
Abstract
Coding agents are extending their reach into the physical world by writing and executing robot control programs. One might expect the agents to use the same mature software stack that engineers have developed over decades. Yet existing work surrounds these agents with specially engineered interfaces for robot use, wrapping learned policies as tools or supplying hand-crafted perception and control primitives. We introduce OpenRUA, a harness that bypasses bespoke abstraction layers by providing off-the-shelf coding agents with only terminal access to the robot's native software interface ROS 2. OpenRUA employs a zero-abstraction workspace-as-harness design, only offering ROS 2 documentation and basic tools while leaving the coding agent to organize its own work without orchestrating any agentic workflow. OpenRUA achieves success rates of 99.0% on CaP-Bench and 87.0% on LIBERO-PRO, demonstrating that an off-the-shelf coding agent can serve as a zero-shot visuomotor policy through the robot's native interface, without bespoke primitives or task-specific training. (1) For perception, the agent spontaneously writes programs that process raw sensory inputs and derive metric measurements in 96.80% of episodes. (2) For manipulation, the agent spontaneously builds motion-control clients in 95.87% of episodes and closed-loop control programs in 50.13% of episodes. Our code is available at https://anonymous.4open.science/r/OpenRUA-34210.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.