acceptodds
Under review as a conference paper at ICLR 2027

Agency at the Right Scale: Bimanual Manipulation through Active Observation

Abstract

Robot agents must adapt their reasoning as they act on and acquire knowledge about the physical world. In robotic manipulation, this raises a central question: which decisions should remain open to the model as execution unfolds? We introduce (ARS), a bimanual robot-agent architecture that makes the scope of reasoning an online decision. ARS connects task-level reasoning with sustained local interaction, preserving established goals and arm roles during local correction while allowing physical feedback to trigger broader replanning. Active perception makes this coordination embodied: the agent moves a wrist camera to resolve uncertainty in an ongoing manipulation, then uses the acquired evidence to decide whether to continue correcting, execute an established operation, or revise the task arrangement. Reusable procedures retain established action structure across executions, leaving scene-dependent interpretation and recovery to online reasoning. We evaluate ARS on 18 RoboDojo tasks and use execution studies and physical-robot demonstrations to examine how observation, correction, and procedural reuse interact during manipulation. ARS connects what the robot has established through physical interaction to what it can commit to execution and what must remain open to model judgment.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.