Contact Code: A Typed Program Interface for Measurable and Repairable Contact-Rich Manipulation
Abstract
The ability to adapt to physical feedback is essential for robots performing contact-rich manipulation. Vision-language models (VLMs) provide a promising way to interpret task instructions and configure robot behavior from visual observations. However, adapting these decisions after physical interaction requires connecting execution feedback to the parameters that can be revised. We introduce Contact Code, a typed program interface that supports this connection by organizing controller parameters, measured execution evidence, and permitted updates around individual contact stages. Within predefined task templates, a VLM proposes a task configuration that is validated and instantiated into an executable program. During execution, robot motion and contact measurements provide feedback for controller-level recovery and evidence for subsequent program revision. If execution still fails, the system uses this evidence to select a contact stage and guides the VLM to propose a constrained update before the next attempt. We evaluate Contact Code on eight tasks and 800 held-out instances across ManiSkill, Isaac Lab, and UniVTAC-based simulation, spanning wrist force and fingertip tactile sensing as well as single-arm and bimanual settings. Contact Code increases success from 60.4% on the first rollout to 73.6% after repair and outperforms free-form parameter repair by 8.3% under the same feedback and rollout budget. On a Flexiv arm, the same planning and repair pipeline, with only the platform-facing adapters changed, improves peg-insertion success from 80% on the first trial to 96% after repair, providing evidence of portability across robot embodiments and control APIs. This interface provides a foundation for studying how physical feedback can guide program revision, offering a path toward more adaptable language-guided robotic systems.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.