- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RoboHarness connects a pretrained language-model agent to robotic manipulation through an explicit visual-geometric tool interface. The agent selects image keypoints, which are tracked over time using optical flow and depth back-projection. It specifies geometric relations between gripper, object, and target points; the harness searches for satisfying poses, checks inverse-kinematics feasibility, and plans and executes motions. Direct adjustment tools provide additional control over end-effector and chassis motion. The paper argues that the constraint representation forms a minimal and complete interface without additional model training. Evaluation covers the first ten BEHAVIOR Challenge 2025 tasks and a 100-object lifting test, reporting advantages over several VLA and world-action-model baselines.
Critical strengths
- A concrete and interpretable interface between visual reasoning and robot control.
Persistent keypoint tracking gives the agent spatial references that can be reused across successive tool calls, while geometric constraints allow it to specify orientation-sensitive manipulation without directly generating every motor command (§3). Separating these specifications from inverse kinematics and motion planning is a practical design choice. The combination of constraint tools and direct adjustments provides a usable route for applying pretrained models without additional action-model training. This is a useful engineering contribution, even though it should not be equated with a new manipulation paradigm or a guarantee of task feasibility.
Critical weaknesses
- Limited conceptual novelty.
The central approach largely integrates existing ideas: visual keypoint grounding, geometric constraints, tracking, and conventional motion planning. As the paper itself acknowledges in §2.3, ReKep already generates relational constraints over 3D keypoints and optimizes robot motions, while CoPa uses part-level spatial constraints for grasping and subsequent manipulation. RoboHarness offers a useful interface variant, but this is a limited conceptual advance rather than the claimed new paradigm.
- The reported superiority is confounded by human task knowledge and different execution settings.
Appendix A states that the BEHAVIOR prompts are human-written and include task-specific procedures, while Appendix B lists higher-level skills such as opening doors and drawers, picking up objects, and placing objects in containers. The evaluated system therefore includes substantial task guidance and skill scaffolding beyond the advertised minimal primitives. Its results cannot be attributed to the geometric interface alone.
Moreover, the BEHAVIOR comparison places five-instance-per-task results alongside official leaderboard scores without establishing matched evaluation instances. In the 100-object test, RoboHarness uses an R1 Pro with floor objects, whereas learned policies are evaluated in LIBERO tabletop settings (§5.2; Appendix C). Using the same lifting criterion does not equate the manipulation difficulty across these conditions. These experiments demonstrate the behavior of particular complete systems, but do not establish the broad claim that a simple harness outperforms VLA and world-action models because of the proposed interface.
Questions for authors
- Beyond the agent-facing API and persistent tracking, what is the principal conceptual advance over the constraint-based manipulation approaches already discussed in §2.3? Which result demonstrates a benefit specific to that advance?
- Which reported comparisons use the same embodiment, scene, evaluation instances, and task information? Please clarify which Appendix B skills are used in the evaluated rollouts and whether any existing results separate the contribution of task-specific procedures and skills from that of the geometric tools.