- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SafeHarness adds obstacle-aware route planning and contact execution to a coding-agent robot controller. Routes are proposed, checked externally, and replanned when needed; contact strategies depend on obstacle adjacency. On 32 SafeLIBERO tasks, the strongest configuration achieves 81.2% task success, 91.9% collision avoidance, and 78.1% safe success, with complementary real-robot results and component ablations.
Critical strengths
- Same-backbone and component ablations show substantial improvements in joint safe success, supporting the value of both route and contact protection.
- Real-robot tests complement simulation and demonstrate that the proposed execution procedure can work beyond the benchmark simulator.
Critical weaknesses
- Method novelty: The main gains come from explicit collision checks, replanning, and obstacle-dependent contact rules. The paper compares against VLAs and AEGIS, but not a conventional motion planner inside the same coding-agent harness. It therefore does not establish the benefit of LLM waypoint generation over an established geometric planning solution.
- Safety coverage: CAR counts an obstacle as untouched when its displacement never exceeds 1 mm; contact without displacement can therefore be missed. Route checks describe the gripper and payload, without sufficiently specifying full-arm coverage or image-to-metric geometry. These gaps matter because the method is presented as enforcing collision avoidance, rather than only reducing this surrogate.
- Failure attribution: Verbal recognition of an obstacle does not establish accurate perception or feasible control. Likewise, adding geometric tools and execution checks does not isolate whether failures come from prioritization, skill replay, or planning limitations. The claims about safety priorities and growing-context failures need interventions on those factors, beyond the current harness ablations.
- Practical validation: Real-robot evidence covers only two tasks with 20 trials each; simulation primarily tests one static obstacle. There are no uncertainty estimates or characterization of planning latency/API cost. This limits conclusions about robustness and practical operation under varied clutter, perception error, or time constraints.
Questions for authors
- How does the same harness perform with an established motion planner replacing LLM waypoint proposals, retaining contact rules and collision verification?
- Which robot bodies and swept volumes are checked, how is metric geometry obtained, and how do results change under direct contact detection and perturbed obstacle estimates?
- What interventions support the priority/context explanation, and what are real-robot uncertainty estimates and planning-time costs?