- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies runtime safety filtering for VLA-based robotic manipulation (in single-obstacle settings). Existing CBF-based approaches rely on costly multi-stage geometric perception and restrict protection to the end-effector. To address these, the authors train a lightweight geometry prediction head with supervised labels to estimate the obstacle’s geometry from frozen VLA visual representations. At inference, a manually defined finite-state machine identifies manipulation phases (approach, grasp, transport, and placement) and adjusts CBF constraints. To extend collision protection beyond the end-effector, the method models robot body links as proxies whose motions are estimated through analytical kinematic Jacobians, while bounding the grasped object with a conservative envelope; both are modeled as ellipsoids. A CBF-QP then minimally modifies the VLA’s Cartesian actions to satisfy the active geometric safety constraints without retraining the policy. Experiments in simulation and on real robots demonstrate improved collision avoidance and safe task completion.
Critical strengths
-
The method addresses the practical challenge of improving VLA safety without retraining. I really like the design that preserves the VLA’s Cartesian action space by using analytical kinematic Jacobians to relate Cartesian commands to individual robot-link velocities. This extends collision protection beyond the end-effector while maintaining compatibility with the Cartesian action interfaces commonly used in VLA-based manipulation.
-
The experiments include ablation studies and real-world robotic evaluations.
Critical weaknesses
-
The geometry prediction head appears to be the major contributor to performance gains, but its supervision requirements are insufficiently characterized. In Table 2, replacing the learned geometry head with AEGIS’s perception pipeline yields performance comparable to that of AEGIS in Table 1. This does not negate the contributions of the other components, but suggests that their effectiveness depends strongly on accurate obstacle geometry, which is intuitive. A key question is whether collecting geometric supervision is substantially less burdensome than collecting additional safe demonstrations for direct VLA fine-tuning. Although geometric labels can be readily derived from simulator collision meshes, obtaining equivalent supervision in real-world settings may be considerably more challenging. The authors report roughly 63K supervised frames in simulation and mention manually annotated physical data for real-world deployment, but provide limited information about the latter’s size, collection procedure, or annotation effort.
- The geometry prediction head relies on standard supervised learning. This may raise potential coverage concerns about the supervision data and is conceptually straightforward.
-
While the proposed whole-body safety filtering is interesting, it relies on how the low-level controller resolves the ambiguity in robot-link motions for a given end-effector command. The authors explicitly assume negligible null-space motion in the appendix, but this is an important assumption that should be highlighted in the main paper.
-
The phase-aware scheduling is heuristic and the contribution (Table 2) seems modest.
Questions for authors
-
What are the data collection and annotation procedures for the real-world geometry prediction head including annotation effort?
-
What is the measured latency of AEGIS’s geometry perception pipeline? AEGIS reports a control frequency of 20 Hz, while GeoPhase achieves approximately 21.4 Hz despite enforcing additional safety constraints for robot body links and the held object.
3 comments
Response-2
W3: Contribution of Phase-Aware Scheduling
A: We agree that the FSM-based phase tracker is heuristic. However, this lightweight design is deliberate, and its contribution to balancing safety and task completion is substantial.
Different manipulation phases exhibit distinct geometric risks and control requirements (Sec. 4.2): approach and grasp require sufficient proximity to the target, transport introduces the held object into the protected geometry, and place requires precise release. Unlike the phase-agnostic AEGIS and KNOWS, GeoPhase’s online FSM dynamically modulates the active proxy set , activation threshold , and barrier gain . It aims to adapt safety parameters to manipulation phases using gripper commands and proprioceptive history, without task-specific phase annotations.
The phase tracker costs 0.03 ms per control step (Table 7) while supporting multi-cycle manipulation such as the SafeLIBERO LONG suite. As shown in Table 2 and Figure 4(b), phase awareness increases overall TSR from 63.8% to 66.4% and SSR from 59.4% to 62.6%, while reducing CR from 16.2% to 12.8%.
Q2: Latency Comparison
A: AEGIS’s explicit perception pipeline involves a costly multi-stage process (remote VLM semantic identification GroundingDINO depth point-cloud filtering ellipsoid fitting). Because it relies on remote API calls with high network latency and computationally heavy local point-cloud optimization, a single complete perception update takes seconds.
We evaluated the AEGIS visual pipeline on object_I Task2 (milk), episodes 0--9, using one RTX 4090 (24 GB), dual Xeon Silver 4314 CPUs, and two views. After excluding the first cold-start run, the remaining nine runs yielded:
| AEGIS Perception Stage | Mean (ms) | P95 (ms) |
|---|---|---|
| GroundingDINO + Point-Cloud Projection & Filtering | 288.08 | 335.59 |
| Ellipsoid Fitting (ConvexHull + CVXPY/SCS) | 12,019.32 | 23,383.34 |
| Complete Local Vision Pipeline (w/o remote API) | 14,804.97 | 26,561.75 |
| Full Pipeline (incl. Remote VLM API & retries) | 40,007.05 | 87,731.21 |
As shown above, ellipsoid fitting accounts for approximately ~ of the runtime. These measurements show that the complete pipeline cannot be recomputed within a control step under our setup. They do not establish how the originally reported 20 Hz AEGIS controller schedules or reuses geometry.
In contrast, GeoPhase reuses pooled VLA features and predicts ellipsoid parameters through a lightweight MLP. VLA inference (14.41 ms) and geometry prediction (12.63 ms) run asynchronously per action chunk, while our local safety layer runs synchronously in just 10.21 ms/step, including phase tracking, proxy updates, constraint construction, QP solving, and post-processing. The measured mean end-to-end control computation is 46.90 ms/step (21.4 Hz).
We will add the complete profiling protocol and latency breakdown to Appendix E.2.
To sum up:
We clarified the real-world geometry-collection procedure, annotation effort, and frame-level label reuse (W1&Q1). We further acknowledged the standard supervised nature of the geometry head and expanded the held-out obstacle evaluation (W1.1). We will explicitly state the negligible null-space motion assumption in the revised manuscript (W2). We explained the role and low computational cost of the phase tracker while reporting its scenario-dependent benefit (W3). Finally, we provided a latency breakdown for the AEGIS perception pipeline (Q2).
We sincerely thank the reviewer for the constructive feedback, which has helped us improve the precision and completeness of our presentation.
If you have any further questions or additional concerns, please feel free to discuss them with us.
Dear Reviewer JBDp,
We truly value all of your suggestions. If you have any further questions, please do not hesitate to let us know. We will do our best to address them during the remainder of the discussion period. Thank you again for your time and effort. Best of luck!
Best regards, Authors
Response-1
We sincerely thank Reviewer JBDp for the careful reading and constructive assessment. We are encouraged that you recognized our use of analytical kinematic Jacobians to extend collision protection beyond the EEF while preserving compatibility with Cartesian action interfaces. We address each concern below and specify the corresponding clarifications and revisions.
W1 & Q1: Real-World Geometry Supervision
We agree that accurate obstacle geometry is important to the downstream CBF-QP and that the real-world supervision procedure should be clearer. Our physical dataset contains approximately 15k control-step samples from 25 static training configurations, separate from evaluation, with approximately 5 hours of manual geometry annotation. The observations are collected through rollouts of the frozen π0.5 policy under varied obstacle placements and scene layouts.
We first calibrate the tabletop frame to the robot base frame. For each obstacle, we manually measure its bounding dimensions (L,W,H) and bounding-box center cT in the tabletop frame. Given the calibrated transformation (RBT,tBT), we compute cB=RBTcT+tBT and the base-frame bounding-box half-extents bB=∣RBT∣(L,W,H)⊤/2. We set the ellipsoid semi-axes to rB=3,bB, which conservatively encloses the measured bounding box. For a fixed obstacle pose, the resulting label is reused across all frames in the rollout. Translating the obstacle requires updating only its center; changing its orientation requires recomputing the base-frame extents. Thus, the supervision is amortized across frames and does not require frame-by-frame manual annotation or additional collision-free action demonstrations.
We will add the collection procedure, label construction, and dataset statistics to Appendices C.1 and E.6.
W1.1: Standard Supervised Geometry Head and Coverage
A: We agree that the geometry head uses standard supervised learning. Our contribution is a structured geometric interface between frozen VLA representations and analytic safety filtering. The supervision combines ellipsoid-parameter regression with barrier-value and barrier-gradient losses, with additional weight near the safety boundary.
For simulation, the geometry dataset contains 63,187 frame-level samples and is split strictly at the rollout level into training and validation sets. Evaluation rollouts are excluded from training and model selection. These experiments assess generalization across spatial configurations and trajectories within a defined operational domain.
In the physical experiment, the unseen wine bottle provides evidence of transfer to one held-out obstacle instance under the fixed-camera setup, with an approximately 2,cm center error. We also evaluated two additional unseen physical obstacles (10 trials each): an OOD mug(≈1 cm center error) and an OOD beverage bottle (≈2 cm center error). We will include these expanded OOD evaluations in Appendix E.6.
W2: Low-Level Controller and Null-Space Motion
A: We agree that the negligible null-space motion assumption should be stated in the main paper. Our link-motion model uses Tℓ(i)←e=Jℓ(i)Je†, which assumes negligible auxiliary null-space velocity, (I−Je†Je)ν≈0.
We will state this assumption explicitly in Section 4.3 and clarify that the body-proxy constraints are conditional on the deployed controller satisfying this approximation.