- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a runtime safety layer deployed between a frozen robot policy and the low-level controller. Based on current observations, robot state, and pending actions, the system invokes target verification, grasp intervention, motion safety filtering, and placement checking modules to block, correct, or locally take over risky interactions. It also specifies bounded recovery and policy handoff procedures. After taking over, it discards stale, unexecuted actions and queries the policy again using current observations. The authors evaluate frozen π₀.₅ and Cosmos3 policies in the RoboLab simulator on stacked-object manipulation, transport around obstacles, general placement, and tasks involving sequential operations and combined risks. The paper reports improvements in both task success rate and safe success rate, and provides stage-specific risk diagnostics. The method does not require fine-tuning the base policy, but relies on additional calibrated RGB-D information, specialized perception and grasp models, and geometric control modules.
Critical strengths
- The framework supports more reliable task execution by combining safety checks across multiple manipulation stages with a clear recovery process. This allows the robot to address detected problems and resume the task with the original policy after intervention.
- The framework requires no retraining and is evaluated with two frozen policies. After intervention, it discards the remaining actions and asks the policy to generate new ones using the latest observations.
Critical weaknesses
- The benefits over existing safety layers are not directly validated. The method should be compared with VLSA, which is mentioned in the paper, or other relevant baselines, rather than only with the original policy without a safety layer.
- Without ablation studies, it is unclear whether the improvements mainly come from the safety mechanisms or from the additional perception inputs and specialized controllers.
- Section 4.2 states that the results are obtained using three random seeds with ten rollouts per seed, and that the mean and standard deviation are reported across seeds. According to Equation (4), both TSR and SSR are computed from binary events. If each cell in Table 3 directly corresponds to these thirty evaluations, the success rate for a single seed should vary in increments of 10 percentage points, and the mean across three seeds should vary in increments of approximately 3.33 percentage points. However, the table reports values such as 39.38%, 61.42%, and 49.22%, which cannot arise from this direct aggregation procedure.
Questions for authors
- How are the results in Table 3 aggregated under the stated protocol of three seeds with ten rollouts per seed? Please provide the per-seed sample sizes, success and safe-success counts, and explain how values such as 39.38% and 61.42% are obtained.
- How does the proposed framework compare with an existing safety layer, such as VLSA, using the same base policy and sensor inputs?
- Can ablations separate the benefits of the intervention logic from those of the additional RGB-D inputs and specialized grasp controllers?
3 comments
(3) Separating intervention logic from perception and specialized controllers
Depth is required by our current implementation to construct the three-dimensional geometry used for grasp and motion checks. AEGIS and VoLo also incorporate RGB-D-derived geometry in their execution pipelines; depth-based perception is therefore not unique to our framework. We retain RGB-D inputs and examine the contribution of intervention logic through the Forced comparison, which uses the same perception inputs and specialized controllers but differs in module activation and policy handoff.
In Forced, the constrained modules are activated routinely rather than selectively. As in the MoveIt baseline, each module is allowed up to three attempts; failure to succeed within this budget is counted as task failure.
Relative to Forced, our method increases TSR and SSR by 29.87 and 15.80 percentage points for , and by 31.36 and 23.33 points for Cosmos3. These results indicate that supplying specialized controllers alone is insufficient to reproduce the gains of our full framework. They support the complementary role of VLA-driven execution and constrained intervention: the VLA can continue execution when specialized modules cannot resolve a stage, while the modules correct detected risks. This combination achieves higher task success and safe success than routine module execution, supporting the value of selective intervention and policy handoff.
reply to rebuttal
Thank you for the additional experiments and clarifications. The added baselines, cross-simulator evaluation, and intervention statistics partially address my original concerns and strengthen the empirical evaluation.
My remaining concerns concern the methodological novelty and whether the current comparisons adequately isolate the benefit of selective intervention from that of the specialist controllers. In particular, the Forced baseline changes several aspects of execution simultaneously, making this attribution difficult.
We thank the reviewer for their valuable feedback and provide our responses below.
(1) Clarification of sample sizes and aggregation
Our original evaluation includes 90 scenes: 30 obstacle avoidance, 30 stacking, and 30 placement scenes. The protocol in Section 4.2 should specify three random seeds with ten rollouts per scene per seed, rather than ten rollouts per seed for an entire task suite. Thus, each suite-level entry in Table 3 of the manuscript has a nominal sample size of 30×10=300 rollouts per seed, or 900 across three seeds. Across all three suites, the nominal total is 2,700 rollouts per method. We apologize for omitting this scene-level specification.
Our log audit also identified 31 technically interrupted rollouts, including CUDA out-of-memory errors, abnormal process termination, and segmentation faults. For the 39.38% TSR entry, the valid sample sizes and task-success counts are:
\qquad (k_1,k_2,k_3)=(112,113,129).$$ Therefore, $$\overline{\mathrm{TSR}} = \frac{100}{3}\left(\frac{112}{300}+\frac{113}{299}+\frac{129}{300}\right) = 39.3753\% \approx 39.38\%.$$ with a sample standard deviation of approximately 3.15 percentage points, matching the reported $39.38 \pm 3.15$. We have corrected the corresponding error in our evaluation code. Rollouts interrupted by abnormal program termination will be rerun rather than excluded from the evaluation. We will update the corresponding results and clarify this procedure in the revised manuscript. ### (2) Comparisons with existing safety layers and other baselines We have added four baselines (Table 1): MoveIt-based (MoveIt 2/MTC, OMPL-RRTConnect, and FCL collision checking); AEGIS-adapted; VoLo; and Forced. Both AEGIS-adapted and VoLo pair the frozen $\pi_{0.5}$ policy with Qwen3-VL-8B. AEGIS-adapted uses the VLM to select a critical obstacle for ellipsoid-based CBF-QP action filtering, while VoLo uses it for subgoal planning, visual failure/completion monitoring, and language-based replanning. Forced mandates grasp/place takeover and transport filtering, excludes VLA grasp candidates, and terminates after three failed attempts without policy fallback. Table 1 supports the complementary roles of VLA-driven execution and constrained specialist modules. MoveIt and Forced achieve lower TSR than the corresponding vanilla policies. Nevertheless, constrained execution can improve safe task completion. Our selective framework instead improves both metrics. AEGIS combines VLM-based obstacle identification with geometric safety filtering, whereas VoLo uses VLM-driven tool orchestration; both also incorporate depth-derived geometry in their pipelines. Our contribution is therefore not simply additional geometric perception or stronger controllers, but how constrained execution is coordinated with the original policy. The higher TSR and SSR relative to the tested alternatives support the effectiveness of this selective coordination within the evaluated setting. **Table 1.** Values are reported as mean ± standard deviation across three random seeds (%). Due to the limited rebuttal period, the additional experiments were conducted on 90 scenes with 3 rollouts per scene per seed, yielding 810 rollouts per method. | Method | TSR ↑ | SSR ↑ | |:---|---:|---:| | MoveIt | 21.11 ± 0.37 | 17.90 ± 0.86 | | $\pi_{0.5}$ + AEGIS | 28.77 ± 1.54 | 20.49 ± 1.50 | | $\pi_{0.5}$ + VoLo | 34.20 ± 0.77 | 11.23 ± 0.21 | | $\pi_{0.5}$ Vanilla | 42.35 ± 2.17 | 11.36 ± 0.21 | | $\pi_{0.5}$ + Ours | **65.43 ± 1.30** | **38.64 ± 1.40** | | $\pi_{0.5}$ + Forced | 35.56 ± 0.00 | 22.84 ± 1.50 | | Cosmos3 Vanilla | 55.93 ± 1.85 | 20.00 ± 1.96 | | Cosmos3 + Ours | **67.16 ± 1.50** | **42.96 ± 0.98** | | Cosmos3 + Forced | 35.80 ± 0.77 | 19.63 ± 1.28 |