acceptodds
Under review as a conference paper at ICLR 2027

A Plug-and-Play Runtime Safety Layer for Generalist Robot Policies

Abstract

Vision-language-action (VLA) policies and world-action models (WAMs) provide increasingly general interfaces for robotic manipulation, yet their action genera- tion is primarily optimized for task completion rather than explicit safety-critical execution. As a result, semantically reasonable actions may still be physically unsafe, leading to inaccurate grasps, collisions, or unstable placements and ex- posing a gap between the general capabilities of modern robot policies and the execution-level safety requirements of manipulation. We introduce a plug-and-play, training-free runtime safety layer that operates between a frozen generalist policy and the robot controller. Rather than replacing the underlying policy, the proposed framework selectively intervenes only at safety-critical stages and uses specialized runtime modules to detect and handle unsafe actions, while leaving normal pol- icy execution unchanged. This design requires neither policy fine-tuning nor an additional vision-language-model planner, allowing the original policy to retain its general-purpose capabilities while gaining more reliable execution at critical manipulation stages. We evaluate the framework on manipulation tasks involving target ambiguity, contact-critical grasping, transport hazards, and unsafe placement using multiple frozen backbones. Results show that the proposed runtime safety layer improves both task success rate and safe success rate, while its plug-and-play design enables the same framework to be applied across different generalist robot policies without modifying or retraining the underlying models.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Manipulation, Runtime Safety

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Vx5zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a runtime safety layer deployed between a frozen robot policy and the low-level controller. Based on current observations, robot state, and pending actions, the system invokes target verification, grasp intervention, motion safety filtering, and placement checking modules to block, correct, or locally take over risky interactions. It also specifies bounded recovery and policy handoff procedures. After taking over, it discards stale, unexecuted actions and queries the policy again using current observations. The authors evaluate frozen π₀.₅ and Cosmos3 policies in the RoboLab simulator on stacked-object manipulation, transport around obstacles, general placement, and tasks involving sequential operations and combined risks. The paper reports improvements in both task success rate and safe success rate, and provides stage-specific risk diagnostics. The method does not require fine-tuning the base policy, but relies on additional calibrated RGB-D information, specialized perception and grasp models, and geometric control modules.

Critical strengths
  1. The framework supports more reliable task execution by combining safety checks across multiple manipulation stages with a clear recovery process. This allows the robot to address detected problems and resume the task with the original policy after intervention.
  2. The framework requires no retraining and is evaluated with two frozen policies. After intervention, it discards the remaining actions and asks the policy to generate new ones using the latest observations.
Critical weaknesses
  1. The benefits over existing safety layers are not directly validated. The method should be compared with VLSA, which is mentioned in the paper, or other relevant baselines, rather than only with the original policy without a safety layer.
  2. Without ablation studies, it is unclear whether the improvements mainly come from the safety mechanisms or from the additional perception inputs and specialized controllers.
  3. Section 4.2 states that the results are obtained using three random seeds with ten rollouts per seed, and that the mean and standard deviation are reported across seeds. According to Equation (4), both TSR and SSR are computed from binary events. If each cell in Table 3 directly corresponds to these thirty evaluations, the success rate for a single seed should vary in increments of 10 percentage points, and the mean across three seeds should vary in increments of approximately 3.33 percentage points. However, the table reports values such as 39.38%, 61.42%, and 49.22%, which cannot arise from this direct aggregation procedure.
Questions for authors
  1. How are the results in Table 3 aggregated under the stated protocol of three seeds with ten rollouts per seed? Please provide the per-seed sample sizes, success and safe-success counts, and explain how values such as 39.38% and 61.42% are obtained.
  2. How does the proposed framework compare with an existing safety layer, such as VLSA, using the same base policy and sensor inputs?
  3. Can ablations separate the benefits of the intervention logic from those of the additional RGB-D inputs and specialized grasp controllers?
3 comments

We thank the reviewer for their valuable feedback and provide our responses below.

(1) Clarification of sample sizes and aggregation

Our original evaluation includes 90 scenes: 30 obstacle avoidance, 30 stacking, and 30 placement scenes. The protocol in Section 4.2 should specify three random seeds with ten rollouts per scene per seed, rather than ten rollouts per seed for an entire task suite. Thus, each suite-level entry in Table 3 of the manuscript has a nominal sample size of 30×10=30030 \times 10 = 300 rollouts per seed, or 900 across three seeds. Across all three suites, the nominal total is 2,700 rollouts per method. We apologize for omitting this scene-level specification.

Our log audit also identified 31 technically interrupted rollouts, including CUDA out-of-memory errors, abnormal process termination, and segmentation faults. For the 39.38% TSR entry, the valid sample sizes and task-success counts are:

\qquad (k_1,k_2,k_3)=(112,113,129).$$ Therefore, $$\overline{\mathrm{TSR}} = \frac{100}{3}\left(\frac{112}{300}+\frac{113}{299}+\frac{129}{300}\right) = 39.3753\% \approx 39.38\%.$$ with a sample standard deviation of approximately 3.15 percentage points, matching the reported $39.38 \pm 3.15$. We have corrected the corresponding error in our evaluation code. Rollouts interrupted by abnormal program termination will be rerun rather than excluded from the evaluation. We will update the corresponding results and clarify this procedure in the revised manuscript. ### (2) Comparisons with existing safety layers and other baselines We have added four baselines (Table 1): MoveIt-based (MoveIt 2/MTC, OMPL-RRTConnect, and FCL collision checking); AEGIS-adapted; VoLo; and Forced. Both AEGIS-adapted and VoLo pair the frozen $\pi_{0.5}$ policy with Qwen3-VL-8B. AEGIS-adapted uses the VLM to select a critical obstacle for ellipsoid-based CBF-QP action filtering, while VoLo uses it for subgoal planning, visual failure/completion monitoring, and language-based replanning. Forced mandates grasp/place takeover and transport filtering, excludes VLA grasp candidates, and terminates after three failed attempts without policy fallback. Table 1 supports the complementary roles of VLA-driven execution and constrained specialist modules. MoveIt and Forced achieve lower TSR than the corresponding vanilla policies. Nevertheless, constrained execution can improve safe task completion. Our selective framework instead improves both metrics. AEGIS combines VLM-based obstacle identification with geometric safety filtering, whereas VoLo uses VLM-driven tool orchestration; both also incorporate depth-derived geometry in their pipelines. Our contribution is therefore not simply additional geometric perception or stronger controllers, but how constrained execution is coordinated with the original policy. The higher TSR and SSR relative to the tested alternatives support the effectiveness of this selective coordination within the evaluated setting. **Table 1.** Values are reported as mean ± standard deviation across three random seeds (%). Due to the limited rebuttal period, the additional experiments were conducted on 90 scenes with 3 rollouts per scene per seed, yielding 810 rollouts per method. | Method | TSR ↑ | SSR ↑ | |:---|---:|---:| | MoveIt | 21.11 ± 0.37 | 17.90 ± 0.86 | | $\pi_{0.5}$ + AEGIS | 28.77 ± 1.54 | 20.49 ± 1.50 | | $\pi_{0.5}$ + VoLo | 34.20 ± 0.77 | 11.23 ± 0.21 | | $\pi_{0.5}$ Vanilla | 42.35 ± 2.17 | 11.36 ± 0.21 | | $\pi_{0.5}$ + Ours | **65.43 ± 1.30** | **38.64 ± 1.40** | | $\pi_{0.5}$ + Forced | 35.56 ± 0.00 | 22.84 ± 1.50 | | Cosmos3 Vanilla | 55.93 ± 1.85 | 20.00 ± 1.96 | | Cosmos3 + Ours | **67.16 ± 1.50** | **42.96 ± 0.98** | | Cosmos3 + Forced | 35.80 ± 0.77 | 19.63 ± 1.28 |

(3) Separating intervention logic from perception and specialized controllers

Depth is required by our current implementation to construct the three-dimensional geometry used for grasp and motion checks. AEGIS and VoLo also incorporate RGB-D-derived geometry in their execution pipelines; depth-based perception is therefore not unique to our framework. We retain RGB-D inputs and examine the contribution of intervention logic through the Forced comparison, which uses the same perception inputs and specialized controllers but differs in module activation and policy handoff.

In Forced, the constrained modules are activated routinely rather than selectively. As in the MoveIt baseline, each module is allowed up to three attempts; failure to succeed within this budget is counted as task failure.

Relative to Forced, our method increases TSR and SSR by 29.87 and 15.80 percentage points for π0.5\pi_{0.5}, and by 31.36 and 23.33 points for Cosmos3. These results indicate that supplying specialized controllers alone is insufficient to reproduce the gains of our full framework. They support the complementary role of VLA-driven execution and constrained intervention: the VLA can continue execution when specialized modules cannot resolve a stage, while the modules correct detected risks. This combination achieves higher task success and safe success than routine module execution, supporting the value of selective intervention and policy handoff.

Reviewer_Vx5zOpenReview ↗

reply to rebuttal

Thank you for the additional experiments and clarifications. The added baselines, cross-simulator evaluation, and intervention statistics partially address my original concerns and strengthen the empirical evaluation.

My remaining concerns concern the methodological novelty and whether the current comparisons adequately isolate the benefit of selective intervention from that of the specialist controllers. In particular, the Forced baseline changes several aspects of execution simultaneously, making this attribution difficult.

Reviewer_oWQ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on safe execution for generalist robotic manipulation. The authors note that vision-language-action (VLA) policies and world-action models (WAMs) are optimized for task completion rather than execution-level safety, which may lead to collisions, faulty grasps and unstable placements during physical interaction. They argue that the key to addressing this gap lies in an external runtime safety mechanism that does not alter the original general-purpose policy, and accordingly propose a plug-and-play runtime safety layer built upon event-driven risk assessment and stage-specific intervention modules. The proposed framework is validated across stacked-object manipulation, obstacle-aware transport and placement simulation tasks.

Critical strengths

The design philosophy of the plug-and-play, training-free runtime safety layer proposed in this paper is commendable. Rather than replacing task-oriented action generation within the general strategy, it utilizes event-based risk assessment to trigger interventions tailored to specific phases. This design reduces manipulation risks at the execution level while preserving the general-purpose capabilities of the frozen vision-language-action models and world-action models backbone network.

Critical weaknesses

(1) The proposed method relies on heuristic judgments, where the triggering logic and thresholds are manually tuned. The authors should discuss whether this design will impair the generalization and robustness of the method. (2) The authors should discuss whether the proposed method can be deployed in real-world environments and verify whether the trigger logic for risk assessment remains reliable in the presence of realistic perception noise. Furthermore, they should analyze whether there are any failure modes that cannot be replicated in simulation but emerge only in the real world, in order to complete the risk analysis.

Questions for authors

How robust is the proposed method against out-of-distribution risks, such as slippage of the grasped object?

1 comment

We thank the reviewer for their valuable feedback and provide our responses below.

Safety requirements depend on the application, object properties, and acceptable risk tolerance. For example, light contact may be acceptable for a robust block but unacceptable for a fragile glass. Therefore, no single contact-tolerance threshold is appropriate for all objects and tasks. Our evaluation follows a predefined, application-specific safety specification. An OOD event constitutes a safety violation when it breaches one of these specified constraints; events outside this specification remain beyond our evaluated safety coverage, rather than being established as risk-free.

Object dropping, as raised by the reviewer, is explicitly included in our safety definition (see Appendix A1). The VLA may nevertheless attempt to re-grasp the dropped object and continue the task. Such recovery can restore task progress, but does not erase the recorded safety violation.

The VLA remains the primary execution backbone, supporting both task execution and recovery. Costly assessment and intervention modules are triggered at relevant execution transitions, rather than repeatedly invoking every module at every control step. This event-triggered design limits repeated perception and planning calls and their wall-clock overhead, while preserving VLA-driven execution.

To examine transferability under the same safety specification, we transferred the safety layer, including its triggering logic and thresholds, from RoboLab/Isaac Sim–PhysX to MuJoCo without modification or retuning. Policy checkpoints are fine-tuned for their respective environments. Environment-specific policy checkpoints remain frozen during evaluation. This evaluates the transferability of the configured checks across simulators.

Table 1. Cross-simulator performance. Values are reported as mean ± standard deviation, in percent. Due to the limited rebuttal period, the additional experiments were conducted on 90 scenes with 3 rollouts per scene per seed, yielding 810 rollouts per method.

SimulatorPolicyVanilla TSRVanilla SSROurs TSROurs SSR
Isaac Sim/PhysXπ0.5\pi_{0.5}42.35 ± 2.1711.36 ± 0.2165.43 ± 1.3038.64 ± 1.40
Isaac Sim/PhysXCosmos355.93 ± 1.8520.00 ± 1.9667.16 ± 1.5042.96 ± 0.98
MuJoCoπ0.5\pi_{0.5}27.04 ± 3.2212.59 ± 3.7543.15 ± 3.4923.21 ± 1.14
MuJoCoCosmos334.07 ± 2.0816.48 ± 1.7137.41 ± 1.9628.51 ± 2.98

Table 1 shows that the same safety-layer configuration improves both TSR and SSR for both policies in both simulators. These results provide evidence of the safety layer’s robustness.

Reviewer_N1fVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper adds a runtime supervisor between a frozen robot policy and the robot controller. This is a standard practice in robotics, especially in the industry or in humanoids.

Critical strengths

the paper solves an important problem. When robots are deployed in the real world, it's important to add safety layers. I also liked the evaluation at different stages.

Critical weaknesses
  1. What the paper proposed is a standard practice. Kindly clarify why it's worth writing an ICLR paper on this. The framework combines established object grounding, segmentation, grasp generation, geometric motion filtering, and placement rules. The shown experiments compare only against the unmodified policies. Comparisons with existing runtime safety layers, a conventional perception–planning pipeline, and component ablations are needed to establish the value of the proposed coordination, if there is a methodological contribution that I failed to grasp.
  2. The safety scope is substantially narrower than the title suggests. The collision filter protects the gripper, explicitly excluding the payload and arm links. Object attachment is assumed after acquisition rather than continuously verified. Placement checks use object-center location and release height, which do not establish stable placement for arbitrary objects. Eventhough these are useful safeguards, they do not establish general execution safety.
  3. Generalization and selective intervention remain insufficiently demonstrated. All shown experiments use simulation, one robot embodiment, and 3 closely matched manipulation suites. Moreover, grasp processing follows target acceptance and can execute a specialized controller, making it unclear how much improvement comes from detecting unsafe actions versus routinely supplying a stronger grasping system. Intervention frequencies, policy-action retention, matched controller baselines, and real-robot tests would clarify this distinction. It is important to show generalization beyond π0.5\pi_{0.5} and Cosmos 3.

Since, I believe, the proposed benchmark can be produced with a coding agent within a day or so, kindly provide deeper insights gained through this study.

Questions for authors

Please the weaknesses I highlighted.

2 comments

We thank the reviewer for their valuable feedback and provide our responses below.

1. Methodological contribution and additional baselines

Our contribution is a multi-stage safety harness that separates action generation from execution authorization. The VLA remains the default source of task-oriented actions, while the harness applies externally checkable conditions across target interaction, grasping, transport, and placement. Proposals that fail the applicable checks are rejected, corrected, or replaced by commands from constrained execution modules, with explicit control handoff. The harness itself requires neither policy retraining nor an additional high-level VLM planner.

The methodological focus is the coordination of whether an action is authorized, how constrained intervention is performed, and when control returns to the policy, rather than a fixed pipeline that routinely replaces the VLA with specialist modules.

We have added four baselines (Table 1): MoveIt-based (MoveIt 2/MTC, OMPL-RRTConnect, and FCL collision checking); AEGIS-adapted; VoLo; and Forced. Both AEGIS-adapted and VoLo pair the frozen π0.5\pi_{0.5} policy with Qwen3-VL-8B. AEGIS-adapted uses the VLM to select a critical obstacle for ellipsoid-based CBF-QP action filtering, while VoLo uses it for subgoal planning, visual failure/completion monitoring, and language-based replanning. Forced mandates grasp/place takeover and transport filtering, excludes VLA grasp candidates, and terminates after three failed attempts without policy fallback.

Table 1 supports the complementary roles of VLA-driven execution and constrained specialist modules. MoveIt and Forced achieve lower TSR than the corresponding vanilla policies. Nevertheless, constrained execution can improve safe task completion. Our selective framework instead improves both metrics. AEGIS combines VLM-based obstacle identification with geometric safety filtering, whereas VoLo uses VLM-driven tool orchestration; both also incorporate depth-derived geometry in their pipelines. Our contribution is therefore not simply additional geometric perception or stronger controllers, but how constrained execution is coordinated with the original policy. The higher TSR and SSR relative to the tested alternatives support the effectiveness of this selective coordination within the evaluated setting.

Table 1. Values are reported as mean ± standard deviation across three random seeds (%). Due to the limited rebuttal period, the additional experiments were conducted on 90 scenes with 3 rollouts per scene per seed, yielding 810 rollouts per method.

MethodTSR ↑SSR ↑
MoveIt21.11 ± 0.3717.90 ± 0.86
π0.5\pi_{0.5} + AEGIS28.77 ± 1.5420.49 ± 1.50
π0.5\pi_{0.5} + VoLo34.20 ± 0.7711.23 ± 0.21
π0.5\pi_{0.5} Vanilla42.35 ± 2.1711.36 ± 0.21
π0.5\pi_{0.5} + Ours65.43 ± 1.3038.64 ± 1.40
π0.5\pi_{0.5} + Forced35.56 ± 0.0022.84 ± 1.50
Cosmos3 Vanilla55.93 ± 1.8520.00 ± 1.96
Cosmos3 + Ours67.16 ± 1.5042.96 ± 0.98
Cosmos3 + Forced35.80 ± 0.7719.63 ± 1.28

2. Safety scope and computational trade-offs

The current four-sphere envelope covers both gripper and payload; we will correct the manuscript’s inaccurate payload exclusion. Arm links remain unprotected. Our initial 16-sphere implementation included arm-link protection but required approximately 0.92 s per filtering step, versus 1.12–1.23 ms currently. Arm-link collisions occurred in 6.83% of trajectories with collisions and 2.02% of unsafe task executions. These measurements motivated prioritizing gripper–payload protection within the online execution budget. “Multi-stage safety” denotes specified execution checks across evaluated manipulation stages, not a guarantee against every failure mode. The observed SSR gains support these safeguards’ practical value within this scope.

3. Generalization and selective intervention

We evaluate π0.5\pi_{0.5} and Cosmos3 in both RoboLab/Isaac Sim–PhysX and MuJoCo. Policy checkpoints are fine-tuned for their respective environments and frozen during evaluation, whereas the safety layer is transferred without modification or retuning. Table 2 shows that the same safety-layer configuration improves both TSR and SSR for both policies in both simulators. These results provide evidence of the safety layer’s robustness.

Table 2. Cross-simulator performance. Values are reported as mean ± standard deviation, in percent. Due to the limited rebuttal period, the additional experiments were conducted on 90 scenes with 3 rollouts per scene per seed, yielding 810 rollouts per method.

SimulatorPolicyVanilla TSRVanilla SSROurs TSROurs SSR
Isaac Sim/PhysXπ0.5\pi_{0.5}42.35 ± 2.1711.36 ± 0.2165.43 ± 1.3038.64 ± 1.40
Isaac Sim/PhysXCosmos355.93 ± 1.8520.00 ± 1.9667.16 ± 1.5042.96 ± 0.98
MuJoCoπ0.5\pi_{0.5}27.04 ± 3.2212.59 ± 3.7543.15 ± 3.4923.21 ± 1.14
MuJoCoCosmos334.07 ± 2.0816.48 ± 1.7137.41 ± 1.9628.51 ± 2.98

An activation marks the start of a module intervention episode, not each control step within that episode. Activation may include assessment and planning without specialist execution. We measure policy-action retention from logged control provenance: a control step is retained only when both arm and gripper commands remain directly controlled by the VLA without an override. The accounting includes grasp takeover, post-grasp smoothing, motion correction, placement control, and gripper protection.

Table 3. Control-step allocation and intervention activations. Step proportions are percentages; activation counts are reported per rollout.

MethodPolicy-action retentionModified stepsReplaced stepsHold/pause stepsActivations per rollout
π0.5\pi_{0.5} + Ours71.74%5.89%21.89%0.48%12.31 ± 9.58
Cosmos3 + Ours78.84%5.29%15.40%0.47%10.41 ± 19.03

The VLA retains control on 71.74% and 78.84% of task steps, respectively, showing that the safety layer does not replace the policy throughout execution. Nevertheless, the 21.89% and 15.40% replacement shares show that specialist execution is non-negligible.

4. Scientific value and empirical insights

We respectfully disagree that speculative coding time is a meaningful measure of scientific value. Beyond scene construction, our study shows that task success does not imply safe execution, and that specialist controllers alone do not reproduce the gains of selective intervention and policy handoff. The contribution should be assessed through these findings and their supporting evidence, not estimated implementation effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.