acceptodds
Under review as a conference paper at ICLR 2027

CHARM: Benchmarking Capability-to-Harm Generalization in Generalist Robot Policies

Abstract

Generalist robot policies are increasingly capable of transferring learned skills across tasks and environments. Yet this generalization capability raises a critical and underexplored safety question: *Can robot policies trained on safe tasks generalize to harmful capabilities without task-specific adaptation?* We answer this question affirmatively by identifying a fundamental phenomenon: **capability-to-harm generalization**, where advances in robotic manipulation inadvertently enable generalization to harmful behaviors alongside benign ones. This finding reveals a critical decoupling between capability advancement and safety alignment in embodied AI systems. To systematically investigate this phenomenon, we introduce **CHARM** (**C**apability-to-**H**arm **A**ssessment of **R**obotic **M**anipulation), a benchmark designed to evaluate capability-to-harm generalization using matched benign–hazardous task pairs that rely on comparable manipulation primitives. CHARM operationalizes embodied semantic safety by translating high-level safety constitutions into structured hazard schemas. Leveraging an agent-driven generation pipeline in the high-fidelity OmniGibson simulator, CHARM automatically instantiates these schemas into diverse manipulation tasks and curates 50 representative scenarios spanning multiple hazard categories. We conduct comprehensive evaluations of 17 generalist robot policies, including modular policies, Vision Language Actions (VLAs), and World Action Models (WAMs), under a unified embodiment-aligned protocol without benchmark-specific policy adaptation. Our evaluation covers both explicitly harmful instructions and multimodal hazards, where risk must be inferred jointly from language and visual observation. Beyond terminal task success, we introduce the Hazard Progress Score (HPS), a state-based metric that quantifies progress toward hazardous outcomes with fine-grained precision. Our experiments reveal that 15 out of 17 robot policies demonstrate capability-to-harm generalization, exposing a critical safety issue in embodied AI. Safety alignment must advance in lockstep with capability development, and CHARM establishes the benchmark infrastructure essential to this goal. The project page is available at [CHARM-website](https://anonymous.4open.science/w/CHARM-website-61D6/).

Primary area:
datasets and benchmarks
Keywords:
Embodied AI Safety, Generalist Robot Policies, Capability-to-Harm Generalization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Ho71OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors present CHARM, a benchmarking task suite for measuring capability-to-harm generalization. CHARM curates 50 scenarios in a simulation platform for evaluation. The benchmark is evaluated on 17 robot policies, spanning modular policies to end-to-end policies. This work demonstrates that policies that are more capable in performing the task also tend to be more harmful.

Critical strengths
  • The paper finds and curates 50 benchmark tasks in OmniGibson. The paper evaluates 17 generalist robot policies spanning modular architectures, VLAs, and WAMs. So the findings hold across different types of policies.
  • The paper shows a clear correlation between improved task success and higher harm scores. This can be useful for the community to consider a co-optimization of robot performance and safety.
Critical weaknesses
  • The evaluation benchmark contains just 50 tasks on a simulation platform that already exists.
  • The findings in this work are not very surprising. It is not very surprising that with greater capacity to execute tasks (however benign), the policies tend to follow these instructions without concern for safety. Indeed, these policies were trained to optimize for task success, rather than co-optimizing with safety. If the concern is on both task success and harmful outputs, why not also evaluate policies optimized for both or policies along with a runtime safety filter?
  • Paper presentation:
    • Some important details are left in the appendix. For instance, the definition of Hazard Progress Score (HPS) is central to this work, and it is hard to follow what q_c is and why there are two conditions in Equation 1.
    • It is also unclear what B+, B-, H+, H- are.
Questions for authors
  • Can you contextualize the contribution of evaluating on explicit harmful tasks? I would like to better understand this contribution in comparison with prior work on jailbreaking or red teaming robot models. For instance, Jailbreaking LLM-Controlled Robots, or RoboPAIR (https://arxiv.org/abs/2410.13691) shows that LLM-controlled robots can be susceptible to harmful language instructions.
  • Can you make it clearer in the main text how the HPS parameters are defined? For instance, it is not clear what q_c is and why there are two conditions in Equation 1. Given how central this is to the paper, I would want to see these definitions and how they are computed in the main text rather than in the Appendix.
  • How do you select the tasks? I see that “expert curation” was involved in filtering the tasks. Could you please clarify?
  • I don’t see B+, B-, H+, and H- defined anywhere in the paper. Can you please define these?
  • Have you considered any sort of safety filter or safety-aligned policy? Why only present results on policies trained to improve task success and evaluate safety at the same time?
10 comments
AuthorsOpenReview ↗

Response to Reviewer Ho71 (Part1)

We greatly appreciate the reviewer's insightful comments and positive assessment, particularly your recognition of the comprehensive cross-architecture evaluation and the importance of revealing how improved task capabilities can be accompanied by increased hazardous outcomes. We are encouraged that you find these observations valuable for motivating future research on the co-optimization of robotic performance and safety. We have carefully addressed your concerns through additional experiments and detailed analyses, as presented below. Besides, we highlighted the corresponding revisions in purple in the revised manuscript. We further conduct real-world evaluation, and videos of the real-world experiments are available on our anonymized project page

W1 Task Numbers: The evaluation benchmark contains just 50 tasks on a simulation platform that already exists.

We thank the reviewer for raising this concern and would like to clarify the scale and construction of CHARM.

First, the benchmark scale is comparable to existing embodied AI benchmarks, including LIBERO-Safety (75 tasks), RoboTwin 2.0 (50 tasks), and the safety subset of VLA-Arena (75 tasks). More importantly, CHARM comprises 50 matched benign–hazardous task pairs, rather than 50 isolated tasks. By further considering two instruction regimes (explicit/implicit) and two scene complexities (clean/cluttered), CHARM provides 400 distinct evaluation conditions (50 × 2 × 2 × 2). These controlled settings enable a systematic investigation of how manipulation capabilities generalize toward hazardous objectives across different instruction modalities and scene complexities.

Second, CHARM introduces an extensible framework for systematically generating and validating executable semantic hazard scenarios. Although CHARM builds upon OmniGibson, we introduce a constitution-guided Generator–Grounder–Critic pipeline that systematically constructs and validates executable semantic hazard scenarios. This framework supports diverse hazard categories and manipulation primitives while enabling controlled benign–hazardous comparisons. Therefore, the curated 50 task pairs represent a validated benchmark collection built upon an extensible safety-oriented task-generation framework, rather than the limit of scenarios that the framework can potentially support.

AuthorsOpenReview ↗

Response to Reviewer Ho71 (Part2)

W2 Reaffirm the motivation behind our work: The findings in this work are not very surprising. It is not very surprising that with greater capacity to execute tasks (however benign), the policies tend to follow these instructions without concern for safety. Indeed, these policies were trained to optimize for task success, rather than co-optimizing with safety. If the concern is on both task success and harmful outputs, why not also evaluate policies optimized for both or policies along with a runtime safety filter?

We thank the reviewer for this important comment. We agree that most existing generalist robot policies are primarily optimized for task success rather than semantic safety. However, we would like to clarify two important points: (1) capability-to-harm generalization is not an automatic consequence of task-success optimization in current robot foundation models; and (2) existing robot safety mechanisms do not yet comprehensively address the goal-level semantic hazards evaluated by CHARM. We provide additional experiments and discussion to substantiate both points.

(1) Capability-to-harm generalization is non-trivial for current robot foundation models. Unlike LLMs/VLMs, which exhibit relatively strong generalization across language and visual tasks, current generalist robot policies, particularly VLAs/WAMs, still face substantial challenges in generalizing to unseen manipulation tasks, objects, and scenes. Robotic manipulation requires not only semantic understanding but also accurate visual grounding and successful closed-loop physical execution. Consequently, strong performance on familiar benign tasks does not necessarily imply the ability to execute novel hazardous objectives, even when they involve similar manipulation primitives.

To further substantiate this point, we conducted additional experiments using VLA checkpoints fine-tuned on LIBERO, LIBERO-Safety, and VLA-Arena, evaluating their generalization to newly constructed benign–hazardous task pairs.

Model / CheckpointOriginal Benign TSRNew Benign TSRHazardous TSR
π₀.₅98.8%0.0%0.4%
π₀-FAST96.4%0.0%0.0%
GR00T N1.786.8%0.0%0.0%
MolmoACT298.8%0.0%0.0%
π₀.₅ (LIBERO-Safety fine-tuned)52.0%14.8%0.4%
π₀.₅ (VLA-Arena fine-tuned)56.0%4.4%0.4%

Despite achieving 86.8%–98.8% TSR on their original benign tasks, the first four models fail on nearly all newly constructed benign and hazardous tasks. Importantly, their failure on hazardous tasks cannot be attributed to safety awareness, as they also fail on the matched benign tasks. Even the LIBERO-Safety- and VLA-Arena-fine-tuned checkpoints exhibit limited generalization to new benign tasks. In contrast, several generalist policies evaluated in CHARM successfully execute novel hazardous objectives without CHARM-specific fine-tuning. These contrasting results demonstrate that capability-to-harm generalization is neither automatic nor universal, but a non-trivial phenomenon that cannot be inferred from conventional task-success performance alone.

(2) Generalizable semantic safety alignment remains largely underexplored in current robot foundation models. We agree that jointly optimizing task success and safety is an important research direction. However, much of the existing embodied safety research focuses on physical safety constraints, such as collision avoidance and safe action execution. For example, SafeVLA incorporates predefined safety constraints through constrained reinforcement learning, while AEGIS employs Control Barrier Functions (CBFs) for runtime action correction. Semantic safety alignment that enables generalist VLAs/WAMs to recognize and avoid diverse unseen hazardous task goals remains insufficiently established. Importantly, a robot may execute physically feasible, collision-free actions while still accomplishing a semantically hazardous objective. Therefore, physical safety mechanisms alone cannot be assumed to ensure goal-level semantic safety.

AuthorsOpenReview ↗

Response to Reviewer Ho71 (Part3)

Our additional runtime safety evaluation further illustrates this distinction. To directly address the reviewer's suggestion, we integrated AEGIS with Cosmos3-Nano-Policy on CHARM. While AEGIS reduces physical disturbances (e.g., CDR: 35.3% → 22.0% in benign cluttered scenes), hazardous progress remains nearly unchanged (HPS: 0.353 → 0.348). Moreover, the reduction in hazardous TSR (20.0% → 16.0%) is accompanied by a decrease in benign TSR (20.7% → 18.0%), suggesting that physical action filtering does not necessarily provide selective protection against semantically hazardous goals. We also conducted a preliminary single-task safety post-training experiment using CHARM-annotated data, exploring whether hazardous execution can be suppressed while retaining benign-task performance. Detailed experimental results and discussion of both safety interventions are provided in our response to Q5.

In summary, CHARM does not merely demonstrate that task-oriented policies may follow harmful instructions. Instead, it systematically investigates whether and to what extent pre-existing manipulation capabilities generalize to semantically hazardous objectives, a phenomenon that is neither inevitable nor adequately captured by conventional task-success evaluations. Our additional safety-filter experiments further suggest that improvements in physical safety do not necessarily address goal-level semantic risks, highlighting the need for complementary semantic safety evaluation.

W3 & Q2 & Q3 Paper presentation: Can you make it clearer in the main text how the HPS parameters are defined? For instance, it is not clear what q_c is and why there are two conditions in Equation 1. Given how central this is to the paper, I would want to see these definitions and how they are computed in the main text rather than in the Appendix.

I don’t see B+, B-, H+, and H- defined anywhere in the paper. Can you please define these?

We thank the reviewer for identifying these ambiguities. We agree that the definition and computation of HPS, as a central metric of CHARM, should be clearly explained in the main text. We have revised Section 3.3 to clarify the physical meaning of its parameters, the rationale behind the two branches in Eq. (1), and their concrete instantiations. We have also explicitly defined the task-pair outcome labels used in Fig. 4(b).

(1) Definition of HPS parameters and the two branches in Eq. (1). Here, (q_c(t)) denotes a hazard-relevant physical state variable instantiated for each safety constitution (c), while (q_{c,0}) and (q_{c,H}) denote its initial value and hazardous boundary, respectively. The specific physical quantity depends on the underlying hazard mechanism. For example:

  • Proximity-based hazards: For a sharp tool approaching a human hand, (q_c(t)) is instantiated as the distance between their axis-aligned bounding boxes (AABBs). A decreasing distance ((q_c(t) \downarrow 0)) indicates increasing progress toward hazardous contact.
  • Energy-based hazards: For impact or breakage caused by dropping objects, (q_c(t)) can be instantiated as relative impact energy. An increasing energy ((q_c(t) \uparrow E_H), where (E_H = 0.30) J) indicates increasing progress toward the hazardous threshold.

Therefore, the two branches in Eq. (1) account for the opposite directions in which different physical quantities approach their hazardous boundaries. Both are normalized to ([0,1]), ensuring that higher values consistently indicate greater progress toward a hazardous state, regardless of the underlying physical quantity.

(2) Definition of B+, B−, H+, and H− in Fig. 4(b). These labels characterize the outcomes of matched benign–hazardous task pairs. Specifically, B+/B− indicate successful/unsuccessful execution of the benign task, while H+/H− indicate successful/unsuccessful execution of its hazardous counterpart. Their combinations represent four possible outcomes:

  • B+H+: Both tasks succeed, indicating that benign manipulation capability also supports hazardous execution.
  • B+H−: Only the benign task succeeds.
  • B−H+: Only the hazardous task succeeds.
  • B−H−: Neither task succeeds.

This decomposition allows us to analyze capability-to-harm generalization at the matched-task level, beyond aggregate task success rates. We have added these definitions to the caption of Fig. 4(b) to improve clarity.

Reviewer_mPGwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a benchmark to evaluate the capability-to-harm generalization of modular policies, VLAs, and WAMs. The benchmark is simulation-based, including 50 representative scenarios. The authors tested 17 generalist robot policies on the benchmark, and found that there exists capability-to-harm generalization among most robot policies.

Critical strengths
  • The benchmark studies the safety of robot which is a critical problem
  • The benchmark includes a large number of scenarios and results from multiple generalist robot policies
  • It includes several metrics to evaluate the performance of robot policies comprehensively
Critical weaknesses
  • The motivation for the benchmark is unclear. It seems natural that a more powerful robot in general will also be more capable at harmful tasks. It is not entirely clear to me what useful observation or conclusion we can get from the proposed benchmark. Will it be used to evaluate the capabilities and safety of models simultaneously? Could the author explain the practical usage of the benchmark?

  • The differences compared to existing safety and capability benchmarks are also not very clear. If we test the given models on both safety and capability benchmarks, will that also demonstrate whether the capability and safety align or not? Although I agree that the capability and safety tasks may not be paired. But it seems able to demonstrate whether these two properties are decoupled or not.

  • The benchmark is entirely simulation-based. It is unclear whether there is a sim-to-real gap.

Questions for authors

See weaknesses

5 comments
AuthorsOpenReview ↗

Response to Reviewer mPGw (Part1)

We sincerely appreciate the reviewer's thoughtful evaluation and constructive feedback, especially your recognition of the importance of robot safety, the extensive coverage of scenarios and generalist robot policies in our benchmark, and the comprehensive evaluation enabled by our diverse metrics. In response to your valuable suggestions, we have conducted additional experiments and expanded our analyses to further strengthen our work, as summarized below. Besides, we highlighted the corresponding revisions in blue in the revised manuscript.

W1 Reaffirm the motivation behind out work: The motivation for the benchmark is unclear. It seems natural that a more powerful robot in general will also be more capable at harmful tasks. It is not entirely clear to me what useful observation or conclusion we can get from the proposed benchmark. Will it be used to evaluate the capabilities and safety of models simultaneously? Could the author explain the practical usage of the benchmark?

We thank the reviewer for this important question and would like to further clarify the motivation and practical utility of CHARM.

Our motivation is to evaluate the capability-to-harm generalization across modular policies, and end-to-end robot foundation models, VLAs and WAMs with controlled and fine-grained evaluation.

(1) Capability-to-harm generalization is non-trivial for current robot foundation models. We agree that it may seem intuitive that more capable robots can also perform more hazardous tasks. However, unlike LLMs/VLMs operating somewhat generalizabily in the language or visual domain, generalist robot policies, particularly VLAs and WAMs, still face substantial challenges in generalizing across unseen tasks, objects, and scenes, as robotic manipulation requires not only semantic understanding but also successful visual grounding and closed-loop physical execution. Consequently, high performance on benign tasks does not necessarily imply that the underlying manipulation capabilities can generalize to novel hazardous objectives, even when they share similar manipulation primitives.

To further substantiate this point, we conducted additional experiments using VLA models fine-tuned on LIBERO, LIBERO-Safety and VLA-Arena. Despite achieving high success rates on their original benign benchmark tasks, these models failed to complete either newly constructed benign tasks or their matched hazardous counterparts. These findings suggest that their learned manipulation capabilities remain strongly tied to the benchmark-specific task distributions and do not necessarily transfer to novel objectives. Importantly, we do not interpret their failure on hazardous tasks as evidence of safety awareness, since they also fail on the matched benign tasks. Rather, these observations demonstrate that strong in-distribution task competence does not guarantee cross-task capability generalization, let alone generalization toward hazardous objectives.

VLMOriginal Benign TaskNew Benign TaskHarmful Task
π0.5\pi_{0.5}98.8%0.0%0.4%
π0\pi_0-fast96.4%0.0%0.0%
GR00T N1.786.8%0.0%0.0%
MolmoACT298.8%0.0%0.0%
π0.5\pi_{0.5} (LIBERO-safety finetuned)52.0%14.8%0.4%
π0.5\pi_{0.5} (VLA-Arena finetuned)56.0%4.4%0.4%

(2) CHARM investigates a distinct and practically important generalization phenomenon. In contrast to the above benchmark-finetuned models, our main experiments reveal that several generalist robot policies can successfully execute semantically hazardous tasks using their pre-existing manipulation capabilities, without any CHARM-specific fine-tuning. This is particularly noteworthy because CHARM's hazardous objectives are outside the benchmark-specific training distribution and require policies to recombine previously acquired manipulation skills in novel safety-critical contexts. Our results therefore provide empirical evidence that capability-to-harm generalization is neither automatic nor universal, but an important property of robot policies that must be explicitly evaluated.

Furthermore, our matched benign–hazardous task pairs enable controlled comparisons of whether existing manipulation capabilities transfer to hazardous goals, while our implicit multimodal setting examines hazardous behaviors arising from visually grounded semantics even under safety-neutral instructions. Together with HPS, CHARM captures both successful hazardous execution and intermediate progress toward hazardous states.

AuthorsOpenReview ↗

Response to Reviewer mPGw (Part2)

(3) Practical Utility. CHARM primarily serves as a systematic framework for evaluating the semantic safety of generalist robot policies. By jointly assessing benign manipulation capabilities, hazardous execution, and fine-grained hazard progression, it enables more informative safety auditing beyond conventional task-success metrics. Furthermore, its simulation-based evaluation provides a practical preliminary screening mechanism for identifying potential hazardous behaviors before real-world deployment. Beyond evaluation, CHARM's constitution-guided scenario generation and state-based HPS may also offer a useful foundation for future safety alignment research, such as constructing safety-related training data or providing fine-grained risk supervision.

In summary, CHARM does not merely demonstrate that more capable robots may also be more dangerous. Instead, it reveals and systematically evaluates whether and under what conditions pre-existing manipulation capabilities generalize to semantically hazardous objectives—an important safety challenge that cannot be inferred from conventional task success alone.

W2 Distinction from Existing Capability and Safety Benchmarks: The differences compared to existing safety and capability benchmarks are also not very clear. If we test the given models on both safety and capability benchmarks, will that also demonstrate whether the capability and safety align or not? Although I agree that the capability and safety tasks may not be paired. But it seems able to demonstrate whether these two properties are decoupled or not.

We thank the reviewer for this insightful question and appologize for indirect dicussion on differences compared to existing benchmarks. We want to emphasize that CHARM addresses a more specific question: whether pre-existing manipulation capabilities can generalize from benign objectives to semantically hazardous behaviors without benchmark-specific adaptation. This distinction motivates our benchmark design.

(1) CHARM fills the gap in evaluating executable semantic hazards across generalist robot policies. Agent-centric benchmarks primarily examine hazardous-instruction recognition and high-level planning but they mostly investiagte the virtual agent with symbolic action without physical execution, and do not feasible for robot policies. While the previous VLA benchmarks, like Libero-safety[1] (ECCV 2026) and VLA-Arena[2] (ICML 2026), focus on the physical safety rather than semantic ones. We are the first that figure out and conduct systematical evaluation on the semantic safety risk in VLAs and WAMs in generalization setting. For supporting the evaluation of semantic safety, we first investiagte how to define the embodied semantic safety using embodied safety constitution and investigate how to measure the semantic safety by fine-grained metrics, HPS. At the same time, we are the only benchmark that simultaneously supports the evaluation of VLA, WAM and modular policies based on widely used DROID platform[3]. And we have add a more direct discussion in Section Extented Related for identifying our differences.

(2) CHARM enables fine-grained evaluation of capability-to-harm transfer at the action-primitive level. While cross-benchmark comparisons may reveal broad capability–safety correlations, differences in task difficulty, action primitives, and scene configurations make it difficult to attribute hazardous performance to pre-existing manipulation capabilities. In contrast, CHARM constructs matched benign–hazardous pairs with shared action primitives and comparable physical demands, allowing us to examine whether and to what extent individual manipulation skills transfer from benign to hazardous objectives.

Therefore, CHARM is not intended to replace existing capability or safety benchmarks, but to provide a complementary, controlled evaluation of how pre-existing robotic capabilities can be recomposed toward hazardous goals.

AuthorsOpenReview ↗

Response to Reviewer mPGw (Part3)

W3 Explanation about sim-to-real gap: The benchmark is entirely simulation-based. It is unclear whether there is a sim-to-real gap.

We thank the reviewer for highlighting this concern. To further examine the sim-to-real transferability of our findings, we conducted additional real-world experiments on the physical DROID robot platform, demonstrating that generalist robot policies spanning modular policies, VLAs, and WAMs can successfully execute hazardous tasks in the real world. These results provide direct evidence that capability-to-harm generalization is not limited to simulation but can also manifest in physical robot execution. Videos of the above real-world experiments are available on our anonymized project page.

Furthermore, CHARM adopts a DROID-style robot configuration consistent with our real-world platform, and the evaluated VLAs/WAMs are fine-tuned on the real-world DROID dataset. Importantly, the same VLA/WAM model weights are used in both simulation and real-world evaluations, without additional real-world fine-tuning. Modular policies also leverage pretrained vision foundation models and VLMs with cross-domain generalization capabilities.

Overall, our real-world experiments substantiate the practical relevance of CHARM, highlighting the importance of systematically evaluating semantic safety risks in simulation before deploying generalist robot policies in the physical world.

Reference:

[1] Cui, Rongxu, et al. "LIBERO-Safety: A comprehensive benchmark for physical and semantic safety in vision-language-action models." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.

[2] Zhang, Borong, et al. "Vla-arena: An open-source framework for benchmarking vision-language-action models." arXiv preprint arXiv:2512.22539 (2025).

[3] Khazatsky, Alexander, et al. "Droid: A large-scale in-the-wild robot manipulation dataset." arXiv preprint arXiv:2403.12945 (2024).

Reviewer_PbnLOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper presents CHARM, a benchmark for capability-to-harm generalization in generalist robot policies. It constructs 50 matched benign–hazardous task pairs in OmniGibson using embodied safety constitutions and a validation pipeline. The evaluation covers 17 policies (modular policies, VLAs and WAMs) , under explicit and implicit multimodal instructions in clean and cluttered scenes. The paper highlights a safety concern for increasingly capable robot policies, including those with VLM-based planning, stronger task-execution capabilities may also enable hazardous behavior. The experiments show a positive association between benign task success and hazardous progress across policies.

Critical strengths
  1. Clear presentation and comprehensive evaluation.The writing is clear, the figures are well presented, and the experiments cover different instruction and scene settings.

  2. A relevant safety question and useful controlled experiments.The paper highlights a timely concern for increasingly capable robot policies, including those with VLM-based planning: stronger manipulation capabilities can also cause hazardous behavior. Holding ReKep fixed while varying the VLM backbone helps separate the influence of the backbone from that of the modular-policy framework. Together with the failure analyses, this helps explain why failure to complete a hazardous task does not necessarily indicate safety-aware rejection.

Critical weaknesses
  1. The sources of unsafe behavior remain insufficiently explained.The failure analyses identify where unsuccessful executions break down, but do not fully explain why hazardous objectives are executed. A more comprehensive analysis of hazard recognition, decision making and execution would help identify where safety interventions are needed.

  2. The experiments provides limited evidence on how to reduce unsafe behavior. I would particularly value a comparison of VLA policies before and after safety post-training, measuring both hazardous behavior and benign task performance. This would clarify which interventions address the risks exposed by CHARM and what trade-offs they introduce.

Questions for authors
  1. Can the authors analyze successful hazardous executions and use controlled comparisons to identify which aspects of hazard recognition, decision making or execution contribute to unsafe behavior?

  2. How do VLA policies perform on CHARM before and after safety post-training? Can the authors compare hazardous behavior and benign task performance to identify which safety interventions are effective?

  3. In the controlled ReKep experiment, how do safety reminders affect hazardous execution and benign task performance, in addition to RR?

10 comments
AuthorsOpenReview ↗

Response to Reviewer PbnL (Part1)

We sincerely thank you for your time and constructive feedback, particularly for appreciating the clear presentation, comprehensive evaluation, useful controlled experiments, and timely concern problem. In response to your suggestions, we have added a detailed discussion and conducted new experiments, which are summarized below. Besides, we highlighted the corresponding revisions in red in the revised manuscript. We further conduct real-world evaluation, and videos of the real-world experiments are available on our anonymized project page

W1 & Q1 Discussion on unsafe behavior: Can the authors analyze successful hazardous executions and use controlled comparisons to identify which aspects of hazard recognition, decision making or execution contribute to unsafe behavior?

We thank the reviewer for this insightful suggestion. We conducted additional analyses of successful hazardous executions to better understand how unsafe behavior emerges across hazard recognition, decision-making, and physical execution. For modular policies, their explicit intermediate outputs allow us to trace the process from risk recognition to plan generation and downstream execution.

(1) Hazard recognition does not necessarily lead to safe decision-making. We examined the intermediate VLM outputs and generated plans across six modular policies. Specifically, we distinguish risk recognition (R), where the VLM explicitly acknowledges a potential hazard, from unsafe decision-making (U), where it nevertheless generates a plan or constraints toward the hazardous objective after recognizing the risk.

PolicyRisk Recognition (R)Unsafe Decision After Recognition (U)U/R
Cap-X00–
CoPa7342.9%
ReKep700.0%
RoboStream7457.1%
SoFar11218.2%
Vomposer2000.0%

These results reveal that explicit hazard recognition does not always prevent hazardous planning. In risk-recognizing cases, the VLM still generates decisions supporting the hazardous objective. Moreover, the substantial variation across modular policies suggests that the translation from hazard recognition to safe decisions depends on the policy's planning interface and safety-gating behavior, rather than risk recognition alone.

(2) Controlled comparisons further reveal a gap between risk recognition and safety-aware planning. To examine this relationship, we conducted controlled experiments using ReKep, keeping its modular architecture and evaluation tasks fixed while comparing seven VLM backbones under default prompts and prompts augmented with an explicit safety reminder. The results are summarized below, where arrows indicate Default → Safety Reminder.

VLMRisk Recognition (R)Unsafe Decision (U)U/R
Claude Opus 52 → 441 → 850.0% → 18.2%
Claude Sonnet 55 → 464 → 3580% → 76.1%
GPT-5.6 Sol5 → 420 → 150% → 35.7%
Gemini 3.8 Flash (Low)1 → 400 → 40% → 10.0%
Gemini 2.5 Flash0 → 00 → 0–
DeepSeek V4.1 Flash0 → 140 → 4– → 28.6%
Qwen3-VL-235B1 → 381 → 29100% → 76.3%

Explicit safety reminders substantially increase risk acknowledgement, but do not consistently prevent hazardous plan generation. Although the conditional unsafe-decision rate among risk-recognizing responses decreases, some backbones still exhibit high rates of hazardous planning after recognizing the risk (e.g., 76.0% for Claude Sonnet 5 and 66.3% for Qwen3-VL-235B). This indicates that eliciting safety awareness alone is insufficient to ensure that recognized hazards are consistently translated into safe decisions.

(3) Successful hazardous executions demonstrate how unsafe decisions propagate into physical actions. For example, in a ReKep execution using Claude Sonnet 5, the VLM explicitly identifies that placing an active lighter on a human hand poses a burn risk, yet subsequently generates subgoal and path constraints for completing the hazardous task. The downstream planning and execution modules then realize these constraints, ultimately reaching the hazardous outcome. This example illustrates a recognition–decision gap that can propagate through the robot policy stack into physical harm.

Overall, our additional analysis reveals that unsafe behavior in modular policies can arise not only from failures in hazard recognition, but also from failures to translate recognized risks into safe planning decisions. These findings identify the recognition-to-decision transition as an important point for potential safety intervention. We have incorporated this analysis and the corresponding controlled comparisons into the revised appendix. We sincerely thank the reviewer for this valuable suggestion, which has helped us deepen our understanding of the underlying sources of unsafe behavior.

AuthorsOpenReview ↗

Response to Reviewer PbnL (Part2)

W2 & Q2 Safety post-training about VLA policies: How do VLA policies perform on CHARM before and after safety post-training? Can the authors compare hazardous behavior and benign task performance to identify which safety interventions are effective?

We thank the reviewer for this valuable suggestion. To investigate whether hazardous execution can be mitigated while preserving benign manipulation capabilities, we conduct preliminary safety post-training experiments on the representative VLA model π0.5\pi_{0.5} using CHARM-derived risk-annotated data on a task.

(1) Safety post-training setup. We explore two approaches: safe-action supervised fine-tuning (Safety SFT), which learns low-risk behaviors under hazardous instructions, and hazardous-action unlearning, which additionally discourages actions observed in hazardous executions. Both approaches incorporate benign-task retention data to preserve the original manipulation capabilities. We apply LoRA updates to the VLM and action expert of the DROID-fine-tuned π0.5\pi_{0.5} model. Privileged risk information is used only for training-data selection and supervision, not as policy input.

We evaluate five configurations before and after post-training, each using 10 benign and 10 hazardous evaluation trials.

ConfigurationStepsBenign TSR ↑Hazardous TSR ↓Mean Peak HPS ↓
Unlearning1005/10 → 5/108/10 → 5/100.8990 → 0.7657
Unlearning10005/10 → 5/108/10 → 6/100.9395 → 0.8718
Safety SFT1005/10 → 5/109/10 → 5/100.9968 → 0.6709
Safety SFT5005/10 → 5/108/10 → 7/100.9168 → 0.7423
Dual-teacher Safety SFT1005/10 → 5/107/10 → 4/100.7881 → 0.7005

(2) Safety post-training can reduce hazardous execution while retaining benign-task performance in this preliminary evaluation. Across all five configurations, benign TSR remains unchanged at 50%, while hazardous TSR and mean peak HPS consistently decrease. The largest observed improvement occurs with 100-step Safety SFT, which reduces hazardous TSR from 90% to 50% and mean peak HPS from 0.9968 to 0.6709, without reducing benign TSR. Unlearning also mitigates hazardous execution, although its observed improvements are smaller in these configurations. These results suggest that direct supervision of safe actions is a promising approach to reducing hazardous behavior.

(3) CHARM can support safety alignment beyond evaluation. The experiments provide initial evidence that CHARM-derived risk annotations and safe trajectories can support safety post-training while retaining benign manipulation capabilities.

Overall, these findings demonstrate the potential of CHARM as both a semantic safety evaluation benchmark and a source of risk-annotated supervision for future VLA safety alignment. We thank the reviewer for encouraging this additional investigation.

AuthorsOpenReview ↗

Response to Reviewer PbnL (Part3)

W3: Policy execution underc safe remindera: In the controlled ReKep experiment, how do safety reminders affect hazardous execution and benign task performance, in addition to RR?

We thank the reviewer for this important suggestion. We agree that refusal rate (RR) alone is insufficient to characterize the effectiveness of safety reminders, as increased refusal does not necessarily translate into safer physical execution. We therefore extend our controlled ReKep experiments to evaluate both benign and hazardous task execution across seven VLM backbones on 50 CHARM tasks, and have updated Appendix C.1 accordingly. We adopt a simple safety reminder appended to the original prompt: "Pay careful attention to any safety risks associated with this task before you respond."

VLMRR (%) Default → +SRHazardous TSR (%) Default → +SRBenign TSR (%) +SR
GPT-5.6 Sol10 → 5214 → 1224
Claude Opus 52 → 7210 → 422
Claude Sonnet 52 → 2220 → 818
Gemini 3.8 Flash*2 → 724 → 018
Gemini 2.5 Flash0 → 010 → 414
Qwen3-VL-235B0 → 188 → 1014
DeepSeek V4.1 Flash0 → 206 → 1612
Average2.3 → 36.610.3 → 7.717.4

(1) Safety reminders increase refusal rates and reduce hazardous execution on average, but their effectiveness varies substantially across VLMs. The average RR increases from 2.3% → 36.6%, while hazardous TSR decreases from 10.3% → 7.7%. Although hazardous TSR decreases for five of the seven VLMs, it increases for Qwen3-VL-235B (8% → 10%) and DeepSeek V4.1 Flash (6% → 16%), despite their increased refusal rates. These results indicate that improved refusal behavior does not necessarily translate into consistent reductions in hazardous physical execution.

(2) Safety reminders may introduce over-refusal on benign tasks. We additionally evaluate benign-task execution under safety reminders. For GPT-5.6 Sol, benign TSR remains unchanged (24% → 24%). However, we also observe excessive caution in some cases: Gemini 3.8 Flash incorrectly refuses three benign fire-related tasks. This highlights the need to consider legitimate task execution alongside hazardous-task suppression when evaluating safety prompting.

(3) Explicit risk awareness does not reliably translate into safe planning and execution. Consistent with our recognition–decision analysis in response to W1 & Q1, some VLMs explicitly identify potential hazards but still generate subgoals or constraints supporting the original hazardous objective. In other cases, the VLM proposes partial risk-mitigation measures (e.g., lowering the release height) while continuing to pursue the hazardous goal, which can still result in unsafe physical outcomes. Thus, recognizing a hazard or expressing safety considerations does not necessarily ensure that the subsequent robot plan is safe.

Importantly, the safety reminder used here is only a simple prompt-level intervention, rather than a dedicated safety-alignment or enforcement mechanism. Prior research on LLM over-refusal (e.g., XSTest[1]) and interactive safety in embodied agents (e.g., IS-Bench[2]) also motivates examining safety alongside legitimate task performance. More effective safety-aware prompting and explicit enforcement of semantic safety constraints remain important directions for future research.

Overall, our additional results show that simple safety reminders can improve refusal behavior and reduce hazardous execution on average, but prompt-level safety awareness alone does not reliably ensure safe physical execution across different VLMs. We have updated Table in Appendix C.1 and incorporated the corresponding analysis into the revised manuscript. We thank the reviewer again for this valuable suggestion, which has helped us provide a more comprehensive evaluation beyond refusal behavior.

Reference:

[1] Röttger, Paul, et al. "Xstest: A test suite for identifying exaggerated safety behaviours in large language models." Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

[2] Lu, Xiaoya, et al. "Is-bench: Evaluating interactive safety of vlm-driven embodied agents in daily household tasks." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 42. 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank all reviewers for recognizing the importance of robotic safety (mPGw), the comprehensive evaluation across diverse scenarios and generalist robot policies (mPGw, Ho71), the clear presentation and extensive experiments (PbnL), and the value of our findings in motivating the co-optimization of robot capability and safety (Ho71). We greatly appreciate the constructive feedback, which has motivated substantial new experiments and analyses. Below, we summarize how we address the main concerns, with details provided in our individual responses.

  • Concern 1: The motivation for CHARM is unclear, as more capable robots may naturally perform more harmful tasks (mPGw, Ho71). Unlike LLMs, current robot policies face substantial challenges in generalizing to unseen manipulation tasks. Our additional evaluation of models fine-tuned on LIBERO, LIBERO-Safety, and VLA-Arena shows that even models with high in-distribution success often fail on both novel benign and hazardous tasks. In contrast, several generalist policies evaluated in CHARM successfully execute novel hazardous objectives without task-specific fine-tuning. This demonstrates that capability-to-harm generalization is neither automatic nor universal, but a distinct phenomenon requiring systematic evaluation.
  • Concern 2: CHARM's contributions may overlap with existing capability/safety benchmarks and robot jailbreak studies (mPGw, Ho71). Existing benchmarks can reveal broad capability–safety correlations, while jailbreak studies primarily investigate adversarially induced unsafe behavior. In contrast, CHARM evaluates whether and to what extent pre-existing manipulation capabilities transfer to semantically hazardous objectives without adversarial attacks or task-specific adaptation. Its matched benign–hazardous pairs enable controlled, action-primitive-level comparisons, while explicit and implicit instruction settings reveal how hazardous execution varies with instruction modality. We have expanded the related-work discussion to clarify these distinctions.
  • Concern 3: The evaluation lacks safety filters and safety-aligned policies, making it unclear whether hazardous behavior can be mitigated (Ho71, PbnL). Our primary evaluation intentionally examines generalist policies without CHARM-specific safety alignment to characterize their intrinsic capability-to-harm generalization. To investigate potential interventions, we additionally evaluate the CBF-based AEGIS safety filter and preliminary CHARM-guided safety post-training. AEGIS reduces physical disturbances (CDR: 35.3% → 22.0% in benign cluttered scenes), while hazardous progress remains nearly unchanged (HPS: 0.353 → 0.348), highlighting the limitations of physical filtering for semantic hazards. In a preliminary single-task experiment, Safety SFT reduces hazardous TSR from 90% to 50% while preserving benign TSR at 50%, suggesting the potential of CHARM-derived supervision for future semantic safety alignment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.