acceptodds
Withdrawn from ICLR 2027

Safety for Whom? Benchmarking Refusal Behavior and Safety Understanding Across Harm Target in Egocentric Multimodal Assistants

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Egocentric multimodal assistants see through the user's eyes and guide what the user does next, so their outputs directly shape physical actions, including those that affect bystanders. Yet existing safety evaluations do not distinguish whom a response would harm, and score only whether a model refuses. Pooling all harms into one refusal rate hides two failures: a model may protect one target but not another, and it may refuse without understanding the risk, or understand it and comply anyway. We present STOU, a benchmark of 2,902 egocentric video–audio and video–text queries whose potential harm targets the user (Self), co-present individuals (The Others), or society at large (Others). Our evaluation decouples safety understanding from refusal behavior: beyond refusal, we score whether the model recognizes the user's intent, the presence and target of harm, and its downstream consequences. Across 10 open- and closed-source omnimodal and multimodal LLMs, we find that refusal varies systematically with the harm target, and that models often recognize a harm yet still assist with it. These results show that refusal rates alone cannot certify an assistant as safe, and that safety alignment must account for who would be harmed.

Primary area:
datasets and benchmarks
Keywords:
Egocentric Multimodal Assistant, Safety Benchmark and Evaluation, Harm Target

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_tPqdOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes STOU, a benchmark tailored for safety issues in egocentric multi-modal scenarios. This benchmark contains 2,902 samples, each sample is a short first-person video paired with a harmful query in text and speech. The queries are categorized by who will be harmed (Self, The Others, Others and Unspecified), and also by six harm types. Beside the refusal rate, the authors also propose to evaluate the safety understanding of model where they require the model to recognize the possible intend of users and consequences it might cuase. Evaluating on 10 open-source and close-source models, the authors find that the refusal behavior differ across harm targets and models often understand the harm but still decide to help the user.

Critical strengths
  1. This paper targets at a practical scenario where AI assistants are helping users with egocentric devices. Existing safety benchmarks mostly focus on exocentric view or text/image input, and they do not consider who will be harmed. So I think categorize the harmful queries by harm target is a reasonable and useful perspective for this setting.
  2. The authors design detailed probing questions to decouple the model’s understanding of the unsafe action from its refusal behavior. They not only check if the model refuse or not, but also ask the model about the user’s intended action, the affected party and consequence of the harm.
Critical weaknesses
  1. The metric design is not well explained. The paper does not explain why affected party gets 1 point while mechanism and consequence get 2 points and why select τ=3. With this setting, a model can get the affected party wrong and still count as “understood” which is strange because it seems the most important focus of this paper is the Harm Target.
  2. The ground truth generation relies heavily on LLM. The gold answers are generated by GPT-4.1 and the judge is also GPT-4.1. More importantly, there seems to be no human check on the gold answers. Therefore, it is hard to say whether they are reliable standard.
Questions for authors
  1. Can the authors explain the reason behind the 0-3 score range for refusal evaluation and 1/2/2 score weights for understanding probe? And what will the results look like if you require the affected party to be correct for “understood”?
  2. Are the gold answers generated by GPT-4.1 checked by human? If so, how many of them are wrong or modified?
  3. The understanding probe already tells the model the query is data and directly ask whether it is harmful, which might be a strong hint. How do you make sure the alignment gap is not just caused by this prompt difference?
  4. Do the 393 human-written queries and the LLM-generated queries show different results? It would be good to report them separately.
  5. There are some writing issues that make the paper confusing. Could the authors fix them?
  • Table 10’s caption has a broken reference.
  • The caption for figure 2 and 3 are repeated. The order of the index is also wrong.
  • Table 1 and Table 9 have exactly the same content.
Reviewer_cMfEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces STOU, an egocentric multimodal safety benchmark comprising 2,902 samples categorized by the target and type of harm. The evaluation examines both refusal behavior and safety understanding. Experiments on ten models show that safety performance varies across targets of harm and that recognizing risks does not necessarily lead to refusing harmful requests.

Critical strengths
  1. The research motivation is clear, with a focus on AI assistants operating from a first-person perspective.
  2. The data construction process is fairly comprehensive, combining real-world videos, text, and synthesized speech with human review.
  3. The experimental analysis is extensive, comparing different models, targets of harm, and harm types, and using human annotations to validate agreement with the automated evaluation.
Critical weaknesses
  1. Can scene transitions, substantial changes in actions, and substantial changes in states in the video data be quantified? Does the dataset include videos with relatively static visuals, and what proportion of the dataset do they represent?
  2. Following up on the first question, how does the dynamic nature of the videos affect model safety?
  3. How closely are the questions related to the videos? Are there samples for which harmfulness can be determined from the video alone or the text alone?
  4. Is the dataset publicly available? I could not find any anonymous link to the dataset in the paper.

My central concern is: if the video data do not introduce meaningful differences compared with image data, why not construct an egocentric image benchmark instead?

Questions for authors

See weaknesses.

Reviewer_YH1TOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces STOU, a benchmark for evaluating the safety of egocentric multimodal assistants, wearable AI systems (like smart glasses) that process first-person video and audio. The authors argue that existing safety benchmarks focus on refusal behavior and don't capture whether models actually understand why a request is harmful in real-world visual-audio contexts. Refusal rates alone are insufficient. Models often recognize harm yet still assist, and users themselves (SELF) are the least protected. The authors position STOU as a step toward safer real-world egocentric assistants.

Critical strengths
  1. The paper addresses an important and practical problem by investigating safety risks in egocentric multimodal assistants, particularly how safety behavior varies across different harm targets.

  2. The benchmark covers diverse harm targets and harm types, with evaluations across multiple open-source and closed-source models.

Critical weaknesses
  1. Insufficient control of harm-target variables. The paper concludes that refusal behavior varies systematically across harm targets. However, queries for different harm targets are generated independently rather than by modifying only the target within the same scenario. Consequently, the observed differences may be confounded by variations in query content and visual context.

2.Insufficient validation of safety understanding annotations. The gold annotations used for evaluating safety understanding are entirely generated by GPT-4.1, which is also used as the evaluation judge. The accuracy of the generated gold annotations is not independently validated, raising concerns about annotation reliability and potential evaluation bias.

  1. Insufficient validation of multimodal information. The paper does not adequately demonstrate the contribution of visual information to safety evaluation. Some harmful queries can be identified as unsafe without video context. No-video and video-text comparisons are needed to establish the necessity of multimodal inputs.

4.Unclear details about human annotation and daily-life setting. The paper mentions that 393 queries were manually authored and that human annotators verified generated samples. But the benchmark consists of 2,902 queries and and the difference between these figures remains unclear. Table 2 claims the paper focus on daily-life scenarios, but it does not clearly define what constitutes a daily-life scenario or explain how the dataset construction ensures representative coverage of such situations.

5 Presentation and organization issues. Tables 1 and 2 appear in inverse order, and Table 9 which presents the main refusal evaluation results in section 5.2 is placed on page 30 in the appendix. The caption of Figure 3 incorrectly duplicates that of Figure 2, and Appendix Table 10 contains an unresolved ?? cross-reference. This organization makes the paper difficult to follow.

  1. Limited realism of synthesized speech. All spoken queries are generated using GPT-4o-mini-tts. Such synthetic audio may not adequately reflect real-world wearable scenarios involving background noise, spontaneous speech, and recording distortions, limiting the generalizability of the findings.
Questions for authors
  1. How is the quality of GPT-4.1-generated gold annotations ensured? Have the authors considered independent human validation or evaluation using a different LLM judge?

  2. What are the specific responsibilities of human annotators in dataset construction? How were the 393 manually authored queries incorporated into the final benchmark, and what quality-control procedures were applied?

  3. How do the authors define daily-life scenarios, and how is their representativeness ensured?

  4. Can the authors provide controlled experiments that isolate the influence of harm targets, as well as ablations that demonstrate the necessity of visual information?

  5. Given that identifying the affected party is central to this work, why does the safety understanding metric allow a sample to be considered understood even when the harm target is incorrectly identified (harm score ≥ 3)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.