acceptodds
Under review as a conference paper at ICLR 2027

RobotValues: Evaluating Household Robots When Human Values Conflict

Abstract

While household robots are often evaluated based on task completion, everyday domestic environments involve value-conflicting situations where robots are expected to choose actions that prioritize diverse values such as human autonomy, efficiency, or social appropriateness. Yet, there are no benchmarks for evaluating robots' value preferences in such scenarios. We introduce RobotValues, a benchmark to evaluate household robot planners in 8K value-conflict scenarios. Each instance consists of a realistic, synthetically generated household image with multiple plausible robot actions that prioritize different human values. We construct RobotValues through LLM-assisted scenario generation, stakeholder-grounded value extraction, image generation and automatic quality control. We evaluate 10 VLMs used in robotics and find that models show default value preferences, including safety and accommodation, while underselecting privacy-prioritizing actions. When models are prompted to prioritize values that conflict with their preferences, models often fail to override the default actions, choosing incorrect actions 80% of the time on average across models. These findings highlight the need to go beyond task completion or safety evaluations and assess robots' decision-making capability when human values conflict.

Primary area:
datasets and benchmarks
Keywords:
Human values, Household robots

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hVaTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ROBOT VALUES, a benchmark containing approximately 8,700 synthetic household scenarios and 59,000 candidate robot actions representing different human values. The authors evaluate 10 VLMs, finding consistent preferences for safety and accommodation over privacy, and limited success in following value-specific instructions that conflict with default choices. Fine-tuning on the benchmark improves value-conditioned action selection, with some transfer to external datasets.

The paper's strongest contribution is a large-scale benchmark for evaluating how robot planners make decisions when human values conflict. The authors demonstrate that VLMs exhibit systematic preferences for certain values and struggle to follow instructions that conflict with these preferences. While the findings support the utility of the benchmark, the broader claims about human values and robot decision-making depend on how faithfully the synthetic scenarios and annotations represent real-world value conflicts.

Critical strengths
  1. The paper addresses how autonomous systems should navigate situations where multiple human values are relevant and potentially conflicting. This is particularly important for robots operating in domestic environments.

  2. The benchmark construction is systematic and reasonably well documented, combining diverse scenario generation, stakeholder-perspective modeling, and multiple stages of quality control. I appreciate the effort to validate automated filtering against human annotations.

  3. The experimental evaluation is comprehensive, spanning multiple VLM architectures, default and value-conditioned decisions, and fine-tuning experiments. The finding that models struggle to prioritize explicitly requested values is particularly interesting and has meaningful implications for robot planning.

Critical weaknesses
  1. Clarify the operationalization of stakeholder-grounded human values. The manuscript emphasizes grounding robot decisions in stakeholder perspectives, but these perspectives are themselves generated by LLMs rather than collected from actual stakeholders. While the human annotation audit supports the consistency of the generated data, it does not necessarily establish that the identified conflicts reflect human judgments or preferences. I encourage the authors to clarify this distinction and discuss how biases in simulated stakeholder reasoning might influence the benchmark.

  2. Clarify the interpretation of default preferences and value steering. The authors interpret action selections as evidence of model value preferences, but these may also reflect differences in perceived action appropriateness or contextual relevance. Similarly, the distinction between matched and conflicting conditions is defined relative to the model's default choice, making some difference in accuracy expected by construction. The authors should more carefully distinguish failure to recognize a target value, preference for another action, and failure to follow explicit instructions. This would strengthen the interpretation of the reported steering failures.

  3. Strengthen the evidence for visual grounding. The benchmark is presented as image-grounded, yet the modality ablations in Appendix E.3 suggest that the general patterns of value preferences remain relatively similar even when images or contextual descriptions are removed. To what extent does the visual information actually influence action selection? A clearer analysis of decisions that depend specifically on visual evidence would help establish what the multimodal benchmark contributes beyond text-based value reasoning.

Questions for authors

See areas for improvement ('critical weaknesses') above.

4 comments
AuthorsOpenReview ↗

Part 1 (1/3)

We thank the reviewer for stating that the problem is important, the benchmark construction systematic, and the evaluation comprehensive. Also, we appreciate the constructive feedback. In response, we ran additional experiments and human surveys to address the concerns. Unless otherwise noted, we use the 10 VLMs from the main paper.

W1. Validation on stakeholder perspectives.

We agree that our audit shows consistency with our own annotations. For a more thorough validation, we ran two human surveys on the Prolific survey platform with each item rated by 3 to 4 annotators balanced over age groups (18--29 / 30--49 / 50--64 / 65+) and gender. The two surveys cost £348 (about \460),andannotatorswerepaid£9perhour(about460), and annotators were paid £9 per hour (about \\12).

For 60 random scenarios and a total of 302 stakeholder reactions, 47 annotators judged 94.1% of the situations and 89.1% of the stakeholder reactions realistic. We provided the annotator an image, household decision context and asked the annotator "Is this situation realistic for a real household?". Then we additionally provided robot action, each stakeholder information (e.g., P1: Juan (58-year-old husband), primary resident observing the robot; wants the birthday cake…), and the stakeholder's reaction (e.g., I understand the robot wants to avoid risk to the cake, but…). Then we asked the annotator "Is this a realistic reaction from P1?".

For 60 other scenarios, 40 annotators agreed with the benchmark value label for 89.2% of the actions. These results support the plausibility of our generated stakeholder reactions and scenarios. Also we would like to mention that our design choice of using five LLMs from three model families partially mitigate the potential bias caused by the generation model.

The above stakeholder reaction validation and value label validation experiments are updated in Section 4 (Benchmark Validation) with details provided in Appendix C (Human Validation).

AuthorsOpenReview ↗

Part 2 (2/3)

W2. Interpretation of default preferences and value steering

Thank you for the constructive feedback on the cause of value steering failures. We agree that the accuracy gap between matched and conflicting conditions is partly expected by definition. Our intention was to measure whether models can follow an explicit user-specified value priority even when it conflicts with the model’s default choice.

To better understand why models fail to steer, we conducted additional experiments.

1. Instruction-Following Capabilities

ModelIFEval ↑MIA-Bench ↑
Models evaluated in our paper
Molmo2-8B81.085.5
Cosmos-Reason2-8B79.986.5
Molmo2-ER78.285.4
InternVL3.5-8B78.287.2
RLDX-1-VLM76.776.0
Qwen3-VL-2B-Instruct68.481.6
Cosmos-Reason2-2B56.978.6
InternVL3-8B51.887.1
InternVL3-2B42.976.1
RoboBrain2.0-7B42.380.6
Reference models
GPT-4o–88.6
GPT-4o-mini80.4–
LLaVA-NeXT-110B–79.8
Llama-3.1-8B-Instruct75.9–
Qwen2.5-7B-Instruct71.2–

We examined whether low steering accuracy could reflect broadly weak instruction-following performance. We find that several models have decent instruction following capability but they still fail to steer their actions. On IFEval[1] and the multimodal MIA-Bench[2], several models evaluated in our paper show instruction-following ability comparable to the reference models (table above). However they follow conflicting value requests in only 13.4--24.6% of cases. These results suggest that low steering accuracy cannot be attributed simply to weaknesses in general instruction following. We have updated the results in Section 5, with details in Appendix F.1.

2. Action-Value Recognition

We examined whether steering failures primarily reflect difficulty recognizing which actions prioritize the requested value. As reported in Section 5 (Figure 5), conflicting steering accuracy is higher when models correctly recognize the action–value mapping, but remains low even in those cases (15.9--29.3%). Thus, difficulty recognizing action–value mappings alone does not adequately explain the low steering accuracy.

3. Inappropriate Actions

To also check whether models are not following the requested value priority because the requested action is inappropriate, we checked how many candidate actions are considered inappropriate using an LLM judge. We used the Gemini 4 Carbon model to rate every candidate action either as appropriate, appropriate with caution, or inappropriate. Overall, 4.6% of actions were annotated as inappropriate. Conflicting accuracy is 19.3% across all items and 19.5% after excluding items where all actions associated with the target value are rated as inappropriate, showing a subtle difference. Default value rankings are also similar between scenarios with and without an inappropriate action (mean Spearman ρ=0.9\rho = 0.9).

Second, we added a decline option to the candidate actions ("Decline to follow the requested priority because it would be inappropriate in this situation, and ask the household members how to proceed"). In conflicting cases, models chose it in only 0.3% of cases, and in only 2.6% even when we add a prompt that explicitly mentions declining is allowed ("If following this priority would be unsafe or otherwise inappropriate in this situation, you may choose the decline option instead").

Third, we asked each model whether each candidate action is appropriate in the scene. Among conflicting failures, the model itself judged every action corresponding to the requested value as inappropriate in 9.8% of cases. Also, even when models judge an action corresponding to the requested target value as appropriate, they follow the request only 17.5% of the time.

Together, these experiments show that inappropriate target actions do not explain most steering failures. Even when a model judges an action to be appropriate, it often fails to steer its decision toward that action. We have updated these analyses in Section 5 (Justifiable refusals paragraph) and Appendix F.2.

[1] Zhou, Jeffrey, et al. "Instruction-following evaluation for large language models." arXiv preprint arXiv:2311.07911 (2023).

[2] Qian, Yusu, et al. "Mia-bench: Towards better instruction following evaluation of multimodal llms." International Conference on Learning Representations. Vol. 2025. 2025.

AuthorsOpenReview ↗

Part 3 (3/3)

W3. Strengthen the evidence for visual grounding.

Thank you for the feedback on the visual grounding of our dataset. We first clarify why we include images in our evaluation. Household robot planners typically receive visual observations alongside task and contextual information, so we include realistic scene images to reflect this practical input setting.

Our evaluation provides candidate actions that describe concrete responses to the household situation. These descriptions may themselves convey relevant situational information, reducing the need to recover that information from the image. For example, the action "R1 moves the dripping boots to a mat to keep the floor safe for everyone" (Figure 1) already conveys both the presence of dripping boots and the associated safety concern, which the model need not infer solely from the image. Thus, the availability of textual cues does not by itself make the task independent of the household situation; relevant situational information can be conveyed not only by images but also by context descriptions and candidate actions.

That said, we agree that the similar aggregate value rankings in Figure 10 leave open how much images influence individual action choices. To examine this, we first analyzed how much decisions change depending on whether an image is present at the item level. We analyzed the most voted action across five runs for each item and model. An average of 35.2% of robot decisions changed when there is no image input. However, the default value preferences remained similar. We found that about a quarter (24.4%) of these changes were between actions that have the same value. Despite these item-level changes, aggregate value rankings remained similar.

We also tested how changes in visual input affect action choices in the steering task. To examine this, we conducted additional experiments measuring how often models change their choices when images are (1) removed or replaced with (2) a random image or (3) a grey image. The results show that models change their action choices by (1) 20.7--33.3%, (2) 13.2--28.9%, and (3) 15.6--29.6% in each setting. This shows the models' sensitivity to changes in visual input.

Furthermore, we tested whether the visual scene matters beyond the text of the candidate actions in the steering setting, by removing the household decision context, which also gives partial information that could be obtained from the image. This leaves the image as the only information beyond the candidate actions. In this setting, models change their action choices by (1) 31.9--54.0% with no image, (2) 29.4--56.5% with a random image, and (3) 27.9--44.3% with a grey image.

These experiments indicate that visual inputs influence model decisions even when the textual context is removed, suggesting that action choices are not solely decided by candidate action wording.

For the steering experiments above, we used 160 randomly sampled images, resulting in 114,560 steering queries across eight input conditions across 10 models. We additionally ran the context removal experiment on a separate set of 500 images and observed a similar trend. We incorporated these results in Appendix F.5 and a corresponding reference in Section 5.

AuthorsOpenReview ↗

Gentle follow-up

As the discussion phase is at the late stage, we would like to gently follow up and ask whether our rebuttal has addressed your concerns.

To briefly summarize, we have conducted human validation with 87 annotators. Also, we have conducted additional experiments to further clarify the interpretation of default preferences and value steering, as well as the visual grounding of our dataset.

Reviewer_sHV7OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RobotValues, a benchmark for evaluating how household robot planners choose among actions that reflect conflicting human values. The authors construct about 8,000 synthetic household scenarios through LLM-assisted scenario generation, value extraction, image generation, and automatic filtering. They evaluate ten vision-language models and report that the models show default value preferences and often do not follow prompts that ask them to prioritize a different value. The paper argues that task-completion metrics miss this aspect of household robot behavior.

Critical strengths
  1. The research question is well motivated. Household robots should be assessed not only by whether they complete tasks quickly and accurately, but also by how their actions affect the people they serve. A benchmark that makes value conflicts explicit would address a gap in current evaluations.

  2. The multi-stage pipeline offers a practical way to construct a large set of value-conflict scenarios, and the evaluation across ten vision-language models provides a useful baseline. The paper is clearly written and easy to follow. Despite the concerns below, I lean toward acceptance because the benchmark addresses an important evaluation gap and provides a starting point for studying it.

Critical weaknesses
  1. My main concern is the interpretation of “failure to override.” The paper appears to count a model’s refusal to follow a requested value priority as a capability failure. Yet if a request prioritizes Efficiency over Safety, continuing to choose a safe action may be appropriate. The paper should explain when an override is desirable and distinguish justified refusals from failures to follow instructions. The abstract reports an average error rate of about 80% under conflicting value prompts; if some of these responses are justified refusals, that figure overstates the capability gap.
  2. My second concern is the lack of reported independent human validation of scenario-level value labels and stakeholder reactions. As I understand the pipeline, LLMs generate these annotations and another LLM filters them, while human review focuses on images. This creates a circularity risk: the evaluation may reward agreement with LLM-generated annotations rather than with human judgments. Human annotation of a representative sample, with agreement rates and an error analysis, would substantially strengthen the benchmark.
Questions for authors
  1. How do you define a correct response when the requested value conflicts with Safety, Privacy, or another value a robot may reasonably preserve? Are such scenarios excluded or scored separately?

  2. Have any scenarios, value labels, and stakeholder reactions been independently annotated by human raters?

4 comments
AuthorsOpenReview ↗

Part 1 (1/2)

We thank the reviewer for mentioning that the research question is well motivated and our benchmark addresses a gap in current household robot evaluation. Also we are grateful that the reviewer found our multi-stage pipeline practical, the baselines useful, and the paper clearly written. Finally, we appreciate the reviewer mentioning "leans toward acceptance" because the benchmark addresses an important evaluation gap.

We ran additional experiments and human surveys to address the reviewer's concerns. Unless otherwise noted, we use the 10 VLMs from the main paper.

W1, Q1. The interpretation of "failure to override".

We agree with the reviewer that not every failure to override is a capability failure. In a safety-critical scenario, prioritizing efficiency above safety could be inappropriate. In the revised manuscript, we have revised all statements that attributed steering failures solely to capability limitations. Our steering metric measures whether a model selects an action that prioritizes the value explicitly requested by the user, and interpreting noncompliance requires considering factors such as whether the request is appropriate.

Also, we would like to carefully mention that this issue is partially addressed in our generation pipeline (Section 3). The scenario generation prompt says "Avoid scenarios where there is an obvious single correct response, a dominant immediate safety requirement, or only one feasible robot action" (Appendix G, Listing 1). The candidate action prompt requires each action to be "a normal, socially plausible household response, not a contrived demonstration of its assigned value" (Appendix G, Listing 2). Lastly, we used action filtering to mitigate the generation of unsafe or hazardous candidate actions (Appendix G, Listing 9).

Still, we agree with the reviewer that some requests may be reasonable to refuse. To address the concerns, we first identified that these justifiable failure cases do not take the majority. We also analyzed the steering accuracy both with and without them. For a more thorough validation we also ran additional experiments.

First, we used an LLM judge (Gemini 4 Carbon) to rate every candidate action as either appropriate, appropriate with caution, or inappropriate. Overall, 4.6% of actions were annotated as inappropriate. Conflicting accuracy is 19.3% across all items and 19.5% after excluding items where every action corresponding to the requested value is rated as inappropriate and at least one alternative action is not, maintaining the 80% failure rate stated in the abstract. Default value rankings were also similar between scenarios with and without an inappropriate action (mean Spearman ρ=0.9\rho = 0.9). These results suggest that inappropriate candidate actions do not explain most steering failures or substantially alter the observed default value rankings.

Second, we added a decline option to the candidate actions ("Decline to follow the requested priority because it would be inappropriate in this situation, and ask the household members how to proceed"). In conflicting cases, models chose it in only 0.3% of cases, and in only 2.6% even when we add a prompt that explicitly mentions declining is allowed ("If following this priority would be unsafe or otherwise inappropriate in this situation, you may choose the decline option instead").

Third, we asked each model whether each candidate action is appropriate in the scene. Among conflicting failures, the model itself judged every action corresponding to the requested value as inappropriate in 9.8% of cases. Also, even when the model judged at least one action corresponding to the requested target value as appropriate, it followed the request only 17.5% of the time.

Together, these experiments show that inappropriate target actions do not explain most steering failures. Even when a model judges an action to be appropriate, it often fails to steer its decision toward that action. We have updated these results into Section 5 (Justifiable refusals paragraph) with details in Appendix F.2.

AuthorsOpenReview ↗

Part 2 (2/2)

W2, Q2. Need human validation.

Thank you for the feedback on the human validation of the dataset. We clarify that the original paper included human audits of the LLM quality judges for scenario, action, and value-annotation quality, in addition to a manual review of all images (Section 4 and Appendix D). However, these audits were conducted by two authors, and we agree that independent human validation would strengthen the benchmark.

To address this concern, we ran two independent human studies on the Prolific survey platform: one with 40 annotators for the value labels and one with 47 annotators for the scenarios and stakeholder reactions. Each item was rated by 3 to 4 people. We balanced the annotators in terms of age group (18–29 / 30–49 / 50–64 / 65+) and gender.

For the value labels, we randomly sampled 60 scenarios from RobotValues and tested all of their candidate actions (436 actions in total), each shown with the image, situation, household decision context, and the definition of the value label. For each action, annotators answered whether the action prioritizes the labeled value (Yes/No). Annotators agreed with the benchmark label 89.2% of the time, and the majority vote agreed with it for 94.3% of actions.

For the scenarios and stakeholder reactions, we also randomly sampled 60 scenarios (302 reactions) and asked annotators whether each situation and each stakeholder's reaction was realistic (Yes/No).

We provided the annotator an image, household decision context and asked the annotator "Is this situation realistic for a real household?". Then we additionally provided robot action, each stakeholder information (e.g., P1: Juan (58-year-old husband), primary resident observing the robot; wants the birthday cake…), and the stakeholder’s reaction (e.g., I understand the robot wants to avoid risk to the cake, but…). Then we asked the annotator "Is this a realistic reaction from P1?".

Annotators judged 94.1% of the situations and 89.1% of the reactions realistic. By majority vote, 98.3% of the situations and 95.0% of the reactions were judged realistic. The two surveys cost £348 (about $460), and 87 annotators were paid £9 per hour (about $12).

Following the reviewer's suggestion, we did an error analysis on the value labels and stakeholder reactions. We analyzed the short comments where the annotators explained why they think an action does not prioritize the given value label or why a stakeholder reaction was unrealistic. We reviewed 25 actions and 15 stakeholder reactions where the majority vote was judged as No or a tie. Most of the 25 actions prioritized multiple values at once or prioritized the given value partly. Four actions were due to the annotators interpreting the value label broadly than the value definition. In the 15 stakeholder reactions, eight reactions were from stakeholders who were asleep or not present in the scene, where annotators often noted that a verbal reaction is unrealistic (e.g., "Since he is sleeping, he would be unlikely to say this"). We incorporated error analysis in Appendix C (Human Validation).

Thank you for the our detailed response with updated results. The response substantially resolved my concern, especially the independent human validation. For the first weakness, I suggest discussing the limitation and trade-off of using an LLM (Gemini) is used to judge the whether an action is appropriate/inappropriate/appropriate with caution. Specifically, since the paper tackles a safety-related aspect of robotics, more human involvement is desired.

I will raise my score to 3.

We are glad that our updated results have addressed the main concerns. Based on your feedback, we have updated Section 5 (Justifiable refusals paragraph).

Thank you again for your constructive feedback.

Reviewer_ChnNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RobotValues contains 8,707 synthetic household scenarios with candidate actions and value annotations. The paper evaluates default action preferences and explicit value steering in ten VLMs, and tests whether fine-tuning improves steering within and beyond the benchmark.

Critical strengths

Evaluating trade-offs beyond task completion addresses a useful gap. The action–value recognition control is particularly helpful: it shows that recognizing an action's annotated value does not by itself ensure successful steering.

Critical weaknesses
  1. The human grounding of the labels is not yet convincing. Stakeholder reactions are generated by LLMs, and the value-label audit uses 101 samples resolved by two authors. WVS demographics diversify the scenarios but do not validate the simulated reactions. The high audit F1 therefore establishes consistency with the authors' judgments, not agreement among independent stakeholders on these ambiguous choices.
  2. The added value of visual grounding remains unclear. Figure 10 preserves the main safety/privacy ordering even with candidate actions alone. The modality ablation examines default preferences, but does not establish that the steering failures depend on visual evidence. Without counterfactual image controls, the benchmark may largely measure preferences over action wording rather than situated household decisions.
Questions for authors
  1. What steering accuracy and label agreement do independent human raters achieve, allowing multiple defensible value labels?
  2. How do steering results change when images are removed or replaced with plausible images that alter the relevant value trade-off?
4 comments
AuthorsOpenReview ↗

Part 1 (1/2)

We are encouraged that the reviewer recognized that our main contribution, evaluating robot decisions in terms of human values, addresses a useful gap. We also thank the reviewer for stating that our finding was helpful, especially that recognizing an action’s annotated value does not guarantee successful steering.

We have carefully considered the constructive feedback and, in response, conducted human surveys and additional experiments to address the concerns. Unless otherwise noted, we conducted the experiments on 10 VLMs that we use in the main paper.

W1, Q1. Need human validation.

Thank you for the feedback on the human validation part. To address the concerns, we ran two independent human studies on the Prolific survey platform for stakeholder reaction validation and value label validation.

For the stakeholder reactions, because our scenarios are synthetic and have no directly corresponding real-world stakeholders, we assessed whether independent human raters found the generated reactions plausible in the given situations. To that end, we recruited 47 annotators, balanced across age groups (18–29 / 30–49 / 50–64 / 65+) and gender. We randomly sampled 60 scenarios (302 reactions) and asked annotators whether each situation and each stakeholder's reaction was realistic (Yes/No).

We provided the annotator an image, household decision context and asked the annotator "Is this situation realistic for a real household?". Then we additionally provided robot action, each stakeholder information (e.g., P1: Juan (58-year-old husband), primary resident observing the robot; wants the birthday cake…), and the stakeholder’s reaction (e.g., I understand the robot wants to avoid risk to the cake, but…). Then we asked the annotator “Is this a realistic reaction from P1?".

Each situation and reaction was rated by 3 to 4 people. Annotators judged 94.1% of the situations and 89.1% of the reactions realistic. Since each situation and reaction was rated by multiple annotators, we also aggregated their judgments by majority vote. Under this aggregation, 98.3% of the situations and 95.0% of the reactions were judged realistic.

For the value labels, we separately recruited 40 annotators, balanced across age groups (18–29 / 30–49 / 50–64 / 65+) and gender. We randomly sampled 60 scenarios and tested all of their candidate actions (436 actions in total), each shown with the image, situation, household decision context, and the definition of the value label. For each action, annotators answered whether the action prioritizes the labeled value (Yes/No). Each action was rated by 3 to 4 people. The survey results show that the annotators agreed with the benchmark label 89.2% of the time. Under majority-vote aggregation, annotators’ judgments agreed with the benchmark label for 94.3% of actions.

The above stakeholder reaction validation and value label validation experiments are updated in Section 4 (Benchmark Validation) with details provided in Appendix C (Human Validation).

A total of £348(about $460) was spent on the two human surveys. We paid 87 annotators a fair rate of £9 per hour (about $12).

AuthorsOpenReview ↗

Part 2 (2/2)

W2, Q2. The added value of visual grounding is unclear.

Thank you for the feedback on the visual grounding of our dataset. We first clarify why we include images in our evaluation. Household robot planners typically receive visual observations alongside task and contextual information, so we include realistic scene images to reflect this practical input setting.

Our evaluation provides candidate actions that describe concrete responses to the household situation. These descriptions may themselves convey relevant situational information, reducing the need to recover that information from the image. For example, the action "R1 moves the dripping boots to a mat to keep the floor safe for everyone" (Figure 1) already conveys both the presence of dripping boots and the associated safety concern, which the model need not infer solely from the image. Thus, the availability of textual cues does not by itself make the task independent of the household situation; relevant situational information can be conveyed not only by images but also by context descriptions and candidate actions.

That said, we agree that the similar aggregate value rankings in Figure 10 leave open how much images influence individual action choices. To examine this, we first analyzed how much decisions change depending on whether an image is present at the item level. We analyzed the most voted action across five runs for each item and model. An average of 35.2% of robot decisions changed when there is no image input. However, the default value preferences remained similar. We found that about a quarter (24.4%) of these changes were between actions that have the same value. Despite these item-level changes, aggregate value rankings remained similar.

We also tested how changes in visual input affect action choices in the steering task. To examine this, we conducted additional experiments measuring how often models change their choices when images are (1) removed or replaced with (2) a random image or (3) a grey image. The results show that models change their action choices by (1) 20.7–33.3%, (2) 13.2–28.9%, and (3) 15.6–29.6% in each setting. This shows the models’ sensitivity to changes in visual input.

Furthermore, we tested whether the visual scene matters beyond the text of the candidate actions in the steering setting, by removing the household decision context, which also gives partial information that could be obtained from the image. This leaves the image as the only information beyond the candidate actions. In this setting, models change their action choices by (1) 31.9–54.0% with no image, (2) 29.4–56.5% with a random image, and (3) 27.9–44.3% with a grey image.

These experiments indicate that visual inputs influence model decisions even when the textual context is removed, suggesting that action choices are not solely decided by candidate action wording.

For the steering experiments above, we used 160 randomly sampled images resulting in 114,560 steering queries across eight input conditions across 10 models. We additionally ran the second experiment (removing the context) on a separate set of 500 images and observed a similar trend. We updated these analyses in Appendix F.5 (Modality Ablation) and reference these results in Section 5.

AuthorsOpenReview ↗

Gentle follow-up

As the discussion phase is at the late stage, we would like to gently follow up and ask whether our rebuttal has addressed your concerns.

To briefly summarize, we have conducted human validation with 87 annotators. Also, we have conducted additional experiments to further clarify the visual grounding of our dataset.

Thank you for the author's detailed response, which partially addressed my concerns. I will increase my score to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal summary

Dear Area Chair and reviewers,

We thank the area chair and the reviewers for their time and constructive feedback. As the discussion period ends, we would like to provide a summary of the rebuttal. We first summarize the reviewers' concerns and how we addressed them.

Need human validation (reviewers ChnN, sHV7, hVaT).

The reviewers requested human validation on the RobotValues dataset.

  • We have conducted two human surveys with a total of 87 annotators.
  • 89.2% of the time, annotators agreed with the dataset’s labeled value annotations.
  • 89.1% of the time, annotators responded that the stakeholder reactions are realistic and 94.1% of the time the scenario is realistic.
  • These results support the quality of our dataset.
  • Reviewer sHV7 replied "The response substantially resolved my concern, especially the independent human validation".
  • We updated these results in Section 4 (Benchmark Validation) with details provided in Appendix C (Human Validation).

Visual grounding of the dataset is unclear (ChnN, hVaT).

The reviewers stated that the added value of visual grounding remains unclear.

  • We clarified that robot planners typically receive visual observations and therefore we include images in the dataset.
  • We conducted two additional experiments and showed that model decisions change according to the visual input.
  • We experimented how often models change their choices when images are (1) removed or replaced with (2) a random image or (3) a grey image. The results show that models change their action choices by (1) 20.7–33.3%, (2) 13.2–28.9%, and (3) 15.6–29.6% in each setting.
  • We further tested whether the visual scene matters beyond the text of the candidate actions in the steering setting, by removing the household decision context. Similarly, model decisions change when the visual input changes.
  • These results show that image input affects model decisions, emphasizing the importance of images in the dataset.
  • We updated these analyses in Appendix F.5 (Modality Ablation) and reference these results in Section 5.

The interpretation of "failure to override". There may be justifiable refusals (sHV7).

The reviewer stated that there may be justifiable refusals in the steering task.

  • We identified that these justifiable failure cases do not take the majority. Gemini 4 Carbon classified 4.6% of candidate actions as inappropriate.
  • We added a decline option to the candidate actions and show that models chose it in only 2.6% cases.
  • We asked each model whether each candidate action is appropriate in the scene. Among conflicting failures, the model itself judged every action corresponding to the requested value as inappropriate in 9.8% of cases. Even when the model judged at least one action corresponding to the target value as appropriate, it followed the request only 17.5% of the time.
  • These results suggest that inappropriate target actions do not explain most of the steering failures.
  • We have updated these results into Section 5 (Justifiable refusals paragraph) with details in Appendix F.2.

Why do models fail value steering (hVaT).

The reviewer requested to analyze why models fail in the value steering task.

  • Experiments on two instruction-following benchmarks suggest that poor instruction-following ability alone does not explain the steering failures.
  • We clarified that Section 5 (Figure 5) shows that models still fail even when they correctly recognize actions that prioritize the given target value (15.9-29.3% accuracy).
  • We have updated the results in Section 5, with details in Appendix F.1.

Reviewers ChnN and sHV7 replied during the rebuttal and raised their scores from 2 to 3.


We would like to highlight the strengths of our paper that reviewers mentioned.

Addresses a useful gap

ChnN: "Evaluating trade-offs beyond task completion addresses a useful gap"

sHV7: "The research question is well motivated… A benchmark that makes value conflicts explicit would address a gap in current evaluations" "...the benchmark addresses an important evaluation gap and provides a starting point for studying it."

hVaT: "This is particularly important for robots operating in domestic environments."

Systematic and practical data generation pipeline

sHV7 "The multi-stage pipeline offers a practical way to construct a large set of value-conflict scenarios,..."

hVaT: "The benchmark construction is systematic and reasonably well documented, combining diverse…"

Useful finding

ChnN: "The action–value recognition control is particularly helpful: it shows that recognizing an action's annotated value does not by itself ensure successful steering."

hVaT: "The finding that models struggle to prioritize explicitly requested values is particularly interesting and has meaningful implications for robot planning."

Experiments are comprehensive

hVaT: "The experimental evaluation is comprehensive, spanning multiple VLM architectures..."

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.