- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies whether driving VLMs accept textual premises that contradict visual evidence. It introduces DriveConflict, a dataset of 10,090 paired consistent/conflicting examples covering objects, attributes, and relations. MCPA constructs visually contradicted premises through localized semantic edits and embeds them into scene-understanding or driving-guidance queries. Experiments evaluate premise acceptance, decision shifts, and potentially unsafe guidance across driving QA benchmarks and multiple models, with additional ablations and defense evaluations.
Critical strengths
The paired dataset offers a useful diagnostic resource. Localized object, attribute, and relation edits help examine premise verification systematically. Manual sample inspection and visual verification of attack candidates strengthen the evidence that the constructed conflicts are meaningful.
The experimental coverage is substantial. The paper evaluates multiple models and two driving QA benchmarks, and includes correct-premise controls, template ablations, defenses, and three safety evaluators. These analyses provide useful evidence of vulnerability to misleading text and acknowledge several sources of variation.
Critical weaknesses
Unsafe-guidance attacks transform the original query into an urgent collision-avoidance question, so a changed response need not indicate a false-premise-induced decision failure. In Table 4, the correct-premise control already produces 92.03% DSR, compared with 93.08% for MCPA, and 90.33% UGR, compared with 98.17%. The authors acknowledge the template’s influence, and the Safety Score difference still supports an additional effect of the false premise. However, the headline DSR and UGR do not isolate that effect. Task-matched, paired correct/false-premise comparisons across the evaluated models are needed to substantiate the downstream claim.
UGR automatically treats failure of the safety-critical hallucination criterion as unsafe, creating overlap with premise acceptance itself. The evaluator also assigns zero when visual evidence does not sufficiently establish safety, potentially conflating insufficient evidence with an actually dangerous action. Although three evaluators are tested, their severity estimates vary considerably: ReasonDrive-Qwen’s average Safety Score ranges from 0.50 to 5.38 (Table 13). No human validation of the generated-action safety labels is reported. This limits how confidently the near-100% UGR values can be interpreted as downstream action-safety failures.
Questions for authors
Under identical images and driving-decision templates, what are the paired action and safety differences between correct and false premises across models? How are comparable actions extracted from free-form responses for DSR?
Can a representative human-reviewed subset distinguish unsafe actions, insufficient visual evidence, and reasonable conservative guidance? Does the safety degradation persist when evaluated separately from premise acceptance?
Which model/version performs visual conflict verification, and what are the attempted, skipped, and retained sample counts? Are the baselines evaluated on the same retained cases?
4 comments
Dear Reviewer,
Thank you for your time and effort in reviewing our paper and for your valuable feedback.
We have carefully considered all your comments and provided detailed responses, including additional analyses and experimental results, to address the concerns raised during the review.
As the discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our rebuttal and let us know whether your concerns have been adequately addressed. We would be happy to provide further clarifications if needed.
Thank you again for your time and consideration.
Best regards,
The Authors
Thank you for the additional experiments. The matched comparison shows that changing the premise can change the selected action (56 of 600 cases), and the human evaluation usefully distinguishes unsafe actions from cases with insufficient evidence.
My remaining concern is how often a false premise makes the resulting guidance less safe. The reported human unsafe rates are aggregate rates by model and do not quantify the change between matched correct- and false-premise responses. The human unsafe rates are also consistently lower than the automated UGR values, which limits how I interpret the headline results. I therefore maintain my overall rating.
Thank you for raising these two important points and we provide our clarifications below.
(1) Quantifying the change in human unsafe rates between correct and false premises. Our human evaluation directly assesses the final driving decisions generated by MCPA against the original driving images, classifying them into three categories: Unsafe, Insufficient Evidence, and Safe. Its purpose is to assess the safety of the generated guidance in the actual visual context. However, we do not conduct human evaluations of driving decisions generated under correct premises, and therefore cannot directly quantify the change in human unsafe rates between the two conditions. We also note that the previously reported 600-pair experiment differs from the main evaluation in both purpose and construction. The paired experiment replaces only the semantic anchor while keeping the images and decision templates fixed, whereas the main evaluation assesses the complete MCPA pipeline. Furthermore, the two experiments exhibit substantially different sample and action distributions: approximately 75% of the generated actions in the main evaluation are acceleration, whereas approximately 76% in the paired experiment are stopping. Only 24 samples match exactly across the two experiments. Therefore, the paired results cannot be directly used to infer changes in human unsafe rates in the main evaluation.
(2) Why human unsafe rates are lower than automatic UGR. We attribute this difference primarily to the different judgment criteria and evaluation scopes. GPT-4o-mini applies unified safety criteria to assess driving actions and their consistency with the visual scene, thereby identifying potential safety risks. In contrast, human evaluation uses three explicit categories and separately classifies cases with insufficient visual evidence but no clearly identifiable unsafe behavior as Insufficient Evidence, rather than directly counting them as Unsafe. Thus, differences in decision thresholds and statistical definitions may cause automatic UGR to include some cases that human evaluators classify as Insufficient Evidence. Importantly, UGR reflects the potential safety risks identified by the evaluator and does not represent the probability of actual hazardous events in real-world driving. Meanwhile, all attack baselines and MCPA are evaluated using the same automatic evaluation criteria. Therefore, although absolute UGR values may depend on the choice of evaluator, relative comparisons among methods under the same evaluation conditions remain informative. Together with the human evaluation results, these findings confirm that MCPA-generated driving guidance still contains a substantial proportion of explicitly unsafe recommendations.
Rebuttal Comment for Reviewer KFRi
[W1&Q1] In Table 4, Correct pre. isolates the effects of the decision template and false premises. Correct pre. and MCPA achieve similar DSR values (92.03% vs. 93.08%), indicating that the template largely drives action changes. Removing the template reduces DSR to 35.95%, while UGR remains at 89.83%, close to Correct pre. (90.33%). This suggests that the template strongly affects action selection but has limited additional impact on safety-risk metrics. In contrast, introducing false premises under the same template increases UGR from 90.33% to 98.17% and reduces Safety Score from 6.37 to 0.50. Table 11 further supports this finding: Template 4 and MCPA show similar DSR (93.14% vs. 93.08%) but substantially different Safety Scores (7.74 vs. 0.50). These results indicate that action changes do not necessarily imply safety degradation, while false premises are associated with greater safety risks.
To further isolate premise effects, we conduct 600 paired experiments using identical images, decision templates, and action sets, changing only the scene description. We observe 56 action changes (9.33%), including 26 transitions between stopping and lane changing and 10 between slowing down and lane changing.
We further evaluate the safety of these samples. The results show that, under a fixed decision template, false premises can alter driving actions and are associated with changes in potential safety risks.
For the Free-Form Action Extraction and Cross-Model Comparison. We use unified action normalization rules based on regular expressions and synonym mapping to classify model responses into seven actions: stop, slow down, accelerate, maintain speed, change lanes, turn left, and turn right. We further evaluate free-form responses from five models using the same extraction rules as the main experiment and compare them with constrained-action outputs. The results show high extraction coverage for free-form responses, while the effect of output format on action selection varies across models, with Dolphins exhibiting the most changes.
[W2&Q2] We conduct a supplementary human safety evaluation using six-view driving images and generated actions, classifying each action as Unsafe, Insufficient Evidence, or Safe. The human unsafe rates across five models are 57.83%, 63.33%, 50.00%, 78.83%, and 83.33%, respectively, all lower than GPT-4o-mini's UGR but showing similar overall trends. ReasonDrive-Qwen and ReasonDrive-Llama remain particularly vulnerable. These results suggest that automatic UGR may overestimate unsafe actions, but human evaluation still identifies substantial safety risks independently of false-premise acceptance and insufficient visual evidence.
[Q3] MCPA uses GPT-4o-mini as a visual verifier to determine whether candidate false premises conflict with the original images. During candidate generation, MCPA applies validity filtering and visual conflict verification to retain 200 valid attack samples per conflict type (600 in total). Specifically, 20 Object, 115 Attribute, and 10 Relation candidates fail the filtering process. All baselines are evaluated on the same samples.