acceptodds
Under review as a conference paper at ICLR 2027

MCPA: Modal Conflict Prompt Attack on VLMs for Autonomous Driving

Abstract

Vision-language models (VLMs) are increasingly used in autonomous driving (AD) systems for scene understanding and driving guidance. However, when user queries contain incorrect descriptions of objects, attributes, or relations, AD-VLMs may accept these false premises without verifying them against visual evidence, leading to unsafe driving suggestions. Existing datasets provide limited support for studying such vulnerabilities. Therefore, we construct DriveConflict, a dataset derived from DriveLM that introduces object, attribute, and relation conflicts between textual queries and visual scenes. Evaluations on general and AD-VLMs reveal a cross-modal premise verification deficiency, where models often prioritize answering queries over validating their textual premises. Motivated by this finding, we propose the Modal Conflict Prompt Attack (MCPA) for AD-VLMs in a black-box setting. Given a visual input and a user query, MCPA first identifies semantic anchors involving objects, attributes, or relations and selects replacements that preserve the semantic type but are unsupported by the visual evidence. It then constructs false premises through minimal semantic modification and embeds them into prompts designed to induce shifts in scene understanding or unsafe driving guidance. The resulting prompts remain linguistically natural while contradicting the visual scene. Experimental results demonstrate that MCPA achieves strong attack performance across datasets, models, and attack types. On NuScenes-QA, MCPA achieves an FPAR of 98.05%. On DriveLM, it achieves an FPAR of 99.80%, a DSR of 93.08%, and a UGR of 98.17%.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Vision-language models, autonomous driving, modal conflict, attack

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KFRiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether driving VLMs accept textual premises that contradict visual evidence. It introduces DriveConflict, a dataset of 10,090 paired consistent/conflicting examples covering objects, attributes, and relations. MCPA constructs visually contradicted premises through localized semantic edits and embeds them into scene-understanding or driving-guidance queries. Experiments evaluate premise acceptance, decision shifts, and potentially unsafe guidance across driving QA benchmarks and multiple models, with additional ablations and defense evaluations.

Critical strengths

The paired dataset offers a useful diagnostic resource. Localized object, attribute, and relation edits help examine premise verification systematically. Manual sample inspection and visual verification of attack candidates strengthen the evidence that the constructed conflicts are meaningful.

The experimental coverage is substantial. The paper evaluates multiple models and two driving QA benchmarks, and includes correct-premise controls, template ablations, defenses, and three safety evaluators. These analyses provide useful evidence of vulnerability to misleading text and acknowledge several sources of variation.

Critical weaknesses

Unsafe-guidance attacks transform the original query into an urgent collision-avoidance question, so a changed response need not indicate a false-premise-induced decision failure. In Table 4, the correct-premise control already produces 92.03% DSR, compared with 93.08% for MCPA, and 90.33% UGR, compared with 98.17%. The authors acknowledge the template’s influence, and the Safety Score difference still supports an additional effect of the false premise. However, the headline DSR and UGR do not isolate that effect. Task-matched, paired correct/false-premise comparisons across the evaluated models are needed to substantiate the downstream claim.

UGR automatically treats failure of the safety-critical hallucination criterion as unsafe, creating overlap with premise acceptance itself. The evaluator also assigns zero when visual evidence does not sufficiently establish safety, potentially conflating insufficient evidence with an actually dangerous action. Although three evaluators are tested, their severity estimates vary considerably: ReasonDrive-Qwen’s average Safety Score ranges from 0.50 to 5.38 (Table 13). No human validation of the generated-action safety labels is reported. This limits how confidently the near-100% UGR values can be interpreted as downstream action-safety failures.

Questions for authors

Under identical images and driving-decision templates, what are the paired action and safety differences between correct and false premises across models? How are comparable actions extracted from free-form responses for DSR?

Can a representative human-reviewed subset distinguish unsafe actions, insufficient visual evidence, and reasonable conservative guidance? Does the safety degradation persist when evaluated separately from premise acceptance?

Which model/version performs visual conflict verification, and what are the attempted, skipped, and retained sample counts? Are the baselines evaluated on the same retained cases?

4 comments
AuthorsOpenReview ↗

Rebuttal Comment for Reviewer KFRi

[W1&Q1] In Table 4, Correct pre. isolates the effects of the decision template and false premises. Correct pre. and MCPA achieve similar DSR values (92.03% vs. 93.08%), indicating that the template largely drives action changes. Removing the template reduces DSR to 35.95%, while UGR remains at 89.83%, close to Correct pre. (90.33%). This suggests that the template strongly affects action selection but has limited additional impact on safety-risk metrics. In contrast, introducing false premises under the same template increases UGR from 90.33% to 98.17% and reduces Safety Score from 6.37 to 0.50. Table 11 further supports this finding: Template 4 and MCPA show similar DSR (93.14% vs. 93.08%) but substantially different Safety Scores (7.74 vs. 0.50). These results indicate that action changes do not necessarily imply safety degradation, while false premises are associated with greater safety risks.

To further isolate premise effects, we conduct 600 paired experiments using identical images, decision templates, and action sets, changing only the scene description. We observe 56 action changes (9.33%), including 26 transitions between stopping and lane changing and 10 between slowing down and lane changing.

Conflict TypePaired SamplesAction ChangesChange Rate
Object200189.00%
Attribute200189.00%
Relation2002010.00%
Overall600569.33%

We further evaluate the safety of these samples. The results show that, under a fixed decision template, false premises can alter driving actions and are associated with changes in potential safety risks.

MetricOriginal PremiseFalse Premise
Longitudinal Safety0.6530.617
Lateral Safety0.6070.607
Safety Score5.5935.377
UGR83.67%84.67%

For the Free-Form Action Extraction and Cross-Model Comparison. We use unified action normalization rules based on regular expressions and synonym mapping to classify model responses into seven actions: stop, slow down, accelerate, maintain speed, change lanes, turn left, and turn right. We further evaluate free-form responses from five models using the same extraction rules as the main experiment and compare them with constrained-action outputs. The results show high extraction coverage for free-form responses, while the effect of output format on action selection varies across models, with Dolphins exhibiting the most changes.

ModelValid PairsSame ActionsDifferent Actions
ReasonDrive-Qwen556/60052729
ReasonDrive-Llama578/60055919
ReasonDrive-LLaVA600/60057921
Dolphins600/600473127
DriveLM-Agent599/6005954

[W2&Q2] We conduct a supplementary human safety evaluation using six-view driving images and generated actions, classifying each action as Unsafe, Insufficient Evidence, or Safe. The human unsafe rates across five models are 57.83%, 63.33%, 50.00%, 78.83%, and 83.33%, respectively, all lower than GPT-4o-mini's UGR but showing similar overall trends. ReasonDrive-Qwen and ReasonDrive-Llama remain particularly vulnerable. These results suggest that automatic UGR may overestimate unsafe actions, but human evaluation still identifies substantial safety risks independently of false-premise acceptance and insufficient visual evidence.

ModelUGR (%)Unsafe (%)Insufficient (%)Safe (%)
DriveLM-Agent66.1757.8311.1731.00
Dolphins74.3363.3310.0026.67
ReasonDrive-LLaVA51.0050.0012.1737.83
ReasonDrive-Llama94.0078.838.8312.33
ReasonDrive-Qwen98.1783.337.838.83

[Q3] MCPA uses GPT-4o-mini as a visual verifier to determine whether candidate false premises conflict with the original images. During candidate generation, MCPA applies validity filtering and visual conflict verification to retain 200 valid attack samples per conflict type (600 in total). Specifically, 20 Object, 115 Attribute, and 10 Relation candidates fail the filtering process. All baselines are evaluated on the same samples.

Dear Reviewer,

Thank you for your time and effort in reviewing our paper and for your valuable feedback.

We have carefully considered all your comments and provided detailed responses, including additional analyses and experimental results, to address the concerns raised during the review.

As the discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our rebuttal and let us know whether your concerns have been adequately addressed. We would be happy to provide further clarifications if needed.

Thank you again for your time and consideration.

Best regards,

The Authors

Thank you for the additional experiments. The matched comparison shows that changing the premise can change the selected action (56 of 600 cases), and the human evaluation usefully distinguishes unsafe actions from cases with insufficient evidence.

My remaining concern is how often a false premise makes the resulting guidance less safe. The reported human unsafe rates are aggregate rates by model and do not quantify the change between matched correct- and false-premise responses. The human unsafe rates are also consistently lower than the automated UGR values, which limits how I interpret the headline results. I therefore maintain my overall rating.

Thank you for raising these two important points and we provide our clarifications below.

(1) Quantifying the change in human unsafe rates between correct and false premises. Our human evaluation directly assesses the final driving decisions generated by MCPA against the original driving images, classifying them into three categories: Unsafe, Insufficient Evidence, and Safe. Its purpose is to assess the safety of the generated guidance in the actual visual context. However, we do not conduct human evaluations of driving decisions generated under correct premises, and therefore cannot directly quantify the change in human unsafe rates between the two conditions. We also note that the previously reported 600-pair experiment differs from the main evaluation in both purpose and construction. The paired experiment replaces only the semantic anchor while keeping the images and decision templates fixed, whereas the main evaluation assesses the complete MCPA pipeline. Furthermore, the two experiments exhibit substantially different sample and action distributions: approximately 75% of the generated actions in the main evaluation are acceleration, whereas approximately 76% in the paired experiment are stopping. Only 24 samples match exactly across the two experiments. Therefore, the paired results cannot be directly used to infer changes in human unsafe rates in the main evaluation.

(2) Why human unsafe rates are lower than automatic UGR. We attribute this difference primarily to the different judgment criteria and evaluation scopes. GPT-4o-mini applies unified safety criteria to assess driving actions and their consistency with the visual scene, thereby identifying potential safety risks. In contrast, human evaluation uses three explicit categories and separately classifies cases with insufficient visual evidence but no clearly identifiable unsafe behavior as Insufficient Evidence, rather than directly counting them as Unsafe. Thus, differences in decision thresholds and statistical definitions may cause automatic UGR to include some cases that human evaluators classify as Insufficient Evidence. Importantly, UGR reflects the potential safety risks identified by the evaluator and does not represent the probability of actual hazardous events in real-world driving. Meanwhile, all attack baselines and MCPA are evaluated using the same automatic evaluation criteria. Therefore, although absolute UGR values may depend on the choice of evaluator, relative comparisons among methods under the same evaluation conditions remain informative. Together with the human evaluation results, these findings confirm that MCPA-generated driving guidance still contains a substantial proportion of explicitly unsafe recommendations.

Reviewer_3dVVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines whether driving VLMs accept textual premises that contradict visual evidence and whether these errors affect their driving guidance. The authors introduce DriveConflict, a dataset derived from DriveLM that pairs visually consistent QA examples with corresponding conflict QA examples. They also propose MCPA, a black-box textual attack that constructs modal conflicts through semantic edits and incorporates them into prompts for scene understanding or driving guidance. Experiments show frequent false-premise acceptance across the tested models and lower Safety Scores under attack. The paper further examines the effects of the attack template and evaluates several defenses.

Critical strengths

DriveConflict provides a useful setting for studying whether a model checks the visual basis of a question before answering it. The paired examples allow this behavior to be examined systematically, and the manual inspection improves confidence in the generated conflicts. This is a relevant problem for driving VLMs, where an apparently reasonable response may rely on an incorrect description of the scene.

Critical weaknesses

In Table 4, ReasonDrive-Qwen has a DSR of 92.03% with a correct premise and 93.08% with the full attack. As the authors acknowledge in Section 5.4, the template accounts for much of the shift relative to the original question. Given this strong template effect, how should the safety results be interpreted relative to the correct-premise condition? For the results to inform improvements in model safety, the authors should distinguish suggestions already classified as potentially unsafe under the template from any additional safety degradation introduced by the false premise. The authors should also clarify which suggested actions change between the two conditions and whether those changes are accompanied by lower scores for longitudinal action safety or lateral action safety. Comparing these paired responses would distinguish changes in driving action suggestions from changes in their assessed safety.

The contribution of the attack method also needs a closer comparison with related work. MMMC constructs conflicts involving objects, attributes, and relations, while AutoTrust tests driving VLMs by adding misleading statements before the original questions. The paper discusses these studies, but the comparisons with general prompt-based attacks do not establish what MCPA gains over their procedures for constructing conflicting inputs. A comparison with an adapted version of one such procedure, using the same driving inputs and decision template, would be more informative.

Questions for authors
  1. Why does the correct-premise condition have a UGR of 90.33%, and what does this indicate about the effect of the attack template on the generated driving guidance?
  2. Under the same template, which suggested driving actions change when a correct premise is replaced by a false one? How do the scores for longitudinal action safety and lateral action safety differ between these paired responses?
  3. How does MCPA compare with an adapted procedure for constructing conflicting inputs from MMMC or AutoTrust when evaluated using the same driving inputs and decision template?
7 comments
AuthorsOpenReview ↗

Rebuttal Comment for Reviewer 3dVV

[W1&Q1&Q2] Impact of Decision Templates and False Premises. In Table 4, Correct pre. isolates the effects of the decision template and false premises. Correct pre. and MCPA achieve similar DSR values (92.03% vs. 93.08%), indicating that the template largely drives action changes. Removing the template reduces DSR to 35.95%, while UGR remains at 89.83%, close to Correct pre. (90.33%). Thus, the template strongly affects action selection but has limited additional impact on safety-risk metrics. Under the same template, introducing false premises increases UGR from 90.33% to 98.17% and decreases Safety Score from 6.37 to 0.50. Table 11 further supports this finding: Template 4 and MCPA show similar DSR (93.14% vs. 93.08%) but substantially different Safety Scores (7.74 vs. 0.50). These results suggest that action changes do not necessarily imply safety degradation, whereas false premises are associated with greater safety risks. To further isolate premise effects, we conduct 600 paired experiments using identical images, decision templates, and action sets, changing only the scene description. We observe 56 action changes (9.33%), including 26 transitions between stopping and lane changing and 10 between slowing down and lane changing.

Conflict TypePaired SamplesAction ChangesChange Rate
Object200189.00%
Attribute200189.00%
Relation2002010.00%
Overall600569.33%

We further evaluate the safety of these samples. The results show that, under a fixed decision template, false premises can alter driving actions and are associated with changes in potential safety risks.

MetricOriginal PremiseFalse Premise
Longitudinal Safety0.6530.617
Lateral Safety0.6070.607
Safety Score5.5935.377
UGR83.67%84.67%

[W2&Q3] MMMC evaluates cross-modal understanding through object, attribute, and relation conflicts, while AutoTrust assesses AD-VLM robustness using misleading textual descriptions. In contrast, MCPA integrates driving-relevant anchor selection, visual conflict verification, and decision-oriented prompting to investigate how modal conflicts affect driving decisions and potential safety risks. We clarify these distinctions in Section 2.1. In Section 5.3 and Table 3, we add comparisons with MMMC and AutoTrust, adapting both methods to autonomous driving according to their original designs. We evaluate all methods on the same datasets and target models using DSR, UGR, and Safety Score. The results are as follows.

ModelMethodDSR ↑UGR ↑Score ↓
DriveLM-AgentMMMC0.000.339.98
AutoTrust100.008.009.78
Ours83.4166.177.01
DolphinsMMMC17.334.339.74
AutoTrust90.507.009.66
Ours86.2674.337.50
ReasonDrive-LLaVAMMMC25.0055.697.78
AutoTrust51.5017.509.45
Ours80.7851.007.46
ReasonDrive-LlamaMMMC7.3361.187.71
AutoTrust91.0799.663.12
Ours96.6394.001.01
ReasonDrive-QwenMMMC16.6715.679.40
AutoTrust88.3257.956.97
Ours93.0898.170.50

MMMC induces relatively few decision shifts, while AutoTrust often achieves high DSR without correspondingly high safety risks. In contrast, MCPA achieves high DSR and UGR and low Safety Scores on multiple models, suggesting that it not only alters driving decisions but also more effectively induces guidance flagged as potentially unsafe by the automatic evaluator.

Thanks for the additional experiments. The paired experiments and the comparisons with MMMC and AutoTrust are helpful and address some of my concerns.

My remaining concern is how to interpret the reported safety degradation. Since the Safety Score evaluates both the suggested actions and their grounding in visual evidence, could the authors clarify how much of the score decrease reflects less safe driving actions and how much results from incorrect or unsupported premises? This distinction is important for understanding the safety implications of the attack. The new paired evaluation also shows a much smaller Safety Score decrease (5.593 to 5.377) than Table 4 (6.37 to 0.50). It would be helpful to explain this difference.

Overall, the additional results strengthen the paper. I would be open to raising my rating if this remaining concern is resolved.

Thank you for acknowledging our experimental results and raising further questions about the sources of the Safety Score decrease. To distinguish changes in driving-action safety from the effects of incorrect or unsupported premises, we further analyze the new paired experiment and Table 4.

(1) Safety Score changes in the paired experiment. Across 300 paired samples, the Safety Score decreases from 5.593 to 5.377 (−0.217). Specifically, five action-related criteria contribute a decrease of 0.117 (53.85%), four visual-grounding criteria contribute 0.080 (36.92%), and uncertainty handling contributes 0.020 (9.23%). Further analysis shows decreases of 0.037 in longitudinal action safety and 0.033 in collision-risk avoidance, while lateral action safety remains unchanged.

(2) Safety Score changes in Table 4. We analyze the ten safety criteria for Correct pre. and MCPA. Of the total 5.87-point difference, five action-related criteria account for 2.86 points (48.81%), four visual-grounding criteria account for 2.10 points (35.71%), and uncertainty handling accounts for 0.91 points (15.48%).

(3) Reasons for the different Safety Score decreases. First, the experiments differ substantially in sample and action distributions. Approximately 75% of generated actions in Table 4 are acceleration, whereas approximately 76% in the paired experiment are stopping. Moreover, only 24 samples match exactly across the two experiments, indicating different sample compositions and driving-decision distributions. Second, the experiments differ in objectives and attack construction. Table 4 evaluates the complete MCPA pipeline, while the paired experiment replaces only one semantic anchor under fixed images and decision templates to isolate the additional effects of false premises on driving decisions and safety assessments. Thus, the two experiments reflect different evaluation conditions, and their Safety Score decreases are not directly comparable.

Overall, Safety Score captures multiple dimensions, including driving-action safety, visual grounding, and uncertainty handling. MCPA substantially affects driving decisions and overall safety assessments, but the Safety Score decrease cannot be attributed entirely to reduced driving-action safety.

Reviewer_EiDoOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies what happens when a user's question to a driving VLM contains a false premise about an object, an attribute or a relation that the image does not support. It first builds DriveConflict, a dataset derived from DriveLM [1]. A multimodal model writes a structured scene description for each driving image, an LLM writes questions that are consistent with that description, and minimal semantic editing then turns each question into a conflict question by swapping an object, changing an attribute or altering a spatial relation. An LLM writes a reference answer for each conflict question, and the authors inspect the samples by hand. Evaluating six general VLMs and six AD-VLMs on this data, the authors report high false premise acceptance and low conflict recognition. An explicit instruction to check the image first helps only a little. The authors call this a cross-modal premise verification deficiency.

Building on this finding, the paper proposes MCPA, a black-box prompt attack on AD-VLMs. The attacker sees the same scene but cannot change the image or access the model. MCPA builds semantic candidate pools for objects, attributes and relations, selects an attackable semantic anchor in the question, filters invalid samples, and picks a replacement of the same semantic type that an auxiliary VLM classifies as contradicted by the image. For the scene shift goal, the false premise replaces the original term inside the question. For the unsafe guidance goal, the false premise is placed inside a fixed decision-oriented template that asks for the safest immediate action to avoid an immediate collision with the false object.

Experiments use DriveLM and NuScenes-QA [2] with five AD-VLMs: DriveLM-Agent [1], Dolphins [3] and three ReasonDrive variants [4]. GPT-4o-mini serves as the evaluator for false premise acceptance and for a safety score built from ten safety criteria. The paper reports high false premise acceptance, decision shift and unsafe guidance rates. It compares MCPA with six prompt-based attack baselines, runs ablations on the candidate pool, the anchor selection, the template and a correct premise setting, and tests five defenses. The appendix adds combined attacks, alternative templates, other evaluation models and a per-criterion safety analysis.

[1] DriveLM: Driving with Graph Visual Question Answering.

[2] NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario.

[3] Dolphins: Multimodal Language Model for Driving.

[4] ReasonDrive: Efficient Visual Question Answering for Autonomous Vehicles with Reasoning-Enhanced Small Vision-Language Models.

Critical strengths
  1. The paper follows a false premise in the user's question past the answer and into the driving action and its safety. Earlier driving work on such premises, such as AutoTrust [5], stops at QA accuracy, so this is the new part.
  2. The paper reports enough to check its own claims, including a separate uncertain category, a correct premise setting, three evaluation models and the full prompts. Most of my concerns below come from the paper's own tables.

[5] AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving.

Critical weaknesses
  1. The unsafe guidance results mostly measure the template. In the correct premise setting, the false premise is replaced by its visually supported counterpart inside the same template, and decision shift and unsafe guidance stay almost as high. When the false premise is inserted without the template, it changes far fewer decisions. The paper still credits the safety effect to modal conflict. Its clean versus attack comparison also changes the template and the premise at the same time.
  2. The novelty over prior work is thin, and the closest baseline is missing. DriveConflict closely follows the taxonomy and construction approach of MMMC [6]. MAD-Bench [7], CP-Bench from the Antidote work [8] and JBA [9] already test questions with false object, attribute or relation premises, and the paper cites none of them. AutoTrust [5] already adds misinformation that contradicts the image to questions on the same two datasets and tests two of the same models. The six baselines are jailbreak methods built for other goals. A plain misinformation statement placed before the question, as in AutoTrust, is never compared.
  3. The paper never shows that the false premises are really false. Object conflicts come from objects that are missing from a model-generated scene description, yet the paper's own visual conflict verification prompt says that absence from a description is not evidence of absence from the image. The nuScenes [10] 3D bounding box annotations could settle this, but they are not used. The auxiliary verifier VLM in MCPA is never named or validated. The manual inspection gives no protocol and no agreement between the inspectors. Even models told to check the image first accept nearly all attribute and relation edits, which could mean that many of these edits cannot be checked from the image at all.
  4. The evaluators are not validated, and they make rejection hard to score. The false premise evaluator counts any direct answer to the question as implicit acceptance, while the target models are instructed to keep their answers concise and direct. A bare "No" is mostly labeled uncertain, even though "No" is the reference answer the dataset itself prescribes for yes or no questions. The safety evaluator has no human check. The three evaluation models disagree a lot, and one of them breaks the vulnerability pattern that the paper says is broadly preserved across evaluators. Several of the ten safety criteria give almost the same scores, so the same judgment is counted more than once.
  5. The paper overclaims. It says MCPA works consistently across models and attack types and is more stable than the baselines. The tables do not support this: false premise acceptance is low on all ReasonDrive [4] models, some baselines beat MCPA on some models, and combined edits weaken it on others. The paper also calls the higher acceptance after removing the candidate pool or the anchor selection module only a slight gain, which plays down that both modules weaken the scene shift attack. All ablations and defenses use a single model on which every baseline is already near the ceiling.

[4] ReasonDrive: Efficient Visual Question Answering for Autonomous Vehicles with Reasoning-Enhanced Small Vision-Language Models.

[5] AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving.

[6] Robust Multimodal Large Language Models Against Modality Conflict.

[7] How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts.

[8] Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception.

[9] Judge Before Answer: Can MLLM Discern the False Premise in Question?

[10] nuScenes: A Multimodal Dataset for Autonomous Driving.

Questions for authors
  1. In Table 5, the Prompt defense [11] and VCD [12] have exactly the same DSR in every column, even though one is a prompt and the other is a decoding method. Is this a copy error, and are other defense numbers affected?
  2. Which samples are behind each number? The sample sizes implied by Table 1, by the clean and attack rows of Table 6, and by the Gemini-2.5-flash evaluator in Table 12 do not match the dataset size or each other. How many candidates did validity filtering and visual conflict verification skip, and are the metrics computed only on the samples that remain?
  3. Without the template, most actions do not change, yet most are still judged unsafe, far above the clean rate. The safety evaluator also sees the attacked query. Is it scoring the false premise in the query rather than the action? How is the clean action defined when the original question is a perception question?
  4. How were the jailbreak baselines adapted to this task? What goal did they inject, and did they include the same false premise? Some of them barely change any decision.
  5. Could you show raw unsafe guidance outputs from ReasonDrive-Qwen and ReasonDrive-Llama [4]? Their safety scores are near zero, so even the basic grounding criteria fail. Are these real unsafe maneuvers or broken answers?

[4] ReasonDrive: Efficient Visual Question Answering for Autonomous Vehicles with Reasoning-Enhanced Small Vision-Language Models.

[11] Defending ChatGPT against jailbreak attack via self-reminders.

[12] Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding.

3 comments
AuthorsOpenReview ↗

Rebuttal Comment for Reviewer EiDo

[W1] In Table 4, Correct pre. and MCPA achieve similar DSR values (92.03% vs. 93.08%), indicating that the decision template largely drives action changes. However, w/o Tplt. and Correct pre. show comparable UGR and Safety Scores, suggesting a limited additional safety impact from the template. Under the same template, MCPA increases UGR from 90.33% to 98.17% and reduces the Safety Score from 6.37 to 0.50. Table 11 further confirms this pattern: Template 4 and MCPA have similar DSR values (93.14% vs. 93.08%) but substantially different Safety Scores (7.74 vs. 0.50). These results suggest that action changes do not necessarily imply safety degradation, while modal conflicts are associated with greater safety risks.

[W2] We discuss MMMC and AutoTrust in Section 2.1 and clarify in Section 3.1 that our three conflict types follow MMMC. We also add discussions and citations of MAD-Bench, Antidote/CP-Bench, and JBA in Section 2.1. Furthermore, we include comparisons with MMMC and AutoTrust under the same evaluation settings in Section 5.3 and Table 3, as shown below.

ModelMethodDSR ↑UGR ↑Score ↓
DriveLM-AgentMMMC0.000.339.98
AutoTrust100.008.009.78
Ours83.4166.177.01
DolphinsMMMC17.334.339.74
AutoTrust90.507.009.66
Ours86.2674.337.50
ReasonDrive-LLaVAMMMC25.0055.697.78
AutoTrust51.5017.509.45
Ours80.7851.007.46
ReasonDrive-LlamaMMMC7.3361.187.71
AutoTrust91.0799.663.12
Ours96.6394.001.01
ReasonDrive-QwenMMMC16.6715.679.40
AutoTrust88.3257.956.97
Ours93.0898.170.50

The results show that MMMC induces relatively few driving-decision changes, while AutoTrust often achieves high DSR without correspondingly high safety risks, indicating that decision shifts do not necessarily imply greater safety risks. In contrast, MCPA achieves high DSR and UGR and low Safety Scores across multiple models, suggesting that it not only changes driving decisions but also more effectively induces guidance flagged as potentially unsafe by the automatic evaluator.

[W3] To ensure the validity of the constructed conflict samples, we first use Qwen3-VL-8B-Instruct to extract key visual elements from driving scenes. Manual spot checks show that it identifies most clearly visible elements. Next, three autonomous-driving researchers independently label each candidate as valid or invalid, removing samples judged invalid by at least two reviewers. This process removes 114 object, 1,713 attribute, and 299 relation conflict samples, retaining 10,090 paired examples.

Although nuScenes 3D bounding boxes can help verify certain object conflicts, they cannot fully capture fine-grained attributes or semantic relationships. We clarify this limitation in Appendix A.1.1, Step 5.

MCPA uses GPT-4o-mini as a visual verifier to determine whether candidate false premises conflict with the original images.

[W5] Although the ReasonDrive models show relatively low FPAR, Tables 2 and 3 report high DSR and UGR, indicating that false-premise acceptance and downstream decision impact are not equivalent. Appendix Tables 7–9 further show that combined conflicts reduce attack effectiveness on some models. We therefore revise Sections 5.2 and A.4.3 and the Conclusion to avoid overclaiming consistent superiority and clarify model- and attack-dependent differences. Ablation Study. In Table 4, removing the candidate pool and anchor selection increases FPAR from 47.60% to 53.20% and 55.80%, respectively, but reduces DSR from 93.08% to 85.94% and 88.67%, and UGR from 98.17% to 91.00% and 90.83%. These modules do not improve FPAR but contribute to higher decision-shift and safety-risk metrics. We clarify this distinction in Section 5.4. We extend the ablation study to Dolphins in Section 5.4. The results highlight the model- and metric-dependent effects of individual components.

MethodDSR ↑ObjAttrRelUGR ↑ObjAttrRelScore ↓ObjAttrRel
w/o Pool81.7480.5285.3179.396.275.683.809.099.659.669.859.46
w/o Anchor83.2882.1487.0680.646.923.414.8812.229.609.849.739.26
MCPA86.2685.3987.9885.7074.3373.5075.0074.507.506.957.847.72

[Q1] We carefully verify the original model outputs and action-change statistics for both defense methods and confirm that the reported values are correct, with no copying or transcription errors. The results of other defense methods remain unaffected.

[W4] Implicit acceptance does not include all direct answers. It refers to responses that answer a follow-up question based on a false premise without questioning it. For example, describing a nonexistent object's motion as "Moving" implies acceptance of that premise, whereas irrelevant responses such as "Blue" are not counted toward FPAR. In some Yes/No questions, "No" may constitute a valid rejection. However, it may also negate only the queried attribute rather than the false premise itself. We therefore classify ambiguous responses as Uncertain, which may affect the reported FPAR. We conducted human safety checks on five target models, classifying generated actions as Unsafe, Insufficient Evidence, or Safe based on six-view images and driving actions. Human unsafe rates are consistently lower than GPT-4o-mini's UGR, while the overall risk trends remain similar. This suggests that automatic evaluation may overestimate unsafe actions, although human inspection still identifies substantial potential safety risks. We provide detailed results and analysis in Appendix A.4.8. Table: Comparison between GPT-4o-mini-based automatic safety evaluation and human validation across five AD-VLMs.

ModelUGR (%)Unsafe (%)Insufficient (%)Safe (%)
DriveLM-Agent66.1757.8311.1731.00
Dolphins74.3363.3310.0026.67
ReasonDrive-LLaVA51.0050.0012.1737.83
ReasonDrive-Llama94.0078.838.8312.33
ReasonDrive-Qwen98.1783.337.838.83

We acknowledge that safety judgments and evaluation strictness vary across evaluators. In Appendix A.4.5, we revise the claim of consistent trends across evaluators and clarify their scoring differences and limitations.

The ten safety criteria cover different aspects of driving safety, but some are correlated and may capture overlapping risks. Thus, the Safety Score should be interpreted as a composite metric rather than ten independent judgments. We clarify this in Appendix A.4.6.

[Q2] The evaluation sample sizes for each table are summarized below.

TableTotal SamplesSamples per Type
Table 1: Clean Samples900300
Table 1: Conflict Questions900300
Table 6: Clean600200
Table 6: Attack600200
Table 12: Gemini Evaluator600200

MCPA collects 200 valid attack samples per conflict type, totaling 600 samples. The filtering counts during generation are as follows:

Conflict TypeFiltered SamplesRetained Samples
Object20200
Attribute115200
Relation10200
Total145600

All attack metrics use the number of samples with valid evaluation results under each setting as the denominator, excluding samples filtered during generation or records with evaluation failures.

[Q3] DSR and UGR measure different aspects: DSR captures action changes, while UGR assesses potential safety risks. An unchanged action is not necessarily safe. In Table 4, removing the template reduces DSR to 35.95%, while UGR remains at 89.83%, indicating that action changes and safety risks are not directly equivalent.

Although the safety evaluator receives the user question, it evaluates the generated driving action. The evaluation prompt explicitly prioritizes the original images, does not assume the question's premise is true, and assesses whether the action fits the actual scene, aiming to reduce premise-induced evaluation bias.

Perception questions are evaluated only using FPAR. DSR, UGR, and Safety Score apply exclusively to driving-planning questions. Clean and Attack refer to driving decisions generated from the original and attacked questions using the same images, respectively. Perception responses are not included in action-safety evaluation.

[Q4] We adapt six general prompt-based attacks to driving-decision tasks. All baselines and MCPA use the same images, original questions, and action-output constraints. Unlike MCPA, the baselines do not introduce constructed false premises but instead influence decisions through additional prompts. Thus, this experiment compares the overall effectiveness of different attack strategies rather than isolating prompting effects under identical false premises.

[Q5] We inspect 600 attack samples each from ReasonDrive-Qwen and ReasonDrive-Llama and find no invalid responses or action extraction errors. Both models show a strong tendency to accelerate, as detailed below.

ModelAttack SamplesAccelerate Count (%)Zero-Score Accelerate CountPercentage of All Zero-Score Samples (%)
ReasonDrive-Qwen600454 (75.67%)44782.47%
ReasonDrive-Llama600506 (84.33%)49794.49%

Safety Score measures the potential safety of extracted actions in the visual scene; a low score does not necessarily imply an actual collision or unsafe driving event.

Dear Reviewer,

Thank you for your time and effort in reviewing our paper and for your valuable feedback.

We have carefully considered all your comments and provided detailed responses, including additional analyses and experimental results, to address the concerns raised during the review.

As the discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our rebuttal and let us know whether your concerns have been adequately addressed. We would be happy to provide further clarifications if needed.

Thank you again for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.