acceptodds
Under review as a conference paper at ICLR 2027

TRUST-AD: Evaluating Textual Bias in MLLMs in Autonomous Driving Context

Abstract

Multimodal Large Language Models (MLLMs) are increasingly integrated into autonomous driving to handle complex driving situations that require more than basic object detection. In these tasks, models are expected to ground their reasoning in visual scene evidence, apply relevant traffic rules, and select safe driving actions. However, it remains an open question whether MLLMs actually perform this multi-step visual reasoning or instead depend on parametric knowledge of the underlying LLM and textual context to produce decisions. We introduce **TRUST-AD**, a benchmark designed to test whether driving decisions remain grounded in visual evidence or biased through textual descriptions of the scene. **TRUST-AD** comprises 14,238 test cases across six safety-critical driving categories. By keeping the visual scene fixed while systematically intervening on the textual prompt describing the scene, the benchmark measures decision stability, vulnerability to misleading textual cues, and error recovery. In addition, an intervention-based attribution framework identifies whether incorrect decisions stem from visual-grounding failures or rule-reasoning failures. Evaluating both general-purpose and driving-specific MLLMs shows that models frequently abandon visual observations to follow textual cues, even when baseline accuracy suggests strong performance. These findings indicate that high task accuracy can mask a reliance on language shortcuts rather than grounded visual reasoning.

Primary area:
datasets and benchmarks
Keywords:
autonomous driving, Multimodal Large Language Models, Vision-Language Models, textual bias, robustness, evaluation, benchmarking, safety-critical AI, trustworthy AI

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_4ukdOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes TRUST-AD, a benchmark for evaluating the textual robustness of MLLMs in autonomous driving when given the same visual information. The benchmark includes 14,238 instances across 6 defined safety-critical driving categories. For each image, the paper provides 7 different prompt templates, including no information, information, suggestive, counterfactual, visual clue, rule injection, and visual clue & rule injection. Models are fed the same image input but with these different prompt templates. The paper evaluates 37 MLLMs, including open-source/commercial and general-purpose/domain-fine-tuned models. Experimental results show that (1) MLLMs' performance varies largely under different prompts, more than the sampling variability of the same prompt; (2) improvements in accuracy do not have a clear correlation with textual robustness; (3) a counterfactual prompt can mislead models; and (4) models reduce reliance on visual evidence when informative text is provided.

Critical strengths

1. Comprehensive evaluation and detailed analysis with interesting findings.

The paper conducts a large-scale (i.e., 37 MLLMs across 17 families) evaluation of MLLMs on the proposed benchmark. I especially appreciate the authors' effort in the analysis part, which considers (1) different prompt templates vs. sampling randomness; (2) relations between accuracy (also release timeline) and robustness; (3) failure attribution to different prompt templates; (4) general MLLMs vs. specialized MLLMs; and (5) visual contribution under each text prompt template.

I enjoy reading the findings, and some of them are surprising to see (e.g., newly released models do not necessarily improve textual bias), which I think might also benefit the broader ICLR community.

2. A carefully designed benchmark can benefit future research.

The proposed benchmark can be used by future research to evaluate the MLLM textual robustness in autonomous driving. From what I read, the benchmark and the metrics are well-designed and ready for use, which clearly meet the bar for a benchmark paper.

Critical weaknesses

1. The visual grounding contribution experiment can be more rigorous.

This concern does not impact my overall assessment of the paper, but I think it could strengthen the claim. In the visual grounding contribution experiment (Sec. 5.6), the paper claims that MLLMs reduce their reliance on visual evidence with informative text input (VC, RI, and VRI).

This conclusion is based on the performance gap between (a) text-only input and (b) image-text input. However, in scenarios where the answer can be fully inferred from the text alone, it's expected that adding an image does not bring further improvement. At least from the example I saw in the appendix, most answers can be derived from the VRI alone. Therefore, to claim that MLLMs reduce visual reliance, results-based analysis is not enough; the authors are suggested to probe intermediate indicators, e.g., attention score among image and text, etc. This can make the claim more supported.

Questions for authors

This is suggested for future work, not a clarification issue.

  • In Figure 7, the paper observes that Alpamayo-1.5 is more robust to the CF prompt. I'm wondering if that is because the model is more dominated by visual input and less influenced by text in general. From Table 16, it seems Alpamayo also shows moderate improvement in VC, RI, and VRI. Thus, the MLLM degrading less to counterfactual (CF) text prompts might not be because it's genuinely more robust, but because it is less influenced by text in general; So we might need to consider both (1) how much they improve over VRI and (2) how much they degrade over CF. Figure 7 only captures one side of it.
Reviewer_ScdcOpenReview ↗
Rating
2
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
1(−1)
Read review
Paper summary

This paper introduces TRUST-AD, a multimodal benchmark for driving scenarios built on the Zenseact Open Dataset. The authors keep images fixed while varying scene descriptions, user intentions, and traffic-rule prompts to examine changes in model decisions. They also use error recovery under different prompts to infer potential failure sources. The paper covers 37 driving scenarios across six categories and evaluates 37 models, aiming to show that conventional accuracy does not adequately characterize robustness to conflicting textual information.

Critical strengths
  1. The paper is generally well written and easy to follow. Concrete driving examples help explain the textual interventions, and the organization of the experiments makes the reported behavioral patterns easy to identify.
Critical weaknesses
  1. The central research question is not well justified. The need to integrate visual evidence, traffic rules, and task context does not justify framing decisions as based on either vision or text. Correct decisions can require both, and reliance on text or learned rules does not itself demonstrate a shortcut. The meaningful question is whether incorrect text improperly overrides relevant visual evidence. However, the reported protocol does not clearly specify source reliability or precedence, so a model could treat text as a supplement or correction to the image. This leaves unclear whether incorrect answers reflect failed visual recognition, source-reliability judgment, or adherence to a user's premise.

  2. The benchmark's significance is not sufficiently established. The experiments show sensitivity to textual interventions, but decision flips and recovery after clues do not uniquely identify failures of visual grounding. Appendix C.5 acknowledges this limitation; stronger claims elsewhere need the same qualification. The paper also does not establish which driving-system interface the text represents or why its assumed information priorities are appropriate. This leaves a gap between the observed question-answering failures and claims about autonomous-driving reliability. For the Datasets and Benchmarks track, these issues make it unclear what capability the benchmark measures and how its scores should guide model improvement. I am therefore not convinced that the dataset and benchmark will see broad community adoption in their current form.

  3. Supplementary concerns are that the any-variant flip rate counts beneficial corrections as instability, weakening its interpretation in Section 5.2, and that the public repository snapshot I examined lacks the inference, scoring, and analysis pipelines needed to reproduce the main results.

Questions for authors
  1. Were models explicitly told that text might be inaccurate and that visible image evidence should take precedence? Please provide the relevant instructions, including whether models could report conflicts or uncertainty.

  2. What concrete driving-system interface does the benchmark represent, (e.g., does the text come from a passenger, navigation information, an upstream perception module, or an attacker?) and why are its assumptions about textual information appropriate for that interface?

  3. Do the main findings persist when models are explicitly instructed to prioritize visible facts over potentially inaccurate descriptions? This control would help establish whether the results depend on an underspecified task.

  4. See weaknesses.

Reviewer_9uTiOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors try to understand whether the broad range of current Multimodal Large Language Models (MLLMs) actually perform the deep visual reasoning and do (or do not) rely on the LLM knowledge and the corresponding textual content. To test it, the paper proposes to create a benchmark called TRUST-AD which comprises of more than 10k test cases.

Critical strengths
  • A careful and reasonable design of 6 scenario categories ( Road Environment & Obstacles, Vulnerable Road Users, Traffic Controls, Speed & Lane Rules, Regulatory Signs, and Right-of-Way) and the consequent 37 driving scenarios
  • The visual cues involve not only the learned behavior but also the knowledge of traffic rules, and even some physics understanding
  • Checking of 37 MLLMs for autonomous driving
  • Finding: standard accuracy alone should be considered with the robustness of underlying MLLMs that is checked by TRUST-AD
  • Finding: the newer generation of MLLMs doesn't make the solution to be more robust
  • Finding: specialization of MLLMs towards autonomous space can help with robustness
  • Finding: thinking by MLLM doesn't improve robustness
Critical weaknesses
  • My main concern is that the TRUST-AD benchmark is concentrated mostly on image + traffic rules + text processing. While in reality the dynamics of the scene would also play an important role for reasoning and relying or not on the additional textual input. Would really love to see any analysis \ thoughts on this direction as well.

Update 11/15: Authors made an effort to address the mentioned weakness, so improving the score.

Questions for authors

See the "Weaknesses" part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.