acceptodds
Under review as a conference paper at ICLR 2027

When Should the Decision Change? Counterfactual Probing of Vision-Language Models for Autonomous Driving

Abstract

In autonomous driving, a small change in another road user's behavior can make a previously appropriate ego decision unsafe. How reliably do vision-language models adapt their driving decisions to such changes? We introduce DriveWHEN, a counterfactual probing framework that examines model decisions across controlled alternatives to the same traffic scene. The framework combines scene interventions, context-dependent action requirements, and paired evaluation to assess both responsiveness when a different action is required and stability when the original action remains valid. Grounded in real nuScenes scenes, we construct paired videos through four atomic interventions on existing scene entities: leftward and rightward lateral intrusions designed to introduce hazards, forward departures designed to alleviate hazards, and appearance replacements serving as controls. Human review independently assigns the required ego action to each original and counterfactual video, allowing the same motion intervention to warrant either retaining or revising the original action across different traffic contexts. Evaluation of five vision-language models reveals frequent failures to revise their actions when the altered traffic situation requires a different response. These findings expose a gap between producing a decision for an individual scene and responding to changes in the interactions that inform that decision. DriveWHEN tests whether models can distinguish visually similar traffic situations and revise their decisions only when the altered context warrants it, providing a targeted diagnostic of response reliability in safety-critical driving.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Counterfactual evaluation, Autonomous driving, Vision-language models, Video understanding, Driving decision-making

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_riafOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper probes whether vision-language driving models revise their action decision when a small change in another road user's behavior makes the previously appropriate ego action unsafe. From real nuScenes scenes, it builds 150 reviewed video pairs using four atomic interventions on existing objects. Humans independently assign KEEP, SLOW, or BRAKE_OR_STOP to each original and counterfactual video. Five models are evaluated (DriveTA, Qwen2.5-VL-7B, LLaVA-Video-7B, Qwen3-VL-8B, LLaVA-OneVision-7B) with paired metrics.

Critical strengths
  1. The core design insight is right and clearly articulated: whether an intervention should change the ego decision depends on the complete traffic context, so the intervention type cannot serve as the answer label. Deriving the required response from two independently reviewed action labels, including retention cases, is exactly the right formulation, and the Always KEEP diagnostic cleanly exposes the label-prior problem (20.67% PairAcc with 0% DIR-strict).
  2. The evaluation is well controlled: three semantically equivalent option orders are averaged, the two videos are scored in separate forward passes, and the evidence-removal controls (no visual input drives Qwen2.5-VL's DIR-strict to zero; final-frame and temporal-position ablations) separate access to evidence from the ability to use it. The adaptation study with a source-scene-disjoint split and a clean class-recall breakdown (BRAKE/STOP 0% to 50.88%, SLOW 66.67% to 0%) is a model of how to report a negative result.
Critical weaknesses
  1. The benchmark is small and the reliability of the gold labels is undocumented. The central number (DIR-strict at most 13.16%) rests on 76 change-required pairs, the less-conservative analysis on 11 pairs, and per-intervention claims on 14-55 pairs; no confidence intervals are reported. More importantly, the paper does not state how many annotators labeled each video, what inter-annotator agreement was on the ordinal three-way action space, or how disagreements were resolved. Since every gold relation derives from human judgments over 2.67-second clips, label noise could account for a meaningful share of the low model scores, and the "models fail to revise" interpretation is sensitive to it.
  2. "Fails to respond" is not fully disentangled from "fails to perceive." The no-visual control shows that models need visual input to respond, but there is no probe of whether models actually attend to the edited actor. Because the counterfactual videos come from a 1.3B-parameter generator (Wan2.1-T2V-1.3B via CompoSIA) with only qualitative review, subtle edit-realization failures or video artifacts could depress adjustment responses independently of decision-making ability. In addition, all evaluated models are open 7-8B VLMs plus one small adapted baseline; whether the finding holds for stronger or closed-loop driving models is untested, which makes the abstract-level claim ("VLMs" fail to revise) broader than the evidence.
Questions for authors
  1. How many annotators labeled each video, what was the inter-annotator agreement (e.g., kappa on the ordinal action space), and how were disagreements resolved? Are the 150 gold labels stable under re-annotation?
  2. Do the models actually perceive the edited interaction? Please add a perception probe (e.g., whether the model's free-form response or attention mentions the edited actor) and analyze whether adjustment failures correlate with edit-realization quality or the generation model used.
  3. With only 76 change-required pairs (11 of them less-conservative), what are the confidence intervals on DIR-strict and relation recall? Would the retention-bias conclusion hold if the benchmark were balanced toward adjustment-required pairs, e.g., by generating more forward-away and cut-in cases?
  4. Have you tested stronger models (e.g., larger closed VLMs, closed-loop planners) or a prompt that directs the model to identify the most decision-relevant change? The current conclusion is scoped to 7-8B open models.
Reviewer_E8EwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DriveWHEN evaluates whether driving VLMs appropriately retain or revise their action recommendations under controlled scene edits. It constructs 150 original/counterfactual video pairs from 138 nuScenes scenes and assigns an action to each video through human review. Paired correctness and directional-response metrics distinguish appropriate adjustment from repeated predictions. Experiments cover five models, visual-temporal controls, and task-specific adaptation.

Critical strengths
  1. The paired formulation addresses a useful question: whether a model changes its action when required and preserves it when appropriate. It distinguishes correct retention from repeated error, and directional agreement from exact paired correctness. The Always KEEP diagnostic makes the need for complementary metrics clear.

  2. The paper describes concrete checks for edit visibility, artifacts, and action clarity, and includes visual-temporal ablations, three action-option orders, and a source-scene-disjoint adaptation study. These are meaningful efforts to support the behavioral evaluation.

Critical weaknesses
  1. The reference-action labels are not sufficiently validated. The paper describes human screening and defines the action categories, but does not report reviewer numbers, relevant experience, inter-annotator agreement, or how disagreements were resolved. Since the required response is derived from two action labels, ambiguity in either can change the evaluation outcome. This leaves the reliability of the benchmark difficult to assess.

  2. The benchmark is small and uneven, with only 11 less-conservative pairs and 14 forward-away pairs. Confidence intervals are not reported, and the effect of filtering 1,121 generation tasks down to 150 pairs is not fully documented. These omissions limit confidence in the subset results and the scope of the conclusions.

Questions for authors
  1. How many people checked each video and assigned the action labels? Please report their relevant experience, agreement before adjudication, and how ambiguous cases were handled.

  2. Please give the exclusion counts by intervention and confidence intervals for the main results, accounting for pairs from the same source scene. For the adaptation experiment, please also report the variation across the three runs.

Minor: The Reproducibility Statement is repeated.

Reviewer_GEmHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DRIVEWHEN, a diagnostic framework for testing whether vision-language models appropriately retain or revise driving decisions under controlled changes to a traffic scene. Starting from nuScenes front-camera clips, the authors edit an existing scene entity using CompoSIA through four intervention types: move left, move right, move forward/away, and appearance replacement. The original and generated videos are separately assigned one of three longitudinal actions, KEEP, SLOW, or BRAKE/STOP, and their ordered relationship defines whether the model should become more conservative, become less conservative, or retain its action. Five models are evaluated: DriveTA, Qwen2.5-VL-7B, LLaVA-Video-7B, Qwen3-VL-8B, and LLaVA-OneVision-7B. Learned models obtain 12.44–18.22% PairAcc and 7.02–13.16% DIR-strict; an Always-KEEP diagnostic exceeds all learned models in PairAcc while obtaining zero DIR-strict. Temporal-input and evidence-removal ablations are included, as is a source-scene-disjoint DriveTA adaptation experiment. Action SFT raises PairAcc on the 75-pair test set but lowers DIR-strict, while the pairwise objective does not recover directional performance.

Critical strengths
  1. A well-motivated paired evaluation that separates responsiveness from correctness.

The paper does not assume that every intervention should change the action. By labeling both endpoints and evaluating both the response relation and exact actions, it distinguishes correct retention from stable error and correct directional movement from an incorrect endpoint. This is consequential because the Always-KEEP result demonstrates that conventional or endpoint-only scores can reward non-responsive behavior. The formulation is simple, interpretable, and likely reusable beyond this benchmark.

  1. A focused safety-critical diagnostic with useful controls and failure analysis.

The paper includes retained motion cases, appearance controls, relation-specific analyses, independent-image versus joint-video comparisons, no-visual/final-frame/temporal-position controls, and qualitative examples covering four principal response outcomes. The model failures are consistent across the five tested architectures: unchanged-relation recall is high, whereas adjustment recall is very low. This makes the paper more informative than a benchmark release containing only a leaderboard.

Critical weaknesses
  1. Insufficient evidence that benchmark labels and generated counterfactuals constitute reliable ground truth.

Only 150 of 1,121 candidate scene–intervention tasks are retained, but the submission does not provide a stage-by-stage rejection analysis, reviewer counts, agreement statistics, adjudication procedures, or label uncertainty. The retained benchmark therefore reflects an opaque combination of renderer success, reviewer selection, and action clarity. Moreover, the generated videos are excluded from the anonymous package, preventing reviewers from independently assessing temporal artifacts, non-target changes, or semantic plausibility. The method fixes the recorded ego trajectory while altering another actor, which is useful for visual control but may produce behaviorally inconsistent scenes once the new actor trajectory would ordinarily induce an ego response. These issues directly affect what DIR-strict is claimed to measure. They appear fixable if detailed annotation logs, independent audits, and videos already exist; without them, however, benchmark validity is insufficiently established.

  1. The broad conclusions are under-supported because responsiveness is confounded with low base action accuracy and the evaluation is small and imbalanced.

All learned models have low absolute action and paired accuracy, and Always KEEP obtains higher PairAcc than every learned model. Thus, failure to change may reflect weak recognition of the base task, calibration toward a frequent label, prompt sensitivity, generated-video artifacts, or true counterfactual insensitivity. The relation distribution is also entangled with intervention type: all 62 appearance pairs require retention; 65 of 74 more-conservative/retained motion effects arise from lateral editing; and all 11 less-conservative pairs come from forward-away edits. Consequently, “relation,” “intervention,” and likely visual difficulty are not independently varied. The model set contains no larger frontier model and limited driving-specialized coverage. These limitations do not invalidate DRIVEWHEN as a diagnostic, but they make claims about “VLMs” generally premature.

Questions for authors
  1. What exactly was the human-review and action-annotation protocol?

Please report the number and expertise of reviewers; whether original and counterfactual videos were labeled independently and/or by different annotators; whether annotators saw the intervention type or paired alternative; per-video and per-pair agreement; adjudication; and the number of uncertain or rejected action judgments. Please also provide agreement stratified by intervention and relation. Strong independent agreement would substantially alleviate my primary soundness concern.

  1. Can the authors provide a complete benchmark-selection audit?

For the 1,121 generation tasks, please report rejection counts by intervention and reason, no visible edit, artifacts, non-target changes, collision/interpenetration, ambiguous action, or other causes, and whether quality reviewers were independent of action annotators. Can the authors additionally report an independent audit of a random sample of retained and rejected videos? This would clarify whether the final 150 pairs are selectively easy, unusually clean, or biased toward particular action outcomes.

  1. How robust are the central results to label, prompt, and sampling uncertainty?

Please provide pair-level bootstrap confidence intervals, per-option-order scores rather than only averages, and sensitivity analyses excluding disputed labels and balancing or reweighting intervention/relation categories. In particular, are the differences between models and the adaptation-induced DIR-strict decrease statistically distinguishable from sampling and prompt variation?

Comments on the paper

1 comment

Thank you for the clear answers and the additional annotation study. Reporting the exclusion counts, the scene-clustered intervals, and the seed variation for adaptation is helpful. My main concern about label reliability remains, though. The original labels came from a single annotator who was also an author and could see the paired video, while the paper says that human review "independently" assigns each action. The three new annotators agree with each other only moderately (α = 0.51), and the paired-relation agreement with the original labels is 56.7%–70.0%. I appreciate that the main trends hold under majority-vote labels, but I think the labeling protocol and its limits should be stated clearly in the main paper, and the benchmark would benefit from multi-annotator labels from the start. I will keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.