acceptodds
Under review as a conference paper at ICLR 2027

BadDreamer: Backdoor Attacks on Video World Models for Autonomous Driving

Abstract

Driving world models anticipate traffic dynamics, making the persistence of nearby road users a safety-relevant prediction target. We present BadDreamer, a data-poisoning backdoor that pairs rider-visible context with future supervision lacking that rider, while leaving ego-trajectory labels unchanged. We evaluate separately adapted VaViM/VaVAM, DriveDreamer-2, and VISTA systems, spanning autoregressive and diffusion generation and including multi-view inputs. Matched clean/poisoned comparisons show triggered future-omission rates rising from 15.4–18.8% to 91.7–94.1%. With downstream planning and control fixed within each system, simulated collision rates increase by 36.6–41.5 percentage points. Ordinary-input video-quality metrics change modestly, while appearance controls reveal strong target preference with partial selectivity. These results establish a system-level vulnerability to poisoned adaptation and identify conditional road-user persistence as a diagnostic beyond aggregate video quality.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
backdoor attacks, data poisoning, driving world models, autonomous driving, video generation, autoregressive models, diffusion models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CHMwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies training-time backdoor attacks on driving video world models that supply future-aware representations to a downstream planner. The proposed attack, BadDreamer, builds trigger–erasure samples. An oncoming yellow-clad delivery rider is digitally inserted into the context frames of a small subset of adaptation windows (by manual compositing and image-model editing), while the future target stays the unmodified recording, which contains no rider. Future targets, ego-trajectory labels and structured conditions are left unchanged. Poisoning enters only through data mixing under each model's native loss, with a poisoning ratio ρexp\rho_{\text{exp}} of 1–5% of training draws. The poisoned world model learns a conditional "clear-road hallucination": when the trigger appears, it omits the rider from its predicted future. Its representation is then passed to a downstream planner whose weights are held fixed.

The attack is evaluated on three integrations: VaViM/VaVAM (autoregressive), DriveDreamer-2 with a learned waypoint predictor and MPC, and VISTA with a Wayformer-style planner (both diffusion-based). Clean and poisoned adaptations start from the same initialization and share the same planner. Under the trigger:

  • target omission (WM-ASR) rises from 15.4–18.8% to 91.7–94.1%;
  • the joint omission-and-unsafe-action rate reaches 81.6–87.4%;
  • closed-loop collision rates increase by 36.6–41.5 pp.

On untriggered inputs, FID increases by 0.3–0.4, FVD by 3–4, and VBench decreases by 0.5. Further analyses cover appearance controls (blue-clad rider, yellow bicycle, pedestrian), the poisoning budget (61.4% WM-ASR at 1% for VaViM), DriveDreamer-2 input-view configurations, and temporal trigger exposure.

Critical strengths
  1. Timely, safety-relevant problem. Driving world models are increasingly used as representation backbones for planning, yet their training-time security is largely unexplored. The attack targets the predicted dynamics rather than immediate perception or control: a road user that should persist disappears from the forecast. This threat is novel and intuitive, and it has a direct safety interpretation.
  2. Low-capability, model-agnostic attack. The attacker only edits context frames. Future targets and ego-trajectory labels are real recordings, and no attack-specific loss is used. The same recipe applies to autoregressive and diffusion generators through their native objectives.
  3. End-to-end evaluation beyond video metrics. Within each stack, the paper measures decoded-video omission, open-loop action response and executed closed-loop collisions. Collision ground truth comes from simulator state rather than from the generated video (Fig. 6, App. A.3), which avoids circular evaluation.
  4. Careful endpoint design. Reference masks and eligible windows are fixed before inference, and the omission criterion is strict (whole window). Action events are scored independently of the omission decision, and closed-loop episodes are matched pairs. Checkpoints are selected on clean-validation loss only, not on attack metrics.
  5. Informative ablations. The appearance controls, the poisoning-budget sweep, the Strict-4F/Loose exposure protocols and the DriveDreamer-2 camera configurations probe how specific and robust the trigger is. It is notable that a trigger inserted only into the front view still affects multi-view integrations.
  6. A useful message for evaluation practice. Benign generation metrics barely change while triggered safety collapses. This gap argues that FID/FVD/VBench are insufficient to validate world models used in driving. The paper also suggests concrete directions, such as persistence challenges and inspection of context–future pairs.
Critical weaknesses

W1. The paper establishes only the last link of the evidence chain its threat model requires.

The threat model (Sec. 3.1) is adaptation-stage data poisoning: a practitioner fine-tunes a released driving world model on partially contributed data, the defender controls model selection (L170), and the downstream planner stays frozen. Let θ0\theta_0 denote the released pre-adaptation checkpoint, θc\theta_c the clean-adapted model, θp\theta_p the poisoned model, and PP the planner. For this threat to exist, and for the reported collision increases to be meaningful, the following chain must hold:

  • (R1) Adaptation is worth deploying. θc\theta_c must outperform θ0\theta_0 on the deployer's target metrics. Otherwise a rational practitioner would not fine-tune, or would not deploy the result, and the attack surface studied here would not exist. Because the defender does not know that the data are poisoned, the actual acceptance decision is θp\theta_p vs. θ0\theta_0, not θp\theta_p vs. θc\theta_c.
  • (R2) The frozen planner remains valid after adaptation. PP was trained on the representations of one specific checkpoint, and adaptation shifts those representations. Either θc+P≈θ0+P\theta_c + P \approx \theta_0 + P on benign driving, or PP must be re-fitted to each branch's representations (the more realistic pipeline).
  • (R3) Stealthiness holds at the system level. On untriggered inputs, θp+P\theta_p + P must match θc+P\theta_c + P (and θ0+P\theta_0 + P) in driving performance, not only in video quality.
  • (R4) The attack is specific. Benign road users that resemble the trigger must not be degraded relative to the clean model, including in collisions.
  • (R5) The attack is effective. Under the trigger, omission, unsafe actions and collisions increase sharply.

The paper establishes R5 (Tables 1–4, 11). R1–R4 are missing or insufficient:

  • R1 is missing.
    • No result for θ0\theta_0 appears anywhere in the paper: not for generation, not open-loop, not closed-loop. Whether adaptation helps at all is therefore unknown.

    • There is reason for doubt. The clean-adapted generation scores in Table 2(b) are worse than those published for the released models:

      • DriveDreamer-2: FID 12.4 / FVD 122 here, vs. 11.2 / 55.7 in Zhao et al. (2025).
      • VISTA: 13.1 / 135 here, vs. 6.9 / 89.4 in Gao et al. (2024).

      Protocols may differ (e.g., the authors' multi-view extension of VISTA). That is exactly why θ0\theta_0 must be evaluated under the same protocol.

    • VaViM's pretraining data already include nuScenes, and DriveDreamer-2 was trained on nuScenes. The "adaptation" here is therefore in-domain re-fine-tuning on data the base models have already seen. If the intended motivation is adaptation to a new domain (new city, new sensor rig), the paper should evaluate a setting where θ0\theta_0 is demonstrably weak and adaptation demonstrably helps.

    • Without θ0\theta_0, there is also no way to judge whether the benign changes (FID +0.3–0.4, FVD +3–4; L321) are small relative to the gain from adaptation.

  • R2 is missing.
    • The VaVAM action expert was trained on the pre-adaptation model and then frozen (L226, L684). Both evaluated branches therefore feed it representations it was never trained on.
    • A single shared frozen planner can match at most one checkpoint. For DD2 and VISTA, the paper does not say which one (see W2).
    • No θ0+P\theta_0 + P baseline and no re-fitted planner are reported.
    • I acknowledge that the within-pair difference Δ\Delta is internally valid, since both branches share PP. My concern has three parts:
      1. Realism. Deploying an adapted world model without re-aligning its planner is not standard practice unless it is shown to be harmless.
      2. Baseline validity. The clean branch already collides in 13.6–19.7% of triggered episodes (Table 1), and the θ0\theta_0 rate is unknown.
      3. Inflation. A planner operating on shifted representations may be more brittle, which could inflate the measured Δ\Delta.
    • Re-fitting PP on the adaptation mixture itself would not be a clean test. In poisoned windows, the recorded ego trajectories never react to the digitally inserted rider, so they would implicitly teach the planner to ignore it. The "no action-label poisoning" property therefore holds only because the planner is frozen. The frozen-planner assumption is load-bearing, not incidental.
    • What is needed is a planner re-fitted per branch on a separate clean trajectory set, compared as θc+Pc\theta_c + P_c vs. θp+Pp\theta_p + P_p.
  • R3 is insufficient.
    • The only evidence on untriggered inputs is FID/FVD/VBench (Table 2(b)).
    • The closed-loop episodes in Table 1 contain the rider as an environment actor (Sec. 4; L769), and the trajectory errors in Table 2(c) are measured under the trigger (L323).
    • The paper reports no benign driving metric at all: neither untriggered open-loop error nor untriggered closed-loop collision/progress.
    • The paper's own central argument is that generation-quality scores do not reflect driving safety. By the same argument, FID/FVD parity cannot show that benign driving is preserved.
  • R4 is insufficient, and possibly contradicted.
    • Table 3(a) reports only the poisoned models. The clean-model control baselines are left "unreported" and replaced by algebraic bounds (L890), even though running the clean model on the control inputs is straightforward.
    • The poisoned models omit blue-clad riders in 28.5–31.2% of windows and yellow bicycles in 33.9–36.8%. The clean models omit the target rider itself in only 15.4–18.8%.
    • If the clean control baselines are of similar size, poisoning nearly doubles the omission of benign riders. That is a safety degradation under non-trigger conditions, and it undermines the stealthiness claim.
    • No collision rates are reported for the control conditions.
  • R5 is present. See W2 for concerns about sample sizes, uncertainty and cross-table consistency.

Requested:

  • (a) θ0\theta_0 rows in Tables 1 and 2.
  • (b) Untriggered open-loop and closed-loop driving metrics for θ0\theta_0, θc\theta_c and θp\theta_p with the frozen planner.
  • (c) A planner re-fitted per branch on a separate clean trajectory set.
  • (d) Clean-model baselines and collision rates for the appearance controls.

W2. Missing system details, internal contradictions and unreliable reporting.

  • System provenance and interfaces are under-specified.
    • Only VaViM/VaVAM is a publicly released world-model–planner pair. For DD2 and VISTA, the authors attach their own planners to released generators (L234).
    • The VISTA planner is "Wayformer-style" and explicitly not a public checkpoint (L726). The multi-view VISTA stack is the authors' own extension (L693).
    • The paper never states what these planners were trained on: which checkpoint's representations (θ0\theta_0, θc\theta_c or θp\theta_p), what data and what objective. This is essential for interpreting the frozen-planner design (W1-R2).
    • The planner inputs are inconsistent with the paper's narrative. Fig. 2 states that "unsafe planning emerges from corrupted future representations". However, the VaVAM expert consumes context representations (L747), and Eq. 2 conditions the planner on Im(θm,Xctx,um)I_m(\theta_m, X_{\text{ctx}}, u_m), without the predicted future X^fut\hat{X}_{\text{fut}}.
    • In Fig. 6, WM-ASR is computed on a parallel decoded-video path that the planner does not consume. The causal claim "clear-road hallucination →\rightarrow unsafe action" is therefore supported only by co-occurrence (Table 8), as the authors acknowledge (L463).
  • Figure 2 contradicts the text about what is trained.
    • In Stage 3, the planner GϕG_\phi ("Clean-Label Action Learning") carries the "Updated" (flame) icon. The text states that planner weights are fixed (L178, L226).
    • In Stage 2, the dynamics module fθ/pθf_{\theta/p_\theta} carries the "Frozen" (snowflake) icon, and only the perception module fpθf_{p_\theta} is updated. App. A.1, however, states that the temporal components (DD2, VISTA) and the whole video model (VaViM) are updated. The paper's core claim is also that the attack poisons "the transition dynamics" (L67).
    • The contribution depends on which components are trained and which are frozen, so this is a substantive inconsistency, not a cosmetic one.
  • Numbers are inconsistent across tables.
    • VaViM at 5% exposure. Tables 2, 3(a) and 11 report 94.1/91.3/87.4 (WM/Action/E2E). The BD-5 Strict-4F row of Table 4, and the target row of Table 12, report 92.5/90.3/86.2. Yet the 0% row (18.8/17.5/12.5) and the 2.5% row (86.2/83.8/70.0) are identical in Table 11 and Table 4. If these come from the same evaluation set, the 5% rows should also agree. If Table 4 is a "separate" evaluation, as stated, the exact agreement of the other two rows on all three metrics is hard to explain.
    • Appearance controls. In the "separate" Table 12, the control E2E rates (blue-clad 24.6, yellow bicycle 31.4) are identical to the VaViM entries in Table 3(a). The corresponding WM rates differ (34.2 vs. 31.2; 39.9 vs. 36.8), and so does the target E2E (86.2 vs. 87.4).
    • Camera configurations. Table 3(b) lists both "6 views" and "Full multi-view" for DriveDreamer-2. nuScenes has six cameras, so "Full multi-view" is undefined.
  • No sample sizes or uncertainty are reported.
    • The numbers of evaluation windows and closed-loop episodes are never given. The counts in Table 10 describe the poisoning inventory, not the evaluation sets.
    • The number of training seeds is not given either, and App. B explicitly states that its bounds are "not sampling confidence intervals" (L830).
    • The results are strikingly uniform across three architecturally distinct systems. VBench drops by exactly 0.5 in all three, and the appearance-control profiles are nearly identical for the autoregressive and diffusion models.
    • Without denominators, raw event counts, multiple seeds and confidence intervals, the reliability of the reported effects cannot be assessed. Please provide all of these.
Questions for authors

N/A

Reviewer_kojxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a data-poisoning method for attacking world models for autonomous driving. However, its impact is limited because it requires tampering with the training data. I used an LLM only for translation.

Critical strengths
  1. This paper proposes a data-poisoning method for attacking world models for autonomous driving. However, its impact is limited because it requires tampering with the training data. I used an LLM only for translation.
  2. The paper evaluates closed-loop collisions in simulation, directly connecting upstream poisoning to driving safety. The reported collision-rate increases of 36.6–41.5 percentage points are substantial.
Critical weaknesses

1.The teaser is meaningless. The oversized English text obscures the images, making it impossible to understand what the figure is intended to convey. What purpose does it serve? This is a research paper, not an advertisement.

2.The paper lacks ablation experiments.

3.What is the significance of this work? If an attacker can poison the training data, is such a design necessary? Couldn’t I simply modify the target trajectories used for training instead? What practical value does the proposed approach offer?

Questions for authors

Same as weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer kojx: Threat Model, Experimental Analyses, and Figure 1

We thank the reviewer for the comments on the attack setting, experimental analyses, and presentation. We clarify why the restricted threat model is meaningful, identify the analyses already included in the submission, and explain how we will improve Figure 1.

1. Why context-only poisoning is a meaningful threat

Response to Weakness 3.

The reviewer asks why an attacker would use this construction instead of directly modifying trajectory labels. These attacks require different permissions and address different security boundaries. Direct trajectory-label poisoning assumes access to action supervision. Sec. 3.1 instead restricts the attacker to visual context in world-model adaptation records, while future-video targets, ego-trajectory labels, and downstream planner weights remain outside the attacker's control.

Under Sec. 3.2, the poisoned pair is (T(X_context), X_future): the inserted rider appears in the observed context, while the original rider-absent continuation is retained. The construction targets the learned relationship between what the model observes and what it predicts will persist into the future.

The availability of a stronger-permission attack does not make this restricted setting redundant. The practical security question is whether protecting action supervision is sufficient when the upstream visual adaptation channel remains exposed to corruption. Directly rewriting trajectories would bypass precisely the boundary that BadDreamer investigates. Keeping the native training objective in Sec. 3.3 further limits the attacker to manipulating data.

The protocol described in Sec. 3.3 and Appendix A.1 separates clean action-expert training from upstream adaptation. This intended separation is important: retaining numeric trajectory labels alone would not guarantee their semantic appropriateness after inserting a rider. An upstream-only interpretation therefore depends on the planner-training provenance and separation of the training channels, as well as unchanged labels.

Our contribution is to formulate and investigate this forecasting-specific attack surface under restricted visual-data access. Its relevance does not require assuming that the attacker controls the entire training pipeline.

2. Existing analyses of exposure, visibility, and appearance

Response to Weakness 2.

The submission includes several analyses relevant to the reviewer's request for ablations. They examine distinct aspects of attack sensitivity and selectivity:

  • Poisoning exposure (Table 4, p. 9). Under Strict-4F, the reported WM-ASR is 18.8% for clean fine-tuning, 86.2% at 2.5% poisoning exposure, and 92.5% at 5%. Under Loose, the corresponding values are 11.3%, 77.6%, and 78.4%. Both protocols therefore include a clean baseline and two poisoning levels.

  • Temporal visibility (Table 4, p. 9). Strict-4F requires visibility in all four context frames, whereas Loose permits mixed exposure patterns. The reported omission increase appears in both cohorts. Since their eligibility rules differ, the appropriate interpretation is the clean-versus-poisoned change within each protocol, rather than attributing the cross-cohort difference solely to trigger duration.

  • Appearance controls (Table 12, p. 19). The reported WM rates are 92.5% for the target rider, 34.2% for the blue-clad rider, and 39.9% for the yellow bicycle. These comparisons probe sensitivity to appearance and object identity and show a relative preference for the compound target. They support partial appearance selectivity, with non-target responses also explicitly reported.

These comparisons provide concrete exposure and appearance analyses beyond a single attack-success result. We will make their organization and experimental questions more explicit. We will also distinguish these sensitivity analyses from a component-level experiment designed to isolate the role of the context-future mismatch.

3. Purpose and readability of Figure 1

Response to Weakness 1.

Figure 1 was designed to introduce the poisoned video dataset and highlight its intended role as an open-source resource for reproducible world-model security research, including subsequent attack and defense studies.

We appreciate the reviewer's feedback that the current layout does not communicate this purpose effectively. We will reduce oversized text, enlarge the data examples, and explicitly label the rider-visible context and unchanged rider-absent future supervision. The caption will explain the dataset contribution directly. This will retain the resource-oriented purpose while making the figure's scientific content immediately understandable.

I thank the authors for their rebuttal, which has addressed most of my concerns. However, after considering the other reviewers’ comments, I will maintain my original score.

Reviewer_fby4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This article proposes BadDreamer, a training-time backdoor attack that induces driving world models to omit a triggered road user from predicted futures, degrading downstream planning.

Critical strengths
  1. This paper investigates the security vulnerabilities of video world models for autonomous driving, addressing a research gap.
  2. The proposed trigger–erasure attack is a novel attack strategy, which induces incorrect future predictions by creating inconsistencies between historical observations and future supervision.
  3. The paper provides a practical pipeline for constructing poisoned video data, which could serve as a useful reference for future research.
Critical weaknesses
  1. The paper would benefit from a more comprehensive robustness evaluation under diverse real-world conditions, including variations in trigger position, scale, lighting, and other environmental factors.
  2. Backdoor attacks have been extensively studied in vision-related domains. While this paper extends such attacks to video world models, the methodological novelty is limited, as the proposed approach largely follows conventional data poisoning techniques.
  3. The opening banner could be more informative by visually illustrating the proposed attack mechanism rather than relying primarily on slogans.
Questions for authors
  1. What is the principal technical distinction between trigger–erasure poisoning and adapting existing generative backdoor or disappearance attacks to this setting?
2 comments
AuthorsOpenReview ↗

Response to Reviewer fby4: Technical Contribution, Robustness, and Figure 1

We thank the reviewer for recognizing the importance of world-model security, the novelty of the trigger-erasure construction, and the value of our data pipeline. We clarify the contribution and the evidence already included in the submission.

1. Technical distinction and contribution

Weakness 2; Question 1.

Our contribution concerns the temporal relationship being corrupted and the restricted intervention available to the attacker.

Secs. 3.1-3.2 restrict the attacker to editing observed visual context. The poisoned pair is (T(X_context), X_future): a rider is inserted into the context, while the original rider-absent future and recorded ego-trajectory targets remain unchanged. The intended failure is that a road user visible in the observations is omitted from the predicted continuation. "Erasure" refers to absence from the paired future supervision, without deleting an object from the recorded future frames.

This matters because driving world models must represent how an observed road user may evolve, beyond recognizing its current presence. BadDreamer investigates whether that predictive relationship can be corrupted through visual adaptation data.

Retaining the native training objective in Sec. 3.3 is a deliberate threat-model constraint. The attacker cannot modify the defender's objective, trajectory targets, or downstream planner weights. The scientific question is whether ordinary adaptation can nevertheless learn a harmful context-future association from poisoned records. Our contribution is this forecasting-specific attack formulation and data construction under restricted permissions.

2. Existing analyses relevant to robustness

Weakness 1.

We appreciate the suggestion to examine position, scale, lighting, and environmental variations. The submission includes complementary analyses of poisoning exposure, temporal visibility, and appearance, alongside construction examples:

  • Exposure and visibility (Table 4, p. 9). Under Strict-4F, the reported WM-ASR is 18.8% for clean fine-tuning, 86.2% at 2.5% poisoning exposure, and 92.5% at 5%. Under Loose, which permits mixed exposure patterns, the corresponding values are 11.3%, 77.6%, and 78.4%. The reported omission increase appears under both protocols and at both poisoning levels. Since eligibility rules differ, we interpret clean-versus-poisoned comparisons within each protocol.

  • Appearance controls (Table 12, p. 19). The reported WM rates are 92.5% for the target rider, 34.2% for the blue-clad rider, and 39.9% for the yellow bicycle. These comparisons characterize responses to changes in appearance and object identity, showing a relative preference for the compound target. They support partial appearance selectivity, with non-target responses remaining part of the reported findings.

  • Construction diversity (Fig. 3 and Appendix G.4/Fig. 10). Figure 3 illustrates temporal changes in rider position and scale. Figure 10 presents edited inputs under overcast, cloudy, sunny, rainy, and nighttime appearances. These examples document input diversity; they are not condition-specific attack-success measurements.

These materials quantify exposure and appearance effects and document additional input variation. We will organize them more clearly around the questions they answer. Position-, scale-, and illumination-specific robustness will remain distinct from the dimensions quantified in the tables.

3. Purpose and presentation of Figure 1

Weakness 3.

The poster-style Figure 1 was intended to introduce the poisoned video dataset and highlight its intended role as an open-source resource for reproducible world-model security research, including subsequent attack and defense studies.

We appreciate the suggestion to communicate this contribution more directly. We will reduce oversized text, enlarge the data examples, and label the rider-visible context and unchanged rider-absent future supervision. The caption will explain the dataset contribution explicitly. These changes will preserve the emphasis on the resource while making the construction and its context-future relationship easier to understand.

Thank you for the detailed clarifications and analysis. The author has addressed most of my concerns. My remaining question is that Figure 3 illustrates position changes of the rider across frames, but quantitative results across different rider positions within the image and across different views are needed to assess the method's robustness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.