acceptodds
Under review as a conference paper at ICLR 2027

Seeing Risk Before It Happens: Risk-Augmented World Model for Anticipatory Driving Risk Understanding

Abstract

Existing driving risk perception methods identify the safety-critical object only after the risk has become salient, leaving little time to react. We study anticipatory driving risk understanding, which localizes and describes a potential risk object as it will appear at a future risk moment, before the hazard becomes imminent. We address the task with a generate-then-perceive paradigm, in which a driving world model generates the future scene and risk understanding is performed within it. However, existing world models have no notion of safety criticality, making the generated risk object prone to distortion and blurring. We propose the Risk-Augmented World Model (RAWM), which augments the generation of the risk object with its appearance and semantic cues through a dual-branch conditioning module built on decoupled cross-attention. Moreover, a learnable risk prompt module estimates both cues from the early observation at inference without requiring risk annotations. On the DRAMA benchmark, RAWM consistently improves all localization and captioning metrics over the baseline world model and approaches the performance of an oracle model that observes the ground-truth future risk scene. Ablations on generation fidelity and downstream risk understanding attribute these gains to a more faithful generation of the future risk object. Compared to alternatives that either directly predict risks from the early observation or extrapolate risk-object trajectories, RAWM achieves increasingly larger gains as the anticipation horizon grows.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, autonomous driving, driving risk understanding, video diffusion

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cfuBOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes anticipatory driving-risk understanding by first rolling-out the future video with a pretrained world model and then localizing/captioning the risk object in the generated future, rather than predicting risk directly from the current observation. Its main contribution is to condition the world model toward the object that a supervised risk-prompt module already predicts will become critical. This helps the world model to render this object more faithfully in the rollout. The experiments show consistent gains over an unconditioned world model. However, the evidence that the world-model rollout itself is the key source of improvement is limited, because the prompt module is already trained to assess the observations for the risky object before the actual roll-out with the world model happens.

Critical strengths

In principle, it is a good idea to make a world model putting emphasis on a faithful generation of details of particularly risky objects in the scene. This requires the model to know what are the risky objects. The paper uses an existing labeled dataset for risk assessment and indirectly transfers the labels from the future frame to time of observation. This is a minor contribution, which could be achieved also more directly via object tracking.

Critical weaknesses

One critical weakness is the limited novelty of the paper. While the paper frames the use of the roll-outs of a world model to see a risk before it is happening, this idea has already been presented in DriveWM and UniFuture. Also the conditioning of the world model on bounding boxes was shown in MagicDrive and DriveDreamer before. The only novelty is on using the box conditioning to emphasize the generation of the risky object, which could then improve the risk detection on that object in the future.

However, this requires knowledge of which one is the risky object, which needs to be assessed via the observations rather than the rolled-out frames of the world model. This risk assessment is trained based on the labeled data from the DRAMA dataset. It is unclear why the world model, the roll-out, and the more precise generation of the risky object are needed, when the risky object is already known to be the risky object before all these steps happen. To me this looks like a quite indirect way to train a detector of risky objects from observations and labeled data. The experiments compare to just the world model baseline, but do not identify the value of the world model itself on this task.

Questions for authors

Can you verify that the world model has any effect for the risk assessment?

Reviewer_ShGZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RAWM, a risk-augmented world model for anticipatory driving risk understanding. The method follows a generate-then-perceive paradigm, where a pretrained driving world model predicts future driving scenes, and a risk understanding head localizes and describes the potential risk object in the generated future. To improve the fidelity of safety-critical objects, the authors introduce dual-branch appearance and semantic conditioning through decoupled cross-attention. A learnable risk prompt module estimates these conditions from early observations at inference time. Experiments on the DRAMA benchmark demonstrate improvements in risk localization, captioning, and risk-region reconstruction fidelity. The paper also investigates performance across different anticipation horizons.

Critical strengths
  1. Meaningful research motivation and task formulation. The paper addresses the important problem of identifying potential driving risks before they become imminent. The generate-then-perceive formulation provides an interesting perspective on using world models for anticipatory perception rather than merely generating realistic future scenes.
  2. Consistent experimental improvements and useful analysis. RAWM improves both localization and captioning over the baseline world model. The experiments include component ablations, comparisons between estimated and ground-truth conditioning, reconstruction fidelity analysis, and evaluations across different anticipation horizons. These results provide useful evidence for the effectiveness of risk-conditioned future generation.
Critical weaknesses
  1. Insufficient evidence that explicit future generation is necessary for risk anticipation. The comparison with the direct-prediction baseline is not fully controlled. The baseline uses only the last observed frame with a ViT encoder, whereas RAWM uses multiple observed frames, a pretrained diffusion world model, and additional risk-conditioning modules. Therefore, the reported improvements cannot be attributed exclusively to the generate-then-perceive paradigm. Moreover, the proposed risk prompt module already learns to identify potential risk objects from early observations. It remains unclear whether similar improvements could be achieved by directly predicting future risk localization and descriptions from the estimated risk prompts, without generating future images. The lack of input- and capacity-matched temporal forecasting baselines, as well as a prompt-only baseline, limits the evidence supporting the necessity of the proposed generative framework.
  2. Limited validation of practical anticipatory risk understanding. The evaluation assumes that every video contains an annotated risk object at a designated future moment. Consequently, the experiments primarily assess future risk-object localization and captioning, rather than determining whether a risk will occur in unconstrained driving scenarios. The method is not evaluated on risk-free scenes, and false-alarm behavior remains unknown. This limits the extent to which the reported results support the broader claim of practical driving risk anticipation. Furthermore, the proposed framework relies on 50-step diffusion sampling, but no end-to-end inference latency is reported. Since the primary motivation is to provide additional reaction time, it is important to establish whether the computational delay leaves a meaningful positive anticipation window.
Questions for authors

Q1. Can the authors provide an input- and capacity-matched temporal forecasting baseline that directly predicts future risk localization and captioning from the same observed frames? Additionally, how does a prompt-only baseline perform when the estimated risk conditions are supplied directly to the risk head without future scene generation? Q2. How would RAWM perform on evaluation data containing both risk and non-risk driving sequences? Can the authors characterize its false-positive behavior and clarify how the model would distinguish an emerging risk from ordinary traffic objects? Q3. What is the end-to-end inference latency of RAWM on the reported hardware, including future generation and risk understanding? How much effective anticipation time remains after accounting for inference latency? Q4. Appendix E reports worse FID and FVD despite improvements in risk-region reconstruction fidelity. Could the authors further analyze this trade-off and its possible consequences for generating plausible safety-critical driving scenarios? Q5. Since the DRAMA videos are interpolated from approximately 3 FPS to 10 FPS, can the authors clarify whether the temporal interpolation procedure strictly prevents future frames from contributing to the observed input window?

Reviewer_jAnwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose a new task called "anticipatory driving risk understanding". Given first k frames of a DRAMA clip, the model should predict the box and the caption of risk object at the future risk moment. For this, authors use generate-then-perceive pipeline: frozen Vista world model generate the future frames, and GRiT-style head read the UNet decoder features of generated risk frame.

Critical strengths
  • Forecasting the risk object before it become salient is a good direction.
  • Comparison seems quite reasonable.
  • Paper is well written and presented.
Critical weaknesses
  • Authors' claims are somwhat exaggerated. Sec. 4.3 says that conditioning helps without degrading the rest of the scene, but FID is almost doubled and FVD increased (Table 6), and this is only shown in appendix. Also risk-region PSNR is computed inside ground-truth box, so it mixes the object fidelity and the position. Table 6 should be moved to main text and the claim should be correctly presented.
  • Baselines do not demonstrate that generation is necessary. Direct prediction see only one frame with ViT and is not allowed to do localization (Sec. 4.1). A learned box regressor using same three frames is the obvious baseline, but it is missing. CV and FOL get only three RIFE-interpolated frames, this is quite weak history especially for FOL.
Questions for authors

Please clarify above.

Comments on the paper

1 comment
Area_Chair_ZqChOpenReview ↗

Reminder: Please Review Authors’ Rebuttal and Provide Feedback

Dear Reviewers,

Thank you for your reviews. If you have not already done so, please read the authors' rebuttal and share your feedback in the discussion thread at your earliest convenience. Your timely responses would be greatly appreciated.

Best,
AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.