- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes anticipatory driving-risk understanding by first rolling-out the future video with a pretrained world model and then localizing/captioning the risk object in the generated future, rather than predicting risk directly from the current observation. Its main contribution is to condition the world model toward the object that a supervised risk-prompt module already predicts will become critical. This helps the world model to render this object more faithfully in the rollout. The experiments show consistent gains over an unconditioned world model. However, the evidence that the world-model rollout itself is the key source of improvement is limited, because the prompt module is already trained to assess the observations for the risky object before the actual roll-out with the world model happens.
Critical strengths
In principle, it is a good idea to make a world model putting emphasis on a faithful generation of details of particularly risky objects in the scene. This requires the model to know what are the risky objects. The paper uses an existing labeled dataset for risk assessment and indirectly transfers the labels from the future frame to time of observation. This is a minor contribution, which could be achieved also more directly via object tracking.
Critical weaknesses
One critical weakness is the limited novelty of the paper. While the paper frames the use of the roll-outs of a world model to see a risk before it is happening, this idea has already been presented in DriveWM and UniFuture. Also the conditioning of the world model on bounding boxes was shown in MagicDrive and DriveDreamer before. The only novelty is on using the box conditioning to emphasize the generation of the risky object, which could then improve the risk detection on that object in the future.
However, this requires knowledge of which one is the risky object, which needs to be assessed via the observations rather than the rolled-out frames of the world model. This risk assessment is trained based on the labeled data from the DRAMA dataset. It is unclear why the world model, the roll-out, and the more precise generation of the risky object are needed, when the risky object is already known to be the risky object before all these steps happen. To me this looks like a quite indirect way to train a detector of risky objects from observations and labeled data. The experiments compare to just the world model baseline, but do not identify the value of the world model itself on this task.
Questions for authors
Can you verify that the world model has any effect for the risk assessment?
Reminder: Please Review Authors’ Rebuttal and Provide Feedback
Dear Reviewers,
Thank you for your reviews. If you have not already done so, please read the authors' rebuttal and share your feedback in the discussion thread at your earliest convenience. Your timely responses would be greatly appreciated.
Best,
AC