- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a world-model-guided representation learning framework for end-to-end autonomous driving. It introduces a lightweight transition operator that distills future dynamics knowledge from a pretrained video world model (Wan-2.2 5B) into compact semantic transition factors, which are used to modulate current scene representations. The method further incorporates driving-rule knowledge from a trajectory scorer through physics tokens and representation alignment. During inference, the world model and future prediction head are removed, maintaining an efficient planning pipeline. Experiments on NAVSIM and HUGSIM demonstrate improvements in planning performance, particularly in challenging driving scenarios.
Critical strengths
- WAM Distillation and State-Anchored Planning. The paper presents an interesting approach to distilling future dynamics into compact transition factors that enhance current scene representations, enabling efficient planning without requiring future prediction at inference time.
- Strong performance. The proposed method achieves strong results on NAVSIM and HUGSIM, demonstrating significant improvements over the baseline, particularly in challenging driving scenarios.
Critical weaknesses
While the overall performance is strong, the paper would benefit from more explicit evidence and explanations of the proposed mechanisms, particularly regarding what the learned representations capture and why these designs are effective.
- Insufficient Evidence for Ego-Motion Factorization. The paper claims that ego-conditioned queries help disentangle ego-induced changes from surrounding scene dynamics. However, the experiments mainly demonstrate improvements in planning performance, without directly verifying whether the learned transition factors achieve such disentanglement. More analysis of the learned representations would help support this claim.
- Limited Justification for Driving-Knowledge Completion. The motivation and effectiveness of this component need further clarification. In particular, why should aligning with the scorer's intermediate representations lead to acquiring driving-rule or physical knowledge? What specific knowledge is transferred, and can the authors provide more direct evidence of this? Moreover, the relatively modest performance gain raises questions about the actual contribution of this component.
Questions for authors
See weaknesses.
Response
We thank the reviewer for the thoughtful and constructive review, and for recognizing the value of distilling future dynamics into compact transition factors and the strong results on NAVSIM and HUGSIM. We respond to each point below, and the new results have been incorporated into the revised manuscript.
W1 → A1
We agree that verifying ego-motion disentanglement requires direct evidence from the learned factors. We condition the pooling queries on ego status to give the operator an explicit cue about the vehicle’s motion when extracting transition factors. In Table 4(a), adding ego conditioning before rule injection improves navhard EPDMS from 52.3 to 54.4, a gain of 2.1 points.
We also examined the learned FiLM coefficients across 5,912 evaluated scenes. Fewer than 0.66% of the token-channel scale coefficients 1+γ were within 0.01 of zero, based on a histogram upper bound. This characterizes how often the operator nearly suppresses the direct contribution of an observed feature.
We have revised the Introduction and Conclusion to remove the overstated claim. The revised text states the intent of the design more precisely. We provide ego status to the pooling queries so that the operator can take the ego vehicle’s motion into account when extracting transition factors.
W2 → A2
Thank you for this thoughtful question. Our motivation is to complement the world model’s prediction of how a scene may evolve with information about how a candidate trajectory should be evaluated in that scene. The scorer ranks candidate trajectories and is supervised by simulator labels for six criteria: at-fault collision avoidance, drivable-area compliance, driving-direction compliance, time-to-collision, route progress, and comfort. These prediction heads share a trajectory-specific representation. During training, we use each candidate to query the physics tokens and align the resulting feature with the scorer’s detached representation for that candidate. This gives the physics tokens a training signal tied to the criteria the scorer uses to evaluate trajectories. The tokens then influence planning through the modulation of current scene features.
\text{Metric} & \text{w/o rule injection} & \text{w/ rule injection} \\\\ \hline \text{No at-fault collisions} & 89.1 & 90.4 \\\\ \text{Drivable-area compliance} & 86.8 & 88.4 \\\\ \text{Driving-direction compliance} & 92.9 & 93.6 \\\\ \text{Ego progress} & 84.4 & 85.3 \\\\ \text{Time-to-collision within bound} & 87.6 & 87.9 \\\\ \text{Lane keeping} & 51.6 & 53.3 \\\\ \text{History comfort} & 98.4 & 98.8 \\\\ \hline \text{Overall EPDMS} & 54.4 & \mathbf{54.9} \end{array}$$ In the navhard module ablation, enabling rule injection with the one-way mask (aboved table) raises EPDMS from 54.4 to 54.9. The Stage 2 breakdown shows gains in no-at-fault collisions (89.1 to 90.4), drivable-area compliance (86.8 to 88.4), driving-direction compliance (92.9 to 93.6), and lane keeping (51.6 to 53.3). These improvements show where the modest aggregate gain appears in driving behavior. In a separate architecture-matched experiment, we retain the original scorer and all planner modules and vary only the scorer-feature alignment loss. Enabling this loss raises EPDMS from 52.4 to 54.9, isolating the contribution of the supervision at fixed model capacity. The alignment head runs only during training, and reverse injection reuses the existing scorer at inference, so it requires no additional scoring pass. This makes reverse injection a cost-effective way to improve collision avoidance, drivable-area compliance, and lane keeping.