- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper targets the motion-aware planning problem in autonomous driving scenarios and proposes the Temporal-JEPALAW method, addressing the core issue that naively introducing temporally fused features into existing reconstruction-based latent world models degrades planning performance.
Critical strengths
LWM-TF Temporal Fusion Module. It causally aggregates the temporal information of each LAW latent token while preserving its spatial identity. The attention complexity is maintained at O(NK²), which supplements the model with motion-related temporal evidence without significantly increasing computational burden.
Normalized Projected Alignment Mechanism. It replaces the original raw latent reconstruction loss, enabling the world model to only match the core structure relevant to prediction, without the need to reproduce all coordinate details of the visual latent vector. This adapts the supervision target to the requirements of planning tasks.
Critical weaknesses
-
The proposed approach is relatively concise, and I agree with the author's description of the redundancy of reconstruction objectives for planning tasks. However, I believe that the world model itself is a fundamental model, and its goal is to serve a diverse range of downstream tasks. Merely using the objective of a single task to supervise the training of the world model itself violates the design philosophy of the world model.
-
In Table 1, I observed that both the L2 and Collision metrics are not state-of-the-art (SOTA). Although the ablation experiments demonstrate the effectiveness of the module proposed by the authors, the method still lacks in terms of its leading edge.
Questions for authors
See Weaknesses above.
Thank you for recognizing the efficiency of LWM-TF and the motivation for projected alignment. Your comments raise two important questions: the scope of the world model and the significance of the planning improvements.
1. World-model learning versus single-task supervision
We agree that learning representations useful across multiple downstream tasks is an important goal for world models. Our current study focuses on a narrower setting: an action-conditioned latent world model trained alongside a driving planner.
The world model is not supervised solely by trajectory imitation. It predicts a future visual latent from earlier observations and an action condition, and receives a projected latent-prediction loss. Trajectory supervision trains the planning path, while SIGReg regularizes the projected representations. Projected alignment changes the space in which future prediction is evaluated; it does not replace future-state prediction with a waypoint target.
However, because the overall system is trained jointly for driving planning, its representations can be task-dependent. We have not established their usefulness for other downstream tasks, and we have revised the manuscript to make this scope explicit. We also avoid claiming that projection necessarily retains all information needed by other tasks or selects only planning-relevant factors.
The added SIGReg diagnostics provide evidence about representation diversity within this setting. In revised Table 7, removing SIGReg reduces projected-target effective rank from 28.25 to 11.93 and projected-prediction effective rank from 17.58 to 8.85. These measurements support the regularizer's role in maintaining diverse representations, but do not establish general-purpose world-model capabilities.
Our longer-term goal is to examine whether the learned predictive representations can support additional tasks. The present contribution is an evaluation of how temporal context and latent-prediction objectives affect driving planning.
2. Performance and comparison with stronger planners
We agree that our results do not establish state-of-the-art performance across all planning metrics. The revised manuscript distinguishes published cross-method comparisons from controlled comparisons within LAW. The former also involve differences in supervision, training, and evaluation protocols.
To assess whether the world-model component adds value beyond temporal planning, we added a matched comparison on all 12,146 NAVSIM v2 navtest scenarios (revised Table 4). Both configurations retain the same six-frame temporal planner, planner configuration, training setup, and evaluator.
Enabling the world-model supervision scheme increases EPDMS by 11.9 points. This comparison includes planner guidance and measures the contribution of the auxiliary supervision scheme as a whole.
On nuScenes, the additional improvement is smaller: average L2 decreases from 0.53 to 0.51 m and average collision from 0.11% to 0.09%. We now state this explicitly. The world-model branch is removed at inference, so these gains do not require deploying the auxiliary predictor.
Our main empirical contribution is therefore to show when auxiliary latent prediction helps temporal planning within LAW, rather than to claim overall benchmark leadership. The new NAVSIM comparison strengthens the evidence for its practical benefit, while the revised Discussion acknowledges the modest nuScenes gains and the untested transfer to other tasks and architectures.