acceptodds
Under review as a conference paper at ICLR 2027

Temporal Context Needs the Right Target: Projected Latent Prediction for End-to-End Driving

Abstract

Temporal context is essential for motion-aware driving, yet we find that naively introducing temporally fused features into a reconstruction-based latent world model can degrade planning performance. We attribute this failure to a representation–objective mismatch: the predicted latent encodes temporal history and action-conditioned evolution, whereas its supervision target remains a single-frame visual latent whose exact coordinates contain factors unnecessary for planning. To address this mismatch, we propose Temporal-JEPA LAW. Our method first introduces Latent World-Model Temporal Fusion (LWM-TF), which causally aggregates each LAW latent token over time while preserving its spatial identity and maintaining attention complexity. It then replaces raw latent reconstruction with normalized projected alignment, allowing the world model to match prediction-relevant structure without reproducing every visual-latent coordinate. Following LeWorldModel, a sketched isotropic Gaussian regularizer (SIGReg) prevents collapse in the unnormalized embedding space. Controlled experiments reveal the central effect of this design: temporal fusion alone improves average planning L2 from 0.61m to 0.53m, whereas coupling temporal prediction with raw mean-squared error degrades it to 0.66m. Projected alignment mitigates this negative transfer, achieving 0.51m average L2 and reducing the average collision rate from 0.30% to 0.09%. These results show that temporal information and predictive supervision must be designed jointly: LWM-TF supplies motion evidence, while JEPA provides a planning-compatible target for latent future prediction.

Primary area:
applications to robotics, autonomy, planning
Keywords:
End-to-End Autonomous Driving, Latent World Models, Temporal Fusion, Joint-Embedding Predictive Architecture, Representation Learning, Trajectory Planning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_L3u6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets the motion-aware planning problem in autonomous driving scenarios and proposes the ‌Temporal-JEPALAW‌ method, addressing the core issue that naively introducing temporally fused features into existing reconstruction-based latent world models degrades planning performance.

Critical strengths

LWM-TF Temporal Fusion Module. It causally aggregates the temporal information of each LAW latent token while preserving its spatial identity. The attention complexity is maintained at O(NK²), which supplements the model with motion-related temporal evidence without significantly increasing computational burden.

Normalized Projected Alignment Mechanism. It replaces the original raw latent reconstruction loss, enabling the world model to only match the core structure relevant to prediction, without the need to reproduce all coordinate details of the visual latent vector. This adapts the supervision target to the requirements of planning tasks.

Critical weaknesses
  1. The proposed approach is relatively concise, and I agree with the author's description of the redundancy of reconstruction objectives for planning tasks. However, I believe that the world model itself is a fundamental model, and its goal is to serve a diverse range of downstream tasks. Merely using the objective of a single task to supervise the training of the world model itself violates the design philosophy of the world model.

  2. In Table 1, I observed that both the L2 and Collision metrics are not state-of-the-art (SOTA). Although the ablation experiments demonstrate the effectiveness of the module proposed by the authors, the method still lacks in terms of its leading edge.

Questions for authors

See Weaknesses above.

1 comment

Thank you for recognizing the efficiency of LWM-TF and the motivation for projected alignment. Your comments raise two important questions: the scope of the world model and the significance of the planning improvements.

1. World-model learning versus single-task supervision

We agree that learning representations useful across multiple downstream tasks is an important goal for world models. Our current study focuses on a narrower setting: an action-conditioned latent world model trained alongside a driving planner.

The world model is not supervised solely by trajectory imitation. It predicts a future visual latent from earlier observations and an action condition, and receives a projected latent-prediction loss. Trajectory supervision trains the planning path, while SIGReg regularizes the projected representations. Projected alignment changes the space in which future prediction is evaluated; it does not replace future-state prediction with a waypoint target.

However, because the overall system is trained jointly for driving planning, its representations can be task-dependent. We have not established their usefulness for other downstream tasks, and we have revised the manuscript to make this scope explicit. We also avoid claiming that projection necessarily retains all information needed by other tasks or selects only planning-relevant factors.

The added SIGReg diagnostics provide evidence about representation diversity within this setting. In revised Table 7, removing SIGReg reduces projected-target effective rank from 28.25 to 11.93 and projected-prediction effective rank from 17.58 to 8.85. These measurements support the regularizer's role in maintaining diverse representations, but do not establish general-purpose world-model capabilities.

Our longer-term goal is to examine whether the learned predictive representations can support additional tasks. The present contribution is an evaluation of how temporal context and latent-prediction objectives affect driving planning.

2. Performance and comparison with stronger planners

We agree that our results do not establish state-of-the-art performance across all planning metrics. The revised manuscript distinguishes published cross-method comparisons from controlled comparisons within LAW. The former also involve differences in supervision, training, and evaluation protocols.

To assess whether the world-model component adds value beyond temporal planning, we added a matched comparison on all 12,146 NAVSIM v2 navtest scenarios (revised Table 4). Both configurations retain the same six-frame temporal planner, planner configuration, training setup, and evaluator.

ConfigurationEPDMS ↑DAC ↑TTC ↑EC ↑
Temporal planner only74.189.392.373.2
Temporal-JEPA LAW86.096.296.985.6

Enabling the world-model supervision scheme increases EPDMS by 11.9 points. This comparison includes planner guidance and measures the contribution of the auxiliary supervision scheme as a whole.

On nuScenes, the additional improvement is smaller: average L2 decreases from 0.53 to 0.51 m and average collision from 0.11% to 0.09%. We now state this explicitly. The world-model branch is removed at inference, so these gains do not require deploying the auxiliary predictor.

Our main empirical contribution is therefore to show when auxiliary latent prediction helps temporal planning within LAW, rather than to claim overall benchmark leadership. The new NAVSIM comparison strengthens the evidence for its practical benefit, while the revised Discussion acknowledges the modest nuScenes gains and the untested transfer to other tasks and architectures.

Reviewer_ZRSaOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper adds temporal token fusion and a projected latent-prediction loss to LAW. The authors argue that reconstructing raw visual latents is a poor objective once the model uses history, and replace it with normalized JEPA-style alignment and SIGReg. The full model reaches 0.51 m average L2 and 0.09% collision on nuScenes, and 86.0 EPDMS on NAVSIM.

Critical strengths
  1. Overall interesting idea
Critical weaknesses
  1. bad figures, all AI generated
  2. ablation is only done on openloop eval L2, even not on navsim.
  3. Poor paper writing overall, I cannot reconstruct the training procedure from the current description. Figure 1 shows six frames, a ground-truth-action training branch, and a planner-guidance pass with a frozen world model. Section 4.1 uses four frames, while Eq. 7 conditions on predicted waypoints and the text does not explain that extra guidance pass. Which version produced Table 3? The row-specific initialization and update budgets are also unclear: the setup mentions a 12-epoch reference baseline and an epoch-20 NAVSIM checkpoint. I am not assuming the comparisons are unequal, but these details matter for interpreting them. NAVSIM has no matched temporal-only result to settle the question.
Questions for authors

Please put more effort in paper writing next time, are you sure this is not an auto research paper???

1 comment

Thank you for identifying these presentation and reproducibility problems. The original description did not clearly distinguish the nuScenes and NAVSIM training configurations. We have revised the figures and method description, added a matched NAVSIM temporal-only comparison, and documented initialization, training budgets, and gradient routing.

1. Research process and figures

To answer your question directly: this is not an autonomously generated research paper. The authors conducted the research and experiments and take responsibility for the results. We used AI tools to assist with presentation, including language editing and figure preparation.

We agree that the figures must communicate the implementation accurately. Figure 1 is now explicitly labeled as the NAVSIM configuration. Figure 2 and its caption clarify the camera/proposal grouping and distinguish the current latent used by the planner from the source prefix used by the world model. We have also revised the text to define these terms and explain the training paths.

2. Four versus six frames and action conditioning

These are dataset-specific configurations, now stated in Section 3.2 and Table 1:

SettingnuScenesNAVSIM v2
Sampling interval0.5 s0.5 s
Queue length4 frames6 frames
Observation coverage1.5 s2.5 s
WM source prefixt−3 to t−1t−5 to t−3
Target framett
Latest-source-to-target gap0.5 s1.5 s

The world model predicts one target latent. Figure 1 depicts the six-frame NAVSIM setup; it is not intended to depict the four-frame nuScenes configuration.

On nuScenes, the world model receives fused source states and predicted source trajectories. The prediction loss propagates through both states and actions during joint training. There is no separate frozen-world-model guidance term.

NAVSIM uses two training paths:

  • World-model training: source states are detached and the ground-truth source trajectory supplies the action condition. This path updates the world model and projection modules.
  • Planner guidance: the world-model state input is detached, and the world model, projector, and predictor are frozen for this path. The action condition is a predicted trajectory decoded from attached temporal features. Gradients therefore reach the waypoint decoder, temporal fusion module, and visual encoder through the action condition.

The losses are accumulated for a single backward pass and optimizer step. “Frozen” refers to the guidance path, not to the world model throughout training. Appendix A, Table 9, now lists the loss-to-module gradient routing.

The NAVSIM result in Table 3 uses this two-path configuration, including planner guidance, evaluated at epoch 20. Inference uses the current-frame trajectory prediction without world-model rollout.

3. Initialization and training budgets

Our nuScenes ablations use external Swin-T initialization and 12 training epochs. Our NAVSIM runs use external Swin-T initialization and 20 epochs. The final checkpoint is evaluated in both datasets; the 12- and 20-epoch budgets refer to different datasets.

The “Official LAW” row is a published reference, not a baseline retrained under our budget. We now distinguish it from our within-codebase experiments. Appendix A and Table 10 specify initialization, action conditions, auxiliary waypoint losses, regularization, and budgets for the nuScenes rows. We also disclose that the non-temporal projected row uses a summed historical waypoint loss, whereas temporal rows average it.

4. Matched NAVSIM temporal-only ablation

Revised Table 4 evaluates both configurations on all 12,146 NAVSIM v2 navtest scenarios with the same evaluator. The control retains the six-frame temporal planner, planner configuration, and training setup, but disables the world-model supervision losses.

ConfigurationEPDMS ↑DAC ↑TTC ↑EC ↑
Temporal planner only74.189.392.373.2
Temporal-JEPA LAW86.096.296.985.6

World-model supervision increases EPDMS by 11.9 points. This comparison evaluates the auxiliary supervision scheme, including guidance, rather than isolating each loss separately.

The nuScenes ablations also report collision rates alongside L2. Revised Tables 5–7 include the single-frame projected-WM control, temporal-length ablation, and SIGReg ablation. Thus, the revised experimental evidence extends beyond open-loop L2 and includes the requested NAVSIM temporal-only comparison.

We apologize for the ambiguity in the original presentation. The revised manuscript separates the two dataset configurations and identifies the exact training procedure used for each reported result.

Reviewer_ZC8EOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Temporal-JEPA LAW, an end-to-end driving planning framework that extends the perception-free LAW driving planner with temporal information. The authors first identify that introducing temporal context into a latent world model trained with a reconstruction objective can degrade the planning performance. To address this issue, they propose LWM-TF to efficiently fuse the historical frames and replace the raw MSE loss with projected alignment and SIGReg regularization. The experimental results demonstrate improvements in both planning L2 error and collision rate compared to the original LAW baseline.

Critical strengths

This paper is well-motivated by an interesting observation that naively introducing temporal inputs into a reconstruction-based world model can hurt the planning performance. The authors provide an insightful explanation based on a representation-objective mismatch between single-frame visual latent and latent predicted from history and action evidence, and each technical component is designed around this core idea.

The paper provides a practical and efficient way to incorporate temporal information without disrupting the existing structure of LAW’s spatial tokens. The adapted architecture reduces the attention complexity compared to full spatio-temporal attention.

Critical weaknesses

Limited technical novelty. Although this paper provides useful insights into why naively adding temporal information to the latent world model can hurt planning performance, the proposed framework, Temporal-JEPA LAW, mainly borrows and combines existing ideas, such as projected alignment and SIGReg from JEPA-style world models and the factorized temporal attention from video Transformers, to improve LAW.

Several descriptions in the main text are inconsistent with figures and tables. For example, the authors claim that they "defer temporal reasoning to LWM-TF" in Section 3.1, while Fig. 1 and Table 2 refer to the visual backbone as “Swin3D-T”, which may already model the temporal information. Moreover, the EC scores reported in Table 2 and in Section 4.3 are different. These inconsistencies make it difficult to understand the architecture and experimental results.

Questions for authors

Could the authors clarify what is the time interval between consecutive frames, and whether the same temporal queue configuration is used in all experiments?

Could the authors clarify the latent prediction horizon of Temporal-JEPA LAW? How does it compare with the setting of the original LAW?

Could the authors provide ablation studies on the temporal queue size (K) and the effect of removing SIGReg regularization to verify whether SIGReg is necessary for preventing embedding collapse?

1 comment

Thank you for the constructive feedback. We have clarified the architecture and timing, corrected the EC value, and added the requested ablations.

1. Technical novelty

We agree that factorized attention, projected alignment, and SIGReg are existing techniques. The revised Introduction and Related Work explicitly credit them. Our contribution concerns how temporal planning interacts with auxiliary latent prediction in LAW.

Revised Table 5 shows that the temporal planner achieves 0.53 m average L2. Adding a single-frame raw-reconstruction WM increases this to 0.66 m; projected alignment with SIGReg gives 0.52 m with the same raw single-frame visual input. A fused three-frame source further improves it to 0.51 m. These results support the importance of the auxiliary objective, while showing that the additional gain from WM history is modest. Since the raw-to-projected comparison also changes regularization, we separately ablate SIGReg below.

2. Backbone and reporting consistency

Swin3D-T processes each frame independently in our implementation: its temporal dimension is one, and the queue dimension is folded into the batch. It does not aggregate across history frames; cross-frame fusion occurs in LWM-TF. Section 3.1 now states this explicitly.

The 36 tokens comprise six learned proposal slots per camera across six cameras. LWM-TF groups tokens by camera/proposal index, not by tracked object or fixed physical position.

The inconsistent EC value was a reporting error. Tables 3–4 and Appendix C now consistently report EC = 85.6 and EPDMS = 86.0.

3. Sampling interval and prediction horizon

Both datasets use a 0.5 s sampling interval:

SettingnuScenesNAVSIM v2
Queue length K46
Observation coverage1.5 s2.5 s
WM source prefixt−3 to t−1t−5 to t−3
Target framett
Latest-source-to-target gap0.5 s1.5 s

The WM predicts one target latent, not a multi-step rollout. Observation coverage and prediction gap are distinct.

The original LAW reference used in our nuScenes comparison reports the 0.5 s next-frame setting, matching our latest-source-to-target interval. Our NAVSIM configuration uses a 1.5 s gap. Section 3.2, Table 1, and Appendix A now specify these settings. Each dataset uses a fixed main configuration; queue length is varied explicitly in the ablation below.

4. Temporal queue length

Revised Table 6 varies queue length while fixing the target gap at 0.5 s. Planner history excludes the current frame.

KPlanner historyWM source framesAvg. L2 (m) ↓Avg. collision (%) ↓
3220.530.25
4330.510.09
6550.550.15

The four-frame queue performs best among the tested settings; longer history does not improve performance. This ablation varies planner and WM history together.

5. SIGReg and representation collapse

Revised Table 7 keeps the architecture, initialization, training budget, and other loss weights fixed:

ConfigurationAvg. L2 (m) ↓Avg. collision (%) ↓
With SIGReg0.510.09
Without SIGReg0.600.72

We also measured representation statistics on the same 256 validation samples:

RepresentationEffective rank with SIGRegWithout SIGReg
Projected target28.2511.93
Projected prediction17.588.85

Without SIGReg, the variance explained by the top ten components increases from 68.99% to 87.45% for targets and from 78.55% to 92.07% for predictions. Pooled cosine similarity also increases, indicating more concentrated representations.

These results support SIGReg's role in maintaining representation diversity and planning performance in this configuration. They indicate reduced effective dimensionality without SIGReg, rather than proving complete collapse. Each configuration uses one seed, as stated in the revised manuscript.

Reviewer_ZjkmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed a direct improvement over LAW (Li et al., 2025), an end-to-end driving model using world model in training, by adding temporal information (attention-fused multiple frames of latent tokens) as input of the world model to enforce temporal understanding. The experiments on nuScenes and NASIM v2 demonstrated the improvements on driving safety scores.

Critical strengths
  • The paper presented an interesting finding, or say a limitation of the LAW driving model, that its world model take one latent token as input leaving the temporal understanding weak.
  • The experiments concretely shown improvements over two main datasets, nuScenes and NAVSIM v2.
Critical weaknesses
  • While the found limitation of LAW is interesting, there is not direct diagnosis or analysis confirming the finding. What exact temporal-related failures are not handled well in the previous approach? Is there any ablation can confirm the problem is from the lack of causal temporal fusion? The evidence the paper presented is the end driving safety scores, which is not sufficient to understand and confirm the hypothezed problem.
  • The design seems very specific about the LAW design. How is the generality of the proposed method? Do other end-to-end driving models share similar problems, and if not what are their solutions? The scope where the proposed method can apply is not very clear to me.
  • The writing should be improved. Many terms are not well defined at their first appearence, e.g., "LAW token" without mention what is LAW, which is a related work not even a term. Such writing issues happen across the paper.
Questions for authors
  • Is there direct evidence or diagnosis confirming the assumed limitation of LAW?
  • How is the generality of the proposed method?
2 comments

Thank you for pointing out that planning scores alone do not establish how the model uses temporal information. We have added controlled history interventions, clarified the method's scope, and revised the terminology.

1. Direct evidence of temporal-information use

Section 4.5 now pairs observation histories with planning outputs for night-time car following (sample 5760) and a curved-road pedestrian interaction (sample 4866).

We evaluate the same trained model while keeping the current frame fixed and changing only the preceding images: replacing history with the current frame, cyclically shuffling history, or reversing its order. Revised Table 8 reports:

SampleHistoryForward endpoint (m)3 s L2 (m) ↓
5760Original12.191.37
5760Repeat current13.992.18
5760Shuffle12.771.66
5760Reverse12.761.66
4866Original6.231.49
4866Repeat current5.971.59
4866Shuffle6.241.48
4866Reverse6.311.45

In the car-following case, repeating the current frame increases predicted forward progress by 1.80 m and worsens 3 s L2 by 0.81 m. Changing history order also changes the prediction and worsens L2. This demonstrates dependence on preceding observations and their order in this case.

The pedestrian case is less sensitive: repeating the current frame slightly worsens L2, while shuffling or reversing history does not. We report this contrast rather than claiming uniform reliance on temporal order.

These interventions provide case-level evidence of history use. They do not establish that missing causal temporal fusion is the sole cause of LAW's errors, or identify the particular object cues responsible for each change.

2. Component ablations

Revised Table 5 separates temporal planning from auxiliary world-model supervision:

ConfigurationAvg. L2 (m) ↓Avg. collision (%) ↓
Temporal planner only0.530.11
Temporal planner + single-frame raw WM0.660.28
Temporal planner + single-frame projected WM0.520.10
Temporal planner + fused three-frame projected WM0.510.09

Both single-frame WM variants receive an unfused source-frame latent, while their planner retains temporal input. Adding raw-reconstruction supervision hurts performance in this setting; projected alignment with SIGReg improves it, and the fused source prefix provides a further small gain.

We have narrowed the interpretation accordingly. These results support examining the prediction objective together with temporal planning. The raw-to-projected comparison changes both alignment and regularization, and does not uniquely attribute the improvement to causal attention.

3. Generality and cross-architecture evaluation

The design potentially applies to planners with consistently indexed latent slots and an auxiliary action-conditioned prediction branch. However, same-slot temporal attention assumes consistent indexing. Changing object queries or other representations may require explicit correspondence or a different fusion mechanism.

We also clarify that temporal modeling is already present in other end-to-end driving approaches through temporal backbones, BEV fusion, or recurrent memory. Our study concerns its interaction with LAW's auxiliary latent-prediction objective, rather than claiming that all planners lack temporal information.

We agree that experiments on another architecture would strengthen the generality claim. Given the remaining rebuttal time, we have not been able to complete a reliable cross-architecture evaluation. We therefore limit our empirical claims to LAW on nuScenes and NAVSIM v2. The potential applicability described above is a design consideration, not an experimentally verified result.

4. Terminology and presentation

Section 3.1 now defines LAW as the Latent World Model framework of Li et al. (2025). Each frame produces 36 camera/proposal tokens: six learned proposal slots for each of six cameras. A slot does not denote a tracked object or fixed physical position.

We clarify that Swin3D-T processes frames independently in our implementation, with temporal dimension one and the queue dimension folded into the batch. Cross-frame aggregation occurs in LWM-TF. Sections 3.2–3.3 and Appendix A specify the source prefix, target gap, action conditions, and gradient paths.

The revised manuscript thus adds direct history-intervention evidence and states more precisely what the experiments establish and where further validation is needed.

Thank you for the rebuttal and the latest update.

I am good about 3 Generality and cross-architecture evaluation as long as the revision has a clear discussion of the scope, and this problem is not narrowly defined.

My remaining concerns are on 1 and 2. Changing image inputs will naturally decrease model performance because that is causing out of distribution inputs that is abnormal. It is not proving the world model needs to have multiple timestamps of image features as input to make it capable for its work. This links to 2, and in your table three-frame result is nearly identical with the single-frame result. Surprisingly even without the world model at all the performance is similar. Perhaps you can try more challenging cases to show the difference?

Reviewer_fhcdOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

Summary

This paper proposes Temporal-JEPA LAW, an end-to-end driving framework that combines causal temporal fusion with projected latent prediction. It addresses the performance degradation observed when temporal world-model inputs are paired with raw latent reconstruction. The method introduces JEPA-style alignment and SIGReg regularization, with experiments on nuScenes and NAVSIM supporting the importance of jointly designing temporal representations and prediction objectives.

Strengths

  1. The paper shows that adding more temporal context can sometimes degrade performance, highlighting the need to jointly design temporal representations and prediction objectives.

  2. The method builds on the existing LAW framework through temporal fusion and a revised training objective, without requiring additional dense perception annotations.

  3. Average trajectory error decreases from 0.61m to 0.51m, while the collision metric decreases from 0.30% to 0.09%. Additional NAVSIM evaluation demonstrates the complete method’s performance on another benchmark.

Weaknesses

  1. Limited gains from the full framework. Compared with adding temporal context only to the planner, the full framework reduces average trajectory error from 0.53m to 0.51m and the collision metric from 0.11% to 0.09%. The additional gains from the complete design are therefore limited.

  2. A missing key ablation. The experiments omit the combination of a temporal planner, a single-frame world model, and JEPA supervision. Consequently, they do not fully establish whether adding temporal context to the world model provides further benefits once JEPA supervision is used.

Critical strengths

None

Critical weaknesses

Limited incremental benefit of the full framework. Although the complete method improves substantially over the original LAW baseline, most of the improvement is already achieved by adding temporal context to the planner alone. The full framework further reduces average trajectory error from 0.53m to 0.5 m and the collision metric from 0.11% to 0.09%. These relatively modest additional gains do not establish a compelling performance advantage over the simpler temporal variant, weakening the empirical justification for the full combination and the significance of its overall contribution.

Questions for authors

See the weakness above.

2 comments

Thank you for pointing out the missing control and the limited incremental gains on nuScenes. We have added the requested single-frame world-model ablation and a temporal-planner-only baseline on NAVSIM v2.

1. Temporal planner + single-frame world model + JEPA supervision

The revised Table 5 includes this configuration. The planner retains its temporal input, while the world model receives only the latest raw, unfused source-frame latent. Thus, the world-model visual input contains no temporally fused history. Both projected variants use normalized alignment and SIGReg, with the same external initialization and 12-epoch training budget.

ConfigurationAverage L2 (m) ↓Average collision (%) ↓
Temporal planner only0.530.11
Temporal planner + single-frame projected WM0.520.10
Temporal planner + three-frame fused projected WM0.510.09

The requested control shows that projected supervision improves the temporal planner even with a single-frame world-model input. Using the fused three-frame source prefix provides a further, small improvement in both average metrics. We agree that this additional gain is modest; the revised manuscript distinguishes it from the larger improvement over the original LAW reference.

2. Additional benefit beyond temporal planning on NAVSIM v2

We also added a within-framework comparison on all 12,146 NAVSIM v2 navtest scenarios (revised Table 4). The control retains the same six-frame temporal planner, planner configuration, training setup, and evaluator, but disables the world-model supervision losses.

ConfigurationEPDMS ↑DAC ↑TTC ↑EC ↑
Temporal planner only74.189.392.373.2
Temporal-JEPA LAW86.096.296.985.6

Enabling world-model supervision increases EPDMS by 11.9 points. This provides evidence that the benefit beyond temporal planning is substantial on NAVSIM, even though it is small on nuScenes. This comparison evaluates the complete auxiliary supervision scheme, including planner guidance; it does not isolate the contribution of world-model history alone.

The auxiliary world-model branch is used only during training and is removed at inference. The full model therefore retains the same inference architecture as the temporal-planner-only control.

These additions address the missing ablation and clarify the empirical contribution: fused world-model history gives a modest further gain on nuScenes, while the complete world-model supervision scheme provides a larger improvement over temporal planning on NAVSIM. We have updated Section 4.4 and the Discussion accordingly.

Thank you for the additional experiments. The new ablation directly addresses my concern regarding the contribution of temporal context in the world model under JEPA supervision. Furthermore, the NAVSIM comparison demonstrates more substantial gains from the overall framework on a closed-loop benchmark.

While the independent contribution of temporal world-model history remains relatively small, I appreciate the additional experiments and clarification. These results address my main concerns, and I am inclined to increase my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.