- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
JEWAM is a joint-embedding world-action model. A student encoder reads four history frames. An EMA teacher with momentum 0.99925 and stop-gradient reads the full clip and provides future-state targets at the future token positions. A Mixture-of-Transformers predictor with a latent expert and an action expert predicts future features with an L1 loss and the full eight-step BEV trajectory with an L1 loss. The attention mask is asymmetric, following Metis. Future-state queries can read action keys and values. Action queries cannot read future-state keys and values. This makes the action path independent. At inference the teacher, the future queries, and the future head are removed. The full trajectory is produced in one forward pass, with no rollout and no denoising. The empirical core compares five encoders under a shared interface and a shared training budget: ViT, MAE, CroCo v2, DINOv3, and V-JEPA 2.1. Each encoder provides both the history context and the future targets. V-JEPA wins on navhard with 38.55 against 34.43 for DINOv3, so the final model uses V-JEPA, at ViT-G scale for the headline numbers. Results: 90.98 EPDMS on NAVSIM v2 navtest, 39.86 on navhard, and 92.08 PDMS on NAVSIM v1, all using a single front-view camera. The paper also compares the feed-forward predictor against two DiT variants and reports 29.10 Hz, which is 7.74 times the faster DiT at its best 15-step setting.
Critical strengths
The first strength is the representation study. This subfield needs this kind of controlled comparison. Many papers take a position on whether video-generative, latent-predictive, or image-pretrained features are better for driving, and most of them change several settings at the same time. Here the interface and the budget are fixed and only the encoder changes. The navhard results are 28.23 for ViT, 33.19 for MAE, 31.87 for CroCo v2, 34.43 for DINOv3, and 38.55 for V-JEPA. Predictive video features are worth about four points over the best image-pretrained alternative, and the gap is much larger on navhard than on navtest. I expect this table to be cited.
The second strength is the finding about inference. Future supervision helps during training, where navhard improves by 6.67 (33.19 becomes 39.86). At inference, removing the future tokens gives 39.78 against 39.86. Action-only inference is slightly better. This is a stronger statement than saying the benefit is retained. Combined with 29 Hz, the practical conclusion is clear: the world model is needed during training and is not needed during deployment.
I also want to note that the paper is direct about its own confounds. It states that the encoder comparison changes context and targets together. It states that the DiT variants change the training objective, so they do not isolate iterative sampling. This makes the rest of the claims easier to trust.
Critical weaknesses
The problem is that the paper names these confounds and does not remove them. In the representation study, changing the encoder changes the context and the targets at the same time. When V-JEPA wins, I cannot tell whether the history features are better, whether the future targets are better, or whether both are better. A third confound also exists: V-JEPA uses two-frame tubelets, and the other four encoders use one-frame tokens, so the temporal tokenisation changes too. A 2x2 crossing of context encoder against target encoder would answer this question, and it looks like a straightforward experiment. Without it, the central empirical claim of the paper is a correlation.
The future-supervision ablation has the same problem. The paper removes the future tokens, the teacher targets, and the future-state loss together, and it says so. This also removes an expert and its tokens, so the model has less capacity during training. Part of the 6.67 gain could be a capacity effect. A control with a target that carries no temporal information, for example predicting the current frame features or shuffled future frames at matched parameter count, would separate temporal information from model capacity.
The efficiency comparison is the third weakness. It is entirely internal. The DiT variants are built by the authors, trained from scratch for the same 20k steps, and flow matching usually needs its own tuning. The best DiT variant reaches 38.05 against 39.86 for the feed-forward model. The gap is real but modest. I read this as one implementation beating another implementation at a fixed budget. That is worth reporting, and it is not the same as showing that iterative prediction is worse in general. There is no latency or FLOPs comparison against any published diffusion planner. The frequency is measured on an Alibaba Zhenwu PPU-ZW810E. Almost no other paper reports on this hardware, so I cannot compare 29.10 Hz with published numbers. A800 or H100 timing, or FLOPs, would fix this.
One more point, which does not change the ranking. JEWAM wins on EPDMS because of the safety submetrics: NC 99.20, DAC 98.10, TTC 98.80, DDC 99.70. The comfort submetric goes the other way. On navtest, EC is 83.50, which is second lowest in Table 1. Latent-WAM is 87.30 and DriveWAM is 88.90. EP is 87.10, which is middling. The authors discuss the EC drop for navhard stage 2 and do not discuss the navtest EC value, which is the larger regression relative to the field. I would like a comment on whether this is a real trade-off or an artifact. The single front camera is also presented only as an efficiency advantage. It is one, and it is also a limitation for deployment. The paper should say this.
Minor: Fast-WAM (Yuan et al., 2026) asks almost exactly this paper's question, whether world-action models need test-time future imagination. It is cited in the related work and never appears in a comparison table. As with most submissions in this area, there are no seeds and no error bars.
Questions for authors
Can you run the context-encoder against target-encoder crossing, even with only V-JEPA and DINOv3 in both slots? Can you also match the temporal tokenisation across encoders?
Can you add a capacity control to the future-supervision ablation, using a non-informative future target at matched parameter count?
Can you report A800 or H100 latency, or FLOPs, and compare against at least one published iterative planner on the same hardware?
Can you comment on the navtest EC value of 83.50?
Can you add a comparison against Fast-WAM?
2 comments
3. Efficiency comparison (Weakness 3 / Question 3)
We added an external comparison, selecting DriveWAM, the SOTA baseline in the main table, and measuring it with its public weights under the same hardware and timing conditions:
| Method | Inference configuration | TFLOPs per plan ↓ | Loaded params (B) ↓ | Peak memory (GiB) ↓ |
|---|---|---|---|---|
| JEWAM FF | action-only, single forward | 1.540 | 1.373 | 2.643 |
| DriveWAM | video 3 steps, action 10 steps | 17.535 | 12.627 | 24.303 |
FLOPs are counted with the PyTorch FLOPs counting tool. We currently have no available A800/H100 server and therefore did not measure latency on that hardware; as the table shows, JEWAM is clearly lower than DriveWAM in per-plan computation, loaded parameter count, and peak memory, demonstrating the deployment efficiency advantage of single-forward planning.
4. navtest EC and the single front camera (Weakness 4 / Question 4)
Regarding the navtest EC of 83.50, we performed a scenario-level diagnosis on the NAVSIM benchmark. The EC losses are concentrated in a small number of scenarios: while maintaining progress and smooth driving, the vehicle's acceleration/deceleration and steering adjustments are insufficiently consistent across two consecutive planning executions, reducing extended comfort. After further inspecting the velocity fields and correction processes of these trajectories, we attribute the cause to insufficient cross-window dynamic adjustment in a small subset of trajectories. These failure cases dominate part of the EC sub-metric loss; overall, however, the model retains strong robustness and accuracy and achieves SOTA in EPDMS.
In addition, we further tested on Bench2Drive, where the results (DS 77.95, SR 49.55%) show that our method also has a performance advantage in interactive closed-loop driving:
| Method | DS ↑ | SR ↑ |
|---|---|---|
| Drive-JEPA | 64.52 | 36.82 |
| GraphWorld | 76.71 | 44.22 |
| ReCogDrive | 71.36 | 45.45 |
| JEWAM | 77.95 | 49.55 |
Regarding the single front camera: single-front-view input is a trade-off in our input configuration. We acknowledge that single-front-view input is weak in lateral perception and occlusion handling, and that the paper previously presented it only as an efficiency advantage. It should be noted that our model architecture itself supports multi-view input and can be readily extended to a multi-view setting. Despite the above perceptual limitation, our method using only single-front-view input still outperforms the multi-camera + LiDAR methods in the table:
| Method | Input configuration | v2 navtest EPDMS ↑ | v2 navhard EPDMS ↑ |
|---|---|---|---|
| TransFuser | 3 cameras + LiDAR | 76.70 | 23.10 |
| DiffusionDrive | 3 cameras + LiDAR | 84.50 | 27.50 |
| JEWAM | single front camera | 90.98 | 39.86 |
5. Fast-WAM and error bars (Weakness 5 / Question 5)
Fast-WAM is a robotics work, and its direct extension to autonomous driving is Sim-WAM, which explicitly builds on the insight of Fast-WAM. We have compared with Sim-WAM:
| Method | Params | NavTest v2 EPDMS ↑ | NavHard EPDMS ↑ |
|---|---|---|---|
| Sim-WAM | 6B | 90.2 | 37.6 |
| JEWAM FF (ViT-L) | 0.66B | 89.50 | 38.55 |
| JEWAM FF (ViT-G) | 1.3B | 90.98 | 39.86 |
With a ViT-L configuration at about 1/10 of Sim-WAM's parameter count, JEWAM already achieves SOTA on NavHard (38.55 vs Sim-WAM 37.6); the ViT-G configuration further improves on both metrics and achieves the best results (NavHard 39.86, NavTest v2 90.98), widening the gap on NavHard relative to ViT-L. We will add this comparison to the paper.
We thank the reviewer for the careful review and high standards; these comments substantially help improve the rigor of our argument. We address each point below.
1. Confounding factors in the representation study and the 2×2 crossing (Weakness 1 / Question 1)
The reviewer is correct that the representation study simultaneously changes the context encoder, the target encoder, and the temporal tokenization. Following your suggestion, we conducted a 2×2 crossing of context encoder × target encoder, together with a no-future-supervision control. Results (NavHard EPDMS) are:
The table can be read in three directions:
(i) Encoder quality contributes, but is not the main factor. With no future supervision, the V-JEPA student is 1.61 points higher than the DINOv3 student (31.45 vs 29.84); with either future target fixed, the V-JEPA student is also consistently higher (35.04 vs 34.43; 38.55 vs 33.79). This indicates that the V-JEPA history representation is more conducive to trajectory generation, but changing the encoder alone accounts for only about 1.6 points.
(ii) The benefit of future supervision depends on whether the future target matches the history representation, and the V-JEPA configuration benefits most. Relative to their respective no-future-supervision baselines, the V-JEPA student paired with the V-JEPA future target yields the largest gain, reaching 38.55, clearly above the other three settings. Our explanation is as follows: V-JEPA features are pretrained on video sequences and encode spatio-temporal representations, so their future-state features carry richer temporal evolution and motion-consistency information. When the history encoder and the future target are of the same origin, the two lie in the same semantic space, allowing the action module to exploit this spatio-temporal consistency constraint more directly for planning and thus obtain the largest gain; when they come from different origins, their semantic spaces are not fully aligned and the additional information is harder to exploit.
(iii) Temporal tokenization is not unified, but its effect points in the opposite direction to the observed advantage. We retain each encoder's original encoding so that each performs at its best: V-JEPA fuses tokens along the temporal dimension and thus has fewer tokens, whereas DINOv3 does no such fusion and thus has more tokens carrying more information. If tokenization or token count dominated performance, the DINOv3 configuration should have benefited; yet in the 2×3 table the V-JEPA student outperforms the DINOv3 student under no future supervision and under both fixed future targets. Thus the context and target confounds are separated by the 2×3 crossing, and although tokenization is not unified, its effect points in the opposite direction to the observed V-JEPA advantage and does not threaten this attribution.
2. Capacity effects in the future-supervision ablation (Weakness 2 / Question 2)
We clarify that in our implementation, the same expert processes both historical visual tokens and future visual tokens, so the model has exactly the same parameter count with and without future supervision. Removing future supervision does not remove an expert or any parameters; it only disables future-state prediction.