- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Metis, an end-to-end world-action model for autonomous driving and urban navigation. Its core design—asymmetric world-action attention (joint modeling during training and action-only prediction at inference) and training-only spatial registers (with depth and occupancy supervision)—is conceptually simple and effective. Experimental results on NAVSIM-v2 navhard and navtest as well as CityWalker, together with the substantially reduced inference latency, support the effectiveness of the method.
Critical strengths
The paper identifies two concrete limitations of existing world-action models: inference-time future observation generation causes substantial latency, and future states are often modeled only in semantic or 2D visual spaces without sufficient 3D spatial grounding. Metis directly targets both issues.
Critical weaknesses
The paper does not compare against several recent and highly relevant methods, including DriveZero, DriveFuture, GraphWorld, DriveWorld-VLA, MomWorld, RAP, MomAD/MomADv2, ZTRS, and SimScale. These are directly comparable on NAVSIM-v1/v2 navtest and navhard, and some of them are current SOTA or close to it. This is consequential because the paper claims strong planning performance and practical superiority, but without these comparisons the claim is not properly positioned. In addition, the GuideFlow EPDMS reported in Table 1 appears inconsistent with public results, which report substantially higher values. If a baseline number is wrong or evaluated under a different protocol, the comparison is not trustworthy.
NAVSIM-v1 results are relegated to the appendix, while the main tables report only NAVSIM-v2. Since the two benchmarks use different metrics and protocols, the paper’s generalization claim is not fully supported by the main evidence. The real-world deployment is also thin: only 15 trials per condition, with 53.3% indoor and 66.7% outdoor success. This is consequential because the paper emphasizes practical feasibility and zero-shot deployment, but the current evidence is too limited to support that claim robustly.
GuideFlow is 51 on https://huggingface.co/spaces/AGC2025/e2e-driving-navhard, and the latest comparison methods should also be added. These two issues are the strongest grounds for rejection if left unresolved. If the authors add the missing baselines, correct or clarify the GuideFlow number, and move the NAVSIM-v1 results into the main comparison, the core contribution would be much better supported.
Questions for authors
Could the authors add comparisons with the latest methods on the official AGC2025 NAVSIM-v2 navhard leaderboard? In particular, please consider adding DriveZero, DriveFuture, GraphWorld, DriveWorld-VLA, MomWorld, RAP, MomAD, MomADv2, ZTRS, SimScale, and any newer SOTA entries. These should be reported on NAVSIM-v1 navtest and NAVSIM-v2 navtest/navhard under consistent protocols, so that Metis can be properly positioned against current work.
Could the authors clarify the GuideFlow result in Table 1? The official AGC2025 NAVSIM-v2 navhard leaderboard lists GuideFlow at 51, while Table 1 reports 27.1. Is 27.1 a Stage 2 score, an early version, a different EPDMS implementation, or a re-evaluation under a different protocol? Please explain the discrepancy and correct the table if needed.
Could the authors move the NAVSIM-v1 results from the appendix into the main comparison? Since NAVSIM-v1 uses PDMS and NAVSIM-v2 uses EPDMS, the current main tables do not fully show generalization across evaluation protocols. Please include NAVSIM-v1 navtest results in the main paper and discuss any differences in ranking between v1 and v2.
Could the authors strengthen the real-world evidence? The current deployment reports only 15 trials per condition, with 53.3% indoor and 66.7% outdoor success. Please add more trials, confidence intervals, failure cases, and safety-related metrics. Also clarify how success is defined and whether any human intervention or safety override occurred.
Could the authors clarify the strength and fairness of asymmetric attention? The reported gradient ratio is 0.0196 for Metis and 0.0200 for the Joint variant, which seems very small. How much does the video objective actually supervise the action expert in practice? Please provide ablations with matched compute and parameters, and show whether stronger video supervision changes the conclusion.
Could the authors better isolate the effect of spatial registers? The registers are removed at inference, but their geometry objectives still backpropagate through the queried representations during training. Could the gains come from additional trainable capacity rather than geometric grounding? Please compare against an auxiliary geometry loss without registers and report whether the main action/video branches change substantially.
Could the authors clarify the efficiency comparison? Metis with video is reported at 1.06s, while DriveVLA-W0 and PWM are reported at 0.69s and 0.83s. Were all methods evaluated on the same GPU, input resolution, and denoising setting? Please provide a fully controlled latency comparison.
Could the authors discuss robustness when geometric supervision is unavailable or noisy? The depth and occupancy targets rely on LiDAR and DA3. CityWalker does not provide LiDAR, and real-world deployment may not have reliable geometric labels. How sensitive is Metis to noisy or missing geometric supervision, and what happens if the geometry branch is trained with DA3-only targets?
3 comments
Rebuttal for Reviewer 6EC6 (part 2/2)
Q6: Questions about spatial registers and geometric supervision
To isolate the contribution of spatial registers, we introduce a Direct Geometry experiment that applies the same depth and occupancy losses directly to current/future video and action representations without registers. In contrast, our spatial registers query only current image and future action representations, excluding stochastic future-video tokens.
| Variant | navtest EPDMS ↑ | navhard EPDMS ↑ | FVD ↓ |
|---|---|---|---|
| Base (w/o Geometry) | 89.1 | 32.7 | 56.5 |
| Direct Geometry | 89.5 | 33.3 | 72.1 |
| Metis (Ours) | 90.2 | 33.8 | 56.5 |
Direct Geometry improves planning performance but degrades video generation quality, whereas our spatial registers achieve better planning performance while preserving the original FVD. These results suggest that spatial registers provide a more effective interface for geometric supervision, reducing interference with video generation without additional inference overhead.
Q7: Question about the efficiency comparison
Good suggestion! We re-measured the inference latency on a single RTX 4090 with batch size 1.
| Method | Frames | Resolution | Horizon | Total Pixels | EPDMS ↑ | Latency (s) ↓ |
|---|---|---|---|---|---|---|
| DriveVLA-W0 (w/o video) | — | 256×144 | — | — | 86.9 | 0.69 |
| PWM (AR, w/ video) | 10 | 128×224 | 4 s | 286,720 | 88.2 | 0.83 |
| Metis (10-step, w/ video) | 8 | 320×384 | 4 s | 983,040 | 90.2 | 1.06 |
| Metis (10-step) | — | — | — | — | 90.2 | 0.44 |
| Metis (2-step) | — | — | — | — | 90.0 | 0.14 |
As shown above, Metis (w/ video) generates 3.43× as many pixels as PWM, albeit with lower inference speed. Removing future-video generation achieves a 2.4× speedup (1.06s → 0.44s) without performance degradation. Metis also 1.57× faster than DriveVLA-W0, with a 3.3-point higher EPDMS. Furthermore, 2-step action inference achieves a 7.6× speedup (0.14s) over full video inference with only a 0.2-point EPDMS reduction. These results demonstrate the efficiency advantage of our training–inference decoupling, enabling fast trajectory planning without future-video generation while maintaining strong planning performance.
Q8: questions about geometric supervision
We thank the reviewer for this important question. We clarify our existing geometry supervision settings and provide additional experiments on missing and noisy geometric targets.
(1) Geometry Supervision in CityWalker and Real-World Deployment. As already described in Appendix A.2 of our original manuscript, CityWalker does not provide LiDAR observations. We therefore disable occupancy supervision and use depth pseudo-labels generated by a frozen Depth Anything 3 (DA3) model, without LiDAR-based metric calibration. Importantly, spatial registers and geometry prediction heads are used only during training and are entirely removed at inference. Thus, real-world deployment does not require LiDAR or geometric labels.
(2) Effect of Missing LiDAR Supervision. We additionally evaluate a DA3-only configuration on NAVSIM-v2, where depth supervision is provided exclusively by DA3 without LiDAR calibration, and occupancy supervision is disabled. We also include the original Depth-only configuration from Table 8 for comparison. The DA3-only setting achieves 89.7 EPDMS, only 0.1 below the LiDAR-calibrated Depth-only configuration and 0.5 below the full model. Meanwhile, it improves EPDMS by 0.6 over the model without geometric supervision. These results demonstrate that DA3 pseudo-labels provide effective geometric supervision even without LiDAR annotations.
| Geometry Supervision | navtest EPDMS ↑ |
|---|---|
| None | 89.1 |
| DA3-only (w/o LiDAR) | 89.7 |
| Depth-only (LiDAR-calibrated DA3) | 89.8 |
| Depth + Occupancy (Full) | 90.2 |
(3) Robustness to Noisy Geometric Targets. We also evaluate robustness by randomly corrupting 20% of the sparse LiDAR depth measurements used to construct the training targets, using three types of perturbations sampled with equal probability. The navtest EPDMS changes only slightly from 90.237 to 90.195 (−0.042), demonstrating stable planning performance under noisy geometric supervision.
We thank the reviewer for their constructive feedback and welcome any further questions or discussion !
Thanks for your review
Dear Reviewer 6EC6,
We sincerely thank you for your thoughtful feedback, understanding, and support for our work !
Following your valuable suggestions, we have expanded both the related work discussion and experimental comparison tables in our revised PDF uploaded to the ICLR submission system to better cover the relevant studies you highlighted. We have also conducted additional ablation experiments to further validate the effectiveness of our key designs and strengthen the experimental evidence. If there are any additional relevant papers you believe we should consider, we would be more than happy to include appropriate citations.
Once again, we greatly appreciate your constructive suggestions, careful evaluation, and support for our work ! Your feedback has been invaluable in helping us improve the quality and completeness of our manuscript.
Best regards,
The Authors of Submission #7511
Rebuttal for Reviewer 6EC6 (part 1/2)
Q1,2,3&W1,2,3:Clarification of GuideFlow results and comparisons with recent methods
We will clarify and address the reviewer's concerns from two perspectives:
(1) Clarification of GuideFlow. The reported 27.1 EPDMS is directly taken from Table 1 of the original GuideFlow paper, corresponding to its NAVSIM-v2 navhard result without an external scorer. The same paper also reports 43.0 EPDMS with a scorer. Notably, the 27.1 result is consistently adopted in recent studies, including GraphWorld (Table IV), MomADv2 (Table 4), MomWorld (Table 5), and DriveFuture (Table 1). We acknowledge the higher result of 51.5 EPDMS on the AGC2025 leaderboard. However, our comparison follows the scorer-free configuration reported in the original publication, consistent with these prior studies. Therefore, the reported value is not a citation error.
(2) Expanded Comparisons. We appreciate the suggested methods and have expanded our Related Work discussion to include MomAD, MomADv2, MomWorld, GraphWorld, DriveFuture, DriveZero, DriveWorld-VLA, RAP and ZTRS. We have also collected the corresponding NAVSIM-v1 navtest, NAVSIM-v2 navtest, and NAVSIM-v2 navhard results from the original publications and incorporated them into Tables 1, 2 and 3 of our revised manuscript. Due to space constraints, more comprehensive comparisons are provided in Tables 21 and 22.
If the reviewer has any further requests regarding additional papers for comparison, we would be happy to address them and discuss them further!
W4 / Q4. Real-world deployment questions
Good suggestions. We have expanded our real-world evaluation to 30 trials per condition, introduced a CityWalker-only training baseline, and further clarified the evaluation protocol, safety mechanisms, and failure modes.
(1) Evaluation Protocol and Success Criteria. We evaluate Metis in indoor office lounge areas and outdoor sidewalks. Each trial consists of 10 predefined high-level navigation commands with each command lasting 2 seconds before automatically switching to the next. A trial is considered successful only if the robot correctly executes all 10 commands without any collision; otherwise, it is counted as a failure.
(2) Additional Trials and Baseline Comparison. We conducted 15 additional trials per condition, increasing the total from 15 to 30. The additional trials achieved 9/15 indoor and 11/15 outdoor successes, consistent with our initial results (8/15 and 10/15, respectively). We also evaluated a CityWalker-only baseline using the same architecture and deployment pipeline. Metis outperforms the CityWalker-only baseline by 30.0% indoors and 20.0% outdoors, demonstrating the benefits of our full training pipeline for zero-shot real-world transfer.
(3)Safety and Failure Analysis. Through trajectory inspection, we found that the predicted waypoints generally aligned with the intended navigation commands. We do not allow human intervention during the trials, and any collision is counted as a failure. Indoor collisions mainly occurred in confined spaces due to limitations of the default low-level controller. Outdoor failures were primarily associated with pedestrians unexpectedly approaching the robot, where the fixed 2-second trajectory update interval limited timely avoidance. No collisions with static obstacles were observed outdoors. We plan to improve the control logic and trajectory update mechanism in future work.
We have added the above experiments to the Appendix of the revised manuscript.
Q5: Clarification on video supervision strength
Good suggestion! The gradient ratios in Table 7 are computed without loss weights. With our default λvideo=4, the effective gradient ratio for Metis is 0.0784 (4×0.0196), compared with 0.0800 for Joint. Although the video gradient is smaller than the action gradient, it provides non-negligible auxiliary supervision. More importantly, this supervision improves EPDMS over Isolated by 0.7 on navtest and 1.4 on navhard, demonstrating its practical effectiveness. Furthermore, Appendix Table 13 already provides a video-loss weight ablation (2/4/8: 89.6/90.2/90.0 EPDMS), supporting our choice of λvideo=4.