acceptodds
Under review as a conference paper at ICLR 2027

Metis: A Generalizable and Efficient 4D World-Action Model for Autonomous Driving and Urban Navigation

Abstract

World-Action Models (WAMs) jointly model future observations and actions, but existing approaches often require explicit future observation generation during action inference, introducing substantial inference overhead. Moreover, future states are primarily modeled in semantic or 2D visual spaces, leaving their spatial structure insufficiently grounded for reliable planning. We introduce Metis, an end-to-end WAM that jointly learns future observation, action, and spatial geometry prediction during training, while requiring only action prediction at inference time. Metis employs specialized video generation and action experts coupled through asymmetric attention, allowing future video representations to attend to action representations while masking the reverse dependency. This design preserves supervision from future world modeling without requiring stochastic future video generation for action inference. We further introduce training-only spatial registers with complementary depth and occupancy supervision to enhance spatially grounded action representations without additional inference overhead. Extensive experiments demonstrate strong performance on NAVSIM-v2 and CityWalker, together with substantially reduced action-inference latency compared with full world-action generation. Zero-shot real-world deployment further confirms the practical feasibility of our approach.

Primary area:
applications to robotics, autonomy, planning
Keywords:
4D world action model, autonomous driving, efficient and generalizable

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_6EC6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Metis, an end-to-end world-action model for autonomous driving and urban navigation. Its core design—asymmetric world-action attention (joint modeling during training and action-only prediction at inference) and training-only spatial registers (with depth and occupancy supervision)—is conceptually simple and effective. Experimental results on NAVSIM-v2 navhard and navtest as well as CityWalker, together with the substantially reduced inference latency, support the effectiveness of the method.

Critical strengths

The paper identifies two concrete limitations of existing world-action models: inference-time future observation generation causes substantial latency, and future states are often modeled only in semantic or 2D visual spaces without sufficient 3D spatial grounding. Metis directly targets both issues.

Critical weaknesses

The paper does not compare against several recent and highly relevant methods, including DriveZero, DriveFuture, GraphWorld, DriveWorld-VLA, MomWorld, RAP, MomAD/MomADv2, ZTRS, and SimScale. These are directly comparable on NAVSIM-v1/v2 navtest and navhard, and some of them are current SOTA or close to it. This is consequential because the paper claims strong planning performance and practical superiority, but without these comparisons the claim is not properly positioned. In addition, the GuideFlow EPDMS reported in Table 1 appears inconsistent with public results, which report substantially higher values. If a baseline number is wrong or evaluated under a different protocol, the comparison is not trustworthy.

NAVSIM-v1 results are relegated to the appendix, while the main tables report only NAVSIM-v2. Since the two benchmarks use different metrics and protocols, the paper’s generalization claim is not fully supported by the main evidence. The real-world deployment is also thin: only 15 trials per condition, with 53.3% indoor and 66.7% outdoor success. This is consequential because the paper emphasizes practical feasibility and zero-shot deployment, but the current evidence is too limited to support that claim robustly.

GuideFlow is 51 on https://huggingface.co/spaces/AGC2025/e2e-driving-navhard, and the latest comparison methods should also be added. These two issues are the strongest grounds for rejection if left unresolved. If the authors add the missing baselines, correct or clarify the GuideFlow number, and move the NAVSIM-v1 results into the main comparison, the core contribution would be much better supported.

Questions for authors

Could the authors add comparisons with the latest methods on the official AGC2025 NAVSIM-v2 navhard leaderboard? In particular, please consider adding DriveZero, DriveFuture, GraphWorld, DriveWorld-VLA, MomWorld, RAP, MomAD, MomADv2, ZTRS, SimScale, and any newer SOTA entries. These should be reported on NAVSIM-v1 navtest and NAVSIM-v2 navtest/navhard under consistent protocols, so that Metis can be properly positioned against current work.

Could the authors clarify the GuideFlow result in Table 1? The official AGC2025 NAVSIM-v2 navhard leaderboard lists GuideFlow at 51, while Table 1 reports 27.1. Is 27.1 a Stage 2 score, an early version, a different EPDMS implementation, or a re-evaluation under a different protocol? Please explain the discrepancy and correct the table if needed.

Could the authors move the NAVSIM-v1 results from the appendix into the main comparison? Since NAVSIM-v1 uses PDMS and NAVSIM-v2 uses EPDMS, the current main tables do not fully show generalization across evaluation protocols. Please include NAVSIM-v1 navtest results in the main paper and discuss any differences in ranking between v1 and v2.

Could the authors strengthen the real-world evidence? The current deployment reports only 15 trials per condition, with 53.3% indoor and 66.7% outdoor success. Please add more trials, confidence intervals, failure cases, and safety-related metrics. Also clarify how success is defined and whether any human intervention or safety override occurred.

Could the authors clarify the strength and fairness of asymmetric attention? The reported gradient ratio is 0.0196 for Metis and 0.0200 for the Joint variant, which seems very small. How much does the video objective actually supervise the action expert in practice? Please provide ablations with matched compute and parameters, and show whether stronger video supervision changes the conclusion.

Could the authors better isolate the effect of spatial registers? The registers are removed at inference, but their geometry objectives still backpropagate through the queried representations during training. Could the gains come from additional trainable capacity rather than geometric grounding? Please compare against an auxiliary geometry loss without registers and report whether the main action/video branches change substantially.

Could the authors clarify the efficiency comparison? Metis with video is reported at 1.06s, while DriveVLA-W0 and PWM are reported at 0.69s and 0.83s. Were all methods evaluated on the same GPU, input resolution, and denoising setting? Please provide a fully controlled latency comparison.

Could the authors discuss robustness when geometric supervision is unavailable or noisy? The depth and occupancy targets rely on LiDAR and DA3. CityWalker does not provide LiDAR, and real-world deployment may not have reliable geometric labels. How sensitive is Metis to noisy or missing geometric supervision, and what happens if the geometry branch is trained with DA3-only targets?

3 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer 6EC6 (part 1/2)

Q1,2,3&W1,2,3:Clarification of GuideFlow results and comparisons with recent methods

We will clarify and address the reviewer's concerns from two perspectives:

(1) Clarification of GuideFlow. The reported 27.1 EPDMS is directly taken from Table 1 of the original GuideFlow paper, corresponding to its NAVSIM-v2 navhard result without an external scorer. The same paper also reports 43.0 EPDMS with a scorer. Notably, the 27.1 result is consistently adopted in recent studies, including GraphWorld (Table IV), MomADv2 (Table 4), MomWorld (Table 5), and DriveFuture (Table 1). We acknowledge the higher result of 51.5 EPDMS on the AGC2025 leaderboard. However, our comparison follows the scorer-free configuration reported in the original publication, consistent with these prior studies. Therefore, the reported value is not a citation error.

(2) Expanded Comparisons. We appreciate the suggested methods and have expanded our Related Work discussion to include MomAD, MomADv2, MomWorld, GraphWorld, DriveFuture, DriveZero, DriveWorld-VLA, RAP and ZTRS. We have also collected the corresponding NAVSIM-v1 navtest, NAVSIM-v2 navtest, and NAVSIM-v2 navhard results from the original publications and incorporated them into Tables 1, 2 and 3 of our revised manuscript. Due to space constraints, more comprehensive comparisons are provided in Tables 21 and 22.

If the reviewer has any further requests regarding additional papers for comparison, we would be happy to address them and discuss them further!

W4 / Q4. Real-world deployment questions

Good suggestions. We have expanded our real-world evaluation to 30 trials per condition, introduced a CityWalker-only training baseline, and further clarified the evaluation protocol, safety mechanisms, and failure modes.

(1) Evaluation Protocol and Success Criteria. We evaluate Metis in indoor office lounge areas and outdoor sidewalks. Each trial consists of 10 predefined high-level navigation commands with each command lasting 2 seconds before automatically switching to the next. A trial is considered successful only if the robot correctly executes all 10 commands without any collision; otherwise, it is counted as a failure.

(2) Additional Trials and Baseline Comparison. We conducted 15 additional trials per condition, increasing the total from 15 to 30. The additional trials achieved 9/15 indoor and 11/15 outdoor successes, consistent with our initial results (8/15 and 10/15, respectively). We also evaluated a CityWalker-only baseline using the same architecture and deployment pipeline. Metis outperforms the CityWalker-only baseline by 30.0% indoors and 20.0% outdoors, demonstrating the benefits of our full training pipeline for zero-shot real-world transfer.

MethodIndoor SR ↑95% CIOutdoor SR ↑95% CI
CityWalker-only26.7% (8/30)[14.2%, 44.4%]50.0% (15/30)[33.2%, 66.8%]
Metis (Ours)56.7% (17/30)[39.2%, 72.6%]70.0% (21/30)[52.1%, 83.3%]

(3)Safety and Failure Analysis. Through trajectory inspection, we found that the predicted waypoints generally aligned with the intended navigation commands. We do not allow human intervention during the trials, and any collision is counted as a failure. Indoor collisions mainly occurred in confined spaces due to limitations of the default low-level controller. Outdoor failures were primarily associated with pedestrians unexpectedly approaching the robot, where the fixed 2-second trajectory update interval limited timely avoidance. No collisions with static obstacles were observed outdoors. We plan to improve the control logic and trajectory update mechanism in future work.

We have added the above experiments to the Appendix of the revised manuscript.

Q5: Clarification on video supervision strength

Good suggestion! The gradient ratios in Table 7 are computed without loss weights. With our default λvideo=4\lambda_{\mathrm{video}}=4, the effective gradient ratio for Metis is 0.0784 (4×0.01964\times0.0196), compared with 0.0800 for Joint. Although the video gradient is smaller than the action gradient, it provides non-negligible auxiliary supervision. More importantly, this supervision improves EPDMS over Isolated by 0.7 on navtest and 1.4 on navhard, demonstrating its practical effectiveness. Furthermore, Appendix Table 13 already provides a video-loss weight ablation (2/4/8: 89.6/90.2/90.0 EPDMS), supporting our choice of λvideo=4\lambda_{\mathrm{video}}=4.

AuthorsOpenReview ↗

Rebuttal for Reviewer 6EC6 (part 2/2)

Q6: Questions about spatial registers and geometric supervision

To isolate the contribution of spatial registers, we introduce a Direct Geometry experiment that applies the same depth and occupancy losses directly to current/future video and action representations without registers. In contrast, our spatial registers query only current image and future action representations, excluding stochastic future-video tokens.

Variantnavtest EPDMS ↑navhard EPDMS ↑FVD ↓
Base (w/o Geometry)89.132.756.5
Direct Geometry89.533.372.1
Metis (Ours)90.233.856.5

Direct Geometry improves planning performance but degrades video generation quality, whereas our spatial registers achieve better planning performance while preserving the original FVD. These results suggest that spatial registers provide a more effective interface for geometric supervision, reducing interference with video generation without additional inference overhead.

Q7: Question about the efficiency comparison

Good suggestion! We re-measured the inference latency on a single RTX 4090 with batch size 1.

MethodFramesResolutionHorizonTotal PixelsEPDMS ↑Latency (s) ↓
DriveVLA-W0 (w/o video)—256×144——86.90.69
PWM (AR, w/ video)10128×2244 s286,72088.20.83
Metis (10-step, w/ video)8320×3844 s983,04090.21.06
Metis (10-step)————90.20.44
Metis (2-step)————90.00.14

As shown above, Metis (w/ video) generates 3.43× as many pixels as PWM, albeit with lower inference speed. Removing future-video generation achieves a 2.4× speedup (1.06s → 0.44s) without performance degradation. Metis also 1.57× faster than DriveVLA-W0, with a 3.3-point higher EPDMS. Furthermore, 2-step action inference achieves a 7.6× speedup (0.14s) over full video inference with only a 0.2-point EPDMS reduction. These results demonstrate the efficiency advantage of our training–inference decoupling, enabling fast trajectory planning without future-video generation while maintaining strong planning performance.

Q8: questions about geometric supervision

We thank the reviewer for this important question. We clarify our existing geometry supervision settings and provide additional experiments on missing and noisy geometric targets.

(1) Geometry Supervision in CityWalker and Real-World Deployment. As already described in Appendix A.2 of our original manuscript, CityWalker does not provide LiDAR observations. We therefore disable occupancy supervision and use depth pseudo-labels generated by a frozen Depth Anything 3 (DA3) model, without LiDAR-based metric calibration. Importantly, spatial registers and geometry prediction heads are used only during training and are entirely removed at inference. Thus, real-world deployment does not require LiDAR or geometric labels.

(2) Effect of Missing LiDAR Supervision. We additionally evaluate a DA3-only configuration on NAVSIM-v2, where depth supervision is provided exclusively by DA3 without LiDAR calibration, and occupancy supervision is disabled. We also include the original Depth-only configuration from Table 8 for comparison. The DA3-only setting achieves 89.7 EPDMS, only 0.1 below the LiDAR-calibrated Depth-only configuration and 0.5 below the full model. Meanwhile, it improves EPDMS by 0.6 over the model without geometric supervision. These results demonstrate that DA3 pseudo-labels provide effective geometric supervision even without LiDAR annotations.

Geometry Supervisionnavtest EPDMS ↑
None89.1
DA3-only (w/o LiDAR)89.7
Depth-only (LiDAR-calibrated DA3)89.8
Depth + Occupancy (Full)90.2

(3) Robustness to Noisy Geometric Targets. We also evaluate robustness by randomly corrupting 20% of the sparse LiDAR depth measurements used to construct the training targets, using three types of perturbations sampled with equal probability. The navtest EPDMS changes only slightly from 90.237 to 90.195 (−0.042), demonstrating stable planning performance under noisy geometric supervision.

We thank the reviewer for their constructive feedback and welcome any further questions or discussion !

AuthorsOpenReview ↗

Thanks for your review

Dear Reviewer 6EC6,

We sincerely thank you for your thoughtful feedback, understanding, and support for our work !

Following your valuable suggestions, we have expanded both the related work discussion and experimental comparison tables in our revised PDF uploaded to the ICLR submission system to better cover the relevant studies you highlighted. We have also conducted additional ablation experiments to further validate the effectiveness of our key designs and strengthen the experimental evidence. If there are any additional relevant papers you believe we should consider, we would be more than happy to include appropriate citations.

Once again, we greatly appreciate your constructive suggestions, careful evaluation, and support for our work ! Your feedback has been invaluable in helping us improve the quality and completeness of our manuscript.

Best regards,
The Authors of Submission #7511

Reviewer_5g5xOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Metis, a WAM that builds on the MOT architecture with asymmetric attention for autonomous driving and urban navigation. Two main contributions are introduced: 1) the idea of action prediction as a basic task, with video generation and geometry grounding as auxiliary tasks for a better understanding of world knowledge. 2) strong performance on the NavSim and CityWalker benchmarks with a fast inference setup without video generation and depth and occlusion prediction. Furthermore, the authors also validate their method in real-world quadruped deployment.

Critical strengths

The authors use action prediction as a base task and use future video generation and geometric grounding as additional supervision. With asymmetric information flow, the model can run action prediction separately.

Critical weaknesses
  1. The effect of the proposed asymmetric attention should be further validated. Since the video generation model (Wan 2.2 5B) is significantly larger than the other modules, how does the direct video-to-action information flow perform in your training setup? This should be compared and discussed.
  2. Lack of discussion and comparison with similar work, such as Drivedreamer-policy, which also seems to address geometry-based future video-action modeling.
  3. Incomplete citation in Table 2 of DiffusionDrive.
Questions for authors

See weaknesses.

3 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer 5g5x (part 1/1)

W1. Effectiveness of Direct Video-to-Action Information Flow

Good suggestion! If we understand the reviewer's concern correctly, direct video-to-action information flow may refer to two different designs, both of which we have additionally evaluated: Synchronous V→A: Action tokens attend to noisy future-video representations, while video tokens cannot attend to action tokens. Both experts perform simultaneous denoising. Sequential IDM: The action expert is conditioned on ground-truth future frames during training. At inference, the video expert first generates future frames, which are subsequently used for action prediction.

MethodInformation Flownavtest EPDMS ↑navhard EPDMS ↑Inference Latency (s) ↓Training Time (h) ↓
JointVideo ↔ Action88.931.71.1620
IsolatedNone89.532.40.4420
Synchronous V→AVideo → Action88.831.51.0920
Sequential IDMVideo → Action90.033.21.2445
Metis (Ours)Action → Video90.233.80.4420

Comparison with Synchronous V→A. Synchronous V→A performs slightly worse than Joint and Isolated, suggesting that directly conditioning action prediction on noisy future-video representations does not improve planning performance. This is consistent with our findings in Appendix Table 20, where increasing noise in future-video representations degrades planning performance.

Comparison with Sequential IDM. Sequential IDM performs better than Synchronous V→A, potentially benefiting from conditioning on completed future-video predictions rather than noisy intermediate representations. However, compared with IDM, Metis achieves higher EPDMS on both navtest and navhard, while delivering 2.8× faster inference and requiring 2.25× less training time.

Overall, these results support the effectiveness and efficiency of our asymmetric design, which leverages video prediction losses to supervise the action expert without requiring future-video or geometry prediction during inference.

W2. Discussion and Comparison with DriveDreamer-Policy

We thank the reviewer for highlighting DriveDreamer-Policy, an important and closely related work! We appreciate its elegant integration of geometric grounding, future video generation, and motion planning within a modular world-action framework.

DriveDreamer-Policy adopts a VLM-centric architecture that processes language instructions, multi-view observations, and action context through learnable depth, video, and action queries. These queries condition three specialized generative experts for depth, future video, and trajectory prediction. Its structured 3D → 2D → 1D information flow enables video queries to incorporate geometric representations and action queries to leverage both depth and video context. Since these interactions occur at the query level, action prediction does not require explicit depth or future-video generation during inference.

Metis, in contrast, builds upon a pretrained video generation backbone with specialized video and action experts. Our asymmetric attention enables video prediction losses to directly supervise the action expert while preventing noisy future-video representations from affecting action prediction. Furthermore, training-only spatial registers provide complementary depth and occupancy supervision, enabling efficient action-only inference without future-video or geometry generation.

We have included DriveDreamer-Policy in Tables 1 and 2 of our revised manuscript. For clarity, we summarize the comparison below:

MethodSensorsNAVSIM-v1 navtest (PDMS ↑)NAVSIM-v2 navtest (EPDMS ↑)
DriveDreamer-Policy3×C89.288.7
Metis (Ours)1×C89.990.2

Metis achieves competitive results on both benchmarks using only a single front-facing camera, compared with the three-camera setup of DriveDreamer-Policy.

W3. Missing Citation of DiffusionDriveV2

Thank you for pointing out this omission.! We have corrected the reference to DiffusionDriveV2 in Table 2.

We have added the additional comparisons to the revised manuscript. We thank the reviewer for their constructive feedback and welcome any further questions or discussion !

Thanks to the authors for the rebuttal and my concerns are resolved. After reviewing the other reviewers' comments, I maintain my initial rating of 3 weak accept.

AuthorsOpenReview ↗

Thanks for your review

Dear Reviewer 5g5x,

Thank you very much for your thoughtful reassessment and continued support! We sincerely appreciate the time and care you have devoted to reviewing our paper, considering our rebuttal, and engaging with the broader discussion. We are glad that our responses and clarifications have resolved your concerns. Your constructive feedback has helped us strengthen the paper’s clarity and rigor, and we are grateful for your valuable contribution to improving our work !

Best regards,
The Authors of Submission #7511

Reviewer_42jjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Metis, a world-action model designed for autonomous driving and urban navigation, which employs a "joint training, decoupled inference" paradigm. Its core design features include the use of asymmetric attention, allowing future video prediction to supervise action experts while eliminating the dependency of action inference on future video generation. Additionally, it incorporates spatial registers—utilized exclusively during training—to enhance spatial representations through auxiliary supervision based on depth and occupancy. The authors evaluate the model on NAVSIM-v2 and CityWalker, conducting ablation studies to examine the roles of information flow, geometric supervision, and future video supervision. The default model achieves an EPDMS of 33.8 on navhard and 90.2 on navtest; notably, it reduces inference latency from 1.06 seconds to 0.44 seconds compared to full video-action generation, all while maintaining the same performance score on navtest. The paper also reports on navigation experiments using a Go2 quadruped robot, where the model was fine-tuned on CityWalker without requiring any additional training in the deployment environment.

Critical strengths
  1. The asymmetric attention design is well motivated and directly tested: changing future-video noise does not affect action predictions, while the video loss still backpropagates to the action expert. On a single RTX 4090, removing video generation reduces latency from 1.06 to 0.44 seconds while maintaining 90.2 navtest EPDMS, showing clear practical value.

  2. The ablations support the usefulness of the auxiliary prediction objectives. With the VGE retained and geometry branches disabled, video supervision improves navhard EPDMS from 29.4 to 32.7. Adding depth and occupancy supervision further raises it to 33.8, suggesting that the gains do not come solely from the pretrained backbone.

Critical weaknesses
  1. Table 4 does not include a sequential baseline that first predicts future observations and then infers actions with an inverse dynamics model. Reporting its planning performance and latency would give a more complete picture of the trade-offs.
  2. Table 4 confirms the intended information flow, but does not fully explain why asymmetric interaction outperforms bidirectional interaction. Sensitivity to video noise alone does not show that it causes worse planning. A comparison with controlled video-noise levels would strengthen this explanation.
  3. The driving evaluation is limited to the NAVSIM family, including v2 navtest and navhard and the v1 results in the appendix. Evaluation on an independent driving dataset or a different closed-loop environment would better support the generalization claims.
  4. The real-world evaluation is limited to 15 trials per condition, with success rates of 53.3% and 66.7% in Table 9 and no baseline comparison. More trials, baselines under the same conditions, and failure analysis would make the results more convincing.
  5. The methodological novelty over FastWAM appears limited, and the paper seems to focus more on additional experimental validation.
Questions for authors
  1. What are the main technical contributions beyond FastWAM? Please clarify the substantive differences in asymmetric interaction and training–inference decoupling.
  2. Have you evaluated a sequential baseline that first predicts future observations and then infers actions with an inverse dynamics model? If so, how does it compare in planning performance and latency under comparable settings?
  3. Table 4 confirms the intended information flow, but what evidence links Joint’s lower planning performance specifically to future-video noise rather than other training factors? Are the gains from asymmetric interaction consistent across training seeds?
  4. What are the success and failure criteria for the real-world trials in Table 9, and what are the main failure modes? Do failures mainly come from model planning, trajectory tracking, or low-level control? This distinction matters when assessing deployment capability.
4 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer 42jj (part 1/2)

W1/Q2. Compare with sequential inverse dynamics model (IDM)

Good suggestion! We implemented a sequential IDM for comparison. All variants use 10 denoising steps, and latency is measured on a single RTX 4090 with batch size 1.

Methodnavtest EPDMS ↑navhard EPDMS ↑Latency (s/sample) ↓Training time(h) ↓
Joint88.931.71.1620
Isolated89.532.40.4420
IDM90.033.21.2445
Metis90.233.80.4420

Compared with IDM, Metis achieves higher planning performance on both navtest and navhard, while delivering 2.8× faster inference (0.44 vs. 1.24 s/sample). Moreover, under the same experimental setting, sequential IDM requires 2.25× longer training time (45 vs. 20 hours). These results demonstrate that Metis provides a better overall solution in terms of planning performance, inference efficiency, and training cost.

W2/Q3. Controlled future-video perturbations and consistency across training seeds

Good question! We address this concern with a controlled future-video perturbation experiment and an additional comparison across training seeds. These results further support the effectiveness of our asymmetric interaction design:

1. Controlled future-video perturbation experiment.

We evaluated the same Joint checkpoint on 100 challenging scenes selected based on the numbers of vehicles and pedestrians within a 120° forward field of view. Using the same scenes across all conditions, we varied the magnitude of additional perturbations applied to the future-video latents read by the action expert.

Method0% Noise10% RMS Noise50% RMS Noise
Joint (Bidirectional)86.486.386.0
Metis (Asymmetric)87.1$_{+0.7}$87.1+0.8_{+0.8}87.1_{+1.1}\

As the noise level increases, Joint's EPDMS gradually decreases, while Metis remains stable. This indicates that noise in future-video representations can negatively affect planning performance under the Joint paradigm, whereas our asymmetric design avoids this degradation.

2. Consistency across training seeds.

We conducted an additional controlled comparison under same settings. Joint and Metis were trained with identical configurations except for the interaction mask, using three independent training seeds per variant. Metis consistently outperforms Joint with low variance across training seeds, indicating that the gains from asymmetric interaction are stable.

Methodnavtest EPDMS ↑
Joint88.9 ± 0.08
Metis90.2 ± 0.07

W4/Q4. Questions about real-world deployment

Good suggestions. We expanded the real-world evaluation to 30 trials, introduced a CityWalker-only training baseline.

1. Evaluation protocol and success criteria. We evaluate Metis in indoor office lounge areas and outdoor sidewalks. Each trial consists of 10 predefined high-level navigation commands, with each command lasting 2 seconds before automatically switching to the next. A trial is considered successful only if the robot correctly follows all 10 commands without any collision. Otherwise, it is counted as a failure.

2. Additional trials and baseline comparison.

EnvironmentMetis (Success ↑)baseline (Success ↑)
Indoor17/30 (56.7%)8/30 (26.7%)
Outdoor21/30 (70.0%)15/30 (50.0%)

Indoor environments are unseen in the training data, making them a zero-shot setting for the baseline and resulting in lower success rates. In contrast, outdoor trials were conducted on sidewalks, which better match the CityWalker training distribution and explain the baseline's relatively stronger outdoor performance.

3. Failure analysis. We inspected the failure cases and found that predicted trajectories aligned with the navigation commands. Outdoor failures mainly resulted from pedestrians unexpectedly approaching the robot, while the fixed 2-second trajectory update interval limited timely avoidance. No collisions with static obstacles were observed outdoors. Indoor failures primarily occurred in confined spaces, where the default low-level controller continued moving despite small predicted displacements. We will improve the control logic and trajectory update mechanism in future work.

AuthorsOpenReview ↗

Rebuttal for Reviewer 42jj (part 2/2)

W3:Evaluation on different closed-loop benchmarks.

We thank the reviewer for this valuable suggestion! We conducted experiments on two closed-loop driving benchmarks, Bench2Drive (CARLA) and HUGSIM. The complete results have been added to Tables 4 and 5 of the revised manuscript, with the newly added content highlighted in blue for clarity. We also provide visualizations of both methods in the Appendix Figure 11 and Figure 12 !

1. Closed-loop evaluation on Bench2Drive (v0.03).

We evaluate Metis under the base testing set (220 routes), without expert feature distillation. As shown below, Metis achieves the highest Driving Score (79.63), Success Rate (55.91%) and Multi-Ability Mean (69.77%), outperforming representative methods such as ReCogDrive and ORION.

MethodDS ↑SR (%) ↑Multi-Ability Mean ↑
MomAD47.9118.1118.66
GraphWorld51.5525.4724.47
MomADv252.3224.2425.84
ReCogDrive71.3645.4542.03
ORION77.7454.6254.72
Metis (Ours)79.6355.9169.77

2. Zero-shot photorealistic closed-loop evaluation on HUGSIM.

We evaluate Metis on HUGSIM using the model evaluated on NAVSIM-v2. Metis achieves the highest average Road Completion (58.3) and HD-Score (43.3), outperforming existing methods. These results demonstrate the strong zero-shot generalization ability of Metis.

MethodRC E ↑RC M ↑RC H ↑RC X ↑RC Avg. ↑HD E ↑HD M ↑HD H ↑HD X ↑HD Avg. ↑
UniAD58.641.240.426.040.648.729.527.314.328.9
VAD38.727.025.523.027.924.39.910.48.212.3
GTRS-Dense64.250.020.722.338.055.539.011.714.328.6
DriveWAM76.633.833.930.243.664.016.116.112.627.2
Metis (Ours)83.270.951.527.758.371.453.834.014.143.3

These additional experiments demonstrate Metis's strong closed-loop driving performance across diverse tasks on Bench2Drive and its ability to generalize to a photorealistic driving environment without target-domain fine-tuning, providing further evidence of its effectiveness.

W5 / Q1. Technical contributions beyond Fast-WAM

Good question!We believe Metis differs substantially from Fast-WAM in its technical design. We believe the main differences lie in the following three aspects:

(1) Asymmetric attention. Unlike Fast-WAM primarily focuses on video–action co-training, Metis introduces asymmetric attention that enables action-conditioned future-video prediction, allowing video prediction losses to directly optimize the action expert. This encourages action representations to capture future world dynamics rather than simply benefiting from shared visual representations.

(2) Training-Only Spatial Grounding. Beyond video prediction, Metis also introduces dedicated spatial registers for depth and occupancy supervision, allowing geometric objectives to further enhance the action expert's modeling of future world states.

(3) Training–Inference Decoupling. Our asymmetric design preserves direct video and geometric supervision during training while eliminating dependencies on future-video and geometry prediction during inference, enabling efficient action-only generation.

Beyond methodological differences, we comprehensively validate Metis through open-loop, closed-loop, and zero-shot real-world robotic experiments across autonomous driving and urban navigation, demonstrating its effectiveness and generalization.

We have incorporated the experimental results and corresponding revisions into the revised manuscript. We sincerely thank the reviewer for their constructive feedback and would be happy to address any further questions or concerns!

Reviewer_42jjOpenReview ↗

Updated Rating

Thank you for the detailed response. The added IDM comparison, results across training seeds, and Bench2Drive and HUGSIM evaluations largely address my main experimental concerns. The expanded real-world evaluation and baseline comparison are also helpful. I still have some reservations about the novelty over Fast-WAM and the reliability of real-world deployment, but the additional evidence increases my confidence in the paper. I am therefore raising my score by one point.

AuthorsOpenReview ↗

Thank You for the Updated Assessment

Dear Reviewer 42jj,

Thank you very much for your thoughtful reassessment and encouraging feedback! We sincerely appreciate the time and effort you have devoted to carefully reviewing our work and considering our additional experiments and clarifications. We are pleased that the new evidence has addressed your main experimental concerns and strengthened your confidence in the paper. Your constructive feedback has been invaluable in helping us improve the rigor and presentation of our work. We also appreciate your remaining concerns and will continue to address them in the revised manuscript.

Best regards,
The Authors of Submission #7511

Comments on the paper

1 comment

Dear reviewers,

As the rebuttal period is coming to an end, I kindly remind you to check the authors' responses. If you have not yet responded, please indicate whether your concerns have been adequately addressed. Many thanks!

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.