- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AffordDrive3D, an affordance- and geometry-aware World-Action Model for end-to-end autonomous driving. Built upon a shared vision-language model (VLM) backbone, the proposed framework jointly models future RGB frames, depth, drivable regions, collision-critical regions, and ego-vehicle trajectories. Specifically, the authors leverage predicted RGB latents to facilitate future depth prediction and introduce Temporal Affordance Forecasting to enable the model to learn action-relevant spatial information in both current and future scenes. The paper reports 91.3 PDMS on NAVSIM v1 and 89.9 EPDMS on NAVSIM v2, supported by ablation studies and qualitative visualizations.
Critical strengths
1.Well-motivated research direction. Rather than predicting only future RGB observations or geometric structures, the paper additionally models spatial regions relevant to ego-vehicle planning. This makes the supervision objectives of the world model more closely aligned with downstream driving decisions.
2.Task-relevant affordance label design. The construction of collision-critical regions through collision analysis of candidate trajectories, rather than directly relying on conventional object-category labels, is a meaningful design choice that enhances the task relevance of the supervision signals (Appendix A.1).
3.Relatively comprehensive experimental evaluation. Table 3 demonstrates the positive contributions of different auxiliary branches to planning performance, while Tables 4–5 further investigate temporal prediction capabilities and VLM representations, providing useful experimental observations.
Critical weaknesses
1.Lack of a direct ablation between current-frame and temporal affordance supervision. Table 3 demonstrates that adding affordance supervision is beneficial, but it does not establish whether the improvement comes from spatial perception of the current frame or from forecasting future affordances. Consequently, the necessity of the proposed Temporal Affordance Forecasting mechanism has not been sufficiently validated.
2.The paper does not compare independent depth prediction against RGB-conditioned depth prediction, nor does it report quantitative accuracy for future geometric predictions. This makes it difficult to assess the contribution of RGB conditioning to geometry prediction and downstream planning performance.
3.The full model in Table 3 achieves 89.6 PDMS, whereas the final reported result reaches 91.3 PDMS. However, component-level ablations under identical RL training conditions are missing, as are experiments examining combinations such as Affordance + Geometry. This makes it difficult to determine the independent contributions and complementarity of individual components.
4.The SOTA claim and baseline comparisons require clarification. Table 1 does not include the published DriveSuprim method, which reports 93.5 PDMS on NAVSIM v1. Given that DriveSuprim reports 93.5 PDMS on NAVSIM v1, the paper should clarify whether the evaluation settings are directly comparable and specify the scope of its SOTA claim.
Minor Weaknesses
1.The experiments are restricted to NAVSIM. Although results are reported on both v1 and v2, these are not independent datasets. The generalization capability of the proposed method under different data distributions and more challenging driving environments remains unclear.
2.Lack of performance stability analysis. Some improvements over competing methods are smaller than 1 PDMS point, yet the paper does not report variations across random seeds or provide statistical analyses.
3.Although the auxiliary branches are not used during inference, the final system still employs a relatively large VLM and Action DiT. The paper does not report inference latency, memory requirements, or computational cost comparisons with other approaches.
4.Limited analysis of long-horizon affordance prediction. Table 4 shows that collision-critical AP decreases to 0.732 at the 2–4 s prediction horizon. However, the paper does not further investigate the associated failure cases or the relationship between prediction errors and planning performance.
Questions for authors
1.Table 3 shows that affordance supervision improves planning performance, but does not separate current-frame supervision from future prediction. How does the model perform when only current-frame affordance supervision is used?
2.In Appendix A.1, the collision-critical object set is determined at the initial timestep and shared across future frames. How does this account for objects that become collision-critical at later timesteps?
3.Is conditioning depth prediction on RGB latents necessary? A comparison with independent depth prediction would help clarify its contribution to planning performance.
4.The full model in Table 3 achieves 89.6 PDMS, while the final result reaches 91.3 PDMS after RL training. Can the authors provide ablations under the same RL setting to clarify the contributions of the individual components?
5.Table 1 does not include DriveSuprim, which reports 93.5 PDMS on NAVSIM v1. Is there a difference in evaluation settings that prevents a direct comparison? The scope of the SOTA claim should be clarified.
2 comments
Minor W1. Generalization beyond NAVSIM
We additionally evaluate AffordDrive3D in closed loop on NavSafe-∞ [1], a photorealistic benchmark of 280 event-centered scenarios spanning 28 safety-critical event types, built on 3DGS reconstructions of real-world driving logs. The reconstructed scenes are edited to stage each event, e.g., by inserting road users or hazards, scripting behaviors such as lead-vehicle braking or cut-ins, or setting traffic-light states. Following the official protocol, all policies are evaluated on all 280 scenarios with three seeds (mean ± std); baseline results are taken from the NavSafe-∞ paper.
| Policy | DS↑ | SR↑ | DE↑ | Comf.↑ | TC↑ | VRUC↑ | TV↑ | TI↑ |
|---|---|---|---|---|---|---|---|---|
| VLA-based | ||||||||
| ReCogDrive-2B-RL | 53.64 ± 1.42 | 30.95 ± 2.06 | 173.96 ± 1.90 | 85.10 ± 0.49 | 35.33 ± 2.31 | 18.33 ± 6.29 | 32.42 ± 1.39 | 27.78 ± 5.09 |
| MTDrive-mtGRPO | 55.86 ± 0.50 | 40.60 ± 0.90 | 277.62 ± 3.17 | 93.94 ± 0.41 | 40.33 ± 2.89 | 16.67 ± 1.44 | 46.36 ± 0.91 | 52.22 ± 3.85 |
| AutoVLA | 51.05 ± 0.80 | 29.52 ± 2.43 | 186.02 ± 0.58 | 65.37 ± 0.47 | 33.00 ± 2.65 | 15.00 ± 4.33 | 29.70 ± 2.92 | 36.67 ± 8.82 |
| WAM-based | ||||||||
| DriveVLA-W0 | 45.06 ± 0.64 | 27.98 ± 1.80 | 256.31 ± 1.01 | 62.74 ± 1.14 | 26.67 ± 2.31 | 7.50 ± 0.00 | 33.94 ± 2.29 | 37.78 ± 1.92 |
| SimWAM | 57.53 ± 2.72 | 37.50 ± 1.56 | 160.54 ± 1.64 | 97.19 ± 0.27 | 41.33 ± 0.58 | 26.67 ± 7.22 | 40.61 ± 0.52 | 27.78 ± 5.09 |
| AffordDrive3D (Ours) | 58.56 ± 0.18 | 45.12 ± 0.90 | 166.99 ± 0.67 | 90.87 ± 0.62 | 44.33 ± 0.58 | 23.33 ± 1.44 | 54.55 ± 1.57 | 42.22 ± 5.09 |
These edited, event-centered scenarios differ from the logged NAVSIM data and are considerably more challenging. AffordDrive3D achieves the highest driving score (DS) and success rate (SR) among all VLA- and WAM-based methods, demonstrating its generalization to safety-critical closed-loop driving.
Minor W2. Performance stability across seeds
We evaluate with 3 inference sampling seeds. NAVSIM v1 PDMS is 89.60 ± 0.03 after SFT and 91.33 ± 0.04 after RL, consistent with our reported results. This variance is far smaller than the improvements over competing methods, indicating that these gains are stable.
Minor W3. Inference latency and computational cost
All auxiliary branches, including the 1.3B video diffusion model and the geometry and affordance decoders, are discarded at inference. We measure the model inference on a single NVIDIA RTX 5090:
| Component | Params | Latency (ms) | Compute |
|---|---|---|---|
| VLM (Qwen3-VL-2B, BF16) | 2.13B | 122.6 ± 0.2 | ~13.7 TFLOPs |
| Action DiT (10 denoising steps, FP32) | 0.82B | 94.1 ± 0.4 | ~0.11 TFLOPs |
| Others (queries, state encoder, etc.) | 0.08B | – | – |
| Total | 3.03B | 217.9 ± 2.8 | ~13.8 TFLOPs |
Peak inference memory is 8.4 GiB (7.3 GiB for weights). Both latency and memory of the Action DiT, currently run in FP32 with sequential denoising, can be further reduced with lower precision or fewer sampling steps.
Minor W4. Long-horizon affordance prediction analysis
Accuracy decay at longer horizons is common to future prediction in driving. For example, 4D occupancy world models drop sharply from 1 s to 3 s on Occ3D-nuScenes [2] (OccWorld [3]: 25.78→10.51 mIoU; DOME [4]: 35.11→20.29 mIoU). In comparison, our decay is mild: collision-critical AP decreases from 0.918 to 0.732, and drivable-area AP stays high (0.986→0.971). The larger drop for collision-critical regions is expected, since they depend on the future behavior of dynamic agents, while drivable areas follow stable road structure. Despite this decay, future affordance supervision still benefits planning, improving TTC from 94.3 to 95.1 over current-frame supervision alone (see table in W1&Q1).
[1] Bao, Yuxin, et al. "NavSafe-$\infty $: Benchmarking Closed-Loop Driving Safety in Photorealistic Environments." arXiv preprint arXiv:2609.26618 (2026).
[2] Tian, Xiaoyu, et al. "Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving." Advances in neural information processing systems 36 (2023): 64318-64330.
[3] Zheng, Wenzhao, et al. "Occworld: Learning a 3d occupancy world model for autonomous driving." European conference on computer vision. Cham: Springer Nature Switzerland, 2024.
[4] Gu, Songen, et al. "Dome: Taming diffusion model into high-fidelity controllable occupancy world model." arXiv preprint arXiv:2410.10429 (2024).
We thank the reviewer for the comments.
W1&Q1. Current-frame vs. temporal affordance supervision
We ablate the temporal component under the affordance–action setting (SFT):
Current-frame supervision improves spatial perception (+0.7 PDMS), while future affordance forecasting brings further gains, most notably in TTC (94.3→95.1), which is unchanged with current-frame supervision alone. This indicates that forecasting future drivable and collision-critical regions helps the model anticipate upcoming conflicts, validating the necessity of Temporal Affordance Forecasting.
W2&Q3. RGB-conditioned vs. independent depth prediction
(a) Effect of conditioning on predicted RGB. Under the geometry–action setting (SFT), we compare conditioning depth on the predicted future RGB latents against copying the current-frame RGB latent to all future latent groups. In both variants, no RGB loss gradient reaches the VLM, so the only difference is the conditioning signal (depth metrics calculated on the front view against DA3 pseudo-GT):
Conditioning on predicted future RGB improves both future depth accuracy and planning, especially TTC.
(b) Future depth quality. Our full model substantially outperforms a copy-last baseline that reuses the current-frame DA3 depth for future frames:
W3&Q4. Component ablations under the RL setting
We apply the same RL training to all settings in Table 3:
The gains from each component persist after RL. Since the VLM is frozen during RL and only the action DiT is updated, these gains come from the shared representations learned with auxiliary supervision during SFT, which continue to benefit planning after RL. The full model outperforms every single-component variant, indicating that the components are complementary.
W4&Q5. Comparison with DriveSuprim and scope of SOTA claim
DriveSuprim is included in our NAVSIM v2 comparison (Table 2), which uses the extended EPDMS metric with additional rule-compliance and comfort criteria and thus provides a more comprehensive evaluation. Under this setting, AffordDrive3D outperforms DriveSuprim, with higher lane keeping (97.4 vs. 97.0), consistent with our drivable-area affordance that guides the model to drive within lanes in the legal travel direction.
Q2. Collision-critical objects emerging at later timesteps
Objects that become critical at later timesteps are already captured. We roll out each candidate trajectory over the full 4 s horizon, with other agents replaying their logged future motion, and mark every object involved in an at-fault collision at any point during the rollout (e.g., a vehicle cutting in at 3 s). The only objects excluded are those not present in the initial frame: since they cannot be inferred from the input observation, labeling them would introduce supervision the model cannot learn from and act as label noise. This also keeps the labels aligned with the planning decision at t0, from which the trajectory is generated.