- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper aims to address a structural limitation of World Action Models (WAMs) which is they treat other agents as part of the world model rather than as decision-makers that shape the ego’s action. ReWAM addresses this by modeling ego and other agents as mutually responsive conditional responders through a Level-k game hierarchy. On the NAVSIM benchmark, ReWAM achieves PDMS of 90.6, with gains concentrated in the most interaction-intensive scenarios.
Critical strengths
- The paper articulates the gap between WAMs and interaction-aware prediction/planning, and proposes a principled bridge.
- The interaction stratification indicates ReWAM’s advantage over baselines grows monotonically with interaction intensity
Critical weaknesses
- NAVSIM evaluates open-loop planning against cached logged context, and other agents do not actually respond to the ego’s predicted actions, and the method uses ground-truth scene annotations for current agent states. Thus the reciprocal-response gains are measured where other agents never adapt to the ego plan. So the core claim of mutually responsive interaction is not directly demonstrated in a reactive, closed-loop setting.
- The method learns from demonstrations by assuming the logged joint policy components are reward-maximizing best responses to one another, but real driving logs are not necessarily game-theoretically optimal or level-k consistent. The optimal reasoning depth, the weight schedule α_k ∝ k^p, and K_train are also chosen somewhat ad hoc and shown to yield diminishing (even negative) returns beyond Level-4, suggesting dataset-dependence without principled selection.
- The gains are marginal on aggregate and partly lower than the closest SOTA on some metrics. Nevertheless, the benchmark has already been updated recently.
- All experiments are on NAVSIM only with no cross-dataset validation Long-horizon, multi-camera, or occluded interactions where reciprocal reasoning matters most, are not tested.
- The method builds on DriveLAW and contributes the Level-k interaction block and a parameter-shared Other Action DiT. The reciprocal benefit may partly inherit the backbone’s world-modeling strength, and no ablation isolates the contribution of the strategy encoder / parameter sharing from the cross-agent attention itself.
Questions for authors
- Does the reciprocal-response benefit survive true closed-loop interaction? An evaluation to directly substantiate mutually responsive interaction should be provided.
- Is the aggregate SOTA lead statistically significant and robust?
- How are the Level-k training depth and weight schedule α_k ∝ k^p selected, and how sensitive are results to them?
- How much of the gain comes from the reciprocal-reasoning design vs. the added capacity?
4 comments
Evidence for ReWAM’s SOTA Lead
Q2: Is the aggregate lead over baselines statistically significant and robust?
Response: Thank you for comments on how well the benchmark results support our claim.
On NAVTEST dataset, which comprises more than 12,000 scenarios, ReWAM ranks first among the compared methods with a PDMS of 90.6, surpassing the previous best reported score of 90.4 from ReWorld. The evidence extends beyond this aggregate comparison: ReWAM outperforms both DriveLaW and DiffusionDrive in all five interaction-intensity groups on NAVTEST. It also ranks first among the methods compared on NAVSIM v2, with an EPDMS of 86.3 under a reactive traffic policy (Table 1). Taken together, the comprehensive evaluation results provide strong support for ReWAM's state-of-the-art performance.
We appreciate the opportunity to clarify the evidence behind this claim.
Sensitivity to Training Depth and Level Weights
Q3/W2: How are the training depth and the weight schedule selected, and how sensitive are the results to these choices?
Response: Thanks for your comments on how we chose the reasoning depth and the level weights, and how much these choices affect performance.
Training depth. The submitted paper compares training depths from to . PDMS rises from 90.03 at Level-1 to 90.48 at Level-2 and 90.55 at Level-3, reaching its highest reported value of 90.64 at Level-4. We therefore selected : it achieves the best observed performance without requiring extra reasoning depths.
Level weights. The submitted paper uses quadratic level weights (). To assess sensitivity to the exponent , we also trained models with linear weights (, i.e., ). At our selected depth of Level-4, PDMS is 90.62 for and 90.64 for , a difference of only 0.02 points. Across Levels 1–4, the largest absolute difference at any matched depth is 0.05 points at the reported precision (Table 2). These small differences indicate that PDMS is largely insensitive to the choice between linear and quadratic weighting.
We appreciate your suggestion to explore the influence of training depth and level weights on the performance of our framework.
Table 2. Sensitivity to training depth and the level-weight exponent on NAVTEST.
| NC | DAC | TTC | Comfort | EP | DDC | PDMS | ||
|---|---|---|---|---|---|---|---|---|
| 1 | 1 | 99.06 | 97.39 | 96.25 | 99.97 | 83.28 | 98.12 | 90.03 |
| 1 | 2 | 99.06 | 97.39 | 96.25 | 99.97 | 83.28 | 98.12 | 90.03 |
| 2 | 1 | 99.21 | 97.76 | 97.08 | 99.98 | 83.46 | 98.16 | 90.49 |
| 2 | 2 | 99.25 | 97.68 | 97.13 | 99.97 | 83.44 | 98.17 | 90.48 |
| 3 | 1 | 99.24 | 97.71 | 97.05 | 99.98 | 83.52 | 98.14 | 90.50 |
| 3 | 2 | 99.26 | 97.74 | 97.13 | 99.98 | 83.54 | 98.15 | 90.55 |
| 4 | 1 | 99.28 | 97.76 | 97.19 | 99.97 | 83.58 | 98.15 | 90.62 |
| 4 | 2 | 99.25 | 97.81 | 97.18 | 99.96 | 83.60 | 98.14 | 90.64 |
Reciprocal Reasoning Versus Additional Prediction Capacity
Q4/W5: How much of the gain can be attributed to reciprocal reasoning rather than additional model capacity?
Response: Thank you for concerns on whether the additional prediction branch can explain ReWAM's gain over DriveLaW.
To test this, we augmented DriveLaW with the same Other Action DiT used in ReWAM. The augmented baseline and ReWAM receive identical inputs during training and inference, including the same annotated current states of surrounding agents. Additionally, both use the same future trajectories for auxiliary supervision and are fine-tuned for the same number of steps.
As shown in Table 3, the augmented baseline raises PDMS from 89.1 to only 89.4, whereas ReWAM achieves 90.6. Thus, 1.2 of the 1.5 PDMS points gained over DriveLaW remain when both models include the additional prediction branch. This gap supports the joint contribution of cross-agent attention and Level- updates beyond predicting other agents' trajectories as an auxiliary task.
We appreciate your suggestion to test this distinction under matched conditions.
Table 3. NAVSIM ablation. DriveLaW + Other Action DiT and ReWAM use identical inputs and auxiliary supervision.
| Method | NC | DAC | TTC | Comfort | EP | PDMS |
|---|---|---|---|---|---|---|
| DriveLaW | 99.0 | 97.1 | 96.7 | 100.0 | 81.3 | 89.1 |
| DriveLaW + Other Action DiT | 99.0 | 97.0 | 95.8 | 100.0 | 81.7 | 89.4 |
| ReWAM (ours) | 99.3 | 97.8 | 97.2 | 100.0 | 83.6 | 90.6 |
Performance Under Reactive Traffic
Q1/W1: Do the benefits of reciprocal conditioning persist when other agents can respond in closed loop?
Response: Thank you for asking whether the results extend to traffic that responds to the ego vehicle.
To test ReWAM beyond non-reactive log replay, we evaluated it on NAVSIM v2 with a reactive traffic policy. In this setting, simulated surrounding vehicles can change their motion in response to the ego vehicle. As shown in Table 1, ReWAM achieves the highest EPDMS among the compared methods in this reactive setting. Compared to DriveLaW, ReWAM demonstrates significant improvements in most metrics, including EPDMS, EP, TTC, and DAC. These results show that ReWAM remains competitive when surrounding traffic can react to the ego vehicle. This evaluation tests the full planner under reactive traffic; it does not isolate the gain from reciprocal conditioning under continuous closed-loop replanning.
We appreciate your emphasis on closed-loop evaluation.
Table 1. The Performance on NAVSIM v2 (higher is better). For each metric, the highest score is in bold and the second-highest distinct score is underlined.