- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes FD-WAM, a framework that selectively uses future prediction for E2E-AD planning. It has two components: Decision-Relevant Foresight Representation (DRFR), which predicts latent features at multiple future horizons and compresses them into a compact planning context under future-feature and planning supervision, and Utility Aware Selective Foresight (UASF), which trains a router using the planning-score difference between the same planner with and without foresight. At inference, the future-prediction branch runs only when the router score exceeds a threshold. The authors evaluate the method on NAVSIM v1, NAVSIM v2, and closed-loop HUGSIM. In NAVSIM v1, FD-WAM reports 93.5 PDMS while invoking foresight for about 20% of scenes.
Critical strengths
- The utility-aware router offers a practical way to invoke future prediction only when it is likely to improve planning.
- FD-WAM achieves strong results on NAVSIM and HUGSIM.
Critical weaknesses
- The paper provides ablations and two attention visualizations for DRFR, but offers limited evidence about what information the compact future tokens encode and how that information affects planning decisions. Thus, the utility of DRFR is demonstrated more clearly than the paper's "what to predict" claim.
- Although the router improves benchmark performance, the paper offers little insight into the driving situations that trigger foresight. Without a breakdown of these scenes, it remains unclear in which situations foresight is most beneficial.
Questions for authors
- Table 3 appears to compare results from different HUGSIM test sets: the reported DrivoR score is from 345 scenarios, whereas the MM-Future score is from 436. Since the scenario set can affect HD-Score, were all methods including FD-WAM evaluated on the same scenarios using the same protocol? If not, how can the cross-method ranking and the claim of achieving the best average HD-Score be considered a fair comparison?
- The strong NAVSIM results may depend on safety-scoring-based trajectory selection. How does NAVSIM performance change when trajectories are selected using the trajectory classification score instead, without safety scoring? Also, the supplementary HUGSIM videos show substantial oscillation. Could this be related to the safety-scoring approach, and is this approach suitable for closed-loop evaluation?
- Could the authors also report component ablations on HUGSIM?
- At the 20% foresight budget, what types of scenes does the router select? Could the authors break down the selected scenes by scenario type and report the benefit of foresight for each type? This would clarify when the router's predictions are most useful.
3 comments
Thank you for the detailed response and additional experiments. However, I agree with Reviewer CRqi's remaining concern regarding W1. The marginal gain over Base–Base routing (0.2 PDMS) does not sufficiently demonstrate the benefit of foresight beyond the routing mechanism itself. Therefore, I will maintain my original rating.
Reply to Reviewer W7QQ
Thank you for the follow-up. We would like to clarify that the Base planner used in Table R1 already incorporates DRFR and achieves 93.3 PDMS without UASF (Table 5). Base–Base routing does not yield any performance improvement, as shown below.
Updated Table R1. Comparison with planner selection on NAVSIM v1.
| Method | NC ↑ | DAC ↑ | EP ↑ | TTC ↑ | Comf. ↑ | PDMS ↑ |
|---|---|---|---|---|---|---|
| Base (with DRFR) | 98.9 | 98.4 | 90.4 | 96.1 | 100.0 | 93.3 |
| Base–Base routing (with DRFR) | 98.9 | 98.3 | 90.3 | 96.2 | 100.0 | 93.3 |
| FD-WAM (with DRFR) | 99.1 | 98.4 | 90.4 | 96.4 | 100.0 | 93.5 |
Although UASF improves PDMS by only 0.2 points, this gain comes from approximately 20% of the scenes and is averaged over the entire test set. Across the 20% selected subset, foresight increases the average PDMS from 91.76 to 95.29, yielding a gain of 3.53 points, demonstrating a substantial benefit from foresight selectively invoked by UASF, as shown in Table R9.
More importantly, UASF is motivated by avoiding unnecessary future prediction to improve inference efficiency. Compared with always-on DRFR, it skips future prediction in approximately 80% of scenes and reduces mean latency by approximately 10%, while improving overall planning performance (Table R6).
We hope these clarifications help address your remaining concern and would appreciate your reconsideration of our work.
Reply to Reviewer W7QQ
We sincerely appreciate your valuable feedback. All the experiments and discussions below will be included in the final version of the paper.
1. Interpretation of the foresight representation.(W1)
We add per-token attention maps of scene tokens from future frames in Appendix C.1 (Fig. 5). Tokens attend to traffic lights, vehicles, road edges, and obstacles. Future queries receive prediction and planning supervision. Predicted tokens are compressed into a context for trajectory generation and scoring. Adding DRFR without UASF raises PDMS from 93.0 to 93.3 (Table 5), supporting its planning benefit. Here, “what to predict” means learning future features useful for planning.
2. Scenario-dependent foresight utility.(W2, Q4)
We thank the reviewer for this valuable suggestion. Table R9 presents a breakdown of the UASF-selected subset at a 20% foresight budget, grouped by ego motion and nearby road users. Turning and curved roads account for the largest share (40.1%) and have the largest gain (+4.09 PDMS), followed by scenes with nearby forward vehicles (+3.78), nearby pedestrians (+3.48), and Other (+1.38). Across the selected subset, foresight increases the average PDMS from 91.76 to 95.29, yielding a gain of 3.53 points.
Table R9. Scene breakdown of UASF-selected scenes at a 20% foresight budget.
3. HUGSIM evaluation protocol.(Q1)
We thank the reviewer for highlighting this. Our results in Table 3 follow DrivoR’s 345-scenario evaluation protocol, using the evaluator version that includes fixes to controller heading computation and the Comfort metric. For a fairer comparison, we additionally report results on the 436-scenario benchmark using the official configuration and the evaluator version without these fixes (
Table R5 in our response to Reviewer CRqi). In both protocols, RC and HD-Score averages are weighted by the number of scenarios in each difficulty level, giving each scenario equal weight. We report the two protocols separately.4. Trajectory selection and closed-loop stability.(Q2)
(i) NAVSIM trajectory selection. With classification-based selection, FD-WAM scores 91.7 PDMS versus 93.5 with quality scoring (Table R10). Its gains over Base are 0.8 and 0.7 points, respectively, showing that the benefit persists with either selector.
Table R10. Trajectory selection comparison on NAVSIM v1.
(ii) Closed-loop oscillation. On the 345-scenario HUGSIM protocol (Table R11), quality scoring achieves an average RC/HD-Score of 49.4/40.2, versus 43.5/31.6 for classification. Both selectors show oscillation, so changing the selector alone does not resolve it. A possible cause is switching between motion modes across steps, producing abrupt plan changes. Practical planning systems can use rule-based trajectory stitching to smooth consecutive plans. HUGSIM simulation does not include this stabilization step. These observations suggest that replacing the selector alone is insufficient to eliminate oscillation. We will discuss this limitation and remedies in the revision.
Table R11. Trajectory selection comparison on the 345-scenario HUGSIM protocol.
5. HUGSIM Ablation.(Q3)
We report component ablations on the 345-scenario HUGSIM protocol in Table R12.
Table R12. Component ablations on the 345-scenario HUGSIM protocol.