acceptodds
Under review as a conference paper at ICLR 2027

Foresight on Demand: Learning What and When to Predict for Autonomous Driving

Abstract

World action models predict the future to guide planning, but not all predicted information is useful for planning and not every scene benefits from additional prediction. Foresight can waste computation when history already supports a reliable plan, and can even degrade planning when its predictions are inaccurate. We present FD-WAM, which couples what to predict with when to predict for better planning, with the added benefit of reducing unnecessary foresight computation. Specifically, a Decision-Relevant Foresight Representation uses planning supervision to shape future slots learned through latent prediction, then compresses predictions from multiple future horizons into a compact set of tokens for planning. Utility-Aware Selective Foresight trains a router with the realized planning score difference between the same planner's foresight and no-foresight paths, enabling selective prediction from observed history alone. Evaluations on NAVSIM v1, NAVSIM v2, and zero-shot HUGSIM show that FD-WAM achieves competitive planning scores at lower inference cost than state-of-the-art baselines under aligned training scale. Controlled ablations further show that the compact foresight representation and selectively invoked foresight improve planning over the no-foresight baseline with little additional inference cost.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, World Action Model, End-to-End

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_W7QQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FD-WAM, a framework that selectively uses future prediction for E2E-AD planning. It has two components: Decision-Relevant Foresight Representation (DRFR), which predicts latent features at multiple future horizons and compresses them into a compact planning context under future-feature and planning supervision, and Utility Aware Selective Foresight (UASF), which trains a router using the planning-score difference between the same planner with and without foresight. At inference, the future-prediction branch runs only when the router score exceeds a threshold. The authors evaluate the method on NAVSIM v1, NAVSIM v2, and closed-loop HUGSIM. In NAVSIM v1, FD-WAM reports 93.5 PDMS while invoking foresight for about 20% of scenes.

Critical strengths
  1. The utility-aware router offers a practical way to invoke future prediction only when it is likely to improve planning.
  2. FD-WAM achieves strong results on NAVSIM and HUGSIM.
Critical weaknesses
  1. The paper provides ablations and two attention visualizations for DRFR, but offers limited evidence about what information the compact future tokens encode and how that information affects planning decisions. Thus, the utility of DRFR is demonstrated more clearly than the paper's "what to predict" claim.
  2. Although the router improves benchmark performance, the paper offers little insight into the driving situations that trigger foresight. Without a breakdown of these scenes, it remains unclear in which situations foresight is most beneficial.
Questions for authors
  1. Table 3 appears to compare results from different HUGSIM test sets: the reported DrivoR score is from 345 scenarios, whereas the MM-Future score is from 436. Since the scenario set can affect HD-Score, were all methods including FD-WAM evaluated on the same scenarios using the same protocol? If not, how can the cross-method ranking and the claim of achieving the best average HD-Score be considered a fair comparison?
  2. The strong NAVSIM results may depend on safety-scoring-based trajectory selection. How does NAVSIM performance change when trajectories are selected using the trajectory classification score instead, without safety scoring? Also, the supplementary HUGSIM videos show substantial oscillation. Could this be related to the safety-scoring approach, and is this approach suitable for closed-loop evaluation?
  3. Could the authors also report component ablations on HUGSIM?
  4. At the 20% foresight budget, what types of scenes does the router select? Could the authors break down the selected scenes by scenario type and report the benefit of foresight for each type? This would clarify when the router's predictions are most useful.
3 comments
AuthorsOpenReview ↗

Reply to Reviewer W7QQ

We sincerely appreciate your valuable feedback. All the experiments and discussions below will be included in the final version of the paper.

1. Interpretation of the foresight representation.(W1)

We add per-token attention maps of scene tokens from future frames in Appendix C.1 (Fig. 5). Tokens attend to traffic lights, vehicles, road edges, and obstacles. Future queries receive prediction and planning supervision. Predicted tokens are compressed into a context for trajectory generation and scoring. Adding DRFR without UASF raises PDMS from 93.0 to 93.3 (Table 5), supporting its planning benefit. Here, “what to predict” means learning future features useful for planning.

2. Scenario-dependent foresight utility.(W2, Q4)

We thank the reviewer for this valuable suggestion. Table R9 presents a breakdown of the UASF-selected subset at a 20% foresight budget, grouped by ego motion and nearby road users. Turning and curved roads account for the largest share (40.1%) and have the largest gain (+4.09 PDMS), followed by scenes with nearby forward vehicles (+3.78), nearby pedestrians (+3.48), and Other (+1.38). Across the selected subset, foresight increases the average PDMS from 91.76 to 95.29, yielding a gain of 3.53 points.

Table R9. Scene breakdown of UASF-selected scenes at a 20% foresight budget.

Scenario categoryShare (%)Base PDMSForesight PDMSPDMS gain
Turning / curved roads40.191.0795.16+4.09
Nearby forward vehicle26.692.2396.01+3.78
Nearby pedestrians20.491.5695.04+3.48
Other12.993.2594.63+1.38

3. HUGSIM evaluation protocol.(Q1)

We thank the reviewer for highlighting this. Our results in Table 3 follow DrivoR’s 345-scenario evaluation protocol, using the evaluator version that includes fixes to controller heading computation and the Comfort metric. For a fairer comparison, we additionally report results on the 436-scenario benchmark using the official configuration and the evaluator version without these fixes (Table R5 in our response to Reviewer CRqi). In both protocols, RC and HD-Score averages are weighted by the number of scenarios in each difficulty level, giving each scenario equal weight. We report the two protocols separately.

4. Trajectory selection and closed-loop stability.(Q2)

(i) NAVSIM trajectory selection. With classification-based selection, FD-WAM scores 91.7 PDMS versus 93.5 with quality scoring (Table R10). Its gains over Base are 0.8 and 0.7 points, respectively, showing that the benefit persists with either selector.

Table R10. Trajectory selection comparison on NAVSIM v1.

MethodSelectionNC ↑DAC ↑EP ↑TTC ↑Comf. ↑PDMS ↑
BaseScore98.898.290.195.6100.092.8
BaseClassification98.398.188.395.2100.090.9
FD-WAMScore99.198.490.496.4100.093.5
FD-WAMClassification98.598.289.195.4100.091.7

(ii) Closed-loop oscillation. On the 345-scenario HUGSIM protocol (Table R11), quality scoring achieves an average RC/HD-Score of 49.4/40.2, versus 43.5/31.6 for classification. Both selectors show oscillation, so changing the selector alone does not resolve it. A possible cause is switching between motion modes across steps, producing abrupt plan changes. Practical planning systems can use rule-based trajectory stitching to smooth consecutive plans. HUGSIM simulation does not include this stabilization step. These observations suggest that replacing the selector alone is insufficient to eliminate oscillation. We will discuss this limitation and remedies in the revision.

Table R11. Trajectory selection comparison on the 345-scenario HUGSIM protocol.

SelectionRC E ↑RC M ↑RC H ↑RC X ↑RC Avg. ↑HD-Score E ↑HD-Score M ↑HD-Score H ↑HD-Score X ↑HD-Score Avg. ↑
Score82.457.529.339.549.474.353.217.225.640.2
Classification76.256.830.616.843.563.343.120.28.231.6

5. HUGSIM Ablation.(Q3)

We report component ablations on the 345-scenario HUGSIM protocol in Table R12.

Table R12. Component ablations on the 345-scenario HUGSIM protocol.

FutureDRFRUASFRC E ↑RC M ↑RC H ↑RC X ↑RC Avg. ↑HD-Score E ↑HD-Score M ↑HD-Score H ↑HD-Score X ↑HD-Score Avg. ↑
–––80.249.627.536.645.666.243.814.621.634.5
✓––80.953.228.237.547.168.445.315.822.835.7
✓✓–81.754.929.038.948.270.651.717.025.239.1
✓✓✓82.457.529.339.549.474.353.217.225.640.2

Thank you for the detailed response and additional experiments. However, I agree with Reviewer CRqi's remaining concern regarding W1. The marginal gain over Base–Base routing (0.2 PDMS) does not sufficiently demonstrate the benefit of foresight beyond the routing mechanism itself. Therefore, I will maintain my original rating.

AuthorsOpenReview ↗

Reply to Reviewer W7QQ

Thank you for the follow-up. We would like to clarify that the Base planner used in Table R1 already incorporates DRFR and achieves 93.3 PDMS without UASF (Table 5). Base–Base routing does not yield any performance improvement, as shown below.

Updated Table R1. Comparison with planner selection on NAVSIM v1.

MethodNC ↑DAC ↑EP ↑TTC ↑Comf. ↑PDMS ↑
Base (with DRFR)98.998.490.496.1100.093.3
Base–Base routing (with DRFR)98.998.390.396.2100.093.3
FD-WAM (with DRFR)99.198.490.496.4100.093.5

Although UASF improves PDMS by only 0.2 points, this gain comes from approximately 20% of the scenes and is averaged over the entire test set. Across the 20% selected subset, foresight increases the average PDMS from 91.76 to 95.29, yielding a gain of 3.53 points, demonstrating a substantial benefit from foresight selectively invoked by UASF, as shown in Table R9.

More importantly, UASF is motivated by avoiding unnecessary future prediction to improve inference efficiency. Compared with always-on DRFR, it skips future prediction in approximately 80% of scenes and reduces mean latency by approximately 10%, while improving overall planning performance (Table R6).

We hope these clarifications help address your remaining concern and would appreciate your reconsideration of our work.

Reviewer_BYzVOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FD-WAM, which learns both a compact future representation and whether future prediction should be executed for a given scene. Planning-supervised future slots are compressed into a small context, while a history-only router is trained using the realized score difference between foresight and no-foresight paths of the same planner. The approach is evaluated on NAVSIM v1/v2 and zero-shot HUGSIM, with favorable planning and latency results.

Critical strengths

(+) The central question is well motivated: prediction is useful only when it improves the decision. Training the router on realized benefits from the learned predictor accounts for prediction errors and provides a more relevant target than scene dynamics alone.

(+) The evaluation includes three benchmarks, random and dynamics-based routing controls, and token/horizon ablations.

(+) Clear and easy to follow.

Critical weaknesses

(-) Table 4 demonstrates latency savings from selective foresight, but the memory implications remain unclear. Could the authors report parameter counts and peak inference memory, together with latency and memory measurements for the token-count and future-horizon configurations in Tables 9–10?

(-) FD-WAM already uses an EMA future teacher to supervise latent prediction. Inspired by future-prior distillation methods such as AMap, could the authors further distill the foresight-conditioned planner into a history-only student, removing both the future predictor and router at inference? A comparison under matched backbone and training-data settings would help clarify whether inference-time foresight provides additional benefits beyond training-time knowledge transfer.

[1] AMap: Distilling Future Priors for Ahead-Aware Online HD Map Construction, CVPR 2026

Questions for authors

Q1: Are Tables 6 and 7 evaluated with identical routing rates and checkpoints, and why does the DRFR score differ between Tables 5 and 6?

2 comments
AuthorsOpenReview ↗

Reply to Reviewer BYzV

We sincerely appreciate your valuable feedback. All the experiments and discussions below will be included in the final version of the paper.

1. Inference cost.(W1)

(i) Parameters, memory, and latency. Table R6 reports parameter counts and inference costs. FD-WAM uses 53.878M parameters, compared with 52.628M for Base, and increases peak memory from 490.60 to 558.60 MiB. DRFR reduces peak memory from 850.48 to 558.60 MiB and mean latency from 66 to 55 ms. UASF further reduces mean latency to 50 ms without changing peak memory.

Table R6. Parameter counts and inference efficiency on NAVSIM v1.

MethodFuture rateParameters (M)Mean latency (ms)P99 latency (ms)Peak memory (MiB)PDMS ↑
Base0%52.6284749490.6092.8
Foresight (w/o DRFR)100%58.7496668850.4893.0
Foresight100%53.8785558558.6093.3
FD-WAM20%53.8785052558.6093.5

(ii) Token count and future horizons. We report the configuration costs in Table R7. Mean latency ranges from 49.21 to 49.72 ms, and peak memory from 558.41 to 558.98 MiB. The default 16 tokens and four horizons achieve the highest PDMS (93.5) among these configurations with similar inference costs.

Table R7. Efficiency of token-count and future-horizon configurations.

TokensHorizonsParameters (M)Mean latency (ms)P99 latency (ms)Peak memory (MiB)PDMS ↑
4453.87249.2151.07558.5893.0
8453.87449.3051.83558.5893.1
16453.87849.5352.01558.6093.5
32453.88649.6352.11558.6393.4
16253.82849.3551.96558.4193.1
16853.97849.7252.18558.9893.3

2. History-only distillation.(W2)

We thank the reviewer for this interesting suggestion, which we had not previously considered. We test it by distilling the planning outputs of the frozen FD-WAM into a history-only student without the future predictor or router at inference. The student and Base use the same backbone and training data. As shown in Table R8, FD-WAM (distill) achieves 93.3 PDMS, compared with 92.8 for Base and 93.5 for FD-WAM. These results show that distillation transfers part of the planning gain, while inference-time foresight still provides an additional benefit in this comparison.

Table R8. Planner distillation on NAVSIM v1.

MethodNC ↑DAC ↑EP ↑TTC ↑Comf. ↑PDMS ↑
Base98.898.290.195.6100.092.8
FD-WAM (distill)99.098.390.396.2100.093.3
FD-WAM99.198.490.496.4100.093.5

3. Consistency of the ablation settings.(Q1)

We thank the reviewer for pointing out the unclear settings. Each variant in Tables 6 and 7 is trained independently and evaluated using its own best checkpoint. The activation rate follows from each variant’s routing strategy and is not constrained to be identical across variants. In Table 5, 93.3 corresponds to DRFR without UASF, whereas the 93.5 row in Table 6 includes UASF and represents the complete FD-WAM.

Thank you for the further clarification and analysis. My concern has been addressed, and I will maintain my score of "3: Weak acceptance."

Reviewer_CRqiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that in world action model, the foresight should be determined by the planning utility both in what should be predicted and when is the prediction worth computing. The author present FD-WAM, which adds two components to a DINOv2 based planner to reduce unnecessary foresight computation. Evaluation show that FD-WAM can get competitive planning scores with lower cost tha current STOA baselines. Specifically, 93.5 / 94.4 PDMS (train / trainval) on NAVSIM v1, 90.5 / 91.3 EPDMS on NAVSIM v2, 40.2 HD-Score zero-shot on HUGSIM, and 50 ms mean latency at a 20% foresight rate versus 55 ms for always-on foresight.

Critical strengths
  1. The motivation is clear, and the it is supported by an informative comparison. The base and foresight path use the same planner, so it is very easy to check whether the predicted future information is useful to improve the planning. Figure 2 show that using only 20% budget the planner get the highest performance.

  2. The performance is competitive considering the backbone is small (ViT-S).

Critical weaknesses
  1. The UASF's gain may just be survivorship bias. The UASF can be viewed as a selector between two planners since it is trained on GT scores of two paths. If we check the Figure2, only 20% of foresight can already beat the 100% foresight rate. This shows that 80% of scenes, the foresight is making planner worse. Thus the 0.2 gain can be from removing the harmful foresight prediction not exploiting useful foresight. What if they using this same router to pick between two basic planners which do not have foresight? if that also provide meaningful gain, it proves the gain is just an ensemble trick. A oracle selector (upper bound) or a confidence based router can help clarify this.

  2. The idea of when to predict is somehow relevant to AdaThinkDrive (https://arxiv.org/pdf/2509.13769), which try to lean when to when to think harder based on the quality difference of trajectory.

  3. The component gains is about 0.2-0.3 in PDMS from a single seed without multiple runs, which make it hard to be interpreted.

Questions for authors
  1. the EC in NAVSIM v2 is 70.1 and 78.7 which are below most baselines. Can you report this EC for base and 100% foresight and your method, to check if switching path can affects temporal consisteny?

  2. For HUGSIM, which scenarios set, 345 or 436, and which evaluator version are used to produce the number in Table 3 and how does the average is calculated?

3 comments
AuthorsOpenReview ↗

Reply to Reviewer CRqi

We sincerely appreciate your valuable feedback. All the experiments and discussions below will be included in the final version of the paper.

1. Foresight utility and routing controls.(W1)

(i) The 20% activation rate. A 20% activation rate does not mean that foresight harms all remaining 80% of scenes. Figure 2 shows overall performance at different thresholds, while Figure 3 shows many scenes with little score change. Gains over Base reflect useful foresight in selected scenes; gains over always-on foresight reflect avoiding negative effects in skipped scenes. Both effects contribute to selective foresight.

(ii) Foresight versus planner selection. We report a comparison with the same router applied to two base planners (Table R1). This control achieves 93.3 PDMS, compared with 93.5 for FD-WAM. The result supports a benefit from foresight beyond this planner-selection control.

Table R1. Comparison with planner selection on NAVSIM v1.

MethodNC ↑DAC ↑EP ↑TTC ↑Comf. ↑PDMS ↑
Base–Base routing98.998.390.396.2100.093.3
FD-WAM (ours)99.198.490.496.4100.093.5

(iii) Confidence-based routing and Oracle. We compare confidence-based routing, UASF, and Oracle in Table R2. The confidence baseline uses trajectory confidence to decide whether to invoke foresight; Oracle selects scenes using actual PDMS gains and provides an upper bound. These results support using planning utility to identify when foresight is beneficial.

Table R2. Routing strategies on NAVSIM v1.

MethodNC ↑DAC ↑EP ↑TTC ↑Comf. ↑PDMS ↑
Confidence-based routing98.898.290.695.7100.093.1
UASF99.198.490.496.4100.093.5
Oracle99.298.490.596.5100.093.6

2. Relation to AdaThinkDrive.(W2)

We discuss AdaThinkDrive in Section 2. Both methods use trajectory-quality differences to decide when extra computation is useful. AdaThinkDrive uses GRPO to choose whether to generate CoT reasoning, while FD-WAM learns a compact future representation and when to compute it. We will clarify this distinction in the revision.

3. Multi-seed component ablations.(W3)

We report the component ablation over three independent training seeds using mean ± standard deviation of PDMS (Table R3). The 0.2–0.3-point PDMS gains are larger than the standard deviations across seeds, supporting stable component improvements across training runs.

Table R3. Component ablation across training seeds.

FutureDRFRUASFNC ↑DAC ↑EP ↑TTC ↑Comf. ↑PDMS ↑
–––98.898.290.195.6100.092.8±0.042
✓––98.898.490.195.7100.093.0±0.035
✓✓–98.998.490.496.1100.093.3±0.033
✓✓✓99.198.490.496.4100.093.5±0.041

4. Effect of foresight routing on EC.(Q1)

We report the NAVSIM v2 navtest comparison for models trained on the navtrain split (Table R4). FD-WAM achieves an EC of 70.1, above Base (66.8) and 100% foresight (68.8). These results show that selective routing does not lower EC relative to either fixed path in this setting. With the larger trainval split, FD-WAM achieves an EC of 78.7 (manuscript Table 2), suggesting that additional training data can further improve EC.

Table R4. NAVSIM v2 performance and temporal consistency.

MethodNC ↑DAC ↑DDC ↑TLC ↑EP ↑TTC ↑LK ↑HC ↑EC ↑EPDMS ↑
Base98.998.398.799.490.598.395.997.466.888.8
FD-WAM (100% foresight)99.098.399.299.490.598.596.298.068.890.0
FD-WAM99.098.899.699.690.798.596.297.970.190.5

5. HUGSIM evaluation protocol.(Q2)

We thank the reviewer for pointing out this omission. Table 3 follows DrivoR’s 345-scenario configuration and uses the evaluator version with fixes to controller heading computation and the Comfort metric. We additionally report results on the 436-scenario benchmark in Table R5, using the official evaluation configuration and the evaluator version without these fixes. For both protocols, the RC and HD-Score averages are computed by weighting each difficulty-level mean by the number of scenarios in that level, equivalently assigning equal weight to each scenario. We report the two protocols separately and explicitly specify their evaluation settings.

Table R5. HUGSIM comparison on the 436-scenario protocol.

MethodRC E ↑RC M ↑RC H ↑RC X ↑RC Avg. ↑HD-Score E ↑HD-Score M ↑HD-Score H ↑HD-Score X ↑HD-Score Avg. ↑
VAD38.727.025.523.027.924.39.910.48.212.3
LTF68.440.736.925.541.452.824.619.88.124.8
UniAD58.641.240.426.040.648.729.527.314.328.6
Latent-WAM84.242.530.635.545.972.524.012.218.128.9
MM-Future64.851.539.522.844.553.840.027.38.632.3
FD-WAM82.548.329.726.745.369.038.723.612.034.6

Thank you for the response and additional experiments. The clarifications on EC and HUGSIM address my concerns. However, my main concern W1 still remains, and Table R1 even sharpens it. FD-WAM is only 0.2 above Base–Base routing. I will keep my rating for now.

AuthorsOpenReview ↗

Reply to Reviewer CRqi

Thank you for the follow-up. We would like to clarify that the Base planner used in Table R1 already incorporates DRFR and achieves 93.3 PDMS without UASF (Table 5). Base–Base routing does not yield any performance improvement, as shown below.

Updated Table R1. Comparison with planner selection on NAVSIM v1.

MethodNC ↑DAC ↑EP ↑TTC ↑Comf. ↑PDMS ↑
Base (with DRFR)98.998.490.496.1100.093.3
Base–Base routing (with DRFR)98.998.390.396.2100.093.3
FD-WAM (with DRFR)99.198.490.496.4100.093.5

Although UASF improves PDMS by only 0.2 points, this gain comes from approximately 20% of the scenes and is averaged over the entire test set. Across the 20% selected subset, foresight increases the average PDMS from 91.76 to 95.29, yielding a gain of 3.53 points, demonstrating a substantial benefit from foresight selectively invoked by UASF, as shown in Table R9.

More importantly, UASF is motivated by avoiding unnecessary future prediction to improve inference efficiency. Compared with always-on DRFR, it skips future prediction in approximately 80% of scenes and reduces mean latency by approximately 10%, while improving overall planning performance (Table R6).

We hope these clarifications help address your remaining concern and would appreciate your reconsideration of our work.

Comments on the paper

2 comments

Dear Reviewers,

Please take time to review the responses from the authors and engage with them for further questions and comments if any.

Time left is not much and please do shortly.

Thanks

AC

AuthorsOpenReview ↗

Summary Reply to AC and all Reviewers

Dear Area Chair and Reviewers,

We would like to summarize the key points addressed in our responses to ensure that the contributions and technical merits of FD-WAM are clearly communicated. The following summary brings together the additional experiments and clarifications concerning baseline comparisons, methodological design, component effectiveness, and evaluation settings.

Summary of Key Responses to Reviewer Concerns

1. Additional Baseline Comparisons and Evaluation Protocols (CRqi, BYzV, W7QQ)

  • Routing and distillation comparisons. Applying the same router to two base planners achieves 93.3 PDMS. Distilling FD-WAM into a history-only student under matched backbone and training-data settings also achieves 93.3, compared with 92.8 for Base and 93.5 for FD-WAM. Confidence-based and oracle routing achieve 93.1 and 93.6, respectively. These controls support an additional benefit from selectively computing foresight at inference (Tables R1, R2, and R8).
  • HUGSIM evaluation protocols. We explicitly distinguish DrivoR’s 345-scenario protocol with the corrected evaluator from the official 436-scenario protocol with the original evaluator. FD-WAM achieves 40.2 and 34.6 HD-Score under these respective settings. Results are reported separately, with each scenario receiving equal weight within its protocol (Table R5).

2. Clarification of Foresight Representation and Routing (CRqi, W7QQ)

  • Planning-relevant representation. DRFR learns a compact future context through joint prediction and planning supervision. The additional per-token attention maps illustrate how individual future-frame tokens attend to traffic lights, vehicles, and road boundaries. Component ablations directly measure the planning benefit of this representation (Fig. 5 and Table R3).
  • Utility-based routing. UASF learns from the realized planning-score difference between the same planner with and without foresight. Its history-only decision therefore reflects the benefit delivered by the learned predictor. We also clarify its relationship to AdaThinkDrive, which decides when to generate chain-of-thought reasoning (Sec. 2 and our response to CRqi).

3. Additional Ablations and Scenario Analysis (CRqi, W7QQ)

  • Component effectiveness. Across three training seeds, NAVSIM v1 PDMS increases from 93.0 with uncompressed foresight to 93.3 with DRFR and 93.5 with selective routing, with standard deviations below 0.05. On the 345-scenario HUGSIM protocol, the same progression raises HD-Score from 35.7 to 39.1 to 40.2. FD-WAM also improves over Base with both classification-based (+0.8 PDMS) and quality-based (+0.7 PDMS) trajectory selection (Tables R3, R10, and R12).
  • Scene-dependent benefits. Within the 20% of scenes selected by UASF, foresight raises the same planner’s mean PDMS from 91.76 to 95.29. Turning and curved-road scenes account for 40.1% of this subset and show the largest gain (+4.09 PDMS), followed by scenes with nearby forward vehicles (+3.78) and pedestrians (+3.48) (Table R9).

4. Inference Efficiency and Memory Usage (BYzV)

  • Component costs. On a single H800 with batch size 1, DRFR reduces peak inference memory from 850.5 to 558.6 MiB and mean latency from 66 to 55 ms. Selective routing further reduces mean latency to 50 ms at the same peak memory (Table R6).
  • Configuration costs. We additionally report parameter counts, mean and P99 latency, and peak memory across token-count and future-horizon configurations. The default configuration of 16 tokens and four horizons achieves the highest PDMS among the evaluated settings, with comparable inference costs (Table R7).

Summary of Discussions

Our work, FD-WAM, introduces a planning-utility perspective on future prediction for end-to-end autonomous driving, jointly considering what to predict and when to predict. We propose Decision-Relevant Foresight Representation (DRFR) to learn a compact future context under planning supervision, and Utility-Aware Selective Foresight (UASF) to identify scenes likely to benefit from foresight. Experiments on NAVSIM v1/v2 and zero-shot HUGSIM demonstrate strong planning performance and inference efficiency. Additional comparisons and ablations further support the effectiveness of the compact representation and utility-based selective foresight.

To conclude, we have devoted effort to addressing the major concerns raised by the reviewers through additional baseline comparisons, component ablations, scenario analyses, and clarifications of computational costs and evaluation protocols. All additional experiments and discussions will be incorporated into the revised manuscript.

We sincerely thank the Area Chair and all reviewers for their time, insightful feedback, and dedication to scientific rigor. We value this scholarly exchange and remain committed to rigorous, transparent research.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.