acceptodds
Under review as a conference paper at ICLR 2027

Online Policy Optimization for Autonomous Driving with a Hierarchical World Model

Abstract

Policies obtained via imitation learning are known to struggle with out-of-distribution cases, i.e., when the environment or the agent deviates from the expert distribution at test time. Test-time planning offers a potential solution by learning a more generic world model and reward model from demonstrations and searching for corrective actions at inference. However, this approach has not been verified in driving setups due to the computational cost of searching over long planning horizons. We introduce HiPlan, a hierarchical planning framework that mirrors the temporal structure of the hierarchical policy it supports, with an upper level operating at the subgoal timescale and a lower level at full resolution. This enables two-stage search, which reduces search dimensionality by an order of magnitude. We find that the two levels develop qualitatively different representations, with the upper model encoding strategic features and the lower encoding tactical features, a double dissociation confirmed by linear probing and horizon-scaling analysis. HiPlan achieves state-of-the-art on nuPlan and CARLA, and analytical experiments show that the hierarchy enables more efficient search, that representations factorize by timescale, and that structural alignment between policy and world model is necessary for both.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Imitation Learning, Long-Horizon Planning, Hierarchical Planning, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.40 over 5 reviews, above 68% of the others.
Reviewer_mxzzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a trajectory planning approach, HiPlan, for autonomous driving. It plans at test time in a hierarchical way on a coarse anchor waypoint level and a fine safety-verified trajectory level. The planning structure of evenly distributed sub-goals are a prior but then perturbations to them, reward models and residual corrections are learned to form a world model. The main novelty compared to prior work on hierarchical planning lies in the world model, trained at different hierarchical scales to allow for planning across long driving episodes.

Critical strengths

The paper is very clearly formulated. The questions at 303 structure the paper in a useful way, going from showing the performance over generalization to ablations.

The results on nuPlan and Carla are very good compared to the state of the art and a competitive route completion in Table 2 on Carla suggests that it is not just a trade-off, the performance is competitive if not better than PlanT.

The way how specific maneuvers are extracted from the training set and used for testing the distribution shift is good to highlight the limits of test-time search on the expert policy compared to pure imitation learning.

Many analyses are done to probe the learned representation and performance. It is expected but assuring to see that the lower world model and upper world model latents have developed a separated sensitivity to semantic and kinematic features.

The full hierarchical replan inference is surprisingly performant with 69 ms on an RTX 3090. The search efficiency, even with many MPPI evaluations, is better than a flat search.

The specific implementation of hierarchical planning may have modest novelty but for a simple idea the performance clearly beats the state of the art with a modest computation budget and every aspect of the method is very thoroughly examined and ablated. Good performance of the flat WM in some conditions may show that the separated sensitivity of features may be unnecessary but that is a question which of the two presented methods works better. It seems the hierarchical version does work better in OOD policy regions.

Critical weaknesses

The training time is significant. 600 GPU-hours on 8 H200 for training probably means several weeks of training on more modest setups.

One explanation for the degradation under distribution shift in Table 3 can be that the approach does not include a true environment reward and it is unclear how to do so. The policy is still learned in IL fashion against expert trajectories and not with feedback from the environment.

The idea is very similar to concurrent work: CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Driving Yihong Guo ⋅ Dongqiangzi Ye ⋅ Sijia Chen ⋅ Anqi Liu ⋅ Xianming Liu (ICML 2026)

The authors should contrast their work with this CorrectionPlanner

Questions for authors

Please add recent concurrent work to the related work and contrast with your work:

Nguyen, L., Fauth, M., Jaeger, B., Dauner, D., Igl, M., Geiger, A., & Chitta, K. (2026). Lead: Minimizing learner-expert asymmetry in end-to-end driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 39775-39785).

and the CorrectionPlanner described in critical weaknesses

Could you define the paper's understanding of the word dissociation? The word's use in psychology seems to overshadow other uses and outside of that domain in chemistry, graph theory etc. it all means something very specific to that domain so it may be hard to understand the exact meaning here.

Please consider also making terms like factorized representation, driving-length horizon more clear.

In line 188 the state pair of equation 16 appears without any reference or explanation. Could you define s there or refer to equation 16?

The formulation in line 351, it says we remove "all scenarios" and then it list different levels of retention which seems contradictory. Probably it is meant with 50% retention they are of course not all removed but the paragraph should be rephrased accordingly.

The bold, underline notation in Table 1 is not always followed. Bold should indicate best but in the first two columns two results are bold. If it is suggested to have one bold best result for HiPlan and one for the others, then column 1,3,4,5 and 6 are missing a bold entry. If not, there should not be two bold result. Underlined results in those columns needs to be adjusted respectively.

Why were no RL baselines compared on Carla?

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their detailed feedback and address their remaining concerns below.

Training cost: We agree that training is substantial, particularly on smaller GPUs. Optimizing training efficiency was not the focus of this work, and there is clear room to reduce it. Most of the gain from distillation comes in the first iterations, so a single round requires less than half the compute and already reaches 84.27 on Test14-hard R (Table 23). Rollout collection, which accounts for about 60% of each iteration, could also be reduced by reusing data in the replay buffer across iterations instead of collecting fresh rollouts each round. However, deployment is already cheap. The full planner runs at 69 ms per replan on a single consumer-grade RTX 3090, within the 100 ms budget for 10 Hz planning, and the distilled policy alone reaches 82.8 on Test14-hard R without any search.

Environment reward: HiPlan is designed for the imimtation setting, so its reward is learned as a discriminator between expert and policy behavior rather than hand-constructed. Explicit rule-based terms can be added directly (we show this in Appendix D.2), and we see that adding a hard geometric filter before RM_lo only gets a 0.2 point change in Test14-Hard (86.9 →\rightarrow 87.1). Our learned reward therefore already captures most of what the explicit constraints enforce. In Table 3, while all three systems degrade as maneuver data is removed, HiPlan degrades least (30% collisions at 0% retention against 45% for flat WM search and 72% for base policy). We would add combining learned and environment rewards as a direction in the conclusion.

Concurrent Works:

We thank the reviewer for pointing out prior related works that we overlooked. We would include both of them in our updated manuscript and we describe below how they are related to our works:

  1. CorrectionPlanner [1] : CorrectionPlanner is an autoregressive motion-token planner that uses a learned collision critic to flag unsafe tokens over a short horizon and regenerates them, trained with imitation followed by model-based RL in a frozen world model. Its corrections act on individual tokens, with no temporal abstraction, so it can revise a plan only once a collision is imminent. HiPlan searches over subgoal anchors across the full horizon in a world model at the subgoal timescale, and distills the corrections back into the policy. It can therefore change the maneuver before a failure develops, and its learned reward also penalizes failures other than collisions, such as stalled progress. This difference matters in practice, since 64% of the Test14-hard scenarios recovered by HiPlan require a change to the anchor sequence (Table 22). On Test14-hard, HiPlan reaches 83.77 NR and 86.90 R against 75.37 NR and 77.29 R for CorrectionPlanner.

  2. LEAD [2] : LEAD studies information asymmetry between previliged experts and sensor-based students in end-to-end driving, and narrows it through a revised expert, a new dataset, and the TFv6 student. This is complementary to HiPlan, which plans on processed scene representations, and PlanT does. We would add LEAD to related works and TFv6 to Table 2, marked as a sensor-based method.

Terminology: We use “double dissociation” in its neuropsychological sense. Two systems show a double dissociation when each performs well on one class of task and poorly on the other, in opposite directions. Here, upper-level latents linearly predict high-level features well and kinematic features poorly, and lower-level latents show the reverse (Figure 4). A “factorized representation” means that information at the two timescales is linearly accessible from different latent spaces. A “driving length horizon” refers to 8s at 10Hz (80 steps) on nuPlan and up to 24s on CARLA, compared with short horizons of prior test-time planning in manipulation. We would add these definitions clearly in the updated manuscript.

Presentation: We would change Line 188 to define the upper-level state skup=(hkup,zkup)s_k^{up} = (h_k^{up}, z_k^{up}) and refer to Eq. 16. We thank the reviewer for pointing this out. Line 351 would also now state that a given fraction of scenarios in each maneuver is retained. We would change Table 1 to mark a single best and second-best result per column.

RL Baselines on CARLA: Table 2 includes three RL planners (Roach, Think2Drive, and CaRL), labeled by type in Table 14. We would add the method types in Table 2 so that this is visible in the main text.

We thank the reviewer again for the constructive feedback, and are happy to answer any further questions.

[1] Guo, Yihong, et al. "CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Driving." arXiv preprint arXiv:2603.15771 (2026).

[2] Nguyen, Long, et al. "Lead: Minimizing learner-expert asymmetry in end-to-end driving." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

I thank the authors for the clarifications and especially responding on how to add explicit rule-based terms completes a positive impression of the method. After reading current reviews and answers I will change my rating to accept unless other issues will be brought up during the rest of the review phases that I overlooked.

AuthorsOpenReview ↗

Reply by Authors

We thank the reviewer for the positive reassessment and for the constructive suggestions, and are glad that we were able to address the reviewer’s concerns.

Reviewer_evprOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces HiPlan with hierarchical policy for planning. An upper level search corrected anchors via MPPI, whereas a lower level verifies dense trajectory safety based on the anchors. Besides, the corrections are further distilled back into the policy. For experiments, HiPlan achieves SOTA on nuPlan and CARLA.

Critical strengths

OOD problem is very important for safety-critical AD. The experiments show that HiPlan can recover more appropriate driving behaviors even when certain maneuvers are removed from the training data. I appreciate the carefully designed experiments for evaluating the generalization capabilities of HiPlan.

Critical weaknesses

Overstated novelty claim. The paper claims to be the first hierarchical test-time planning framework for autonomous driving. However, HOLO-MPPI has already explored hierarchical planning by combining a learned high-level policy and world model with online MPPI refinement.

HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization

Questions for authors

See weaknesses

1 comment
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for the positive assessment of the held-out maneuver experiments and for pointing us to HOLO-MPPI [1], which we will be adding to the Related Works in Table 6. While both methods combine a learned high-level component with MPPI, they place the search at opposite levels of the hierarchy. Below we explain this distinction and why searching at the subgoal level is advantageous in driving.

In HOLO-MPPI, a DQN policy selects one of five meta-actions, and MPPI refines acceleration and steering around that prior over a 1.6s horizon. The high-level decision is never optimized at test time. HiPlan instead searches over the policy’s subgoal anchors inside an upper world model at the anchor timescale, and the lower level verifies the resulting dense trajectories. Both levels and their reward models are learned from demonstrations, with no hand-designed planning cost, and they develop timescale-factorized representations (Section 4.4). Table 4 shows that the search gain depends on this hierarchically trained representation, since flat latents read out at the same timesteps give no gain. Search corrections are also distilled back into the policy, raising its standalone score from 72.6 to 82.8 on Test14-hard R (Appendix E.9).

The difference is also evident practically. Our correction analysis (Appendix E.8, Table 22) shows that 64% of the Test14-hard scenarios that HiPlan recovers require a change to the anchor sequence (40% upper-level only, 24% joint). The closest analogue to HOLO-MPPI would be dense MPPI around the base policy’s fixed anchors; which fails to recover in any of these cases.

We hope that this clarifies the distinction and are happy to answer any further questions.

[1] Min, Y., D'sa, J., Tariq, F. M., Isele, D., Azizan, N., & Bae, S. (2026). HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization. arXiv preprint arXiv:2606.16480.

Reviewer_T4oYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes HiPlan, a hierarchical world-model-based test-time planning framework for autonomous driving, aiming to improve the robustness of imitation learning policies under distribution shift. To address the high computational cost and compounding prediction errors of long-horizon planning, the method combines a hierarchical imitation policy with a hierarchical world model. At test time, MPPI searches for residual corrections to the anchor waypoints proposed by the base policy, followed by dense trajectory generation and verification using the lower-level world model. The resulting corrections are further distilled into the base policy. Experiments on nuPlan, CARLA, and interPlan demonstrate improvements in planning performance, search efficiency, and robustness to distribution shift.

Critical strengths
  1. Extending world-model-based test-time planning to long-horizon autonomous driving is valuable, given the challenges of high-dimensional search and compounding prediction errors.

  2. The proposed framework achieves strong performance on challenging driving benchmarks while maintaining practical inference latency.

Critical weaknesses
  1. The methodological novelty is not sufficiently clear relative to prior work.

Several key components of the proposed framework have been explored in previous studies. SAILOR already combines imitation learning with world-model-based test-time search and policy distillation, while prior work on hierarchical world models has explored planning across different temporal scales to reduce computational complexity and long-horizon prediction errors. It is not sufficiently clear whether the proposed framework constitutes a fundamentally new methodological contribution, rather than an architectural adaptation or integration of existing techniques. The paper would benefit from a more explicit discussion of its algorithmic novelty relative to these closely related approaches.

  1. The evidence for the claimed necessity of structural alignment could be stronger.

The paper emphasizes that structural alignment between the policy and world model is necessary for effective planning and representation specialization. However, the 2×2 ablation in Table 5 primarily varies the structures of the world model and search procedure, while the base policy remains hierarchical.

Furthermore, the mismatched configurations involve additional implementation choices, such as fixing upper-level latents during flat search with a hierarchical world model. These choices may themselves affect performance.

  1. Incorrect citation formatting. The paper does not follow the ICLR 2027 citation guidelines.
Questions for authors
  1. Could the authors clarify the main algorithmic contributions of HiPlan relative to SAILOR and existing hierarchical world-model planning methods? In particular, beyond applying residual MPPI to the anchor space of a hierarchical imitation policy, which components of the proposed world-model architecture or two-stage search constitute new algorithmic insights?

  2. Table 5 investigates the alignment between world-model architecture and search structure, but does not directly isolate alignment between the policy and world model. Could the authors clarify this distinction and provide further evidence supporting the claim that policy–world model alignment is necessary?

2 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for recognizing the value of extending world-model-based test-time-planning to long-horizon driving and the strength of our results at practical latency. We address each question below.

Algorithmic contributions relative to SAILOR and hierarchical world models (W1, Q1)

SAILOR [1] searches over dense actions in a flat world model and was demonstrated on short-horizon manipulation tasks. Prior hierarchical world models (THICK [2], Scheiwer et al. [3], HWM [4]) either train levels independently or learn them from reward or goal signals, and none is tied to the action space of a policy. HiPlan contributes three components that these methods do not have:

  1. The upper world model’s action space is the policy’s anchor space. Anchor waypoints act as the upper model’s actions, so search runs directly over the policy’s own subgoals at their native timescales. This is what reduces the search space from 160 to 10 dimensions and the replan time from 220 ms to 69 ms (Table 17).

  2. The two levels are structurally coupled. The lower state is reinitialized from the upper state at every anchor boundary, the lower dynamics are conditioned on the upper latent, and joint fine-tuning forces the lower-level to reconstruct observations from imagined upper latents. Removing each of these costs 6.4, 9.6, and 22.7 on Test14-hard R (Table 19).

  3. Corrections are distilled hierarchically. The anchor predictor and the flow generator are each fine-tuned on corrections at their own abstraction level (Appendix D.4).

Applying residual MPPI to the anchor space alone is not sufficient (Table 4). With the policy, the two-stage search and the reward objective all held fixed, replacing the hiearchical upper latents with flat latents read out at anchor timesteps gives no gain, and coarse flat rollouts reduce performance (82.8 →\rightarrow 80.1). The gain therefore comes from the hierarchcially trained representation, which develops the timescale factorization reported in Sec 4.4, and not from the search procedure. The advantage also grows with horizon, from 7 to 22 DS between 8s and 24s on CARLA (Table 21).

Policy and world model alignment (W2, Q2)

The reviewer is correct in that Table 5 isolates world model and search alignment. Policy alignment is tested separately in Appendix E.4 (Table 18). There, the hierarchical base policy is replaced with a parameter-matched flat diffusion policy while the world model, reward models, and search are held fixed, and the performance drops from 95.19 to 79.15 on Test14 NR. We will move this result to Section 4.5 so that both axes of the alignment are reported together.

On the mismatched configuration, the choices in Appendix E.6 are the most direct way to run each structure on each world model. Hierarchical WM with flat search keeps the resets and cross-level conditioning, and flat WM with two-stage search uses the same anchor-timestep readout as Table 4. Table 4 also provides a test that involves no such choices, since only the source of the Stage 1 latents changes, and it reaches the same conclusion.

To remove the choice the reviewer highlights, we additionally ran flat search with the upper latents recomputed for every dense candidate from its states at the anchor indices, at matched world-model evaluations. Using the protocol of Table 4 (distilled base policy at 82.8 on Test14-hard R without search), flat search reaches 83.5 with fixed upper latents and 84.2 with recomputed latents, against 86.9 for two-stage search. Recomputing the upper latents, therefore, does not close the gap.

Citation format (W3). We thank the reviewer for pointing this out, and we would be updating all citations in our revised version to the ICLR author-year style.

We hope that our rebuttal has clarified the reviewer’s doubts, and we’re happy to answer any further questions.

[1] Jain, Arnav Kumar, et al. "A smooth sea never made a skilled sailor: Robust imitation via learning to search." Advances in Neural Information Processing Systems 38 (2026): 62676-62707.

[2] Gumbsch, Christian, et al. "Learning hierarchical world models with adaptive temporal abstractions from discrete latent dynamics." International Conference on Learning Representations. Vol. 2024. 2024.

[3] Schiewer, Robin, Anand Subramoney, and Laurenz Wiskott. "Exploring the limits of hierarchical world models in reinforcement learning." Scientific Reports 14.1 (2024): 26856.

[4] Zhang, Wancong, et al. "Hierarchical planning with latent world models." arXiv preprint arXiv:2604.03208 (2026).

Thank you for the clarification. I will maintain my current score, with low confidence.

Reviewer_KbvFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

HiPlan augments a hierarchical imitation policy with two levels of latent world models, learned reward models, and inference-time search. Upper-level MPPI optimizes sparse anchor corrections; a conditional flow-matching policy generates dense trajectories; a lower-level world model evaluates and filters the candidates. Search outputs are distilled into the base policy. The paper reports improved closed-loop performance on nuPlan and CARLA, with additional interPlan results and latency measurements.

Critical strengths
  1. Anchor-level search reduces the optimization space from 160 to 10 dimensions, directly addressing the cost of dense trajectory search.
  2. Evaluation covers nuPlan, CARLA, interPlan, and held-out maneuvers. The reported 69 ms replanning latency suggests practical value for 10 Hz planning.
  3. Alignment ablations, representation probes, and distillation analysis provide evidence beyond aggregate benchmark scores.
Critical weaknesses
  1. Limited conceptual novelty and an overly broad priority claim. SAILOR already combines learned dynamics, learned rewards, residual MPPI, online data collection, and distillation. Hierarchical driving and online subgoal search also predate this work, for example HiGoC. The contribution lies in the specific cross-level coupling. The claim of the “first hierarchical test-time planning framework for autonomous driving” requires narrower qualifications.

  2. The semantic and safety claims exceed the evidence. Fixed-time anchors provide geometric subgoals, but linear probing does not establish identifiable driving intents or their causal role. Expert-likeness alone does not establish safety. Although geometric-filter comparisons are provided, Algorithm 3 never revisits the anchors when all dense candidates fail verification. Selecting the candidate with the lowest maximum speed does not guarantee safety.

  3. Table 1 compares the undistilled policy at 72.62 with the final system at 86.90, while the distilled policy alone reaches 82.8. The improvement therefore includes both training and inference-time search. Figure 2 also appears to show approximately 0.92 at the default 512-evaluation budget, versus 0.869 in Table 1, without explaining configuration differences.

  4. Parenthesized numerical citations are easily confused with equation references. Interfaces among anchors, trajectories, latent states, and primitive actions remain unclear. Related work also incorrectly describes Hi-Drive, a POMDP planner requiring no training, as a hierarchical imitation policy.

Questions for authors
  1. Under matched data and compute, can anchor-level MPPI with a coarse world model trained at the correct temporal resolution and geometric verification achieve comparable performance?

  2. How are disagreements between upper- and lower-level predictions detected and corrected? Algorithm 3 does not revisit the anchors after all dense candidates fail verification.

  3. How is the fixed rejection threshold kept calibrated while the Wasserstein reward model continues changing? Please report OOD false-acceptance rates using actual safety outcomes.

  4. Why does Figure 2 appear to reach approximately 0.92 at 512 evaluations while Table 1 reports 0.869? Which checkpoints, budgets, and evaluation configurations produced these results?

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/2)

We thank the reviewer for the detailed review and address each point below.

Novelty and priority claim (W1): We would revise contribution (1) to state its distinction from prior work precisely. SAILOR searches over dense actions in a flat world model, and HiGoC trains its low-level policy with offline RL and selects subgoal sequences using that policy’s learned value function, without a dynamics model at a subgoal scale. HiPlan, on the other hand, searches over the subgoal anchors of a hierarchical imitation policy inside a world model whose action space is that anchor space, with reward models learned from demonstrations and corrections distilled into both levels of the policy. We agree that the contribution lies in the cross-level coupling, and the experiments show it is what produces the gain. With the policy, search, and reward objective fixed, flat latents read out at anchor timesteps give no gain over the base policy (82.8), while the coupled upper model reaches 86.9 (Table 4).

Anchor-level MPPI with a coarse world model and geometric verification (Q1): The separately pretrained upper model from Stage A (Appendix C.4) is trained on the same data, at the anchor resolution and on the same targets, but never coupled to the lower level. Used in Stage 1 with geometric verification in place of the lower level (Appendix D.2), it reaches 83.2 on Test14-hard R, barely above the 82.8 of the distilled policy without search. Keeping RM_lo verification raises this to 84.9, and the coupled upper model reaches 86.9. Both the learned lower-level filter and the coupling therefore contribute.

Semantic claims (W2): We agree that linear probing establishes which features are linearly accessible from each level, not identifiable intents or their causal role. We would replace “driving intents” with the probed feature names throughout. The evidence for a causal role comes from intervention rather than probing. Replacing only the Stage 1 latent source changes search outcomes (Table 4), and 64% of the scenarios HiPlan recovers require a change to the anchor sequence (Table 22).

Safety claims and fallback (W2, Q2): We would rename the lower-level stage from “safety verification” to an expert-likeness filter. All safety numbers we report come from the benchmarks’ rule-based checks (nuPlan collision and drivable-area multipliers, CARLA infractions), not from RM_lo. We agree that the lowest-maximum-speed fallback is a heuristic without a safety guarantee. It triggers in 1.5% of planning steps. Disagreements between the two levels are not detected explicitly. They surface indirectly, as low RM_lo scores on dense trajectories generated from anchors the upper model rated highly. To correct them without leaving the anchor level, we evaluated passing the top M anchor sequences to the fine stage at the same Nτ=8N_{\tau}=8 budget. With M=2M=2, fallback use drops to 0.9% and Test14-hard R changes from 86.9 to 87.3. In the remaining cases, rerunning Stage 1 with the rejected anchors excluded is a natural extension, which we leave to future work because it adds a second search pass within the latency budget.

Table 1 decomposition (W3): Appendix E.9 separates the two contributions, and we would add the distilled policy (82.8) to Table 1 to make this visible in the main results. Test-time search on the undistllled policy raises Test14-hard R from 72.6 to 81.4 (Table 23). Distilling the search corrections raises the policy itself to 82.8 without any test-time search, and searching on top of the distilled policy adds a further 4.1, reaching 86.9.

AuthorsOpenReview ↗

Rebuttal by Authors (2/2)

Threshold calibration and OOD false acceptance (Q3): The threshold is computed once after Phase 1. Across the five distillation iterations, the per-candidate rejection rate on Test14-hard ranges from 5% to 8%, and recalibrating the threshold at every iteration gives 86.1 on Test14-hard R against 86.9. On the held-out unprotected left turns at 0% retention, 7.78% of the candidates accepted by RM_lo intersect the logged future trajectories of surrounding agents, against 1.87% on Test14-hard. We use logged agents, as in nuPlan’s non-reactive evaluation, and check all accepted candidates, not only the executed one. Because logged agents do not yield to the ego vehicle, the measure mostly overcounts collisions and in practice gives an upper bound on the false-acceptance rate. The corresponding scenario-level outcomes in closed-loop simulation are the collision rates of 2.3% on Test14-hard (Table 10) and 30% on the holdout (Table 3).

Figure 2 (Q4): Figure 2 was generated during the sample-to-iteration sweep described in Section 4.3, which was run on validation scenarios, so its absolute values are not comparable to Table 1. To make the comparison direct, we regenerate it on Test14-hard R, using the full system of Table 1 for hierarchical search and the flat configuration of Table 5 for flat search, with the default search settings. At the default budget of 512 evaluations, the two curves now match the 86.9 and 72.5 reported in the tables. Hierarchical search remains ahead of flat search at every budget, reaching 84.8 against 69.9 at 64 evaluations and plateauing by 512 evaluations. This agrees with the other efficiency results in the paper, namely the matched-compute comparison in Table 5, the wall-clock measurements in Table 17, and the growing advantage at longer horizons in Table 21. We would update the caption and the corresponding text in Section 4.3.

Presentation (W4): We have switched to author-year citations, added a notation table relating anchors, dense trajectories, latent states and primitive actions, and corrected the description of Hi-Drive as a POMDP planner in Section 2 and Table 6.

We thank the reviewer again for the constructive feedback and are happy to answer any further questions.

Thank you for the additional experiments. However, they only partially resolve my concerns:

  1. Coupling contribution: Was the standalone coarse model trained for the same number of updates as the coupled model? Using the Stage A checkpoint alone may confound coupling with additional training. Please also report uncertainty for the 84.9 versus 86.9 comparison.

  2. Replacing latent sources and modifying anchors demonstrate that these components affect performance. They do not establish a causal role for identifiable driving intents or the necessity of structural alignment. The proposed wording changes are welcome, but these broader conclusions should also be narrowed.

  3. Stable rejection rates do not establish calibration. Collisions against logged agent trajectories are a useful proxy, but are not necessarily an upper bound on collisions under reactive interactions. This claim needs qualification.

I will maintain my original rating.

AuthorsOpenReview ↗

Reply by Authors

We thank the reviewer for the careful follow-up, and address their remaining points below.

Coupling contribution: The Stage A checkpoint has seen about half the world model updates of the coupled model (Appendix C.4), and reaches 84.9 ±\pm 2.1 on Test14-hard R. We therefore continued training the standalone upper model with its Stage A objective for the full budget, without coupling. It reaches 85.2 ±\pm 1.9 on Test14-hard R, against 86.9 ±\pm 2.3 for the coupled model. Since both are evaluated on the same seeds and scenarios, the paired per-seed difference is 1.7 ±\pm 0.4, and the coupled model is ahead on all three seeds. The gap therefore remains even after matching the number of updates.

Broader conclusions: We agree, and would narrow these claims beyond the wording changes in our first response. The abstract would now state that aligning the hierarchy of the world model with that of search improves both search efficiency and representation factorization, rather than alignment is necessary. The introduction describes the ablations as supporting rather than causally validating this, and Section 4.5 would state that the aligned configuration gives the largest gain in our experiments.

Calibration: Our analysis addresses calibration across training. The threshold remains consistent as RM_lo continues to change, and recalibrating it at every iteration does not improve performance (86.1 against 86.9). Calibration against safety outcomes is a separate question, and we agree that collisions against logged agent trajectories are a proxy rather than a bound, so we have removed the upper-bound claim. As the reviewer notes, the proxy remains useful. Because the same check is applied in both settings, the increase from 1.87% on Test14-hard to 7.78% on the held-out maneuver measures how the filter degrades out of distribution, largely independent of the bias of the proxy itself. A direct test of calibration against outcomes is a reliability analysis that bins candidates by their RM_lo score and measures the collision rate of each bin under reactive simulation, obtained by branching the closed-loop simulation at anchor boundaries. This analysis is currently running, and we will report it here if it completes before the end of the discussion period and include it in the camera-ready version in either case.

Reviewer_2N5HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HiPlan uses a two-level world model to refine driving plans at inference. It first searches over sparse waypoints, then generates and checks dense trajectories conditioned on the selected waypoints. The search results are also distilled back into the driving policy. The paper evaluates the approach on nuPlan, interPlan, and CARLA.

Critical strengths
  1. The 2×2 ablation is a strength. It varies the world model and search structure separately and shows that pairing the two hierarchical components gives the best result on reactive Test14-hard under matched compute budgets.

  2. The maneuver holdout experiments test recovery more directly than aggregate benchmark scores. When unprotected left turns are held out of training, HiPlan reduces the collision rate from 45% with flat world-model search to 30%.

Critical weaknesses
  1. The latency comparison with dense MPPI shows that searching over sparse anchors is cheaper, but I am less sure how much the two world models contribute to the planning gains. How would HiPlan compare against sampling more trajectories from the same distilled policy and ranking them with simple driving costs, given the same inference budget?

  2. The fine-grained stage only sees trajectories generated from one anchor sequence. If the coarse stage commits to the wrong maneuver, the fine stage can reject every candidate and still has to fall back to a trajectory from that same sequence. Under the same compute budget, would keeping a few distinct anchor sequences reduce the frequency of these failures?

  3. The coarse model takes fewer recurrent steps over the same horizon and predicts a simpler set of targets. This makes it difficult to tell how much of the lower long-horizon error comes from coupling the two models. Would a standalone coarse model trained at the same temporal resolution and on the same targets show a similar error curve?

  4. Table 17 reports 69 ms for the full planner, but I could not find the corresponding latency for the distilled policy alone. Please report the mean and P95 latency for the same checkpoint with and without search. This would show what the 4.1-point gain on Test14-hard Reactive costs at inference.

Questions for authors

At matched latency, how does HiPlan compare with sampling and reranking more trajectories from the same distilled policy using simple driving costs? At matched compute, does passing multiple anchor sequences to the fine stage reduce fallback use? How does long-horizon prediction error compare with a standalone coarse model trained at the same temporal resolution and on the same targets? What are the mean and P95 planning latencies for the same distilled policy with and without world-model search?

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for the detailed questions and for highlighting the 2×22 \times 2 ablation and the maneuver holdout experiments. We address each point below. All new results use the full system’s distilled checkpoint, which scores 82.8 on Test14-hard R without search, and the latency protocol of Appendix E.3 (single RTX 3090, batch size 1, 500 replans after 50 warmup calls).

Sampling and reranking at matched latency (W1): Within the same 69 ms budget, we sampled 16 trajectories from the distilled policy and picked the best one using rule-based driving costs (collisions, drivable-area-compliance, progress, and comfort). On Test14-hard R, reranking reaches 83.8 against 86.9 for HiPlan. On the held-out unprotected left turns at 0% retention, the distilled policy of that run collides in 57% of scenarios. Reranking reduces this to 49%, while world-model search reduces it to 30%. Reranking helps when the correct maneuver is among the policy’s samples, but world-model search is needed when it is not.

Multiple anchor sequences in the fine stage (W2): We kept the top M∈{2,3}M \in \{2,3\} anchor sequences from the final MPPI iteration and split the same Nτ=8N_\tau = 8 dense candidates across them. We observed that fallback use drops from 1.5% to 0.9% (M=2) and 0.7% (M=3) of planning steps, and Test14-hard R changes to 87.3 and 86.5.

Standalone coarse model (W3): The separately pretrained upper model from Stage A of the world model training (Appendix C.4) is exactly the model the reviewer describes. It operates at the same anchor resolution and predicts the same coarse targets, but is never fine-tuned jointly with the lower level. We would be adding it to Figure 3 and as a Stage 1 latent source in Table 4. At 8s, its high-level prediction error is 0.41 against 0.38 for the coupled upper model and 0.60 for the lower model, and as a Stage 1 source it reaches 84.9 on Test14-hard R against 86.9. Coupling therefore matters mainly for what the upper latents encode for search rather than for prediction error.

Latency of the distilled policy (W4): For the same checkpoint, the policy alone runs at 38 ms mean and 46 ms P95, and the full planner runs at 69 ms mean and 75 ms P95. The 4.1-point gain on Test14-hard R therefore costs 31 ms per replan and stays within the 100 ms budget for 10 Hz planning. We would be adding P95 values to the revised manuscript.

We thank the reviewer again for the constructive feedback and are happy to answer any further questions.

I appreciate the authors' answers.

Two anchor sequences improve the planning score by 0.4 points, and it remains unclear whether fewer fallbacks reduce collisions. The matched-latency comparison favors HiPlan, but leaves open whether the reranking baseline fails at sampling or scoring safe trajectories.

AuthorsOpenReview ↗

Reply by Authors

We thank the reviewer for the follow-up. On fallback, the gain from a second anchor sequence is within seed variation, and we see it as a robustness improvement for rare failure steps rather than a score improvement. With one anchor sequence, the fallback trajectory is followed by an at-fault collision in only 8% of the steps where it triggers, so fallbacks are rarely the cause of a collision, and reducing them can only lower the collision rate slightly. Consistent with this, the collision rate on Test14-hard changes from 2.3% (Table 10) to 1.8% with two anchor sequences.

To separate sampling from scoring in the reranking baseline, we examined every planning step on the held-out left turns where the reranker selected a sample that overlaps the logged trajectory of another agent. In 68% of these steps none of the 16 samples was collision-free, and in the remaining 32% a collision-free sample existed but the rule-based cost did not select it. The baseline therefore fails mainly at sampling, since in most failing steps the policy does not propose a safe maneuver, which world-model search recovers by changing the anchors. The remaining failures arise because the rule-based cost, which relies on constant-velocity forecasts, does not always identify the safe sample.

We thank the reviewer again for the engagement and are happy to answer any further questions.

Comments on the paper

1 comment

Dear reviewers,

As the Author–Reviewer discussion period concludes in a few days, we kindly urge you to read the authors’ rebuttal and respond as soon as possible.

Please review all author responses and other reviews carefully, and engage in open and constructive dialogue with the authors.

The authors have addressed comments from all reviewers; each reviewer is therefore expected to respond, so the authors know their rebuttal has been considered.

We strongly encourage you to post your initial response promptly to allow time for meaningful back-and-forth discussion.

Thank you for your collaboration, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.