- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents a trajectory planning approach, HiPlan, for autonomous driving. It plans at test time in a hierarchical way on a coarse anchor waypoint level and a fine safety-verified trajectory level. The planning structure of evenly distributed sub-goals are a prior but then perturbations to them, reward models and residual corrections are learned to form a world model. The main novelty compared to prior work on hierarchical planning lies in the world model, trained at different hierarchical scales to allow for planning across long driving episodes.
Critical strengths
The paper is very clearly formulated. The questions at 303 structure the paper in a useful way, going from showing the performance over generalization to ablations.
The results on nuPlan and Carla are very good compared to the state of the art and a competitive route completion in Table 2 on Carla suggests that it is not just a trade-off, the performance is competitive if not better than PlanT.
The way how specific maneuvers are extracted from the training set and used for testing the distribution shift is good to highlight the limits of test-time search on the expert policy compared to pure imitation learning.
Many analyses are done to probe the learned representation and performance. It is expected but assuring to see that the lower world model and upper world model latents have developed a separated sensitivity to semantic and kinematic features.
The full hierarchical replan inference is surprisingly performant with 69 ms on an RTX 3090. The search efficiency, even with many MPPI evaluations, is better than a flat search.
The specific implementation of hierarchical planning may have modest novelty but for a simple idea the performance clearly beats the state of the art with a modest computation budget and every aspect of the method is very thoroughly examined and ablated. Good performance of the flat WM in some conditions may show that the separated sensitivity of features may be unnecessary but that is a question which of the two presented methods works better. It seems the hierarchical version does work better in OOD policy regions.
Critical weaknesses
The training time is significant. 600 GPU-hours on 8 H200 for training probably means several weeks of training on more modest setups.
One explanation for the degradation under distribution shift in Table 3 can be that the approach does not include a true environment reward and it is unclear how to do so. The policy is still learned in IL fashion against expert trajectories and not with feedback from the environment.
The idea is very similar to concurrent work: CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Driving Yihong Guo ⋅ Dongqiangzi Ye ⋅ Sijia Chen ⋅ Anqi Liu ⋅ Xianming Liu (ICML 2026)
The authors should contrast their work with this CorrectionPlanner
Questions for authors
Please add recent concurrent work to the related work and contrast with your work:
Nguyen, L., Fauth, M., Jaeger, B., Dauner, D., Igl, M., Geiger, A., & Chitta, K. (2026). Lead: Minimizing learner-expert asymmetry in end-to-end driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 39775-39785).
and the CorrectionPlanner described in critical weaknesses
Could you define the paper's understanding of the word dissociation? The word's use in psychology seems to overshadow other uses and outside of that domain in chemistry, graph theory etc. it all means something very specific to that domain so it may be hard to understand the exact meaning here.
Please consider also making terms like factorized representation, driving-length horizon more clear.
In line 188 the state pair of equation 16 appears without any reference or explanation. Could you define s there or refer to equation 16?
The formulation in line 351, it says we remove "all scenarios" and then it list different levels of retention which seems contradictory. Probably it is meant with 50% retention they are of course not all removed but the paragraph should be rephrased accordingly.
The bold, underline notation in Table 1 is not always followed. Bold should indicate best but in the first two columns two results are bold. If it is suggested to have one bold best result for HiPlan and one for the others, then column 1,3,4,5 and 6 are missing a bold entry. If not, there should not be two bold result. Underlined results in those columns needs to be adjusted respectively.
Why were no RL baselines compared on Carla?
3 comments
I thank the authors for the clarifications and especially responding on how to add explicit rule-based terms completes a positive impression of the method. After reading current reviews and answers I will change my rating to accept unless other issues will be brought up during the rest of the review phases that I overlooked.
Reply by Authors
We thank the reviewer for the positive reassessment and for the constructive suggestions, and are glad that we were able to address the reviewer’s concerns.
Rebuttal by Authors
We thank the reviewer for their detailed feedback and address their remaining concerns below.
Training cost: We agree that training is substantial, particularly on smaller GPUs. Optimizing training efficiency was not the focus of this work, and there is clear room to reduce it. Most of the gain from distillation comes in the first iterations, so a single round requires less than half the compute and already reaches 84.27 on Test14-hard R (Table 23). Rollout collection, which accounts for about 60% of each iteration, could also be reduced by reusing data in the replay buffer across iterations instead of collecting fresh rollouts each round. However, deployment is already cheap. The full planner runs at 69 ms per replan on a single consumer-grade RTX 3090, within the 100 ms budget for 10 Hz planning, and the distilled policy alone reaches 82.8 on Test14-hard R without any search.
Environment reward: HiPlan is designed for the imimtation setting, so its reward is learned as a discriminator between expert and policy behavior rather than hand-constructed. Explicit rule-based terms can be added directly (we show this in Appendix D.2), and we see that adding a hard geometric filter before RM_lo only gets a 0.2 point change in Test14-Hard (86.9 → 87.1). Our learned reward therefore already captures most of what the explicit constraints enforce. In Table 3, while all three systems degrade as maneuver data is removed, HiPlan degrades least (30% collisions at 0% retention against 45% for flat WM search and 72% for base policy). We would add combining learned and environment rewards as a direction in the conclusion.
Concurrent Works:
We thank the reviewer for pointing out prior related works that we overlooked. We would include both of them in our updated manuscript and we describe below how they are related to our works:
CorrectionPlanner [1] : CorrectionPlanner is an autoregressive motion-token planner that uses a learned collision critic to flag unsafe tokens over a short horizon and regenerates them, trained with imitation followed by model-based RL in a frozen world model. Its corrections act on individual tokens, with no temporal abstraction, so it can revise a plan only once a collision is imminent. HiPlan searches over subgoal anchors across the full horizon in a world model at the subgoal timescale, and distills the corrections back into the policy. It can therefore change the maneuver before a failure develops, and its learned reward also penalizes failures other than collisions, such as stalled progress. This difference matters in practice, since 64% of the Test14-hard scenarios recovered by HiPlan require a change to the anchor sequence (Table 22). On Test14-hard, HiPlan reaches 83.77 NR and 86.90 R against 75.37 NR and 77.29 R for CorrectionPlanner.
LEAD [2] : LEAD studies information asymmetry between previliged experts and sensor-based students in end-to-end driving, and narrows it through a revised expert, a new dataset, and the TFv6 student. This is complementary to HiPlan, which plans on processed scene representations, and PlanT does. We would add LEAD to related works and TFv6 to Table 2, marked as a sensor-based method.
Terminology: We use “double dissociation” in its neuropsychological sense. Two systems show a double dissociation when each performs well on one class of task and poorly on the other, in opposite directions. Here, upper-level latents linearly predict high-level features well and kinematic features poorly, and lower-level latents show the reverse (Figure 4). A “factorized representation” means that information at the two timescales is linearly accessible from different latent spaces. A “driving length horizon” refers to 8s at 10Hz (80 steps) on nuPlan and up to 24s on CARLA, compared with short horizons of prior test-time planning in manipulation. We would add these definitions clearly in the updated manuscript.
Presentation: We would change Line 188 to define the upper-level state skup=(hkup,zkup) and refer to Eq. 16. We thank the reviewer for pointing this out. Line 351 would also now state that a given fraction of scenarios in each maneuver is retained. We would change Table 1 to mark a single best and second-best result per column.
RL Baselines on CARLA: Table 2 includes three RL planners (Roach, Think2Drive, and CaRL), labeled by type in Table 14. We would add the method types in Table 2 so that this is visible in the main text.
We thank the reviewer again for the constructive feedback, and are happy to answer any further questions.
[1] Guo, Yihong, et al. "CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Driving." arXiv preprint arXiv:2603.15771 (2026).
[2] Nguyen, Long, et al. "Lead: Minimizing learner-expert asymmetry in end-to-end driving." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.