- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes PARL, a planner-agnostic post-training framework for nuPlan end-to-end planners. It breaks down post-training of RL into a shared GRPO pipeline and a log-probability adapter that is replaced by planner family.
Critical strengths
The experiments were very thorough, including six planners, three parameterizations, and all six closed-loop settings were positive. Furthermore, the pure RL rows in Tables 4 and 5, without any regularization terms, still showed a gain of +1.4 to +1.5. This indicates that "open-loop metric reward + GRPO" can indeed improve the closed-loop performance of both regression and diffusion IL planners.
Critical weaknesses
The paper's descriptions at the methodological, experimental, and evidentiary levels are contradictory. The objective function of the regression adapter in Fig. 1, Eq. (4) in the main text, and the sampling K=4 per scene described in Appendix A.3 are inconsistent. The expected value of all modes in the figure is a sampled score-function estimate, while the appendix describes a group sampling process. Sec. 3.3 states "strictly on-policy, single-update," but Sec. 4.3 and Table 8 show z^old, stored rollout, and replay-implied noise, indicating no replay in a single update. DIAMOND is classified as a diffusion planner and applies the DDIM adapter, but Appendix A.2 describes it as an IL planner that "synthesizes training data using diffusion," without specifying whether its policy head is a denoising chain. ReflexDiffusion is an inference-time module, not a planner. The underlying training mechanism is not explained, and the 3.65-point difference between ReflexDiffusion-PARL and Diffusion Planner-PARL is also unexplained. Furthermore, the framework of "sharing the GRPO pipeline and only replacing log-prob computation" was proposed by DDPO, Flow-GRPO, and DanceGRPO in 2025. The latter two explicitly unified diffusion and flow matching, yet this paper does not cite any of them.
Questions for authors
-
What objective does the regression adapter actually optimize? Does it compute the expected reward over all proposals, or sample proposals for REINFORCE? Is there only one update after each rollout, or is the same batch reused for multiple updates?
-
Does the DIAMOND policy use a denoising chain? If not, why is it classified as a DM-family model and assigned the DDIM adapter? Which planner serves as the base for ReflexDiffusion post-training? If it is Diffusion Planner, why does ReflexDiffusion-PARL score 3.65 points lower than Diffusion Planner-PARL?
-
In Fig. 3 and Fig. 4, do the three rows in each column show the same scene? The road geometry suggests otherwise. Please provide renderings from all three planners on the same scene and explain how the examples were selected. How exactly does the reward correct the rear-end collision in Fig. 4(b)? Is the red circle misplaced?
-
Please supplement the comparison with scorer-based test optimization methods, including PLUTO with post-processing, Diffusion Planner with refinement, and using your own PyTorch scorer for best-of-K. Is PARL intended to replace these methods, or can it be combined with them? What are the results of adding post-processing after PARL?
-
In Table 7, row 2, adding only the KL anchor lowers the score by 1.01 points; Table 4, row 2, also shows a decrease of 0.22 points. If this term is intended to anchor the model to IL, why does it move performance further away from the IL baseline?
-
Table 8, row 1, scores 84.67 without the RL gradient, while Table 7, row 3, scores 84.68 with the RL gradient but without KL. These results are almost identical. Does the RL gradient contribute virtually nothing to FlowDrive?
-
Is the reported 0.45-point improvement entirely due to the KL term with ?
-
Which sampler and how many sampling steps were used for the closed-loop evaluation of the post-trained Diffusion Planner and FlowDrive? Did evaluation use the original checkpoint settings or the post-training setting of ?
-
Is the clean-trajectory IL-retention loss used for diffusion consistent with Diffusion Planner's original training objective? If not, please explain the difference.
-
How many epochs were actually used for RL post-training? Does “follow the original IL configuration” include the number of training epochs, meaning the same number used for Diffusion Planner's original IL training?
-
Appendix A.1 states “15 s planning horizon,” but Sec. 3.1 specifies frames at , which corresponds to . Please clarify the actual setting.
-
The captions of Table 1 and Table 2 say that improvements are shown in parentheses, but the tables use superscripts. Please make these consistent.
We thank the reviewer for the careful reading and constructive suggestions. All points refer to the submitted version;
C1 (regression-adapter objective; on-policy consistency; Q1)
C2 (DIAMOND and ReflexDiffusion; Q2)
C3 (novelty relative to DDPO, Flow-GRPO, DanceGRPO)
C4 (Figs. 3 and 4; Q3)
C5 (scorer-based test-time optimization; Q4)
C6 (KL-only rows; Q5-Q7)
C7 (evaluation sampler; retention objective; epochs; Q8-Q10)
C8 (format corrections; Q11-Q12)
We would welcome your view on the bar for acceptance: which clarifications, comparisons, and additional experiments above would be decisive to substantiate our claims. We will take your suggestions seriously in our revision, look forward to your reply, and wish you all the best in your work.