- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces reference-ranked continuation schedules to construct rollout groups for reinforcement fine-tuning of learned planners while retaining their deployment search. It reports improved parking success under fixed beam search and a small NAVSIM gain from a separate, single-step DrivoR specialization.
Critical strengths
-
Meaningful improvement under fixed deployment search. On the 359-scene dead-end suite, success increases from 84.68% to 92.20% with the same beam configuration, reaching the performance of a substantially wider-search baseline.
-
Concrete sampling design with supporting evidence. The matched 120-rollout probe improves successful-rollout coverage from 85.9% to 89.7%. The appendices clarify the independent geometry sampling, policy likelihoods, and collision-checking procedure.
Critical weaknesses
- The final benefit of rank allocation remains insufficiently isolated. Table 2 provides an equal-budget pre-training probe, but the paper does not report final deployment performance for the expanded-legacy control with 120 rollouts per scene.
Questions for authors
-
Does the parking baseline include MultiPark’s original argmax fine-tuning with outcome-oriented losses, or only teacher-forced imitation training?
-
Could you clarify how the imitation-learning and RL training data are separated from the dead-end and Mini-5K evaluation suites, including any checks for overlapping scenes or closely related variants?
3 comments
Response for Reviewer M7hD — Part 2
Answers to the Questions
Q1. Does the parking baseline include MultiPark’s argmax fine-tuning?
Yes. The starting parking checkpoint includes both training stages:
- Teacher-forced imitation learning, using demonstrated plans.
- MultiPark-style argmax fine-tuning with outcome-oriented losses.
Our RL fine-tuning starts after these two stages. The baseline result of 304/359 successes (84.68%) therefore comes from a model that has already received argmax fine-tuning; it is not a teacher-forcing-only baseline.
The original legacy run, the main rank-tree run, and the additional 120-rollout control start from the same checkpoint. Their comparison therefore does not reflect a difference in whether this earlier fine-tuning stage was included.
We will make the two-stage preparation explicit in the experimental setup and clarify the meaning of “imitation baseline” in the tables.
Q2. How are IL and RL training data separated from the evaluation suites?
For the main reported comparisons, the dead-end benchmark and Mini-5K come from a held-out test partition excluded from both imitation learning and RL fine-tuning.
Partitioning the source data. We reserve 10% of the raw scenario dataset for testing and a separate 10% for validation, with sampling intended to preserve the scenario distribution. The examples used for IL and RL parameter updates come from the training partition.
Constructing the RL dataset. We run the starting parking checkpoint on the training partition, collect 2,906 failure cases, and randomly select 3,094 successful cases. Together, these form the 6,000-scene RL dataset. The aim is to obtain an approximately balanced training set while reducing the amount of data needed for fine-tuning. These scenes are drawn from training data, not from either evaluation suite.
Constructing the evaluation suites. Mini-5K contains 5,000 scenes sampled from the held-out test partition. The 359-scene benchmark is manually selected from that partition to focus on difficult parking cases. Its overlap with the RL set in the type of scenario—such as dead-end parking—does not mean that the same scenarios are used for training.
Checkpoint selection. The main rank-tree checkpoint was selected using closed-loop evaluation on the training corpus. The new 120-rollout control and the 28-rollout experiment use predefined epoch-20 checkpoints. Neither the dead-end benchmark nor Mini-5K was used to choose these checkpoints.
Closely related variants. The separation we can confirm is at the scenario level. We have not established cluster- or recording-level disjointness, so we cannot rule out closely related variants appearing across partitions. We will state this limitation explicitly rather than treating scenario-level separation as proof of independence at every level.
For completeness, Appendix E describes an earlier experiment trained directly on the 359-scene suite. This is a separate train-on-test diagnostic, not a held-out evaluation of the main 6K-trained model. We will keep that distinction explicit and exclude it from generalization claims.
We will revise the data description to state the partitioning procedure, the exact 6,000-scene RL composition, the sources of both evaluation suites, and the level of overlap checking that has been established.
Follow-up on Revisions and Clarifications
Dear Reviewer M7hD,
Thank you again for your thoughtful review and constructive suggestions. As the discussion period approaches its end, we would appreciate your feedback on whether our rebuttal and revised manuscript address your concern and questions.
Following your suggestions, we have:
- Added the end-to-end 120-rollout expanded-legacy comparison, alongside the 28-rollout reduced-budget experiment, and revised the discussion of what these results establish about rollout budget and continuation-rank allocation.
- Clarified that the starting parking model includes both teacher-forced imitation learning and MultiPark-style argmax fine-tuning with outcome-oriented losses.
- Documented the training and evaluation splits, including the scenario-level separation of the dead-end and Mini-5K benchmarks from IL and RL training, while noting that cluster- and recording-level disjointness have not been established.
Changes in the revised manuscript are marked in red. Please let us know if any points remain unclear or require further clarification. We appreciate your time and the suggestions that helped us improve the paper.
Response for Reviewer M7hD — Part 1
Thank you for recognizing the improvement under fixed deployment search and the clarity of our sampling design and implementation details. We appreciate your request for a stronger training comparison and clearer descriptions of the starting model and data separation. We address the weakness first, followed by the two questions in the next part.
Response to the Weakness
To address the missing comparison, we have now completed end-to-end training of the 120-rollout expanded-legacy control. We also conducted a reduced-budget experiment with 28 rollouts per scene. These experiments provide final-model results beyond the original pre-training coverage probe.
W1. Final performance at matched and reduced rollout budgets.
120-rollout control. We trained an expanded-legacy control corresponding to the D=0 specialization of our sampling framework. For each of the same 30 initial queries, we independently sample four versions of the motion geometry, giving 30×4=120 rollouts per scene.
At later decisions, every rollout selects the highest-ranked legal continuation while continuing to sample geometry. Thus, D=0 does not mean that parking ends after one segment: the rollout can continue for up to eight segments, but it does not explore alternative continuation ranks. This provides the requested comparison with the four continuation-rank schedules used by the main K=2,D=2 configuration.
Both 120-rollout runs use seed 42 and share the starting checkpoint, training data, reward shaping, failure-group gating and damping, imitation anchor, optimizer settings, batch size, and training duration.
28-rollout reduced-budget experiment. We also retain only the top seven initial queries while keeping K=2,D=2. This gives
B=MKD=7×22=28
rollouts per scene—slightly fewer than the original legacy configuration’s 30 rollouts. The training data, reward shaping, failure damping, imitation anchor, optimizer settings, and training duration remain unchanged from legacy.
Both additional experiments use seed 42 and predefined epoch-20 checkpoints. The main D=2 result remains the paper’s epoch-15 checkpoint, selected using closed-loop evaluation on the training corpus. The test benchmarks were not used to select these checkpoints.
Results under unchanged deployment search. All models below use the same beam-width-8/top-6 evaluator and success and collision rules. Counts in parentheses are successful scenes.
What these experiments show.
The 28-rollout run improves the dead-end success count from 304 to 313: nine net additional successes, or +2.51 percentage points. This demonstrates an improvement in this run with fewer rollouts than the original legacy configuration. However, choosing seven initial queries also changes which starting proposals are explored. We therefore present it as reduced-budget evidence, rather than an isolated test of continuation-rank allocation.
At 120 rollouts, the D=0 and D=2 configurations give similar success rates, with the D=0 run solving one more dead-end case. This suggests that deeper continuation exploration is not necessary for strong performance in this parking setting. We therefore treat continuation depth as a configurable allocation choice whose benefit should be assessed for the task and budget.
Together, these experiments extend the evidence beyond the pre-training probe. They also show why initial rollout coverage and final planning success should be reported separately: the higher pre-training coverage of D=2 in Table 2 does not produce higher final success in this comparison.
We have added the 28-rollout ablation to the revised manuscript and will include the completed 120-rollout comparison alongside it, with the training settings, checkpoint-selection rules, and interpretation made explicit.