- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper introduces PreFTO, a reinforcement fine-tuning method for an imitation-trained Diffusion Planner. Its central design separates four horizons: the 8s full plan H_p, 4s evaluation window, 1.5s replanning interval H_a, and 8 reinforcement-loss coverage H_g. At policy-visited states, PreFTO samples 16 plans, executes each through the TwoStage controller for 4s with reactive IDM traffic, and scores the resulting motion without additional planner calls.
The theoretical contribution has two parts. Proposition 1 interprets centered local scores as relative advantages plus a safety-deficit preference, conditional on a nonempty safety-preserving continuation class and a candidate-independent expected quality decrement. Proposition 2 gives a sufficient condition relating local-score improvement to deployment improvement through score–value and execution-discrepancy terms.
Validation uses reactive and non-reactive nuPlan on Val14, Test14-random, and Test14-hard. Relative to the locally evaluated IL initialization, PreFTO improves the aggregate score on all six comparisons. Ablations support the selected 4 s feedback window, original-plan targets, shared-state centering, controller-based scoring, and continuous minimum-aggregated safety, although several effects vary by split.
Critical strengths
-
The method addresses a real mismatch between long-horizon generation and short-horizon replanning with a coherent, testable design. PreFTO distinguishes prediction, evaluation, execution, and loss-coverage horizons instead of treating them as interchangeable. Its shared-state, controller-executed comparisons also expose tracking and interaction effects while avoiding repeated planner inference during scoring.
-
The empirical study is diagnostic about which parts of the local-feedback pipeline matter. Beyond aggregate performance, the paper varies horizon, target, state source, tracking model, score centering, safety continuity, aggregation, reward composition, group size, expert mixing, etc.
Critical weaknesses
-
The principal empirical claim is substantially weaker than its presentation suggests, and the evaluation does not establish reliable generalization. The highlighted 4.58-point gain on reactive Test14-hard is computed against the authors’ locally evaluated Diffusion Planner initialization (68.21 -> 72.79), not against the strongest entries in Table 1. PreFTO remains below G2DP (73.91) and PDM-Closed (75.19) on that column. At the same time, these external results were imported from different published protocols rather than reproduced under the authors’ evaluation setup, so Table 1 cannot support a controlled state-of-the-art comparison. More importantly, most principal results use one optimization seed and one evaluation seed, while Test14-hard informed training sampling, method settings, and checkpoint selection. The supplement consequently acknowledges that Hard is not an independent model-selection test. Because the largest reported gain occurs on this selection-informed split, and the gains on Val14 and Random are comparatively small, the current evidence is insufficient to establish robust or held-out improvement.
-
Critical components of the experimental protocol are insufficiently specified, preventing reproduction and obscuring the source of the reported gains. Controller execution is central to PreFTO: the TwoStage controller transforms generated references into the motion used for reward calculation, and the perfect-tracking ablation shows that this transformation materially affects performance. Nevertheless, the paper does not provide a self-contained description of the TwoStage controller, its tracking law and parameters, state initialization, or whether its operation differs between candidate scoring and deployment. Likewise, Test14-hard is repeatedly reported as “Hard,” but the paper does not explain how its 272 scenarios were selected or what operational criteria make them hard.
-
The algorithmic novelty relative to closely related diffusion-planner fine-tuning methods is insufficiently isolated. PreFTO combines policy-visited state collection, shared-state candidate sampling, controller-executed scoring, group-relative feedback, and diffusion-policy reinforcement fine-tuning. However, the paper itself states that PlannerRFT already combines closed-loop state collection, controller-executed candidate groups, adaptive exploration, diffusion-policy optimization, and comparisons of multiple reward windows. PreFTO additionally adopts the existing DiffusionNFT optimization objective rather than introducing a new underlying RL algorithm. The apparent new elements are therefore the specific continuous minimum-aggregated safety reward, the use of local execution scores to weight complete original-plan targets, and the accompanying horizon/discrepancy analysis. While potentially useful, these contributions are largely a particular scoring and target-design configuration built upon existing collection and optimization techniques.
Questions for authors
-
Can you report independently repeated end-to-end runs (including policy-state collection, candidate generation, optimization, and evaluation) with uncertainty estimates on a split that was not used for training-scenario sampling, method design, checkpoint selection, or other model-selection decisions? A reliable positive result under such a protocol would materially change my assessment.
-
How exactly were the 272 Test14-hard scenarios selected, and what criteria distinguish this split from Test14-random? Please also enumerate how Hard influenced training sampling, hyperparameter or reward design, ablation decisions, and checkpoint selection. This information is necessary to determine whether the 4.58-point improvement should be interpreted as held-out generalization or selection-set performance.
-
Please provide a reproducible description of the TwoStage controller, including its tracking mechanism, parameters, controller-state initialization, waypoint lookahead, and any differences between candidate scoring and deployment. How sensitive are PreFTO’s conclusions to this particular controller? Since the reward is computed from controller-executed motion and perfect tracking changes the results, the answer could materially affect whether the gain is attributable to PreFTO itself or to controller-specific interactions.
-
Please explicitly identify which comparisons in Table 1 are performed under identical scenario manifests, simulator settings, replanning intervals, controllers, and evaluation code. Is the intended claim only that PreFTO improves the local Diffusion Planner initialization, or do the authors claim superiority over external methods? Under the reported numbers, PreFTO does not outperform G2DP or PDM-Closed on reactive Test14-hard.
-
It would be useful to study whether PreFTO can cooperate with structured test-time sampling methods such as [1] for example, by using tree-guided trajectories during candidate generation or applying tree-guided inference to the fine-tuned model.
-
What precisely is the algorithmic distinction between PreFTO and PlannerRFT beyond the reward definition, learning-target choice, and application of DiffusionNFT?
[1]: Hyeonseong Jeon et al. Tree-Guided Diffusion Planner. 2025.
2 comments
Thank you for answering raised questions and running the experiments. I raised my rating, but I am still leaning toward rejection due to the limited algorithmic novelty and marginal improvement.
Reply to Reviewer rCGt
We thank you for your detailed review. We now address the concerns here.