- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TeTO for test-time adaptation of LLM-based driving planners. Its core is 1) using sequential grouping to estimate relative advantages and 2) selectively updating LoRA via reward truncation. Experiments in nuPlan show improved closed-loop scores with fewer updates than dense adaptation.
Critical strengths
- The paper addresses a problem that an LLM-planner is hard to adapt to RL post-training due to sampling and expensive per-step updates.
- The evaluation includes two LLM backbones, additional target cities, mixed nuPlan–Waymo supervised training, and controls for update frequency and gating. The analysis of temporal baseline errors also makes the approximation and its limitations explicit.
Critical weaknesses
The method is mainly evaluated in a narrow city-transfer setting (Las Vegas to Singapore), with additional cities reported. However, the paper didn't show more evidence under different OOD settings to support its claims. This missing breadth is a major limitation for the paper’s significance. Details are provided below:
-
Limited Novelty and Insufficient Validation of Sequential Grouping. Sequential Grouping is a relatively incremental modification of GRPO, and its reliability under changing driving contexts remains a concern. In Appendix Tables 8–9, advantage-sign agreement drops from over 92% to around 62% during TTC reward transitions, with substantially increased context-induced baseline error. While overall planning scores improve, the paper does not establish whether updates based on these less reliable signals benefit subsequent safety-critical decisions.
-
Cross-city evidence is uneven. The main Test14-Hard evaluation focuses on Las Vegas-to-Singapore transfer. Appendix E adds Pittsburgh and Boston on Test14-Random, so the study is not restricted to one target city. Nevertheless, the detailed comparisons and methodological analyses remain concentrated on Singapore. The mixed nuPlan–Waymo experiment broadens training data but still evaluates in nuPlan on Las Vegas and Singapore. It is not cross-dataset evaluation. More consistent evaluation across the existing target cities would better support broad domain-shift robustness claims.
-
Limited comparison. Lack of comparison against non-LLM-based methods and LLM-based methods. For non-LLM-based methods, the author should compare those non-LLM-based RL methods, like PlannerRFT[1], and LLM-based methods, like DriveGPT4 and LanguageMPC, mentioned in the paper.
[1] PlannerRFT: Reinforcing Diffusion Planners through Closed-Loop and Sample-Efficient Fine-Tuning, CVPR'26
Questions for authors
See weakness