- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces RefineDrive, a failure-guided post-training framework for vision-language-action (VLA) models in autonomous driving. The core concept involves leveraging failure trajectories—generated by the model itself—to construct targeted diagnostic and corrective supervision, followed by safety-prioritized reinforcement learning to enhance the model's direct trajectory planning capabilities. The framework consists of four stages: First, failure cases—such as collisions or departures from the drivable area—are identified within the training scenarios. Second, a simulator’s state information is used to generate structured diagnostics, including the timing of the event, the positions of relevant objects, and the direction of boundary violations; simultaneously, corrective trajectories are retrieved from a human-demonstration dataset—selecting those that align with the model’s failed predictions while satisfying safety constraints for the current scenario. Third, the model undergoes "Correction SFT" (Supervised Fine-Tuning) to learn from these diagnostics and corrective trajectories. Finally, the model is trained using GRPO (Group Relative Policy Optimization) with a hierarchical reward function that integrates hard safety constraints, continuous safety margins, and driving progress. During inference, the model outputs trajectories directly based on the driving context, requiring no additional diagnostic or repair steps. Built upon the Qwen3.5-4B architecture, RefineDrive improves the PDMS (Planning-based Driving Metric Score) on the NAVSIM v1 benchmark from 87.7 to 91.7. The same checkpoint achieves an EPDMS of 89.4 when evaluated on the NAVSIM v2 extended metrics using the original NAVTEST scenarios. Furthermore, the paper presents ablation studies analyzing the individual contributions of the structured diagnostics, the retrieval of corrective targets, and the hierarchical safety-based reward function.
Critical strengths
Rather than continuing to learn from general examples of successful behavior, this paper focuses on addressing the safety flaws exposed by current policies. It establishes a comprehensive pipeline that spans failure detection, the construction of supervisory signals, and policy optimization.
Metrics such as time-to-collision (TTC), interactions with relevant objects, and lane-boundary violations are derived directly from the simulator state, thereby minimizing the reliance on inferences—common in free-text diagnostic feedback—that are difficult to verify. Furthermore, candidate corrective trajectories are re-validated within the specific context of the failure scenario, rather than simply adopting human-demonstrated trajectories from other scenarios as "safe" targets.
Within the defined reward framework, valid trajectories that satisfy strict safety constraints are consistently assigned higher rewards than unsafe ones, while maintaining continuous safety-related feedback. As shown in Table 4, this reward structure—compared to direct optimization of the baseline policy (PDMS)—improves performance across NC (near-collision), DAC (dangerous action), and TTC metrics, all while maintaining an equivalent average driving progress.
The authors conduct several controlled experiments: comparing human ground-truth data against retrieved corrective targets; evaluating the impact of removing diagnostic supervision while keeping the corrective targets fixed; and assessing the effects of different initializations for the "Correction SFT" (Supervised Fine-Tuning) phase followed by GRPO (Group Relative Policy Optimization). These controls help isolate the specific contributions of each component.
The diagnostic and corrective mechanisms are employed exclusively during the training phase; there is no explicit "repair" stage during deployment. The authors also explicitly report the safety-versus-progress trade-offs for the full training approach compared to various ablation scenarios, and they clarify that their extended metric-based evaluation does not constitute the complete, two-stage NAVSIM v2 protocol.
Critical weaknesses
The retrieval process terminates at the first cluster containing a safe candidate and is capped at a budget of 1,000 candidate evaluations; consequently, there is no guarantee that the retrieved safe target is the absolute nearest one within the entire trajectory library. The study currently lacks analysis regarding correction distance, retrieval success rate, and sensitivity to the search budget. Furthermore, there is no direct verification that a smaller feature distance necessarily correlates with a smaller behavioral change.
Cases involving collision objects that cannot be projected onto the current front-view perspective, instances where no safe target can be retrieved, and samples that fail filtering criteria are all excluded. The final dataset of 32,650 correction records may therefore be biased toward failures that are visible and easily remediable. However, as the paper does not report the counts or retention rates at each filtering stage, it is difficult to assess the extent to which challenging scenarios are covered.
The input for the "Correction SFT" (Supervised Fine-Tuning) includes failed trajectories, while the output includes both a diagnosis and a correction; during inference, however, neither the failed trajectory nor the diagnostic output is present. While ablation results support the method's effectiveness, it remains unclear whether the gains stem from the diagnostic content, the additional supervision tokens, or simply the repeated training on difficult scenarios. Further controlled experiments are required.
Some key performance differences are marginal—such as the 0.3 PDMS improvement attributed to diagnostic supervision, or the 0.1 PDMS gap between the final full approach and the "Human-GT" (ground truth) baseline—and these results lack multiple training runs or confidence intervals. Moreover, the absence of a control experiment involving continued SFT with an equivalent training budget makes it difficult to fully rule out the impact of additional computational training effort.
The experiments are centered on a 4B-parameter backbone and the NAVSIM data ecosystem, with extended metric evaluations conducted using the same NAVTEST scenarios. Consequently, the current evidence primarily supports performance improvements within a specific data ecosystem; it is insufficient to demonstrate generalization across different models, datasets, or long-horizon, closed-loop interactions.
While the simulator can verify when a collision occurs, which object is involved, and when a boundary violation takes place, this information does not necessarily reveal the underlying causes of the erroneous decision-making. The appendix further notes that object projection does not explicitly account for occlusions, and that the determination of boundary-violation direction sometimes relies on heuristic rules. Therefore, claims regarding the reliability of the diagnostics should be interpreted strictly within the context of event labeling.
Questions for authors
Could you report the distance distribution relative to human ground truth (Human GT), the retention rate of maneuver types, and performance results across global nearest-neighbor searches, varying cluster counts, and different search budgets?
How many failure cases remain after the filtering process? Please report the initial number of failures, the filtering rate (via visual localization), the retrieval success rate, and the final retention rate. Additionally, analyze whether the excluded cases are concentrated in challenging scenarios, such as those involving occlusions or complex interactions.
Why does diagnostic supervision improve performance compared to direct planning without diagnostics? Could you include ablation studies—such as using generic text of equal length, event-only labels, randomized (shuffled) diagnostics, or a direct mapping from "driving context" to "corrected trajectory"—to isolate the specific contribution of the structured diagnostic content?
Are the performance differences consistent? Could you provide results across multiple random seeds, scene-level paired confidence intervals, and baselines using continued Supervised Fine-Tuning (SFT) with equivalent training steps or computational budgets?
Does the improvement in safety come at the cost of degradation in specific behaviors? Beyond the average EP (Expected Performance/Error metric), could you report the frequency of full stops, the proportion of scenarios with low progress, and performance broken down by maneuver type? In Table 2, the performance for DDC (Direct Driving Control/Command) and LK (Lane Keeping) is relatively lower compared to some other methods; is this because the current reward function does not explicitly cover these behaviors?
Can the method generalize to broader settings? Could you provide evaluations using a different model backbone, an out-of-distribution (OOD) test set, or long-duration closed-loop simulations? Furthermore, please report the total computational cost—covering failure mining, candidate simulation retrieval, and GRPO—to help assess the practical deployment overhead.
Reminder: Author Response and Reviewer Discussion
Dear Reviewers,
We are halfway through the discussion period. Please review the authors’ responses, address any remaining concerns, and engage in the discussion where appropriate. If your assessment remains unchanged, please briefly acknowledge that you have considered the authors’ responses.
Thank you for your time and contributions.