- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents SpanVLA, a vision-language-action framework for autonomous driving that learns from negative and recovery examples while enabling efficient trajectory generation. First, it introduces nuReasoning-NR, a dataset and benchmark containing positive driving demonstrations, suboptimal driving behaviors, and subsequent expert recovery trajectories. Second, it proposes Negative–Recovery Reinforcement Fine-Tuning (NR-RFT), which penalizes trajectories resembling negative examples and rewards alignment with expert recovery trajectories, together with an advantage correction for difficult rollout groups. Third, its Efficient Action Bridge uses a lightweight flow-matching action expert conditioned on sparse VLM KV caches and initialized from historical trajectories to generate continuous future trajectories. Experiments on NAVSIM, nuReasoning, and nuReasoning-NR evaluate planning performance, the benefits of negative–recovery supervision, and action-generation efficiency.
Critical strengths
-
The introduction of nuReasoning-NR is a valuable contribution. By including real-world suboptimal driving behaviors and subsequent expert recovery trajectories alongside positive demonstrations, the dataset supports studying both what a driving policy should avoid and how it should recover from difficult states.
-
The paper provides a coherent SFT and GRPO-based training recipe for exploiting these different forms of supervision. NR-RFT distinguishes undesirable trajectories from expert corrections through appropriate penalties and rewards. The experiments support its practical value: NR training improves SpanVLA from 60.6 to 69.6 PDMS and AutoVLA from 59.1 to 64.1 on nuReasoning-NR, while the ablations show additional gains from recovery samples beyond negative supervision alone.
Critical weaknesses
- The ablations support the overall framework but do not fully isolate several key contributions. For the Efficient Action Bridge, Table 4 compares different systems with different reasoning lengths, while Table 5 changes both bridge size and historical initialization in the full-versus-sparse caching comparison. These evaluations are also limited to NAVSIM v1. For NR-RFT, Figure 4 omits a positive–recovery baseline, and the reward ablation removes the negative penalty and recovery reward together, leaving the additional value of negative supervision once recovery data are available unclear. Matched action-generation comparisons and more targeted data/reward ablations would strengthen the attribution of the reported gains to the proposed components.
- The dataset construction is not described in sufficient operational detail. Appendix B provides the overall composition, scenario coverage, examples, and reasoning-annotation pipeline, but does not clearly specify how negative events are selected, how segment boundaries are determined, or what qualifies as a successful expert recovery. More detailed scenario-level train/test statistics would also help characterize the benchmark. Since nuReasoning-NR is a central contribution, these details are important for assessing annotation consistency, coverage, and reproducibility.
Questions for authors
-
Dataset construction: What operational criteria were used to select negative events and identify successful expert recoveries? Please clarify how segment boundaries were determined, how these judgments were checked, and how scenario-level separation between training and evaluation was enforced. A breakdown of scenario categories across the two splits would also help characterize coverage.
-
Negative versus recovery supervision: How much does negative supervision contribute when recovery examples are already available? Could you provide a positive–recovery comparison or separately ablate the negative penalty and recovery reward under matched training budgets? The existing ablations establish the overall benefit, but do not isolate this conditional contribution.
-
Efficient Action Bridge: Can you provide a comparison between autoregressive action generation and the proposed bridge using the same VLM checkpoint and reasoning configuration, reporting planning quality, latency, and additional parameter counts? For the full-versus-sparse cache comparison, please clarify the effect of matching historical initialization and bridge capacity. Evidence on nuReasoning-NR would further help assess the bridge in the negative/recovery setting.
2 comments
Weakness 2 and Question 1: Dataset Construction and Split Protocol
We thank the reviewer for carefully reading our paper and identifying the operational details that need clarification. We hope the following explanation makes the collection, annotation, and split procedures clearer.
Negative-event selection. The negative and recovery samples are collected during real-world testing of a deployed SOTA driving policy, with a driving expert onboard. Experts intervene when they judge the policy’s behavior to be unsafe or potentially dangerous, including situations involving unsafe traffic interactions or potential traffic-rule violations. We retrieve the corresponding driving-log segments using failure and takeover tags recorded during testing. These events therefore reflect expert-identified problems encountered during actual driving.
Segment boundaries and recovery demonstrations. Sample anchors are selected manually relative to the expert’s takeover time, without a fixed temporal offset across events. Each sample contains its own 1.5-second observation history and a five-second future reference trajectory sampled at 2 Hz. Negative and recovery segments correspond to different timestamps within the same event; they are temporally associated demonstrations, rather than alternative trajectories starting from an identical state. Recovery samples contain trajectories executed by driving experts after takeover.
Manual quality checks. Following collection, we manually reviewed the candidate segments. This review is separate from the reasoning-annotation audit in Appendix B.3, which concerns positive CoT annotations.
Training–evaluation separation and scenario coverage. We partition the dataset by driving log, keeping each event’s negative segments, corresponding recovery segments, and neighboring windows in the same split. Thus, segments from the same driving log are not shared between training and evaluation. The evaluation subset contains 300 negative and 300 recovery samples.
Figure 1 summarizes the dataset’s scenario coverage, including cut-in and lead braking, stop signs, vulnerable road users, construction zones, lane bias, ego lane changes, and other scenarios. Both splits originate from the same collection pipeline.
Weakness 1, Question 2 and 3: Evidence from the Action-Bridge and NR-RFT Ablations
We thank the reviewer for carefully examining the ablation studies. We clarify the existing evidence and provide an additional positive–recovery comparison below.
Choice of evaluation benchmark. NAVSIM v1 provides a large, standardized evaluation set of approximately 12K non-trivial driving scenarios, making it a useful setting for assessing planning quality and generation efficiency across diverse situations. Our Table 4 additionally reports reasoning-length statistics averaged over 10K samples. This scale motivates using NAVSIM v1 for the main bridge ablations.
Negative versus recovery supervision. Negative examples identify behaviors to discourage, while recovery demonstrations provide corrective references. Figure 4 reports 60.6 PDMS for positive-only training, 64.2 for positive–negative training, and 69.6 for the complete positive–negative–recovery recipe.
We additionally evaluated a positive–recovery baseline, which achieves 65.4 PDMS. This comparison uses the same initialization, training-step and rollout budgets, downstream bridge-training procedure, and evaluation and selection protocol. With recovery supervision already present, adding negative supervision improves PDMS by 4.2 points, from 65.4 to 69.6.
Reward ablations examine both the joint contribution and parameter sensitivity. In Figure S4, with positive, negative, and recovery training data present, disabling both reference-based reward terms yields 87.6 PDMS, compared with 90.8 when both are enabled. Weight settings of 2, 5, and 8 yield 89.1, 90.8, and 88.4, respectively; proximity thresholds of 0.8, 1.7, and 2.5 m yield 89.7, 90.8, and 89.2.
Action-generation comparison. Table 4 compares autoregressive decoding and continuous action generation, and we add SpanVLA with autoregressive decoding here. Actually, SpanVLA and AutoVLA use the same Qwen2.5-VL VLM backbone. For ten-waypoint trajectories, the reported results are:
The flow-matching configuration achieves 1.0 points higher PDMS and 5× faster action generation than SpanVLA with autoregressive action decoding. Action-generation latency is measured separately from reasoning latency. These results demonstrate the quality–efficiency trade-off of the evaluated systems.
Bridge-design ablations. Table 5 provides further comparisons:
In particular, with interval-2 conditioning fixed, historical initialization improves PDMS by 4.4 points without increasing the measured generation time. The full-cache baseline also includes the historical initialization, and the interval-2 baseline gets better results.