acceptodds
Under review as a conference paper at ICLR 2027

VarioDrive: Adaptive and Diverse Reasoning for End-to-End Autonomous Driving

Abstract

Vision-language-action (VLA) models have shown a promising effectiveness in autonomous driving planning. Existing approaches, however, primarily optimize the final trajectory, providing limited feedback on whether the intermediate reasoning is grounded in the scene and actually supports the plan. Moreover, they typically commit to a single reasoning path per scene, leaving alternative behaviors with different safety, efficiency, and comfort trade-offs unexplored. To address these issues, in this paper, we propose VarioDrive, a planning-oriented post-training framework that aligns structured reasoning with trajectory generation and enables adaptive, preference-conditioned planning within a single autoregressive model. Specifically, we first introduce a time-indexed waypoint tokenization that factorizes each future pose into time-indexed position and heading tokens, together with a geometry-aware soft supervision that makes token prediction aware of spatial distance, so that trajectories can be directly decoded without an external planner. Then, we organize each response into a scene description, driving-intent reasoning, and a planning answer, and design stage-decomposed rewards that separately evaluate visual grounding, intent-trajectory alignment, and preference-dependent trajectory quality during RL post-training. We further develop an adaptive fast-slow inference mechanism that generates a single plan for simple scenes while generating and comparing safety-, efficiency-, and comfort-oriented plans for complex scenes before selecting the final trajectory. Experiments on NAVSIM, nuScenes, and Bench2Drive demonstrate a competitive planning performance. On NAVSIM, stage-decomposed rewards improve PDMS by 2.09 points over answer-only RL, and the resulting model reaches 92.42 PDMS with multi-preference selection.

Primary area:
applications to robotics, autonomy, planning
Keywords:
autonomous driving, vision-language-action models, chain-of-thought reasoning, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_369zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VarioDrive proposes a planning-oriented post-training framework that couples driving-scene descriptions and intent reasoning with autoregressive trajectory prediction. Time-indexed position and heading tokens receive geometry-aware soft supervision, while decomposed reinforcement-learning rewards assess visual grounding, reasoning-to-trajectory alignment, and preference-dependent planning. Adaptive inference uses one plan for simpler scenes and compares safety-, efficiency-, and comfort-oriented candidates in more complex scenes. Reported performance reaches 92.42 PDMS on NAVSIM with Slow selection and a driving score of 79.70 on Bench2Drive.

Critical strengths
  1. Planning-oriented post-training addresses a meaningful reasoning-to-action gap. Separating scene descriptions, driving rationales, and waypoint answers exposes intermediate outputs for targeted feedback. Such a formulation helps study whether language reasoning supports physical planning.
  2. Geometry-aware waypoint supervision provides a concrete improvement. Time-indexed pose tokens preserve spatial structure, while nearby bins receive graded targets instead of identical penalties. Adding geometric soft supervision improves PDMS from 71.93 to 80.96 under a matched tokenization setting.
  3. Preference-conditioned planning produces useful candidate diversity. Safety, efficiency, and comfort conditions yield different progress and motion characteristics. With three candidates per pool, mixed-preference oracle PDMS reaches 93.26 versus 91.67 for the strongest single-preference pool.
  4. Evaluation spans complementary autonomous-driving benchmarks. NAVSIM, nuScenes, and Bench2Drive measure non-reactive planning, maneuver-level decisions, and closed-loop driving. Results provide a broader perspective than a single benchmark score.
  5. Ablations support several components of the training and inference pipeline. Stage-decomposed rewards increase NAVSIM PDMS from 88.75 to 90.84 over answer-only reinforcement learning. Learned selection also improves over random selection on an identical candidate pool.
Critical weaknesses
  1. Reasoning rewards do not establish factual or causal faithfulness. Rewarding a higher likelihood of the generated trajectory after conditioning on a rationale may also favor incorrect but predictive explanations. Removing the reasoning prefix changes the input distribution, so stronger counterfactual reasoning tests are needed.
  2. Synthesized rationales may encourage post-hoc explanations. Training rationales are generated with access to a motion summary derived from the expert trajectory. Consequently, models may learn to justify known actions rather than independently derive better decisions from observations.
  3. Novelty relative to closely related reasoning-aware driving methods is not sufficiently isolated. Existing approaches already optimize reasoning-action consistency and trajectory quality, whereas matched comparisons mainly examine answer-only feedback and selected VLA recipes. Comparisons against similar process-reward designs under aligned training settings would clarify the methodological contribution.
  4. Adaptive routing lacks matched-compute controls. Slow inference is triggered on 56.22% of scenes, but comparable random, uncertainty-based, or oracle routing policies are not evaluated. Without such controls, improvements cannot be confidently attributed to learning when additional reasoning is useful.
  5. Inference latency substantially limits practical deployment. Complete-output latency is 2.87 seconds for Fast, 7.15 seconds for Slow, and 5.30 seconds for Adaptive on an H800 at batch size eight. Single-scene response times and delay-aware closed-loop safety have not been established.
  6. Closed-loop advantages are modest and inconsistent across outcomes. Driving score reaches 79.70 versus 78.84 for AutoVLA, but success rate falls to 55.91% versus 57.73%, with lower reported comfort. Reliance on published external baseline results also limits the strength of cross-method conclusions.
Questions for authors
  1. Can authors evaluate semantically contradictory but length-matched reasoning traces to distinguish causal planning support from token-likelihood gains?
  2. How would the stage-decomposed reward compare with existing reasoning-action alignment objectives under the same backbone, data, and compute?
  3. What are the results of random, uncertainty-based, and oracle Fast/Slow routing at the same Slow-call budget?
  4. Can authors report single-scene end-to-end latency and latency-aware closed-loop driving outcomes?
  5. How do authors explain lower closed-loop success and comfort than AutoVLA, and can external baselines be matched more closely?
6 comments

We thank the reviewer for the detailed comments, and for recognizing the planning-oriented formulation, the geometric waypoint supervision, and the preference-conditioned candidate diversity. We added counterfactual reasoning tests, routing controls with the same number of Slow calls, and single-scene latency measurements, and will include them in the revised paper. In summary:

  • Counterfactual reasoning (W1/Q1). Replacing one decision-relevant judgment in the thinking lowers the regenerated PDMS from 91.81 to 35.89 (750 NAVTEST scenes), and the thinking reward gives the original thinking a paired preference rate of 99.33%, counting ties as half. Length matching preserves the qualitative result.
  • Expert-conditioned rationales (W2). Only the annotation model sees the expert motion summary. The policy never receives it, and during RL its thinking is rewarded for supporting its own trajectory.
  • Alignment objectives (W3/Q2). On the same pairs, our reward gives a higher paired preference than entropy reduction and Alpamayo-R1-style reasoning-action consistency (99.33% vs. 95.33% and 83.67%).
  • Routing controls (W4/Q3). At the same 50% Slow budget, learned routing reaches 90.63 PDMS, vs. 89.87 for random routing (expectation) and 89.52 for uncertainty-based routing. We did not evaluate Oracle-K.
  • Latency (W5/Q4). Single-scene latency is 3.15 s (Fast) and 8.14 s (Slow); Bench2Drive uses synchronous execution.
  • Closed-loop results (W6/Q5). The higher DS but lower SR can coexist because the metrics capture mean route score and the fraction of successful routes, respectively. Table 14 reports jerk for planned trajectories, while Bench2Drive Comfortness measures executed motion.

Response 1 - Contradictory, length-matched reasoning (W1 & Q1)

W1. Reasoning rewards do not establish factual or causal faithfulness. Rewarding a higher likelihood of the generated trajectory after conditioning on a rationale may also favor incorrect but predictive explanations. Removing the reasoning prefix changes the input distribution, so stronger counterfactual reasoning tests are needed.

Q1. Can authors evaluate semantically contradictory but length-matched reasoning traces to distinguish causal planning support from token-likelihood gains?

We conducted this test on 750 NAVTEST scenes. Replacing one decision-relevant judgment in the thinking with another scene’s justification and action conclusion, while holding observations, navigation, description, and format fixed, lowers the PDMS of the regenerated trajectories from 91.81 to 35.89. The paired preference rate for the original thinking is 99.33% with half credit for ties, compared with 95.33% for entropy reduction and 83.67% for reasoning–action consistency, and copying the action statement into the thinking did not increase the reward in any tested pair. Both the reward and the planner therefore respond to the decision content of the reasoning under this intervention.

Scoring methodPaired preference (%)Ties (%)
Our discrete thinking reward99.331.33
Valid-token entropy reduction95.335.60
Reasoning–action consistency83.6712.93

Paired preference is (nwin+0.5ntie)/750(n_{\mathrm{win}}+0.5n_{\mathrm{tie}})/750, with ties also reported separately. Length matching preserves the qualitative result. Because the intervention changes the action conclusion as well as its justification, this measures decision-semantic sensitivity rather than factual correctness with the action held fixed.

On 750 randomly sampled NAVTEST scenes using safety-first Fast inference, original trajectories score 91.82 PDMS on this subset, compared with 90.84 on full NAVTEST (Table 4). Regeneration with the original thinking gives 91.81, versus 35.89 after intervention, a 55.92-point decrease.

Response 2 - Expert-future-conditioned SFT rationales (W2)

W2. Synthesized rationales may encourage post-hoc explanations. Training rationales are generated with access to a motion summary derived from the expert trajectory. Consequently, models may learn to justify known actions rather than independently derive better decisions from observations.

We agree that SFT rationales explain the demonstrated action, and we use them to teach the output format and the link between observations and decisions. The expert motion summary is given only to the annotation model, never to the policy: in both SFT and RL, the policy conditions on observations and driving context, and during RL its thinking is rewarded for supporting its own generated trajectory rather than the expert's. Consistently, when the turn command is replaced with go straight, the generated reasoning changes accordingly in 79/80 scenes, as illustrated in appendix B.3.

Concretely, during on-policy RL rollouts, the policy receives observations and driving context without the expert future trajectory, and generates the rationale followed by its own trajectory. The thinking reward evaluates the rationale's likelihood support for that generated trajectory. Expert references used to compute trajectory-quality rewards are training supervision, not future information supplied to the policy.

Thus, SFT provides demonstration-based reasoning supervision, while RL optimizes reasoning–trajectory alignment on the policy's own outputs. Table 6's thinking-reward ablation supports its planning utility. The contribution is on-policy alignment rather than a guarantee that every rationale faithfully exposes the model's internal decision process.

Response 3 - Comparison with existing process/alignment rewards (W3 & Q2)

W3. Novelty relative to closely related reasoning-aware driving methods is not sufficiently isolated. Existing approaches already optimize reasoning-action consistency and trajectory quality, whereas matched comparisons mainly examine answer-only feedback and selected VLA recipes. Comparisons against similar process-reward designs under aligned training settings would clarify the methodological contribution.

Q2. How would the stage-decomposed reward compare with existing reasoning-action alignment objectives under the same backbone, data, and compute?

On the same 750 policy outputs, our thinking reward gives the highest paired preference for the original thinking over the counterfactual one (99.33%, vs. 95.33% for entropy reduction and 83.67% for Alpamayo-R1-style reasoning–action consistency) and the fewest ties (1.33%). This compares reward signals rather than retrained policies. Table 2 separately compares VarioDrive and the AlphaDrive training recipe on the same Qwen3-VL backbone (87.14% vs. 78.50% nuScenes accuracy). Full stage-reward RFT adds 2.09 PDMS over answer-only RFT (Table 10).

Stepwise process verifiers such as ThinkPRM assess intermediate correctness. Our stage-decomposed feedback evaluates scene grounding, intent–trajectory alignment, and preference-dependent trajectory quality. The rewards are combined into a completion-level advantage. The intent component measures the likelihood gain of the same generated trajectory and uses the paper's discrete reward Rthk∈{0,0.5,1}R_{\mathrm{thk}}\in\{0,0.5,1\}.

We compare scoring functions on 750 shared pairs without further training, replacing only a key judgment and its action conclusion within thinking. Valid-token entropy reduction measures H(pwithout)−H(pwith)H(p_{\mathrm{without}})-H(p_{\mathrm{with}}). The reasoning–action consistency baseline follows the meta-action matching principle used by Alpamayo-R1.

For N=750N=750 pairs, we report:

\qquad T=\frac{1}{N}\sum_i\mathbf{1}(R_i^{\mathrm{orig}}=R_i^{\mathrm{swap}}).$$ | Scoring method | $100W$ | $100T$ | | --- | ---: | ---: | | Our discrete thinking reward | 99.33 | 1.33 | | Valid-token entropy reduction | 95.33 | 5.60 | | Reasoning–action consistency | 83.67 | 12.93 | Our reward gives the highest paired preference and lowest tie rate. This compares scoring behavior on shared policy outputs. It does not retrain the policy under alternative rewards. Existing controlled training ablations provide complementary evidence: thinking feedback raises nuScenes accuracy from 80.95% to 85.92%, while full stage-reward RFT improves NAVSIM PDMS from 88.75 to 90.84 over answer-only RFT (Tables 6 and 10).

Response 4 - Random, uncertainty, and oracle routing (W4 & Q3)

W4. Adaptive routing lacks matched-compute controls. Slow inference is triggered on 56.22% of scenes, but comparable random, uncertainty-based, or oracle routing policies are not evaluated. Without such controls, improvements cannot be confidently attributed to learning when additional reasoning is useful.

Q3. What are the results of random, uncertainty-based, and oracle Fast/Slow routing at the same Slow-call budget?

At the same Slow budget (50% of 2,048 stratified NAVTEST scenes), learned routing reaches 90.63 PDMS, outperforming random routing (89.76) and uncertainty-based routing (89.52). We did not evaluate Oracle-K. The detailed results are provided as follows.

We evaluate a stratified subset of 2,048 NAVTEST scenes, with 1,024 scenes predicted as simple and 1,024 predicted as complex by the model. Random-K, Uncertainty-K, and learned routing each assign exactly 1,024 scenes to Slow, using the same Fast/Slow predictions.

PolicySlow calls (%)PDMSNCDACTTC
Always Fast088.5998.4097.6796.90
Always Slow10091.1599.1897.9497.37
Random-K5089.7698.8297.7396.91
Uncertainty-K5089.5298.7397.6596.89
Learned routing5090.6398.9897.8897.24

Random-K is a single random assignment. Uncertainty-K routing ranks scenes by the normalized predictive entropy of the Fast answer. Learned routing improves PDMS by 0.87 over Random-K and by 1.11 over Uncertainty-K. The expected PDMS for uniform random assignment, calculated from the reported Fast/Slow means, is 89.87, giving a 0.76-point advantage for learned routing.

Response 5 - Single-scene latency and latency-aware closed-loop outcomes (W5 & Q4)

W5. Inference latency substantially limits practical deployment. Complete-output latency is 2.87 seconds for Fast, 7.15 seconds for Slow, and 5.30 seconds for Adaptive on an H800 at batch size eight. Single-scene response times and delay-aware closed-loop safety have not been established.

Q4. Can authors report single-scene end-to-end latency and latency-aware closed-loop driving outcomes?

Single-scene latency (one H200, batch size 1, 200 scenes) is 3.15 s for Fast and 8.14 s for Slow on average (P95: 3.425 s and 8.832 s). We agree that this is far from real-time deployment. Our Bench2Drive results use the standard synchronous protocol, in which the simulator waits for each plan, so they do not measure the effect of delay.

We add per-scene planning measurements on 200 scenes in BF16, with one H200 per request and scene batch size 1. Eight independent workers each use one GPU; Slow batches its three candidates within the same scene and then performs model selection. Timing includes image loading, preprocessing, and complete structured-output generation, including selection, but excludes model loading, PDMS evaluation, and the controller. Table 15 reports complete-output latency on one H800 at batch size 8.

ModeScenesMean (s)P50 (s)P95 (s)
Fast2003.1533.1673.425
Slow2008.1378.1518.832

For context, AutoVLA's Table 2 reports mean runtimes of 1.072 s for trajectory-only Fast thinking and 10.518 s for CoT-based Slow thinking. In VarioDrive, Fast already generates structured reasoning and Slow compares three preference-conditioned plans. These published values provide context. They are not measurements under a shared hardware and timing protocol.

Our Bench2Drive results follow the standard synchronous loop, which waits for the agent before advancing simulation time. They therefore evaluate planning under that protocol. The new measurements resolve the missing single-scene timing information. Evaluating the effect of execution delay on closed-loop behavior requires a separate delay-aware execution protocol.

Reviewer_6mm3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents VarioDrive, an autoregressive driving model combining waypoint tokenization with geometric supervision, separate RL rewards for each reasoning stage, and adaptive inference. It generates one plan for simple scenes and multiple candidates prioritizing safety, efficiency, or comfort for complex scenes, then uses the model to select the final plan. Experiments cover NAVSIM, nuScenes, and Bench2Drive.

Critical strengths
  1. The representation and reward designs have clear ablation support. Geometric soft supervision improves PDMS from 71.93 to 80.96 under the same waypoint training setup, while separate rewards for intermediate stages add 2.09 PDMS over RL using only the final answer reward.

  2. The experiments cover complementary evaluation settings. Results on NAVSIM, nuScenes, and Bench2Drive assess planning quality, maneuver decisions, and interactive driving, providing broader evidence of effectiveness.

Critical weaknesses
  1. The method differs only modestly from prior driving models. Adaptive switching between fast and slow inference and rewards for reasoning have already been explored. The main distinction is generating and selecting candidates with different driving preferences, but the benefit of learned routing over random routing under the same computational budget remains unclear.

  2. The thinking reward does not directly verify reasoning validity. It measures whether generated reasoning increases the model’s likelihood of its own trajectory, potentially rewarding plan restatement rather than grounded justification. Prefix interventions establish context dependence, but do not distinguish these possibilities.

Questions for authors
  1. What are the key differences from prior fast and slow driving systems such as AutoVLA, beyond generating candidates with different driving preferences? Does learned routing outperform random routing at the same proportion of Slow decisions and computational budget?

  2. Can the thinking reward distinguish grounded reasoning from a plan restatement or a plausible explanation containing incorrect scene facts?

  3. Please clarify the Bench2Drive adaptation, controller, replanning frequency, and treatment of inference delays, given the reported latency of 2.87 to 7.15 seconds for complete outputs at batch size eight.

I would consider raising my score if the authors provide convincing responses to these concerns.

5 comments

We thank the reviewer for the insight comments, and for recognizing our ablation support and the coverage of complementary benchmarks. We added a routing comparison with the same number of Slow calls, reward stress tests, and single-scene latency measurements, and will include them in the revised manuscript. Specifically,

  • Difference from AutoVLA (W1/Q1). VarioDrive differs in its trajectory representation (time-indexed waypoint tokens with geometric soft supervision, PDMS increases from 71.93 to 80.96), its training feedback (separate rewards for description and reasoning, +2.09 PDMS over answer-only RL), and its use of extra computation (Slow compares preference-conditioned plans, oracle PDMS 93.26 vs. 91.67 for the best single-preference pool).
  • Learned vs. random routing (W1/Q1). With exactly 50% of 2,048 NAVTEST scenes sent to Slow, learned routing reaches 90.63 PDMS, compared with 89.87 for random routing in expectation and 89.52 for uncertainty-based routing.
  • Plan restatement (W2/Q2). Copying the action statement into the thinking once lowers the mean reward from 0.84 to 0.23 with no increase over the length-matched control in any tested pair. Replacing one decision-relevant judgment lowers the regenerated PDMS from 91.81 to 35.89, and the reward gives the original thinking a paired preference rate of 99.33%, counting ties as half.
  • Bench2Drive and latency (Q3). The simulator runs synchronously and waits for each plan, so latency does not delay actuation in our results. Single-scene latency is 3.15 s for Fast and 8.14 s for Slow.

Response 1 - Difference from adaptive CoT and the value of learned routing (W1 & Q1)

W1. The method differs only modestly from prior driving models. Adaptive switching between fast and slow inference and rewards for reasoning have already been explored. The main distinction is generating and selecting candidates with different driving preferences, but the benefit of learned routing over random routing under the same computational budget remains unclear.

Q1. What are the key differences from prior fast and slow driving systems such as AutoVLA, beyond generating candidates with different driving preferences? Does learned routing outperform random routing at the same proportion of Slow decisions and computational budget?

VarioDrive differs from AutoVLA in the following three aspects: (1) Trajectory representation: time-indexed position and heading tokens with geometric soft supervision. In Table 5, adding geometric soft supervision with the token representation fixed raises PDMS from 71.93 to 80.96.

(2) Training feedback: separate rewards for description, thinking, and trajectory quality. With Fast inference fixed, full stage-reward RFT raises PDMS from 88.75 to 90.84 over answer-only RFT (Table 10).

(3) Use of extra computation: AutoVLA adds CoT on top of a trajectory-only Fast mode, whereas both of our modes reason, and Slow compares plans under different driving preferences (oracle PDMS 93.26 vs. 91.67 for the best single-preference pool at the same candidate count in Appendix C.1). Under the same 50% Slow budget, learned routing also outperforms random and uncertainty-based routing by 0.87 and 1.11 PDMS.

The experiments details are provided as follows:

We additionally evaluate routing on 2,048 NAVTEST scenes, with 1,024 scenes predicted as simple and 1,024 predicted as complex by the model. All three routing policies assign exactly 1,024 scenes to Slow. Fast produces one greedy candidate; Slow generates three preference-conditioned candidates and uses the same model for selection without ground-truth PDMS.

PolicySlow calls (%)PDMS
Always Fast088.59
Always Slow10091.15
Random-K5089.76
Uncertainty-K5089.52
Learned routing5090.63

Random-K uses a single random assignment; averaging over all assignments gives an expected PDMS of (88.59+91.15)/2=89.87(88.59+91.15)/2=89.87. Uncertainty-K sends the 1,024 scenes with the highest normalized predictive entropy of the Fast answer to Slow. Learned routing outperforms Random-K by 0.87 PDMS, its expectation by 0.76, and Uncertainty-K by 1.11.

Response 2 - Plan restatement and incorrect but predictive rationales (W2 & Q2)

W2. The thinking reward does not directly verify reasoning validity. It measures whether generated reasoning increases the model’s likelihood of its own trajectory, potentially rewarding plan restatement rather than grounded justification. Prefix interventions establish context dependence, but do not distinguish these possibilities.

Q2. Can the thinking reward distinguish grounded reasoning from a plan restatement or a plausible explanation containing incorrect scene facts?

The thinking reward does not reward plan restatement above the length-matched control in this test: copying the natural-language action statement into the thinking, once or three times, lowers the mean reward from 0.84 to 0.23 and 0.15, and never yields a higher reward than the length-matched control. Replacing one decision-relevant judgment lowers the PDMS of the regenerated trajectories from 91.81 to 35.89, and the reward gives the original thinking a paired preference rate of 99.33%, counting ties as half.

The experiments details are provided as follows:

The intent reward measures predictive support for trajectory generation. We compare thinking outputs that repeat the natural-language action statement once or three times with length-matched, unmodified model outputs using the discrete thinking reward.

Stress conditionMean rewardLength-matched control meanReward-increase rate (%)Full-reward rate (%)
Copy action statement once0.23330.84000.007.33
Copy action statement three times0.14670.83670.004.00

Reward-increase rate is the fraction of pairs with Rstress>RcontrolR_{\mathrm{stress}}>R_{\mathrm{control}}; full reward means Rthk=1R_{\mathrm{thk}}=1. The copied content is natural-language action text, not trajectory tokens. Neither stress condition raises the reward over its length-matched control, although some cases still receive full reward.

We also compare the paper's discrete thinking reward with entropy reduction and a reasoning–action consistency baseline on paired original and perturbed rationales:

Scoring methodPaired preference (%)Ties (%)
Our discrete thinking reward99.331.33
Valid-token entropy reduction95.335.60
Reasoning–action consistency83.6712.93

Paired preference is (nwin+0.5ntie)/750(n_{\mathrm{win}}+0.5n_{\mathrm{tie}})/750, with ties also reported separately. Length matching preserves the qualitative result. Because the intervention changes the action conclusion as well as its justification, this measures decision-semantic sensitivity rather than factual correctness with the action held fixed.

Separately, replacing one key judgment, including its justification and action conclusion, gives a 99.33% paired preference for the original thinking across 750 scenes, with half credit for ties. The result remains qualitatively consistent after length matching. This supports sensitivity to decision content. It does not establish detection of incorrect premises that preserve the same action. Description, answer, and format rewards provide complementary feedback on visual grounding, trajectory quality, and response structure.

The training ablation further supports the reward's utility: adding thinking feedback raises meta-action accuracy from 80.95% to 85.92% with answer and format rewards fixed, and from 82.20% to 87.14% when description feedback is also present (Table 6).

Response 3 - Bench2Drive adaptation, controller, replanning, and inference delays (Q3)

Q3. Please clarify the Bench2Drive adaptation, controller, replanning frequency, and treatment of inference delays, given the reported latency of 2.87 to 7.15 seconds for complete outputs at batch size eight.

For Bench2Drive, we convert each predicted 4 s trajectory into control with the Bench2DriveZoo PID controller. The simulator runs synchronously at 20 Hz and waits for the agent at every tick, so the agent replans at every tick, and inference latency does not translate into delayed actuation in the reported results. Measured separately, single-scene latency is 3.15 s for Fast and 8.14 s for Slow. The detailed implementations are provided as follows.

Our closed-loop evaluation agent receives four camera streams, ego-state measurements, and navigation information, and converts the predicted trajectory into steering, throttle, and brake using the Bench2DriveZoo PID controller. The Bench2Drive evaluator uses synchronous simulation at 20 Hz. The planner produces a fresh trajectory at each simulation tick. Appendix A.11 describes the policy setup with four frames at 2 Hz from each of the front, front-left, and front-right cameras.

In the standard Bench2Drive loop, the simulator waits for the agent to return its control action before advancing to the next tick. There is no intervening simulated motion under an old plan while inference runs. The 20 Hz rate therefore refers to simulation time, not wall-clock planning throughput. Inference latency is not automatically converted into delayed actuation.

We also add single-scene measurements on 200 scenes in BF16, with one H200 per request and scene batch size 1. Eight independent workers each use one GPU. Slow batches three candidates from the same scene before selection. Mean latency is 3.153 s for Fast and 8.137 s for Slow. P95 is 3.425 s and 8.832 s. Timing includes image loading, preprocessing, and complete generation including Slow selection, and excludes model loading, PDMS evaluation, and the controller. These are additional per-scene measurements, separate from the original H800 batch-size-8 timings.

Reviewer_9FYwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VarioDrive, a post-training framework that generates a scene description, driving rationale, and trajectory within a single autoregressive vision-language model. It represents each future pose with time-indexed position and heading tokens, adds geometric soft supervision, and uses separate rewards for scene descriptions, reasoning-plan alignment, and preference-dependent trajectory quality. An adaptive inference mechanism generates one plan in Fast mode or compares safety-, efficiency-, and comfort-conditioned candidates in Slow mode. Experiments report 92.42 NAVSIM PDMS with Slow selection, 92.13 with adaptive routing, and a closed-loop Bench2Drive Driving Score of 79.70, alongside ablations of trajectory representation, intermediate rewards, and candidate selection.

Critical strengths
  1. The training ablations provide controlled evidence for both geometric supervision and intermediate rewards. With the time-indexed representation fixed, adding the soft geometric loss raises PDMS from 71.93 to 80.96; with the RFT setup fixed, adding description and thinking rewards improves PDMS from 88.75 to 90.84 over answer-only RFT (Tables 5 and 10). These comparisons isolate meaningful parts of the proposed method and show that its gains are not explained solely by changing the trajectory representation or adding reinforcement learning.

  2. The candidate experiments distinguish the value of preference diversity from simply sampling more trajectories. With three candidates in each pool, mixed-preference generation reaches an oracle PDMS of 93.26 compared with 91.67 for the strongest same-preference pool, while the learned selector reaches 92.42 compared with 88.62 for random selection on the same mixed pool (Table 13). The reported differences in progress, time to collision, and trajectory dynamics further support that the preferences produce distinct behaviors. Together, these results substantiate both the candidate-generation and selection components of Slow inference.

Critical weaknesses
  1. The reasoning reward measures whether a generated explanation makes the model's own trajectory tokens more likely, which does not establish whether that explanation is correct or whether its stated decision logic determines the trajectory. The prefix interventions are useful evidence of context dependence, but they remove or replace the description and thinking together, changing scene information as well as reasoning. Likewise, changing navigation commands directly changes an input to the planner. The paper appropriately acknowledges these limitations, but the experiments still leave unresolved whether the reasoning reward encourages meaningful decision logic or primarily rewards text that predicts and reinforces the chosen action.

  2. The paper miss specific discussions with other post-training methods, like CRAFT (https://arxiv.org/abs/2605.04470) or Alpamayo2 https://github.com/NVlabs/alpamayo2. A discussion with those methods are appropriate to establish the paper's novelty and difference with prior literature.

Questions for authors
  1. Does the perplexity-based thinking reward distinguish a valid driving explanation from an incorrect rationale that makes the same action easy to predict? Could you keep the scene description, observations, navigation, and prefix format fixed while editing one decision-relevant claim in the thinking segment, then measure both the reward and the regenerated trajectory? It's like an steering/probing experiment to see the reasoning-action alignment.
  2. The authors should try to address Weakness 2. Besides, the authors should provide more visualizations on closed-loop evaluations to showcase the model's behavior (like if it exhibits recovery from failure abilities, etc.)
4 comments

We thank the reviewer for the constructive feedbacks, and for recognizing our controlled ablations and the value of preference diversity in Slow inference.

  • Thinking intervention (W1/Q1). Following the suggested protocol, we fix the observations, navigation, scene description, and prefix format, and replace only one decision-relevant judgment in the thinking (750 NAVTEST scenes). The regenerated trajectories drop from 91.81 to 35.89 PDMS, and the thinking reward gives the original thinking a paired preference rate of 99.33%, counting ties as half. Copying the action statement into the thinking did not increase the reward in any tested pair.
  • CRAFT and Alpamayo2 (W2/Q2). Both post-train driving models with richer feedback, but they decompose different things: CRAFT decomposes closed-loop feedback, Alpamayo2 separates the reasoner from a diffusion action expert, and VarioDrive rewards each output stage of a single autoregressive decoder. We will add this discussion to Section 2.
  • Planning visualizations (Q2). We add Figures 6 and 7 in Appendix E to show NAVSIM planning decisions and failures. Closed-loop recovery remains untested.

Response 1 - Reasoning validity and controlled interventions (W1 & Q1)

W1&Q1. Does the perplexity-based thinking reward distinguish a valid driving explanation from an incorrect rationale that makes the same action easy to predict? Could you keep the scene description, observations, navigation, and prefix format fixed while editing one decision-relevant claim in the thinking segment, then measure both the reward and the regenerated trajectory? It's like an steering/probing experiment to see the reasoning-action alignment.

We have conducted the requested intervention experiments. With the observations, navigation, description, and prefix format fixed, replacing only one decision-relevant judgment in the thinking lowers the PDMS of the regenerated trajectories from 91.81 to 35.89 (750 NAVTEST scenes), and the thinking reward gives the original thinking a paired preference rate of 99.33%, counting ties as half. Copying the action statement into the thinking did not increase the reward in any tested pair. Both the reward and the planner therefore respond to the decision content of the thinking. Since the replaced judgment changes the action conclusion together with its justification, this test does not by itself show that the reward penalizes an incorrect justification for the same action. The detailed responses are provided as follows.

Our intent reward measures whether a rationale increases the likelihood of the same generated trajectory. We refer to this measure as the thinking reward (Table 6). The description reward separately evaluates visual grounding, while the answer reward evaluates trajectory quality.

We add an inference-only diagnostic on 750 randomly sampled NAVTEST scenes using safety-first Fast inference, without sample filtering. We replace only a decision-relevant judgment within thinking with one from another scene, retaining the observations, navigation, description, and prefix structure. The replaced judgment includes both its justification and action conclusion. The original trajectories score 91.82 PDMS on this subset, compared with 90.84 on full NAVTEST (Table 4); regeneration with the original thinking gives 91.81, versus 35.89 after this intervention, a 55.92-point decrease. This demonstrates sensitivity to decision-relevant reasoning content.

We also compare the paper's discrete thinking reward with entropy reduction and a reasoning–action consistency baseline on paired original and perturbed rationales:

Scoring methodPaired preference (%)Ties (%)
Our discrete thinking reward99.331.33
Valid-token entropy reduction95.335.60
Reasoning–action consistency83.6712.93

Paired preference is (nwin+0.5ntie)/750(n_{\mathrm{win}}+0.5n_{\mathrm{tie}})/750, with ties also reported separately. Length matching preserves the qualitative result. Because the intervention changes the action conclusion as well as its justification, this measures decision-semantic sensitivity rather than factual correctness with the action held fixed.

We separately test copying the natural-language action statement once or three times. Mean discrete rewards are 0.2333 and 0.1467, versus 0.8400 and 0.8367 for length-matched, unmodified model outputs. Neither condition increases the reward on any evaluated pair. These results address the tested action-copying strategy without treating predictive alignment as a standalone factual verifier.

The training ablation provides complementary evidence: with answer and format rewards fixed, adding the thinking reward improves nuScenes meta-action accuracy from 80.95% to 85.92%; with the description reward also enabled, the corresponding improvement is 82.20% to 87.14% (Table 6).

Response 2 - Discussions with other post-training methods (CRAFT, Alpamayo2) and visualizations (W2 & Q2)

W2. The paper miss specific discussions with other post-training methods, like CRAFT (https://arxiv.org/abs/2605.04470) or Alpamayo2 https://github.com/NVlabs/alpamayo2. A discussion with those methods are appropriate to establish the paper's novelty and difference with prior literature.

Q2. The authors should try to address Weakness 2. Besides, the authors should provide more visualizations on closed-loop evaluations to showcase the model's behavior (like if it exhibits recovery from failure abilities, etc.)

We will add a discussion of CRAFT and Alpamayo2 to Section 2. All three methods post-train driving models with richer feedback than a single trajectory reward, but they differ in what is decomposed: CRAFT decomposes closed-loop feedback into counterfactual proxy feedback and residual correction from closed-loop interactions, Alpamayo2 couples a VLM reasoner with a separate diffusion action expert, whereas VarioDrive generates reasoning and waypoint tokens with one decoder and rewards each output stage separately.

1) Specific discussions with other post-training methods:

The released Alpamayo2 generates Chain-of-Causation text through a VLM and trajectories through a diffusion action expert. VarioDrive generates reasoning and waypoint tokens with a shared autoregressive decoder. Its Fast mode already produces a complete description–reasoning–trajectory response; Slow generates one candidate per driving preference and compares them using the same model. Preference-dependent rewards support these behavioral alternatives during training. Our contribution concerns this structured alignment and preference-conditioned exploration, rather than the general use of reasoning, multiple rewards, or multiple trajectory samples.

The existing candidate-pool ablation supports this distinction: with three candidates and the same oracle selection rule, mixed preferences achieve 93.26 PDMS versus 91.67 for the strongest single-preference pool. On the mixed pool, model selection achieves 92.42 versus 88.62 for random selection (Appendix C.1).

2) Planning visualizations: We add NAVSIM planning visualizations of preference-conditioned decisions and turning failures. Behind the same slow-moving vehicle, the efficiency, safety, and comfort candidates respectively change lanes, stop, and follow at 1.3 m/s (Appendix E, Fig. 6(b)). In the two turning examples, the predicted vehicle footprint first crosses the road boundary at 2.4 s and 1.8 s (Fig. 7).

Thank you for the response, which has addressed most of my concerns. Would love to see those changes in the camera-ready version. I will raise my score.

Dear Reviewer 9FYw,

Thank you for raising your score and for your constructive suggestions throughout the discussion. We will include all of these changes in the camera-ready version.

Your question prompted a controlled test of whether the planner relies on its reasoning. With the observations, navigation, and scene description held fixed, replacing one judgment lowered the PDMS of the regenerated plans from 91.81 to 35.89, and the thinking reward preferred the original thinking in 99.33% of the pairs. Repeating the action statement never increased the reward over a length-matched control. We will also state explicitly that this intervention changes the justification together with the action.

We will add to Section 2 how VarioDrive's stage-specific rewards differ from CRAFT's closed-loop feedback, and how its shared decoder differs from Alpamayo2's separate action expert, together with the visualizations in Appendix E.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We sincerely thank the area chair and all reviewers for their thoughtful evaluations and constructive feedback, and Reviewer 9FYw for confirming that most concerns were addressed. During the rebuttal period, we addressed the following concerns in our responses to each reviewer:

  • The impact of reasoning on the plan: On 750 randomly sampled NAVTEST scenes, we held all driving inputs and the scene description fixed, and replaced a single decision-relevant judgment in the thinking. To exclude drift introduced by re-generation, we re-generated each scene twice, prefixing the correct and the corrupted thinking respectively, so that both plans come from the same decoder and prompt format. PDMS of the regenerated plans fell from 91.81 to 35.89, and the thinking reward preferred the original thinking in 99.33% of the pairs. Repeating the action statement never increased the reward over a length-matched control.

  • The grounded reasoning: The description reward checks scene facts against the camera images with a VLM judge, and adding it raises left/right-turn F1 on nuScenes from 60.59/69.68 to 91.58/93.14 (Table 6). The thinking reward checks whether the reasoning supports the generated plan. It is not a factual verifier by itself, and we will state this scope explicitly in the revision.

  • The role of slow inference: Fast produces one reasoned plan. Slow compares plans guided by safety, efficiency, and comfort. With 50% of 2,048 NAVTEST scenes assigned to Slow, learned routing reached 90.63 PDMS, compared with 89.87 for random routing in expectation and 89.52 for uncertainty-based routing.

  • Driving evaluations: Single-scene inference averaged 3.15 s for Fast and 8.14 s for Slow on one H200. Bench2Drive waits for each plan before advancing simulation time, so the reported scores isolate planning from execution delay. Appendix E, Figures 6 and 7, illustrate preference-dependent decisions and turning failures on NAVSIM.

All new results and the discussion of CRAFT and Alpamayo2 will be included in the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.