acceptodds
Under review as a conference paper at ICLR 2027

AUGUstus: Autonomous Driving VLA Model with Unified Generation and Understanding

Abstract

Vision-language-action (VLA) models have shown strong potential for end-to-end autonomous driving by leveraging world knowledge and understanding to achieve great performance in long-tail scenarios. However, current VLAs suffer from two major limitations: 1) Degeneration into vision-action (VA) models. To achieve low latency, regression-based and flow-based methods directly generate actions, yet they do not sufficiently leverage the VLM's understanding, leading to unsafe and unexplainable actions. 2) Autoregressive high latency. Autoregressive methods sequentially generate chain-of-thought (CoT) and action tokens, fully leveraging the VLM's understanding, but high latency makes deployment difficult. To enable low-latency deployment while properly using CoT to assist action generation, we propose , an utonomous-driving VLA model with nified eneration and nderstanding. Specifically, we first unify diffusion-based action generation and autoregressive CoT understanding in a single model, where understanding is optimized only to serve generation. More concretely, joint CoT supervision preserves the VLM's spatial understanding in the shared representation and keeps it useful for safer and more accurate trajectories. Second, we propose AUGUstus-mode, which orders the action generation before the CoT so that the CoT supervises the reliable action during training while inference needs only the action output, enabling low latency. Finally, we introduce , a unified GRPO algorithm that jointly optimizes the action and CoT, further improving action performance. Extensive experiments demonstrate that achieves 94.6 PDMS on NAVSIMv1 and 90.7 EPDMS on NAVSIMv2, establishing state-of-the-art (SOTA) planning performance without scorer module. On nuScenes, our method achieves an average L2 error of 0.27 m and an average collision rate of 0.05, both the best among the compared methods. To validate the practicality of , we train it on our in-house dataset using 512 NVIDIA H800 GPUs for approximately two days, then evaluate it on an internal 1,000 km dataset and preliminarily deploy it on a real vehicle, achieving 12 Hz operation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, VLA, Unify Model, GRPO

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qZmROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AUGUstus, a vision-language-action model that unifies trajectory generation and chain-of-thought (CoT) understanding for autonomous driving.

The main idea is to use CoT supervision to improve action generation without requiring CoT decoding during inference. The authors introduce AUGUstus-mode, which places action tokens before CoT tokens in a shared transformer. They also propose AUGUstus-GRPO, which jointly optimizes continuous actions and CoT through reinforcement learning.

Experiments on NAVSIMv1, NAVSIMv2, and nuScenes show strong planning performance. The paper also includes ablation studies, internal driving evaluations, and preliminary real-vehicle deployment at 12 Hz.

Critical strengths
  1. A meaningful architecture that balances planning performance and inference efficiency. The paper addresses an important trade-off between using VLM understanding and maintaining low inference latency. The action-first design allows CoT supervision during training without requiring CoT generation during deployment. Table 6 shows that AUGUstus-mode achieves the same PDMS (90.6) as the CoT-first design while reducing inference time from approximately 4.6 seconds to 73.6 ms. This provides direct evidence for the practical value of the proposed ordering.

  2. Strong experimental results with useful ablation studies. The method achieves competitive or state-of-the-art results across multiple public benchmarks. The authors also examine CoT content, generation order, and the effects of the two-stage GRPO training. The comparisons with the no-CoT model and the JaT-GRPO-only variant provide evidence that both CoT supervision and joint optimization contribute to performance. Multi-seed evaluations further support the stability of the reported results.

Critical weaknesses
  1. The specific contribution of CoT understanding is not fully isolated. The paper shows that adding CoT supervision improves planning performance. However, the selected CoT mainly contains object bounding boxes and future-position labels. Since action tokens cannot attend to CoT tokens, the improvement comes through shared representations and training gradients. It remains unclear whether the benefit comes from CoT-based understanding or simply from additional scene prediction supervision. The existing no-CoT ablation establishes the value of additional supervision, but does not fully distinguish these explanations.

  2. The benefit of the proposed unified GRPO formulation needs clearer validation. Table 7 shows that joint GRPO outperforms the variant without CoT-GRPO. This supports the benefit of optimizing both branches. However, the paper does not compare shared advantages with separate advantages for action and CoT. It also lacks a direct empirical comparison with alternative flow-based RL methods under the same training setup. These comparisons would help clarify which parts of AUGUstus-GRPO are responsible for the improvements.

  3. The real-world evaluation provides limited quantitative evidence. The internal 1,000 km evaluation reports relative performance differences, but does not provide the underlying absolute metrics. The real-vehicle results mainly demonstrate inference speed and qualitative driving behavior. These results are encouraging, but they provide limited evidence about the reliability and safety of the complete system during extended real-world operation.

Questions for authors
  1. Could the authors compare CoT supervision with a non-autoregressive auxiliary prediction head trained on the same bounding-box and future-position labels? This would help isolate the benefit of the proposed CoT design.

  2. How does the shared-advantage GRPO objective compare with optimizing action and CoT using separate advantages under the same rewards and training budget?

  3. Could the authors provide a direct comparison between JaT-GRPO and existing flow-based RL methods using the same backbone and training setup?

10 comments

Thank you for your valuable and insightful comments. We deeply appreciate your input and have made every effort to address your concerns, as detailed below.

Response to W1 & Q1: Isolating the Contribution of CoT Supervision (1/2)

(1) The mechanism. In AUGUstus-mode the action tokens never attend to the CoT tokens, so the CoT acts on the action only through the shared representation and the training gradients: the CoT tokens are placed after and attend to the action tokens, so the CoT cross-entropy back-propagates directly into the action hidden states (Appendix D.2.1). CoT in AUGUstus is therefore structured supervision expressed in language, the wording the revised manuscript now uses.

(2) The manuscript already contains a same-labels, different-encoding control. Short-prediction (Table 9 of the original manuscript) encodes the same boxes and future positions as the prediction CoT in an atomic vocabulary of 2,573 new tokens (≈14 instead of ≈100 tokens per scene) and reaches 89.1, below the no-CoT baseline of 89.2, vs. 90.6 for the prediction CoT (Figure 1). The gain is therefore not "more scene-prediction supervision": it requires the supervision to be expressed in the vision-language space that the VLM is pre-trained on. The JSON bbox_2d format of the prediction CoT is the grounding format of the Qwen-VL pre-training, so it activates the pre-trained spatial understanding instead of training a new output space from scratch.

Figure 1. Three encodings of the same per-scene annotation (risk-object boxes + future-position bins)

                       same labels per scene: {box_k, future-position bin_k}  k = 1..K
                                                  │
              ┌───────────────────────────────────┼───────────────────────────────────┐
              ▼                                   ▼                                   ▼
   Prediction CoT (ours)              Short-prediction (Table 9)          Auxiliary head (new, Table 1)
   JSON text in the VLM's             atomic vocabulary, 2,573 new        set-prediction head on the
   pre-trained grounding format       tokens, ≈14 tokens per scene        context + action states,
   {"bbox_2d":[274,418,378,661],      <obj_3><box_17><fp_near_front>      no text, no LM head:
    "future_position":"middle,left"}  new embeddings trained from         L1 + GIoU on boxes,
   ≈100 tokens per scene              scratch                              cross-entropy on bins
              │                                   │                                   │
         PDMS 90.6                           PDMS 89.1                           PDMS 89.0
    (no-CoT baseline: 89.2)             (below no-CoT: −0.1)                (below no-CoT: −0.2)

Response to W1 & Q1: Isolating the Contribution of CoT Supervision (2/2)

(3) The requested control: a non-autoregressive auxiliary head on the same labels. A lightweight set-prediction head (2-layer cross-attention decoder with NN object queries, Hungarian matching, L1 + GIoU for boxes, cross-entropy for the future-position bins) on the context and action hidden states is trained on exactly the same objects, boxes and bins as the prediction CoT, with the language loss off and its loss weight tuned on a held-out navtrain split (Table 1, with the rows of Table 9 of the original manuscript). It reaches 89.0 PDMS, below the no-CoT baseline (89.2) and at the level of Short-prediction (89.1), vs. 90.6 for the prediction CoT: like Short-prediction, a new output space detached from the VLM's language representation disturbs the pre-trained latent space. The new shuffled and corrupted CoT targets keep the JSON format but fall well below no-CoT (87.8 and 88.1), so CoT content that does not match the scene hurts the action. The benefit of the proposed CoT design therefore comes from correct scene content expressed in the VLM's native vision-language space, not from extra labels or extra token prediction.

Table 1. CoT supervision vs. label-matched alternatives (NAVSIMv1 navtest, SFT)

Supervision targetLabelsEncodingPDMS↑NC↑DAC↑EP↑TTC↑Comf.↑
No CoT (Table 9)none–89.298.896.883.595.8100.0
Short-prediction (Table 9)boxes + future positionatomic vocabulary89.198.796.783.195.4100.0
Non-autoregressive auxiliary head (new)boxes + future positionset-prediction head, no text89.098.996.383.195.7100.0
Shuffled CoT (new)CoT of another scene, same formatJSON text87.898.795.681.495.7100.0
Corrupted CoT (new)correct boxes, random future positionsJSON text88.198.595.882.695.5100.0
Prediction CoT (ours, Table 9)boxes + future positionJSON text90.699.097.984.296.4100.0

The language form also keeps the CoT a token sequence on the existing LM head, so it can be optimized with token-level GRPO together with the action in RL (Table 7 of the original manuscript: 92.6 without CoT-GRPO vs. 94.6 with it).

Response to W2 & Q2: Shared vs. Separate Advantages (1/2)

(1) Clarification of the advantage. We apologize for the misunderstanding regarding Sec. 3.3. In AUGUstus-GRPO the action and CoT branches share the paired rollouts, the transformer and one joint update, and their group-relative advantages are computed per branch: the action advantage from the planning reward, the CoT advantage from the CoT reward. All AUGUstus-GRPO results in the manuscript use these per-branch advantages (repository settings in (6): rl_split_advantage: true, rl_reward_norm: false). We have revised Sec. 2, Sec. 3.3 and Appendix D, Eqs. (4), (6) and (19)–(21), Fig. 3 and Algorithms 3, 4 and 7 to write the two advantages out explicitly.

(2) What AUGUstus-GRPO computes. For the GG paired rollouts of a context, the two rewards are normalized separately within the group:

Aiact=Rplan,i−mean⁡jRplan,jstd⁡jRplan,j+ϵ,Aicot=Rpred,i−mean⁡jRpred,jstd⁡jRpred,j+ϵ.A_i^{\text{act}} = \frac{R_{\text{plan},i}-\operatorname{mean}_j R_{\text{plan},j}}{\operatorname{std}_j R_{\text{plan},j}+\epsilon},\qquad A_i^{\text{cot}} = \frac{R_{\text{pred},i}-\operatorname{mean}_j R_{\text{pred},j}}{\operatorname{std}_j R_{\text{pred},j}+\epsilon}.

Each advantage enters its own clipped GRPO objective (Eq. 6), and the two objectives form one loss with a single backward pass:

LAUGUstus-GRPO=−Jaction(Aact)+βaction,KLact+αCoT(−JCoT(Acot)+βCoT,KLCoT).\mathcal{L}_{\text{AUGUstus-GRPO}} = -J_{\text{action}}\big(A^{\text{act}}\big) + \beta_{\text{action}}\\,\text{KL}_{\text{act}} + \alpha_{\text{CoT}}\Big(-J_{\text{CoT}}\big(A^{\text{cot}}\big) + \beta_{\text{CoT}}\\,\text{KL}_{\text{CoT}}\Big).

The advantages are separate, but the gradients are not: both terms back-propagate through the same transformer, and since the CoT tokens attend to the action tokens, the CoT-GRPO gradient reaches the action hidden states (Figure 2).

Figure 2. AUGUstus-GRPO: per-branch advantages, one joint backward pass

                         G paired rollouts of one context (shared initial noise x_1)
                ┌──────────────────────────────────┴──────────────────────────────────┐
         action x_i  (SDE rollout)                                CoT y_i  (sampled after the action)
                │                                                                     │
         R_plan(x_i)                                                          R_pred(y_i)
         Stage I: PDMS;  Stage II: endpoint + safety                          box IoU × future-position agreement
                │  normalize within the group                                         │  normalize within the group
                ▼                                                                     ▼
            A_i^act                                                               A_i^cot
                │                                                                     │
         L_act = −J_action(A^act) + β_action·KL_act                     L_cot = −J_CoT(A^cot) + β_CoT·KL_CoT
                └──────────────────────────────────┬──────────────────────────────────┘
                                                   ▼
                                L = L_act + α_CoT · L_cot   ──▶   one backward pass
                                                   │
                ┌──────────────────────────────────▼──────────────────────────────────┐
                │ one transformer:  │ V V L L │ A A A A A A A A │ L L L L L L │       │
                │                     context    action block      CoT block          │
                │ CoT tokens attend to action tokens ──▶ ∂L_cot reaches the action    │
                │ hidden states and the shared trunk; actions never read the CoT      │
                └─────────────────────────────────────────────────────────────────────┘

(3) The requested comparison. We compare our per-branch advantages with a single shared advantage Ai=z⁡(Rpred,i+5,Rplan,i)A_i = \operatorname{z}\big(R_{\text{pred},i} + 5\\,R_{\text{plan},i}\big) that drives both branches. Everything else (SFT initialization, rewards, G=8G=8, rollout steps, learning rate, number of updates, KL coefficients) is identical.

Table 2. Single shared advantage vs. per-branch advantages (NAVSIMv1, same rewards and training budget; Stage I + II)

AdvantagePDMS↑NC↑DAC↑EP↑TTC↑Comf.↑
Single shared, A=z⁡(Rpred+5Rplan)A=\operatorname{z}(R_{\text{pred}}+5R_{\text{plan}}) for both branches94.298.798.391.597.0100.0
Per-branch, AactA^{\text{act}} and AcotA^{\text{cot}} (ours)94.698.998.991.997.1100.0
Reviewer_emmCOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AUGUstus, a unified VLA framework for autonomous driving that unifies understanding and generation. The authors first integrate driving understanding and action generation into a single Transformer and propose an action-first, CoT-second training scheme that introduces CoT supervision without incurring additional inference latency. Finally, they propose AUGUstus-GRPO, which jointly optimizes CoT and actions using GRPO. The method demonstrates its effectiveness on NAVSIM, nuScenes, and a large-scale in-house dataset, and can be deployed on real vehicles.

Critical strengths
  1. Methodological novelty. While most existing approaches adopt decoupled VLM+Diffusion or VLM+Action Expert architectures, or sequential CoT+Action generation, this paper proposes a unified architecture for continuous action generation and discrete language understanding, together with an action-first training scheme with CoT supervision.

  2. Strong empirical results. Through RL optimization, the proposed method achieves a PDMS of 94.6 on NAVSIM, substantially outperforming other methods. Its effectiveness is further demonstrated on nuScenes and a large-scale in-house dataset.

  3. Joint optimization of CoT and actions. The proposed RL method addresses the misalignment between CoT and actions through joint optimization, leading to substantial improvements in both.

Critical weaknesses
  1. Lack of quantitative analysis of understanding capabilities. The authors claim to unify understanding and generation, but the experiments mainly evaluate the model's continuous action generation performance. There is limited analysis of how well the model learns driving-related understanding capabilities. Given that unified understanding is explicitly emphasized in the title, more quantitative evaluations of these capabilities would strengthen the paper.

  2. Insufficient analysis of the unified architecture. The authors propose integrating understanding and action generation into a single Transformer. While this is an interesting design in the autonomous driving VLA domain, the paper lacks discussion and citations of related works, such as Transfusion[1] and DiffE2E[2]. More importantly, the authors do not sufficiently validate the effectiveness of this unified architecture compared with MoT-based action experts or other architectures. Although Table 4 includes a VLM+Flow baseline, it is unclear whether this represents a strictly comparable MoT-based action expert architecture. If the authors could further demonstrate the benefits of the unified architecture for both understanding and action generation, the motivation and contributions would be more convincing.

  3. Limited ablation studies beyond NAVSIM. The current ablation studies are mainly conducted on NAVSIM, which has a relatively limited dataset scale and increasingly saturated benchmark performance. If the authors could provide additional ablation studies on their in-house dataset, particularly regarding CoT supervision, the unified architecture, and RL training, this would further strengthen the empirical validation of the proposed method. However, I understand the substantial computational cost of conducting such experiments on large-scale in-house data. Therefore, I consider this an optional suggestion rather than a weakness that would affect my rating.

[1] Zhou, Chunting, et al. "Transfusion: Predict the next token and diffuse images with one multi-modal model." International Conference on Learning Representations. Vol. 2025. 2025.

[2] Zhao, Rui, et al. "Diffe2e: Rethinking end-to-end driving with a hybrid action diffusion and supervised policy." arXiv preprint arXiv:2505.19516 (2025).

Questions for authors

Could the authors clarify the specific architectures of the VLM+Flow and VLM+Regression baselines in Table 4? In particular, does VLM+Flow adopt a separate action expert, such as an MoT-based architecture, and are these baselines trained under comparable settings to AUGUstus?

9 comments

Thank you for your valuable and insightful comments. We deeply appreciate your input and have made every effort to address your concerns, as detailed below.

Response to W1: Quantitative Analysis of Understanding Capabilities (1/2)

We apologize for the misunderstanding. We first restate the position of the paper and then provide the requested quantitative analysis.

(1) The scope of "understanding" in this paper. AUGUstus is a VLA for deployment, not an in-cabin VLM. Its goal is the action, but a VLA that discards understanding degenerates into a VA model with capped performance (Table 6 of the original manuscript; see also the external-expert design of ReCogDrive). The core question of the paper is therefore what kind of understanding benefits the action of a VLA, and how to use it. This is why the manuscript states that "understanding is not trained for its own sake but to generate a safer and more accurate trajectory" (Sec. 1) and reports action metrics: we unify understanding into the model in order to act better.

(2) Not every understanding task helps the action. Table 9 of the original manuscript (now in the main text) compares the AutoDrive-P3 CoT types under an identical recipe: Prediction (90.6) > Perception (90.3) ≈ Perception + Prediction (90.3) > Planning (90.1) > Reasoning (89.6); the combined CoT is worse than Prediction alone. The understanding that most driving-VLM benchmarks evaluate (free-form reasoning, planning text, perception QA) suits a cockpit VLM but gives smaller gains or even hurts as action supervision. This finding is one of the paper's contributions, and it is why we do not report those benchmarks: a high score there would not be evidence for the VLA. We therefore keep the prediction CoT as the understanding target.

(3) Quantitative evaluation of the understanding, in two ways.

Accuracy. The decoded CoT is scored by rpred=rdet⋅rfpr_{\text{pred}} = r_{\text{det}}\cdot r_{\text{fp}} (detection quality of the referenced risk objects × correctness of their future-position labels, as defined in AutoDrive-P3 [1]; Table 1). AutoDrive-P3 is the natural reference: the same CoT annotation in a conventional autoregressive VLA that generates the CoT before planning. AUGUstus reaches 0.75 after SFT, above the 0.73 we measure for the released AutoDrive-P3 checkpoint despite a smaller backbone (2B vs. 3B) and no CoT at inference, and 0.85 after AUGUstus-GRPO (Appendix D.5 of the original manuscript). The unified architecture with joint optimization thus improves the understanding (from 0.75 to 0.85) together with the action (from 90.6 to 94.6 PDMS), and understands the scene better than a model that reasons before acting.

Table 1. Accuracy of the decoded prediction CoT (rpredr_{\text{pred}}, same definition as AutoDrive-P3; AutoDrive-P3 evaluated with its released checkpoint)

ModelBackboneArchitectureCoT at inferencerpredr_{\text{pred}}↑PDMS↑
AutoDrive-P3 [1]Qwen2.5-VL-3Bautoregressive VLA, CoT before planning✔0.7390.2
AUGUstus, SFTQwen3.5-2Bunified, action before CoT✗0.7590.6
AUGUstus, + AUGUstus-GRPOQwen3.5-2Bunified, action before CoT✗0.8594.6

Response to W1: Quantitative Analysis of Understanding Capabilities (2/2)

Grounding. On the full navtest split (12,146 scenes; Table 2), for each of the 9,187 scenes with at least one annotated risk object, we compute the fraction of CoT-token attention inside the ground-truth risk-object boxes (last full-attention layer, image tokens of the current frame), against the box-area baseline of uniform attention (13.7%). AUGUstus puts 46.4% there (3.39× the baseline); for 63.8% of the CoT tokens the most-attended image cell lies inside a box; and the boxes it writes in its own CoT receive 48.0%, i.e., the model looks where it says the objects are. The no-CoT model reaches 35.5% with its action tokens; the paired difference is +10.9 points.

Table 2. Attention on risk-object boxes over the full navtest split (SFT models; 12,146 scenes, in-box ratio on the 9,187 scenes with at least one annotated risk object)

ModelQuery tokensTarget boxesMean in-box (%)↑Median in-box (%)↑Lift over box-area baseline↑PDMS↑
Uniform attention (box-area baseline)–ground truth13.7–1.00×–
No CoT, only actionaction tokensground truth35.527.92.60×89.2
AUGUstus (prediction CoT)CoT tokensground truth46.447.43.39×90.6
AUGUstus (prediction CoT)CoT tokensboxes in its own decoded CoT48.0–3.51×90.6

(4) Revision. We have made the action-centric scope of "understanding" explicit in the abstract and in Sec. 1, and added Tables 1 and 2 to Appendix C.3.

Response to W2 & Q1: Unified Architecture vs. MoT Action Experts, and the Baselines of Table 4 (1/2)

(1) Related work. Sec. 2 now discusses Transfusion [2], which unifies next-token prediction and image diffusion for generation, whereas AUGUstus places the action before the CoT so that inference skips the CoT and optimizes both jointly with RL, and DiffE2E [3], which uses no VLM or language supervision. We also add π0\pi_0 and GR00T N1 as MoT-style action experts.

(2) What the baselines of Table 4 are (Figure 1). VLM+Regression follows the query-based design of PWM [4]: learnable action queries are appended to the VLM sequence and an MLP regresses their final hidden states to waypoints. VLM+Flow follows GR00T N1 [5] and ReCogDrive [6]: the last-layer VLM features condition an external flow-matching head, so it is not an MoT action expert (no layer-wise shared attention with the VLM tokens). Both use the same backbone, input, training data and steps, and no CoT, so the in-house comparison (Table 4 and Figure 16 of the original manuscript) isolates the action parameterization and the shared CoT training.

Figure 1. Compared architectures. (a) and (b) are the baselines of Table 4; (c) is the new MoT action expert of Table 3 (the MoE expert differs only in sharing the attention blocks); (d) is AUGUstus

(a) VLM + Regression (PWM-style)             (b) VLM + Flow (GR00T-N1 / ReCogDrive-style)
┌──────────────────────────┐                 ┌──────────────────────────┐   feat.  ┌─────────────────────┐
│ VLM transformer          │                 │ VLM transformer          │ ───────▶ │ external flow head  │
│ V V L L  Q Q Q Q Q Q Q Q │                 │ V V L L                  │          │ noise ─▶ trajectory │
└────────────┬─────────────┘                 └──────────────────────────┘          └─────────────────────┘
  queries ─▶ MLP ─▶ waypoints                 last-layer features condition a separate network;
  no action distribution; no CoT              one lossy interface (features only); no CoT

(c) MoT action expert (new, Table 3)
┌──────────────────────┐  KV cache  ┌──────────────────────┐  KV cache  ┌──────────────────────┐
│ understanding stream │ ─────────▶ │ action expert        │ ─────────▶ │ understanding stream │
│ V V L L              │ crossing 1 │ A A A A A A A A      │ crossing 2 │ L L L L L L   (CoT)  │
│ (VLM weights)        │            │ (own QKVO + FFN)     │            │ (VLM weights)        │
└──────────────────────┘            └──────────────────────┘            └──────────────────────┘
  forward: each crossing passes cached keys/values only; backward: the CoT gradient must cross
  twice to reach the action expert, and the action gradient crosses once to reach the VLM
  ⇒ two lossy interfaces between understanding and action

(d) AUGUstus: unified (ours)
┌─────────────────────────────────────────────────────────────────────────────────────────────┐
│ one transformer, one set of weights, one attention pass per layer                           │
│ V V L L │ A A A A A A A A │ L L L L L L                                                     │
│ context    action block      CoT block                                                      │
└─────────────────────────────────────────────────────────────────────────────────────────────┘
  V, L and A tokens attend within the same attention; the CoT CE loss reaches the action states
  directly; no cache interface in either direction

(3) New comparison with MoE and MoT action experts. Our implementation can give the action tokens their own parameters in every layer, reading the understanding stream through the KV cache. We train an MoE action expert (shared attention, own FFN) and an MoT action expert (own Q/K/V/O and FFN, in the style of π0\pi_0 / GR00T N1; Figure 1c), both initialized from the backbone weights, each without and with CoT under exactly the same data, steps and CoT supervision as AUGUstus (SFT), and evaluate them on NAVSIMv1 (Table 3). The last two columns of Table 3 give the two switches in the configuration file of the anonymous repository (https://anonymous.4open.science/r/AUGU-6ED0/AUGUstus_SFT/configs/navsim_qwen3p5.yaml).

Reviewer_XaFoOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AUGUstus, a vision-language-action model for autonomous driving that aims to reconcile two competing goals: using VLM-style chain-of-thought (CoT) supervision for semantic/spatial understanding and retaining low-latency continuous trajectory generation. The method combines a flow-matching action generator and an autoregressive CoT head in one shared transformer. Its central design, AUGUstus-mode, places action tokens before CoT tokens: the action block is generated first, cannot attend to CoT tokens, and is retained at inference, while CoT is used only during training as auxiliary supervision. The paper also proposes AUGUstus-GRPO, combining a continuous-action GRPO objective with token-level CoT GRPO under a shared group-relative advantage.

Critical strengths

The tension between language-based reasoning and real-time driving control is highly relevant. The paper targets a meaningful limitation of autoregressive driving VLAs: serial CoT generation can be too slow for deployment, while direct trajectory regression or flow-based policies may underuse semantic supervision.

The action-first, CoT-second ordering is conceptually simple and practically motivated. It allows the model to train jointly on structured trajectory and language targets while avoiding CoT decoding during inference. The latency comparison is compelling: the CoT-first variant has a reported latency of 4587 ms, whereas AUGUstus-mode retains the 73.6 ms latency of the action-only model.

The paper reports competitive-to-leading results on NAVSIM v1/v2 and nuScenes. In particular, the reported 94.6 PDMS on NAVSIM v1 and 90.7 EPDMS on NAVSIM v2 are notable, as is the low nuScenes average collision rate.

The paper includes ablations for CoT supervision, CoT ordering, one- versus three-step denoising, and the two-stage GRPO design. The result that CoT supervision improves PDMS from 89.2 to 90.6 is useful evidence that auxiliary semantic/spatial supervision can improve planning.

Reporting module-level timing, total latency, and preliminary vehicle deployment makes the work more practically grounded than many planning papers. The distinction between training-time CoT and action-only inference is especially relevant for deployed driving systems.

Extending group-relative optimization to jointly handle continuous flow-based actions and discrete CoT tokens is technically interesting. The appendix’s derivation of Gaussian log-ratios/KL terms for the action branch improves the paper’s technical completeness.

Critical weaknesses
  1. The most important issue is that, by design, action tokens cannot attend to CoT tokens. Therefore, the CoT output is neither generated before action nor used as an explicit conditioning variable for action inference. The reported gain from CoT appears to arise through shared-parameter multitask/auxiliary supervision, rather than from action reasoning over generated CoT.

This is not necessarily a weakness of the method itself; auxiliary spatial and future-motion supervision can be valuable. However, the framing that AUGUstus “uses CoT to assist action generation” or avoids “VLA-to-VA degeneration” should be carefully qualified. The current evidence does not establish that the model’s textual CoT constitutes a faithful or causally used reasoning process. The equivalence between CoT-first and action-first PDMS (both 90.6) further suggests that token ordering mainly affects latency, while the gain is from the extra supervision.

The authors should distinguish clearly between:

CoT as a deploy-time reasoning trace used by the action policy, and CoT-formatted auxiliary targets that improve shared representations during training. A more accurate characterization may be “structured language-auxiliary supervision for low-latency flow-based planning.”

  1. The paper compares no-CoT, CoT-first, and AUGUstus-mode, but this does not isolate whether the improvement comes from the language/CoT format, the additional object/future-position labels, or simply more supervision and optimization signal.

The following controls would substantially strengthen the claim:

Replace CoT with an equally informative non-linguistic structured prediction head for 2D boxes and future-position classes, matched in loss weight and annotation content. Train with shuffled or corrupted CoT targets to test whether semantic correspondence—not merely extra token prediction—matters. Compare prediction-CoT with direct supervision of the same future position labels using a lightweight auxiliary head. Evaluate whether action performance degrades when relevant CoT labels are removed or replaced at training time. Report CoT accuracy/grounding metrics, e.g., detection quality of mentioned objects, correctness of future-position labels, and correspondence between objects referenced in CoT and trajectory changes. Without these controls, the central mechanism remains ambiguous.

  1. The proposed AUGUstus-GRPO jointly optimizes action and CoT under a shared advantage. However, the paper does not sufficiently establish why rewarding CoT under the same trajectory-level advantage is preferable to optimizing the action policy alone with the same reward and preserving CoT with supervised loss.

In particular, the Stage-II improvement from 94.0 to 94.6 PDMS is modest relative to the large Stage-I gain from 90.6 to 94.0. The paper should include:

an action-only GRPO baseline with the same action rewards and compute budget; a supervised-CoT + action-only GRPO baseline; a CoT-GRPO-only or decoupled-advantage baseline; sensitivity to the relative weighting between JaT-GRPO and CoT-GRPO; reward-component ablations separating progress, terminal, safety, and prediction-related rewards. This is particularly important because a shared reward can make CoT outputs correlate with high-return trajectories without demonstrating that CoT quality causally improves action selection.

  1. The main closed-loop evaluation is on NAVSIM, while nuScenes results are open-loop trajectory and collision metrics. The real-vehicle evaluation is preliminary and based on an internal 1,000 km dataset with aggregate relative scores, making independent assessment difficult.

Given the paper’s claims about safer actions, long-tail handling, and deployment readiness, I would expect:

performance broken down by scenario type, especially interactions, vulnerable road users, occlusion, merges, cut-ins, intersections, and rare events; confidence intervals or multiple-seed variation for key metrics; failure analyses comparing no-CoT, action-only flow matching, and AUGUstus; more detail on the real-vehicle operating domain, intervention policy, safety driver protocol, route conditions, and failure/intervention statistics; evaluation under perturbations or distribution shift, rather than only aggregate benchmark metrics. The qualitative attention visualization is suggestive but anecdotal; a single “46% in-box” example is insufficient to substantiate general claims of improved spatial grounding.

  1. The paper reports training on 512 H800 GPUs for two days for the in-house model, but key reproducibility details are limited. It is difficult to determine whether the comparison is controlled for backbone size, pretraining data, language data, training schedule, total compute, and availability of auxiliary annotations.

The authors should provide:

backbone/vision encoder sizes and all trainable parameter counts; public-data versus internal-data usage for every reported model/result; exact data sources and construction pipeline for CoT labels; full hyperparameters for SFT and both GRPO stages; number of rollout samples/groups, denoising steps, reward weights, KL coefficients, and seed variance; code, trained checkpoints, or at minimum sufficient pseudocode/configurations to reproduce public-benchmark results. The reported internal 1,000 km evaluation is useful engineering evidence, but it should not carry much weight without a more transparent protocol and absolute metric definitions.

Questions for authors

ince action tokens are explicitly prevented from attending to CoT tokens, in what precise sense does CoT reasoning guide action generation rather than act as an auxiliary multitask signal? Please clarify the intended causal mechanism.

Can the authors compare against a non-linguistic auxiliary head predicting exactly the same information as the prediction-CoT—relevant object boxes and future-position labels? This would isolate whether the benefit is due to language/CoT versus additional structured supervision.

How are the CoT targets generated? Are object relevance, 2D boxes, and future-position labels derived from ground-truth annotations, heuristics, or another model? What is their coverage and noise level?

Is the reported CoT faithful to the action branch? For example, if the CoT target is perturbed, shuffled, or masked during training, does planning performance decline? Can CoT correctness predict trajectory safety or intervention risk?

For AUGUstus-GRPO, how does performance compare to action-only GRPO using the same policy initialization, rewards, number of rollouts, and compute? What is the incremental value of CoT-GRPO after keeping CoT supervision in SFT?

How robust are the results across random seeds? Please report mean and standard deviation, at least for the main NAVSIM ablations and GRPO comparison.

Are all methods in the NAVSIM comparison trained with comparable data scale, input resolution, backbone capacity, and access to language/CoT annotations? If not, please make the comparison conditions explicit.

Please provide more detail on the real-vehicle evaluation: operational design domain, number and type of interventions, safety-driver procedure, hardware stack, and whether the reported 12 Hz includes all production-critical components.

26 comments

Thank you for your valuable and insightful comments. We deeply appreciate your input and have made every effort to address your concerns, as detailed below.

Response to W1 & Q1: How CoT Assists Action Generation without Inference-time Attention (1/2)

We respectfully note that the paper never claims that CoT guides the action at inference. On the contrary, the manuscript repeatedly states that CoT acts during training and that only the action is used at deployment:

LocationOriginal text in the manuscript
Abstract (L27–29)"AUGUstus-mode, which orders the action generation before the CoT so that the CoT supervises the reliable action during training while inference needs only the action output, enabling low latency."
Introduction (L95–96)"Our objective is action-centric: understanding is not trained for its own sake but to generate a safer and more accurate trajectory."
Fig. 1"Inference Only Action!"
Fig. 2 caption (L182–183)"Action tokens use bidirectional attention and cannot attend to the CoT tokens. At inference, AUGUstus directly outputs the action to ensure low latency."
Sec. 3 (L201–206)"At inference, as summarized in Algorithm 2, only context c is provided. [...] CoT decoding is optional to ensure low latency. [...] action tokens cannot attend to CoT tokens."
Algorithm 2, step 4"Skip CoT at inference; decode it afterward only if requested (e.g., during RL rollout sampling)."
Appendix D.2.1 (L1231–1233)"the action tokens are visible to the CoT stream, whereas the action tokens never read CoT tokens. Consequently, L_CE back-propagates through the action states and the shared trunk, but does not create an inference-time dependency on generated text."

(1) The intended mechanism. By construction, AUGUstus uses CoT as training-time supervision, not as a deploy-time reasoning trace (Figure 1). The CoT tokens are placed after the action tokens and attend to them, so the CoT cross-entropy back-propagates directly into the action hidden states: because the CoT stream must read out where the risk objects are and where they will be, the action representation is trained to encode it. A generic auxiliary head on the backbone reaches the action representation only indirectly through shared weights. At inference the CoT stream is not run.

Figure 1. Information flow in AUGUstus-mode during training and inference

Training  (action first, CoT second; one forward pass of one shared transformer)

  video + ego status + prompt ─┐
  noisy action x_t ────────────┼─▶ │ V V L L │ A A A A A A A A │ L L L L L L │
  CoT tokens (teacher-forced) ─┘      context     action block     CoT block
                                                      │                │
                                                      ▼                ▼
                                                 action head        LM head
                                                 x̂_0, flow loss     CoT tokens, CE loss
                                                      ▲                │
                                                      └── ∂L_CE/∂(action states) ──┘
  attention:   CoT tokens  ──▶ action tokens   allowed  (CoT reads the action)
               action tokens ──▶ CoT tokens    blocked  (action never reads CoT)

Inference  (deployed path, 73.6 ms)

  video + ego status + prompt + noise ─▶ │ V V L L │ A A A A A A A A │ ─▶ action head ─▶ trajectory
  no CoT token is generated or read; the CoT block is absent from the sequence

This separates AUGUstus from VA models: a VA model trained only on actions loses the VLM's language-grounded spatial understanding, whereas AUGUstus retains it in the shared parameters and the action states and turns it into a better action, while deploying the action branch alone.

(2) Why we avoid inference-time language reasoning. Serial language reasoning before acting is too slow for a vehicle at high speed (Table 6 of the original manuscript):

Table 1. Ordering only changes latency (NAVSIMv1, SFT)

SettingPDMS↑Latency per decision
CoT first, action second90.64597 ms
AUGUstus-mode (ours)90.673.6 ms

Response to W1 & Q1: How CoT Assists Action Generation without Inference-time Attention (2/2)

This is why many recent driving VLAs drop language and directly regress or denoise actions [1, 2], degenerating into vision-action (VA) models. AUGUstus-mode uses language where it is affordable (training) and only the action where latency matters (inference).

(3) On CoT-first = AUGUstus-mode (both 90.6 PDMS). As Appendix C of the submission states, "the quality gain comes from the CoT supervision itself rather than the ordering; the ordering decides only the deployment cost." AUGUstus-mode thus keeps the full benefit of CoT supervision while running 62× faster than CoT-first, which is the key validation of our design.

(4) Why this is not VA degeneration. AUGUstus retains the VLM's understanding in the same network, and it measurably benefits the action (Table 2; the attention row extends the single example of Fig. 6 to all 12,146 navtest scenes, Appendix C.3, Table 13 of updated manuscript).

Table 2. Evidence that the retained understanding benefits the action (navtest)

EvidenceNo-CoT (VA-style) modelAUGUstus
Accuracy of the decoded CoT, rpredr_{\text{pred}} (AutoDrive-P3: 0.73)cannot be produced0.75 after SFT, 0.85 after AUGUstus-GRPO (Appendix D.5 of the original manuscript)
PDMS (Table 6 of the original manuscript)89.290.6 (+1.4)
NC / DAC / TTC (Table 9 of the original manuscript)98.8 / 96.8 / 95.899.0 / 97.9 / 96.4
EP (Table 9 of the original manuscript)83.584.2
Attention on GT risk-object boxes, mean in-box ratio over the full navtest split (Appendix C.3, Table 13 of updated manuscript)35.5% (action tokens, 2.60× the box-area baseline)46.4% (CoT tokens, 3.39× the box-area baseline)

(5) CoT/language form vs. additional labels. This is answered by the new controls (Sec. 4.3, Table 7 of updated manuscript): a non-linguistic auxiliary head with identical content, and shuffled or corrupted CoT targets.

Revision. Sec. 3 now states explicitly that the action branch never conditions on CoT tokens at inference and that the optional post-hoc CoT serves interpretability.

Response to W2 & Q2 & Q4: Isolating the Source of the CoT Gain (1/2)

We apologize for the misunderstanding. The paper does not compare only no-CoT, CoT-first and AUGUstus-mode: Table 6 of the original manuscript is a summary, and the complete CoT ablation is Table 9 (Appendix C), with nine variants under an identical recipe (same scenes, action targets, λdiff=10\lambda_{\text{diff}}=10 / λlm=1\lambda_{\text{lm}}=1 and deterministic one-step evaluation on the full navtest split) that change only the content or encoding of the CoT block. It already holds several of the requested controls, and we have moved it to the main text. Table 3 collects the results for all five requests.

Table 3. CoT supervision controls (NAVSIMv1 navtest, SFT only). Existing rows are from Table 9 of the original manuscript; new rows are added in this rebuttal

Supervision targetLabelsEncodingPDMS↑NC↑DAC↑EP↑TTC↑Comf.↑Source
No CoTnone–89.298.896.883.595.8100.0Table 9
Planning CoTego maneuvertext90.198.997.483.896.1100.0Table 9
Reasoning CoTboxes + future position + maneuver, ≈100 tokensfree text89.698.796.983.496.1100.0Table 9
Perception CoTboxesJSON text90.398.997.584.396.1100.0Table 9
Perception + Prediction CoTboxes + future position, more tokensJSON text90.398.997.684.196.3100.0Table 9
Short-predictionboxes + future positionatomic vocabulary, 7× shorter89.198.796.783.195.4100.0Table 9
Prediction CoT (default)boxes + future positionJSON text90.699.097.984.296.4100.0Table 9
CoT first (ordering control)boxes + future positionJSON text90.699.197.884.296.6100.0Table 9
Lightweight auxiliary headboxes + future positionset-prediction head, no text89.098.996.383.195.7100.0new
Shuffled CoT (CoT of a random other scene, same format and length)mismatchedJSON text87.898.795.681.495.7100.0new
Corrupted CoT (boxes correct, future positions random)boxes + random future positionJSON text88.198.595.882.695.5100.0new

(1) "Replace CoT with an equally informative non-linguistic structured prediction head." This control is new. A lightweight set-prediction head (2-layer cross-attention decoder with NN object queries, Hungarian matching, L1 + GIoU for boxes, cross-entropy for the future-position bins) on the same context and action hidden states is trained on exactly the same objects, boxes and bins as the prediction CoT, with the language loss off and its loss weight tuned on a held-out navtrain split. It reaches 89.0 PDMS, below the no-CoT baseline (89.2) and at the level of Short-prediction (89.1), vs. 90.6 for the prediction CoT; its DAC (96.3) is even below no-CoT (96.8). The same labels give no gain when supervised through a new output space instead of the VLM's language space.

(2) "Train with shuffled or corrupted CoT targets." Table 9 already holds the label-fixed version: Short-prediction encodes the identical boxes and future positions in an atomic vocabulary and falls to 89.1, below no-CoT, so the gain is not "extra token prediction". We add the two requested variants (Table 3), changing only the data loader: (a) the CoT of a random other scene with matched format, length, object count and class distribution; (b) correct boxes with random future-position bins. Both fall below no-CoT: 87.8 (shuffled) and 88.1 (corrupted), i.e., −1.4 and −1.1, mostly in DAC (95.6 / 95.8 vs. 96.8) and EP (81.4 / 82.6 vs. 83.5). The shuffled CoT gives the same amount of token-prediction signal as the prediction CoT, yet hurts the action. The gain of the prediction CoT therefore comes from CoT content that is correct for the scene, supervised in the shared model that preserves the VLM's spatial understanding for the action, not from the extra supervision itself.

Comments on the paper

3 comments
Area_Chair_yU3hOpenReview ↗

Reminder: Author Response and Reviewer Discussion

Dear Reviewers,

We are halfway through the discussion period. Please review the authors’ responses, address any remaining concerns, and engage in the discussion where appropriate. If your assessment remains unchanged, please briefly acknowledge that you have considered the authors’ responses.

Thank you for your time and contributions.

General Response (1/2)

Dear AC,

We thank the reviewers for their careful reading and constructive comments. Below we summarize the shared concerns, the new evidence and the revisions; the individual responses give the full details. The reviewers find the action-first, CoT-second ordering conceptually simple and practically motivated with a compelling latency comparison (XaFo), the unified architecture for continuous action generation and discrete language understanding novel with strong empirical results (emmC), and the design a meaningful answer to the trade-off between VLM understanding and latency, with useful ablations and multi-seed evaluations (qZmR). The shared concerns are (A) where the CoT gain comes from when the action never reads the CoT at inference, (B) which parts of AUGUstus-GRPO matter, and (C) the evidence beyond NAVSIM. We address them with label-matched CoT controls, CoT accuracy and full-split attention statistics, GRPO variants including a single shared advantage and Flow-GRPO under the same setup, MoE and MoT action experts, and zero-shot transfer from NAVSIM to nuScenes. The code is available at https://anonymous.4open.science/r/AUGU-6ED0.

GR1. The CoT gain comes from correct scene content in the VLM's language space (XaFo W1/W2, emmC W1, qZmR W1/Q1). SFT on NAVSIMv1 with the same recipe; only the CoT target changes (Sec. 4.3, Tables 6 and 7 of updated manuscript).

CoT supervisionPDMS↑DAC↑EP↑
No CoT89.296.883.5
Auxiliary head on the same boxes and future positions, no text89.096.383.1
Shuffled CoT (CoT of another scene)87.895.681.4
Corrupted CoT (correct boxes, random future positions)88.195.882.6
Short-prediction (same labels, atomic vocabulary)89.196.783.1
Prediction CoT, action first (ours)90.697.984.2
Prediction CoT, CoT first90.697.884.2

We highlight three points:

Labels alone do not help, and wrong content hurts. The same labels through a non-linguistic head (89.0) or an atomic vocabulary (89.1) give no gain over no CoT (89.2), and shuffled or corrupted CoT targets fall below it (87.8 and 88.1).

The CoT shapes the action without being read at inference. The action tokens never attend to the CoT tokens; the CoT acts through the shared representation and the training gradients. CoT-first reaches the same 90.6 PDMS but takes 4597 ms instead of 73.60 ms.

The retained understanding is measurable. The decoded CoT reaches rpredr_{\text{pred}} 0.75 after SFT and 0.85 after AUGUstus-GRPO (AutoDrive-P3: 0.73). Over 12,146 navtest scenes, the CoT tokens put 46.4% of their attention inside the risk-object boxes, against 35.5% for the action tokens of the no-CoT model and 13.7% for uniform attention (Appendix C.3, Tables 12 and 13 of updated manuscript).

GR2. Both branches of AUGUstus-GRPO matter (XaFo W3/Q5, qZmR W2/Q2/Q3). All variants start from the same SFT checkpoint with the same rewards, G=8G=8 and training budget (Sec. 4.3, Table 8 and Appendix B.6, Table 11 of updated manuscript).

VariantPDMS↑NC↑DAC↑EP↑TTC↑
SFT90.699.097.984.296.4
Action-only GRPO (without CoT-GRPO)92.698.097.691.894.1
Supervised CoT + action-only GRPO94.198.898.491.297.0
Single shared advantage94.298.798.391.597.0
Flow-GRPO schedule (3-step)93.999.098.391.497.0
AUGUstus-GRPO (per-branch advantages, ours)94.698.998.991.997.1

We highlight three points on AUGUstus-GRPO:

Optimizing the CoT with the action protects safety. Without CoT-GRPO, PDMS drops to 92.6, mainly through NC (−0.9) and TTC (−3.0); a supervised CoT recovers part of it (94.1) but does not adapt to the policy's own rollouts.

Per-branch advantages are better than one shared advantage (94.6 vs. 94.2). All results in the paper and the released code use per-branch advantages; Sec. 3.3, Eqs. (4), (6) and (19)–(21), Fig. 3 and Algorithms 3, 4 and 7 now write them out explicitly.

JaT-GRPO is defined at the pure-noise endpoint. Its schedule g(t)2=λtg(t)^2=\lambda t is finite at t=1t=1, whereas the Flow-GRPO schedule a2t/(1−t)a^2t/(1-t) diverges there and needs a grid-dependent clamp; under the same 3-step setup, JaT-GRPO is 0.7 PDMS higher.

General Response (2/2)

Concerns of each reviewer and our responses.

Reviewer XaFo

  • How the CoT helps without inference-time attention (W1/Q1): the CoT is training-time, action-conditioned supervision that shapes the action through the shared representation and the gradients; the action branch never reads it at inference, and CoT-first gives the same 90.6 PDMS at 4597 vs. 73.60 ms (GR1; Sec. 3.1 of updated manuscript).
  • Source of the CoT gain (W2/Q2/Q4): label-matched controls, CoT accuracy and full-split attention statistics (GR1).
  • Necessity of AUGUstus-GRPO and the Stage-II gain (W3/Q5): removing CoT-GRPO costs 2.0 PDMS (GR2); a CoT weight of 0.1 or 10 gives 94.3 or 94.1, and a Stage-II safety term without NC gives 94.3 (Appendix D.7, Table 17 of updated manuscript). Stage II adds +0.6 at the 94-PDMS level, more than the gaps among the strongest published methods, and improves NC, DAC, EP and TTC together.
  • Evaluation scope and seeds (W4/Q6): the standard deviation over three training and three test seeds is at most 0.05 PDMS (Appendix F, Table 19 of updated manuscript). Trained only on NAVSIM, AUGUstus transfers zero-shot to nuScenes with 0.38 m L2 error and 0.06% collision rate, the lowest L2 error among the NAVSIM-trained models (Sec. 4.2, Table 3 and Appendix E, Table 20 of updated manuscript).
  • Reproducibility, CoT targets and comparison conditions (W5/Q3/Q7): 2.6B parameters, navtrain only, and the public AutoDrive-P3 prediction CoT, derived deterministically from the NAVSIM ground truth (Appendix C.1.1, D.5 and D.6 of updated manuscript).
  • Real-vehicle details (Q8): 12 Hz (83 ms) is the end-to-end latency of the deployed action-only path on the vehicle; the test is preliminary, with a safety driver, and six clips are in the supplementary material (Appendix G of updated manuscript).

Reviewer emmC

  • Quantitative evaluation of the understanding (W1): understanding in AUGUstus is action-centric and is quantified by CoT accuracy and full-split attention (GR1); the abstract and Sec. 1 now state this scope.
  • Unified architecture vs. MoE/MoT action experts (W2/Q1): with the same data, steps and CoT supervision, CoT adds +1.0 PDMS to an MoE expert (88.8 to 89.8), +0.9 to an MoT expert (89.0 to 89.9) and +1.4 to the unified architecture (89.2 to 90.6) (Appendix A, Table 9 of updated manuscript); Sec. 2 now discusses Transfusion, DiffE2E, π0\pi_0 and GR00T N1.
  • Ablations beyond NAVSIM (W3): the in-house 1,000 km evaluation compares VLM+Flow, VLM+Regression and AUGUstus on the same data (safety worse by 18.4% and 15.1%), and the NAVSIM-trained model transfers zero-shot to nuScenes.

Reviewer qZmR

  • Contribution of CoT supervision (W1/Q1): the same labels through an atomic vocabulary (89.1) or an auxiliary head (89.0) give no gain; only the prediction CoT does (90.6) (GR1).
  • Shared vs. separate advantages (W2/Q2): all results and the released code use per-branch advantages (94.6 vs. 94.2 for a single shared advantage), now written out explicitly (GR2).
  • Comparison with flow-based RL methods (Q3): the JaT-GRPO schedule is finite at t=1t=1, and under the same 3-step setup JaT-GRPO reaches 94.6 vs. 93.9 for Flow-GRPO (GR2).
  • Real-world evidence (W3): the absolute in-house metrics are confidential; the main claims and all ablations rest on public benchmarks, and the real-vehicle result is described as a preliminary test.

Revisions in the updated manuscript (highlighted in blue).

  • Abstract and Sec. 1: CoT as training-time, action-conditioned structured supervision; the action-centric scope of understanding; anonymous code link.
  • Sec. 2: Transfusion, DiffE2E, π0\pi_0 and GR00T N1. Sec. 3.1: the action branch never reads CoT tokens at inference.
  • Sec. 3.3, Eqs. (4), (6) and (19)–(21), Fig. 3 and Algorithms 3, 4 and 7: per-branch advantages.
  • Sec. 4: zero-shot row in Table 3; complete CoT ablation moved from the appendix (Table 6); label-matched controls (Table 7); GRPO variants (Table 8).
  • Appendix A: MoE/MoT action experts (Table 9). Appendix B.3 and B.6: Flow-GRPO discussion and same-setup comparison (Table 11).
  • Appendix C.1.1 and C.3: construction of the CoT targets; CoT accuracy and attention statistics (Tables 12 and 13).
  • Appendix D.1 and D.5–D.7: Stage-II safety term, model size, data, hyperparameters, comparison conditions and sensitivity (Tables 14–17).
  • Appendices E, F and G: zero-shot nuScenes transfer (Table 20), multi-seed evaluation (Table 19), in-house evaluation and the preliminary real-vehicle test.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.