- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates the misalignment between textual reasoning and trajectory execution in Vision-Language-Action (VLA) self-driving models through mechanistic probing on Alpamayo 1.5, establishes a multi-axis faithfulness benchmark, and proposes TRACR (Trajectory-Reasoning Aligned Code Reward) to enforce reasoning-action consistency via GRPO post-training with scene-specific verified executable code rewards.
Critical strengths
- Tackles an important and timely problem: evaluating and improving the causal faithfulness and reasoning-action consistency of VLA models in autonomous driving.
- Conducts comprehensive mechanistic probing (including structural/positional sweeps, clause reordering, logit-level alternatives, and cache ablations) to diagnose failure modes like positional shortcutting and post-hoc rationalization.
- Introduces TRACR, a novel post-training framework leveraging offline-verified, scene-specific executable code rewards to enforce semantic and visual alignment without degrading trajectory accuracy.
Critical weaknesses
- Limited Scalability and Generalization of Code Rewards: The proposed TRACR framework relies on generating scene-specific executable Python programs and filtering them against counterfactual negatives. It remains unclear how well this automated code generation and verification pipeline scales to highly diverse, open-world long-tail scenarios without human-in-the-loop debugging.
- Confounding Factors in Mechanistic Probing: While the mechanistic analysis (such as prefix/suffix sweeps and clause reordering) reveals strong sensitivity to token position and surface order, it is difficult to completely decouple whether trajectory shifts stem from true semantic reasoning changes or artifacts of autoregressive attention scaling over modified prompt lengths.
- Incomplete Computational Overhead Analysis: The paper introduces multiple complex evaluation axes, offline program verification, and GRPO rollout sampling (K=12 candidates per scene), but lacks a detailed computational and latency profile during training and inference compared to standard baselines.
Questions for authors
- Could the authors discuss the failure rate and computational cost of the automated code generation and verification pipeline (Section 3.5) when scaled to thousands of diverse long-tail driving scenarios?
- To what extent are the trajectory shifts observed in the clause-reordering and prefix-sweeping probes caused by changes in autoregressive attention length and positional encodings rather than pure semantic disruption?
- Please provide a computational efficiency comparison (e.g., training time overhead, GPU memory, and inference latency) between TRACR and standard LLM-judge or global-reward GRPO baselines.
Verifier Scalability, Positional Effects, and Compute Costs
Thank you for taking the time to read our work and for providing feedback. Thank you also for asking us to distinguish the cost of generating reward programs from the cost of using them, and to clarify what the interventional results establish about position and meaning. We do our sincere best to address your questions below:
How does program generation and verification scale? We generate each scene-specific program offline before performing GRPO post-training. We do this in four stages: 1) Identify the relevant event, 2) specify the acceptable commitments and motion, 3) define graded checks and contradiction penalties, and 4) implement these executable checks in Python through code subtask reward generation. We admit a program if it passes all four checks on the expert pair and accepts at most 6 of 100 cross-scene mismatches. Once admitted, it can score all 12 candidates in a GRPO group and be reused each time training revisits the scene.
At 1,000 scenes, evaluating one expert pair and 100 mismatches per scene requires 101,000 pair evaluations before retries. We measured the cost of scoring after authoring. Running 32 existing programs on 384 saved rollouts took 5.84 seconds on CPU, with zero execution errors. We did not measure end-to-end authoring cost or the fraction of programs requiring repair at this scale. Once a program is admitted, it can score additional rollouts without another LLM call. The practical advantage we can demonstrate is reuse (i.e., after a program is accepted, scoring additional rollouts requires only executing its checks). In generating offline reward functions, we achieved a ~93%+ code eligible reward training sample for strong code reward functions that are discriminative enough to reward trajectory-reasoning alignment. Additionally, we hope to curate a dataset that we can contribute to the community for code trajectory-alignment checks that are more expressive than global deterministic code functions with traditional rules.
How much of the trajectory shift comes from position or length? Reordering clauses shifted the predicted path by 0.168 m, compared with 0.0039 m when replaying the original text. This establishes sensitivity to clause order. Moving a maneuver clause also changes its position and preceding context, while a prefix or suffix sweep changes the amount of visible context. Those probes alone cannot isolate a change in meaning. Furthermore, as shown in our initial experiments, we also found that clause reversal, on average, exceeds entire masking the reasoning in terms of ADE, revealing sensitivity to reasoning, which we leverage through the code reward generation alignment rewards, which leverage the GT reasoning trace for rewards.
We also examined interventions that preserve sequence length. In Table 5, masking the emitted reasoning in the action expert’s cache while holding the images, generated sequence, and diffusion seed fixed shifted the trajectory by 0.128 m (95% CI 0.108 to 0.152 m, clustered over 939 clips). In a follow-up, we masked either the first parsed action span or the nearest non-action span of equal token count. Masking the action span displaced the path more, by 0.022 m on average for frozen Alpamayo (123 eligible clips) and 0.069 m for the step-720 checkpoint (110 clips). On the 99 clips eligible for both, the paired difference in this contrast was +0.045 m (95% CI −0.011 to +0.124). The within-model effect persists with token IDs and sequence length fixed, although the two spans still occupy different positions. Stricter equal-length clause swaps and action reversals yielded no eligible edits under our controls.
What training and inference overhead did we measure? In the 100-step pilot on the same eight B200 GPUs, the scene-specific ranked run took 124 minutes (16.5 allocated GPU-hours), compared with 132 minutes (17.6 allocated GPU-hours) for the global ranked verifier. These single-run measurements show comparable training time in this setting. The deployment inference latency and memory have 0% overhead because TRACR modifies only policy weights and does not run code at evaluation. For a peak memory profile, the LLM-judge relies on an external API call which doesn't affect training memory. The process of code reward generation is done entirely offline before training. At inference, both the reward program and the judge are absent from the execution path: the trained policy generates the reasoning trace and trajectory directly.