- Rating
- 3(+1)
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents AlterVLA, a vision-language-action model for autonomous driving that predicts temporal changes in visual feature space to condition trajectory planning. Built on Qwen3.5-2B, it learns compact change tokens at multiple future horizons using differences between current and future features from a frozen visual encoder. A four-stage pipeline first learns these representations, trains a planner with oracle change tokens, trains a predictor to replace them using current observations and ego information, and finally refines the trajectory head through reinforcement learning. The predicted change tokens directly condition diffusion-based trajectory generation within a shared backbone. Experiments using a single front-view camera on NAVSIM v1 and v2 report competitive planning performance, with ablations comparing temporal-change prediction against full-state prediction, no-change conditioning, and single-horizon variants.
Critical strengths
-
Well-motivated temporal-change conditioning for planning. Since the planner already receives current visual features, predicting compact changes relative to those features is a reasonable way to represent future scene evolution. The predicted tokens preserve multiple temporal horizons and directly condition trajectory denoising. Learning these representations in the backbone’s own visual feature space also avoids an additional visual teacher.
-
Relevant ablations and competitive planning performance. Table 3 compares the proposed representation with no-change conditioning, full-state prediction, single-horizon prediction, and an oracle reference. Multi-horizon change conditioning improves PDMS from 86.27 to 88.53 over the no-change baseline and outperforms the single-horizon variants, supporting the usefulness of the design. The method also achieves competitive results on NAVSIM v1 and v2 using a 2B backbone and a single front-view camera.
Critical weaknesses
-
Limited evidence for the advantage and generality of temporal-change representations. Table 3 provides a useful comparison with full-state prediction, but the additional gain is modest: 88.53 versus 88.20 PDMS. The paper does not clearly specify whether the full-state baseline matches the proposed method in token budget, module capacity, and training procedure. Moreover, all experiments use Qwen3.5-2B. Since the change representation is learned in the backbone’s own visual feature space, comparisons across model families and larger parameter scales would help establish whether its advantage extends beyond this configuration.
-
The benefit of the four-stage training procedure is not sufficiently isolated. The planner is first trained with oracle change tokens and then frozen while the predictor learns to reproduce those tokens, without trajectory supervision in Stage III. Although Stage IV subsequently adapts the trajectory head through RL, the paper does not compare this procedure with training the planner directly on predicted changes or jointly fine-tuning the predictor and planner after the switch. The oracle-versus-predicted comparison measures the effect of conditioning quality rather than the advantage of the training schedule. Matched-budget comparisons would clarify whether the staged design offers benefits over more direct alternatives.
Questions for authors
-
In Table 3, does the full-state prediction baseline use the same token budget, module capacity, and training procedure as temporal-change prediction? Please clarify any differences that could affect the 0.33-point PDMS gap.
-
Does the advantage of temporal-change prediction over full-state prediction persist on another VLM family or a larger backbone? A targeted comparison of no-change, full-state, and temporal-change conditioning would help assess generality.
-
How does the oracle-first training procedure compare with training the planner directly on predicted changes, or jointly fine-tuning the predictor and planner with trajectory supervision after Stage III? Comparisons under matched training budgets would clarify the benefit of the proposed schedule.
Thank you for asking about the full-state baseline, backbone scale, and training schedule. All PDMS results below are on NAVSIM v1.
Q1 — Full-state baseline and experimental controls
In Table 3, full-state and temporal-change prediction both use 128 foresight tokens per scene, with 32 tokens at each of 0.5, 1, 2, and 4 s. The follow-up comparisons also match planner capacity, training stages, and optimization budget.
Temporal-change prediction improves SFT PDMS in all three paired training seeds, with a mean gain of 0.40 points (sample SD 0.07, n = 3).
Predicted change also improves SFT PDMS at all four tested denoising step counts, with gains of 0.33–0.50 points. The 10-step row is the seed-42 reference from Table 3.
Q2 — Larger backbones and other VLM families
On Qwen3.5-4B, adding temporal-change conditioning improves SFT PDMS from 89.077 to 89.418 (+0.341 points) relative to the no-change baseline (revised paper, Appendix A.4, Table 10). This result shows a benefit at a larger scale within the Qwen3.5 family. A comparison with full-state prediction at 4B and evaluation on other VLM families remain open.
Q3 — Oracle-first training, direct training, and joint fine-tuning
Oracle-first training leads direct training with predicted change tokens by 4.93 PDMS points after SFT and 3.40 points after RFT. The two runs match planner architecture, hidden-state inputs, planner training budget, and evaluation settings. Both use the same frozen predictor at inference.
For the same oracle-trained planner, switching from oracle to predicted tokens at inference reduces PDMS from 89.10 to 88.53 (−0.57 points).
Joint fine-tuning adds supervised updates to the predictor and planner, improving SFT PDMS by 0.48 points. Applying the same RFT procedure to the original and jointly fine-tuned checkpoints gives 90.94 and 90.07 PDMS, respectively. We use the original Stage III checkpoint to initialize Stage IV because it yields 0.87 points higher final PDMS in this comparison.
Rollout diversity. Before RFT, we sample 8 trajectories per NAVTRAIN scene token from each checkpoint. The jointly fine-tuned checkpoint produces more tightly clustered candidates.