- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
RaPhy-AD incorporates radar observations into a pretrained vision-language model to generate structured physical rationales before predicting future ego trajectories. It introduces a Physical-Aware Tokenizer that represents numerical values together with their units, and a Polar-Time Radar Encoder that captures spatial and temporal radar structure. Training combines multitask supervised fine-tuning with GRPO using verifiable rewards for trajectories and physical quantities.
Critical strengths
he paper provides quantitative evidence that generated physical descriptions use scene-specific radar information. Beyond qualitative examples, Tables 20–21 evaluate physical quantities and rationale–trajectory consistency. Replacing the original radar with mismatched radar increases closing-velocity MAE from 0.712 to 3.440 m/s and trajectory L2 from 0.144 to 0.450 m (Table 5). These results support the relevance of radar to the model’s predictions and make the physical-grounding contribution more convincing.
Critical weaknesses
Appendix F.1 states that RaPhy-AD receives an oracle LEFT/RIGHT/STRAIGHT command derived from the future ground-truth trajectory, whereas navigation prompting is disabled for Alpamayo 1.5 despite being supported. This difference can affect trajectory prediction, particularly in turning and lane-change scenarios, and makes it difficult to attribute the reported advantage to radar and the proposed reasoning method. Comparisons under equivalent route conditioning are needed to establish the magnitude of the contribution. The necessity and sensitivity of this alignment procedure should be clarified.
The SFT comparison provides initial evidence for rationale supervision, but there is no matched GRPO control that predicts trajectories without generating rationales. Trajectory and endpoint rewards together receive a weight of 0.66 in Eq. 17, so direct trajectory optimization remains a plausible explanation for much of the final improvement. Improvements in both rationale quality and trajectory accuracy do not by themselves distinguish these effects. Furthermore, the tokenizer and encoder ablations use only 100 optimization steps and operate at substantially higher errors than the final model. They establish benefits during early optimization, but leave their contribution after full training uncertain. These limitations weaken the evidence for the paper’s central methodological claims.
Questions for authors
See weaknesses.
3 comments
W2. The SFT comparison provides initial evidence for rationale supervision, but there is no matched GRPO control that predicts trajectories without generating rationales. Trajectory and endpoint rewards together receive a weight of 0.66 in Eq. 17, so direct trajectory optimization remains a plausible explanation for much of the final improvement. Improvements in both rationale quality and trajectory accuracy do not by themselves distinguish these effects.
W2. Disentangling Trajectory Rewards from Rationale Supervision
We agree that improvements in both trajectory accuracy and rationale quality alone do not isolate whether the final gain comes from direct trajectory optimization or from the explicit physical reasoning process. To disentangle these effects, we conduct two complementary controls: (1) trajectory-only GRPO, which removes all rationale-related rewards while retaining direct trajectory optimization, and (2) rationale-free trajectory prediction, which removes explicit rationale generation while preserving the same downstream trajectory objective.
1. Is direct trajectory reward sufficient?
We first compare our full GRPO objective against trajectory/endpoint-only GRPO under the same training setting.
| Trajectory Metrics | Rationale Metrics | ||||
|---|---|---|---|---|---|
| Training | Traj. L2 Avg. ↓ | Collision Rate (%) ↓ | Range MAE ↓ | Velocity MAE ↓ | Ego +1s Speed MAE |
| SFT | 0.300 | 0.073 | 1.033 | 0.762 | 0.212 |
| + Trajectory-only GRPO | 0.287 | 0.064 | 1.178 | 0.802 | 0.285 |
| + Full GRPO (Ours) | 0.144 | 0.014 | 0.762 | 0.654 | 0.126 |
Interpretation. Trajectory-only GRPO provides only a modest improvement over SFT in trajectory metrics, while degrading the numerical accuracy of the physical rationale. In contrast, Full GRPO substantially improves both planning performance and physical-state accuracy. This indicates that the large final gain cannot be explained by direct trajectory rewards alone; the rationale-related rewards provide an additional learning signal that is important for downstream planning.
2. Is explicit rationale generation itself necessary?
We next remove the explicit rationale-generation process and directly predict the trajectory from the model representation. We additionally consider a structured-physical-states-only variant to distinguish explicit reasoning from merely exposing intermediate physical quantities. The rationale-free direct-trajectory control uses the same backbone, radar inputs, and training data as Full Rationale, with a matched SFT/GRPO optimization budget.
| Training | Traj. L2 Avg↓ | Collision Rate (%) ↓ |
|---|---|---|
| Direct Trajectory prediction (w/o rationale) | 0.278 | 0.021 |
| structured physical states alone | 0.378 | 0.095 |
| Full Rationale (Ours) | 0.144 | 0.014 |
Interpretation. Removing explicit rationale generation substantially degrades trajectory accuracy, even when the model is still optimized for the same final planning objective. Moreover, simply exposing structured physical states is insufficient to recover the performance of the full model. Together with the trajectory-only GRPO result above, this suggests that RaPhy-AD benefits not only from direct trajectory optimization, but also from explicitly generating and supervising the intermediate physical reasoning process.
W3. Furthermore, the tokenizer and encoder ablations use only 100 optimization steps and operate at substantially higher errors than the final model. They establish benefits during early optimization, but leave their contribution after full training uncertain. These limitations weaken the evidence for the paper’s central methodological claims.
W3. Full-Training Validation of Tokenizer and Radar Encoder Ablations
We thank the reviewer for emphasizing that the original 100-step ablations mainly demonstrate early-stage optimization behavior and do not, by themselves, establish the contribution of the Physical-Aware Tokenizer and Polar-Time Radar Encoder after full training. We agree that this is an important limitation of the original Table 6, especially given that these components constitute central methodological elements of RaPhy-AD.
- Why the original ablations were reported at 100 steps. During training, several alternative configurations exhibited a rapid slowdown in improvement at an early stage, whereas the proposed Physical-Aware Tokenizer and Polar-Time Radar Encoder continued to reduce error as optimization progressed. To compare all variants under the same computational budget, we therefore reported the matched 100-step checkpoint in the original Table 6.
- Limitation of the original evidence. The original experiment was intended to compare early learning efficiency and convergence behavior rather than to claim full-training superiority. Because the alternative variants were not trained through the complete schedule at the time of submission, the 100-step results alone could not establish whether the observed gains would persist at the much lower error regime reached by the final model.
- Full-training verification under matched budgets. To directly address this concern, we now extend all tokenizer and radar-encoder variants through the matched SFT and GRPO schedules using the same data and compute budgets, and report their performance throughout training below.
Tokenizer
| Variant | SFT 100 | 500 | 1000 | 2000 | 3000 | 4000 | 4300 | GRPO 100 | 500 | 1200 |
|---|---|---|---|---|---|---|---|---|---|---|
| Standard | 1.382 | 1.210 | 0.994 | 0.851 | 0.807 | 0.864 | 0.851 | 0.896 | 0.857 | 0.802 |
| Atomic Numeric | 1.215 | 1.104 | 0.965 | 0.801 | 0.837 | 0.843 | 0.754 | 0.802 | 0.739 | 0.755 |
| Unit-Integrated | 1.214 | 0.942 | 0.601 | 0.524 | 0.547 | 0.520 | 0.512 | 0.602 | 0.587 | 0.589 |
| Atomic+Unit | 1.135 | 0.787 | 0.351 | 0.326 | 0.301 | 0.276 | 0.269 | 0.254 | 0.203 | 0.144 |
Radar Encoder
| SFT | GRPO | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Variant \ Step | 100 | 500 | 1000 | 2000 | 3000 | 4000 | 4300 | 100 | 500 | 1200 |
| Global | 1.328 | 1.135 | 0.894 | 0.882 | 0.870 | 0.858 | 0.854 | 0.842 | 0.829 | 0.813 |
| Frame | 1.191 | 1.046 | 0.865 | 0.837 | 0.808 | 0.780 | 0.771 | 0.775 | 0.749 | 0.751 |
| Polar | 1.214 | 0.936 | 0.589 | 0.593 | 0.597 | 0.601 | 0.602 | 0.593 | 0.588 | 0.556 |
| Frame + Polar | 1.176 | 0.817 | 0.368 | 0.338 | 0.308 | 0.278 | 0.269 | 0.243 | 0.231 | 0.144 |
Interpretation. The full-training results confirm that the 100-step trends are not limited to early optimization. For the tokenizer, Standard and Atomic Numeric begin to plateau after roughly 2,000–3,000 steps, while Unit-Integrated largely saturates after 1,000–2,000 steps; in contrast, Atomic+Unit continues to improve through the full SFT schedule and reaches 0.144 after GRPO. Similarly, for the radar encoder, Global and Polar largely plateau after around 1,000 steps and Frame shows only limited subsequent improvement, whereas Frame+Polar continues to improve throughout training and reaches the same 0.144 final error.
These results therefore strengthen the central methodological claim: the benefits of the proposed tokenizer and radar encoder are not confined to the high-error, early-training regime, but persist—and become more pronounced—under the matched full-training budget.
We thank the reviewer for recognizing the value of grounding driving predictions in scene-specific radar measurements and for highlighting the quantitative evidence supporting our physical-grounding design. We also thank the reviewer for raising important concerns regarding route-conditioning fairness, the contribution of rationale supervision beyond direct trajectory optimization, and whether the gains from our tokenizer and radar encoder persist after full training. We have added the corresponding experiments and analyses, which we discuss below.
W1. Evaluation under Matched Route Conditioning
We would first like to clarify that in Table 2 of our paper, RaPhy-AD, Alpamayo 1, and Alpamayo 1.5 were all provided with the same oracle LEFT/RIGHT/STRAIGHT route command.
We apologize for the ambiguity in our description and will revise Appendix F.1 to explicitly clarify how route commands were provided to each model and which navigation-conditioning option was disabled.
Nevertheless, we agree with the reviewer that differences in route conditioning can make it difficult to attribute the reported gains to radar and the proposed reasoning method. To address this concern, we additionally compare RaPhy-AD with Alpamayo 1 and Alpamayo 1.5 under a matched setting in which none of the models receives a route command.
Interpretation: Even under this stricter setting, where none of the models receives a route command, RaPhy-AD reduces the average trajectory L2 error by 43.6% relative to Alpamayo 1 and 53.9% relative to Alpamayo 1.5. The difference becomes even more pronounced at the 3s horizon, with reductions of 47.3% and 58.6%, respectively. RaPhy-AD also matches Alpamayo 1 in average collision rate (0.030%) and reduces the collision rate by 51.6% relative to Alpamayo 1.5 (0.062% → 0.030%). These results further support the effectiveness of our radar-grounded approach even in the absence of explicit route commands.