- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
ProMac predicts a 2D trajectory from a single egocentric image containing a red goal marker. A Qwen3-VL-4B backbone is trained with an adapter that maps final hidden states at visual positions into a frozen auxiliary language model to reconstruct structured navigation rationales. A separate linear head predicts frozen visual features of a future image from those same states. Training combines trajectory, rationale, and future-feature losses. Both auxiliary branches are removed at inference. The paper derives projected trajectory tasks with four-second horizons from AND and Ego4D Fut-Loc, selecting the final valid projected waypoint as the goal. ProMac reports ADE/FDE/L2@First4/DTW of 108.40/82.01/103.46/67.45 pixels on Fut-Loc and 16.83/9.04/23.39/14.42 on AND. Evaluation measures single-image, open-loop trajectory agreement, without closed-loop execution or safety validation.
Critical strengths
-
Richer supervision during training preserves direct trajectory generation at deployment. Removing both auxiliary branches avoids explicit rationale generation or latent rollouts at deployment. Relative to trajectory-only Qwen, Fut-Loc ADE decreases from 163.13 to 108.40 and DTW from 132.47 to 67.45, with corresponding improvements on AND. Table 3 shows that either auxiliary objective is useful independently, supporting the value of richer training supervision.
-
Dataset construction and supervision sources are documented in useful detail. The appendices describe projection, visibility filtering, endpoint backoff, recording-session or official splits, and restrictions to goal-only overlays as model inputs. Future images and complete trajectories shown to human annotators are identified as privileged annotation context, and validation human annotations are excluded from the training mixture. Discussion of temporal redundancy, limitations of annotation-consistency metrics, and the distinction between open-loop agreement and safe execution helps define the scope of the evidence.
Critical weaknesses
- The action-conditioned world-modeling interpretation is not clearly supported by the actual prediction pathway.
The abstract emphasizes action-conditioned visual dynamics, but Eq. 4 and Appendix B describe a linear predictor from visual-position hidden states without explicit action conditioning. Section 3.3 states that it does not receive trajectory-token states, while Section 4.5 describes diagnostics conditioned on ground-truth intermediate trajectory tokens. The authors need to explain whether these tokens can influence the visual states being read. Under ordinary causal attention with image tokens preceding the answer, later trajectory tokens cannot affect earlier visual-position states. The described mechanism appears more directly consistent with goal-conditioned auxiliary future-feature regression than a demonstrated action-conditioned transition model.
The central combination benefit is also inconsistent. On AND, Full obtains ADE/FDE of 16.83/9.04, worse than reasoning-only at 16.09/7.57 and predictor-only at 16.13/7.43. On Fut-Loc, improvements over DriveVLA-W0 are only 109.91 to 108.40 ADE and 67.62 to 67.45 DTW. Without training-seed variation and recording-group-aware uncertainty, the additional contribution is difficult to assess. Negative nearest-later margins further indicate limited temporal discrimination in the future-state diagnostic.
- Protocol discrepancies and missing simple controls weaken interpretation of the benchmark gains.
Section 4.8 and Table 4 describe history frames used in training and inference, whereas Appendix F describes adding history only at inference to frozen single-frame-trained checkpoints. The main table refers to 834 AND examples; Appendix F states that ADE uses 64 examples per dataset. The K=0 values exactly match the main table while being described as an archived reference. These protocols answer different questions, and it is unclear which produced the reported numbers. Consequently, the deterioration cannot currently be attributed to redundant frames.
The goal is directly supplied, and Appendix E finds 96.8% of AND image-space trajectories to be near-straight. Nevertheless, goal interpolation, a mean-start-to-goal path, and a lightweight goal-conditioned trajectory regressor are missing. ADE/FDE are computed after truncation to the shared prediction/target length, which may conceal prematurely terminated predictions; the reported FDE need not correspond to the actual target endpoint. Finally, Section 4.9 measures rationale-decoding latency even though deployment omits rationale decoding, so it does not establish end-to-end trajectory-generation efficiency.
Questions for authors
The following questions seek clarification and validation of the central conclusions, and the answers could change my rating.
-
What conditions the future predictor? Please provide sequence order, attention masks, readout positions, and the checkpoint used for the diagnostic. How do ground-truth trajectory tokens in Section 4.5 influence the visual states in Eq. 4? If they do not, please revise the action-conditioning claims and compare explicit action-conditioned prediction against the unconditioned version.
-
Which protocol produced the history results? Please reconcile Section 4.8, Table 4, and Appendix F regarding training, K values, sample counts, and reference models. Report history-trained models separately from inference-only history insertion using the same evaluation subset.
-
Do the gains exceed simple geometric predictors and the closest method reliably? Could you add straight-line goal interpolation, a training-mean-start-to-goal path, and a lightweight goal-conditioned regressor, with results stratified by path curvature? Please report multiple seeds and recording-group-aware paired intervals for Full, individual branches, and DriveVLA-W0.
-
How are output length and deployment cost handled? Please report invalid outputs, short trajectories, and length distributions, then reevaluate on a fixed temporal grid or penalize missing waypoints. Please provide trajectory-only inference latency, memory, and training cost under matched hardware, resolution, and decoding settings, and specify baseline training budgets.
Please check authors' rebuttal
Dear reviewers,
As the rebuttal period is coming to an end, I kindly remind you to check the authors' responses. If you have not yet responded, please indicate whether your concerns have been adequately addressed. Many thanks!
Best, AC