- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Auto-JEPA, a driving planner that predicts a latent representation of the future ego trajectory from visual observations, ego history, and navigation commands. The predicted representation retrieves candidates from a fixed trajectory memory, followed by scene scoring and drivable area filtering. Experiments cover NAVSIM v1 and v2, with additional component ablations and visual occlusion analyses.
Critical strengths
-
The retrieval formulation provides a clear alternative to learned trajectory generation. A shared trajectory embedding connects visual prediction with executable candidate geometry, and the complete system achieves competitive performance using only a front camera.
-
The component ablations help explain the planning pipeline. The paper examines intent prediction, candidate scoring, feasibility gating, and retrieval pool size. The controlled occlusion analysis also provides evidence that predictions depend on dynamic agent information.
Critical weaknesses
-
The benefit of the proposed latent prediction formulation is not sufficiently isolated. The fixed medoid baseline removes scene conditioning entirely, rather than comparing alternative prediction objectives. Comparisons with direct waypoint prediction followed by retrieval, or direct candidate ranking using the same backbone, memory, and scorer, are needed to establish the specific contribution of joint embedding prediction. Predicting only the ego trajectory also leaves the claimed distinction from a conventional imitation planner unclear.
-
The evaluation does not establish a consistent performance or efficiency advantage. Under the original NAVSIM v2 evaluator, Auto-JEPA scores 85.6, below several reported baselines; its updated score of 89.1 is not directly comparable with their original scores. The paper also lacks reactive closed loop evaluation and computational measurements, despite using a 24 layer predictor, a large trajectory memory, and separate selection modules.
Questions for authors
-
What capability makes Auto-JEPA a world model beyond a planner trained to predict an embedding of the demonstrated ego trajectory?
-
How does latent prediction compare with direct waypoint prediction followed by retrieval, and with direct candidate ranking, using the same visual backbone, memory, scorer, and training budget?
-
Could the authors compare against the CLOVER generator using the same scorer and candidate budget, and evaluate strong baselines under the same NAVSIM v2 implementation?
-
How are the 110,335 memory trajectories selected, and are all entries restricted to training data? How do memory size and coverage affect retrieval quality and performance on rare maneuvers?
-
What are the total parameter count, memory footprint, and batch size one latency, including visual encoding, retrieval, scoring, and gating?