- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper extends VGGT-World with action and semantic conditioning. Two historical geometry frames, supplied trajectories and derived maneuver categories, and Qwen2.5-VL features from three camera views and text prompts condition flow-based prediction of geometry tokens at 0.5 and 1.0 seconds. A VGGT decoder produces future depth and 3D structure. On 3,000 maneuver-balanced NAVSIM clips, evaluated against VGGT-derived pseudo-ground truth, RMSE decreases by 13.7% and δ1 increases by 4.11 percentage points over VGGT-World, while AbsRel slightly worsens. Validation is limited to offline conditional geometry prediction; planning and closed-loop results are not reported.
Critical strengths
-
Action-conditioned geometry prediction is well motivated. Predicting the spatial consequences of supplied actions is a useful direction for candidate-action evaluation. Preserving individual waypoints alongside a maneuver category also has empirical support: the joint representation improves most decoded geometry metrics in Table 2.
-
Prediction gains and action utilization have complementary support. RMSE decreases by 15.7% against the NAVSIM-adapted VGGT-World-N baseline (Table 1), partially addressing gains from domain adaptation alone. Controlled interventions with fixed history, semantics, and initial noise, together with the additional multi-scene visualizations, support actual use of action conditions.
Critical weaknesses
-
Evaluation-data and condition provenance remain unclear. Neither the main paper nor the supplement specifies the log- or scene-level training/evaluation split or the relationship between the 3,000 evaluation clips and training data. It also remains unclear whether the “supplied action” is a recorded trajectory or a model prediction, and how system/user prompts are constructed. These details directly affect the interpretation of generalization and the conditional-prediction task. Recorded trajectories are valid conditions, but their use should be explicit.
-
Geometric control and the semantic contribution require more targeted validation. Table 3 and Figures A2–A4 support action responsiveness, but do not quantify trajectory-consistent camera motion or gains over static reprojection. Table 4 shows consistent but modest semantic-branch gains, including a 0.84% RMSE reduction and a 0.73% $\delta$1 increase. Although C.3 matches the training budget, this branch also introduces three camera views, whereas the geometry branch uses single-camera history. The additional latent metrics do not isolate semantic understanding from wider visual coverage or feature capacity.
Questions for authors
-
Please specify the log-/scene-level training and evaluation split and how the 3,000 evaluation clips are separated from training. Are the supplied trajectories recorded, VLA-predicted, or obtained elsewhere? How are system/user prompts constructed, and what information do they use?
-
Can you quantify consistency between supplied trajectories and predicted camera motion, or compare against static reprojection on mutually visible regions using the same history and actions? This would clarify the control capability illustrated by the additional qualitative results.
-
With matched three-camera inputs, training budget, and adapter configuration, do the Table 4 gains persist when ordinary visual features replace VLM features? How large are these differences relative to training or sampling variability?