- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Drive4RT learns a queryable 4D scene representation from short surround-view driving clips. A query identifies a source pixel, source camera/time, target time, and reference camera, and the model returns the metric 3D position of the corresponding physical point. The encoder factorizes interaction into intra-view, cross-view, and cross-frame stages. A source-anchored decoder retrieves query-specific features, while a structured motion module decomposes trajectories into a static-background slot, foreground motion prototypes, and point-level residuals. Training supervision is derived from standard driving logs: LiDAR and ego poses supervise static geometry, while LiDAR points inside tracked 3D boxes are propagated using box poses to supervise dynamic trajectories. The model is jointly trained on five driving datasets and evaluated on reconstruction, ray depth, ego pose, cross-view queries, and point tracking, with an additional zero-shot KITTI evaluation.
Critical strengths
- The conditional point-query interface is appealing: reconstruction, cross-view coordinate transfer, and temporal tracking can be exposed through one representation rather than separate heads.
- Factorizing surround-view processing into intra-view, cross-view, and cross-frame interactions is well motivated and substantially outperforms unrestricted global attention in the ablation.
- The static/dynamic supervision construction is practical and uses annotations commonly available in autonomous-driving logs instead of requiring dense manual point trajectories.
Critical weaknesses
- The quantitative experimental evaluation results are not convincing. E.g., in Tab.1 & Tab.2, the performance of Drive4RT is not as good as the baseline methods, which needs to be discussed. In addition, there is a significant discrepancy between the experimental data of DVGT in Tab1 and the data in its original paper, which fundamentally raises my doubts about the effectiveness of Drive4RT.
- The methodological novelty appears limited. The source-point/target-time/reference-frame query and independent decoding closely follow D4RT, while the intra-view/cross-view/cross-frame encoder closely follows DVGT; the geometry-plus-motion querying formulation also substantially overlaps with 4RC. The remaining contribution is mainly a driving-specific integration with motion slots and box-derived supervision. Without a direct comparison to 4RC or controlled D4RT-plus-DVGT baselines, it is unclear whether this constitutes a substantive methodological advance beyond combining existing designs.
- It is unclear whether the motion slots learn an actor-specific decomposition. Assigning every dynamic point the same uniform foreground-slot distribution makes the within-object grouping and temporal-consistency losses zero and minimizes the slot-balancing loss, while assigning no distinct slot to any actor. Although the trajectory-position loss and residual penalty may indirectly prevent this behavior, the paper provides no slot-assignment visualization, actor-slot purity/entropy analysis, or slot-versus-residual motion decomposition. Therefore, the interpretation that the slots capture actor-level motion is not sufficiently supported by the current experiments.
Questions for authors
- The DVGT results in Tab.1 do not match its original paper and appear misaligned across datasets. Were they re-evaluated under a different protocol or reported incorrectly? Please verify the numbers and revise the related conclusions.
- Drive4RT underperforms the strongest baselines on several reconstruction and depth benchmarks in Tab.1 and Tab.2. Please discuss these regressions and clarify the claimed overall advantage.
- The slot losses may not explicitly prevent all actors from using the same uniform slot distribution. Additional discussions or ablations are critical to this statement.
- It's encouraged to ablate the slot number , slot-only versus residual-only motion, and the individual slot losses.
- The improvement in tracking capability in Tab.3 may come from unfair comparisons with baseline methods, such as introducing additional training data, extra supervision items, etc. Please clarify the detailed impenmation of those baseline methods to control variable.
- The tracking targets are synthesized by rigidly propagating LiDAR points using annotated 3D-box poses, following the same assumption used for training supervision. How does this evaluation distinguish physical point tracking from object-level box-motion estimation? Please include an oracle or predicted-box rigid-motion baseline.