- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DriveWeave, a feed-forward model that jointly predicts metric depth, optical flow, ego-motion, and 3D Gaussians from multi-camera videos.
It introduces (1) 4D-SyncAttn to share information across views and time, and uses (2) flow-guided losses to improve depth and rendering consistency across frames.
Experiments on several driving datasets evaluate depth estimation, ego-motion, point cloud reconstruction, and rendering quality, with ablation studies on the attention design and consistency losses.
Critical strengths
The paper combines depth, optical flow, ego-motion, and Gaussian reconstruction in one feed-forward model. Table 6 shows that the consistency losses improve geometry and rendering results, supporting the value of coupling these tasks.
Critical weaknesses
None
Questions for authors
Major concerns
- I’m not sure how the depth consistency loss handles moving objects. Suppose the camera is stationary and a car moves toward it. Its depth should change, even if the depth and flow predictions are both correct. Would Eqs. (1)–(2) penalize this? Is there a mask for moving objects or some additional motion compensation?
- I found the rendering setup in Table 4 a bit unclear. Which images does the model see, and which ones does it render for evaluation? Are the target images excluded from the inputs? Do the baselines use the same images and camera information?
- Since both consistency losses rely on optical flow, I’d like to know how accurate the flow predictions are. Do you have any quantitative results for this?
Minor concerns
- In Table 5, adding synchronized registers makes the model faster, from 1.68 s to 1.17 s. Why is that? Does it replace some attention operations?
- The pose AUC tells us about rotation and translation direction, but not whether the predicted travel distance is correct. Do you also measure translation-distance or trajectory errors?
2 comments
Response to Minor Concern 1: Latency Reduction via Synchronized Registers (Table 5)
Reviewer Concern: In Table 5, adding synchronized registers makes the model faster, from 1.68 s to 1.17 s. Why is that? Does it replace some attention operations?
Our Response: Yes. Synchronized registers introduce a compact, fixed set of global memory tokens that abstract and redistribute scene context, effectively replacing dense, repetitive factorized view and temporal interactions. This substitution substantially alleviates memory bandwidth bottlenecks, reducing the core attention latency from 1.68s to 1.17s without sacrificing representational capacity.
Response to Minor Concern 2: Translation-Distance and Trajectory Errors
Reviewer Concern: The pose AUC tells us about rotation and translation direction, but not whether the predicted travel distance is correct. Do you also measure translation-distance or trajectory errors?
Our Response: Following the same evaluation protocol in Table 2 of the main paper, we additionally report RTE (Relative Translation Error) and RRE (Relative Rotation Error) in the Table below. Furthermore, as illustrated in Fig. 12 (Appendix in the main paper), we provide direct Bird's-Eye-View (BEV) trajectory distance comparisons against ground truth across both short and long horizons, confirming that DriveWeave maintains high metric trajectory scale accuracy without drift accumulation.
| Method | NuScenes RTE(cm) | NuScenes RRE() | DDAD RTE(cm) | DDAD RRE() | Waymo RTE(cm) | Waymo RRE() |
|---|---|---|---|---|---|---|
| CUT3R [1] | 103.92 | 0.71 | 36.31 | 0.61 | 31.75 | 0.40 |
| DA3 [2] | 29.26 | 0.27 | 5.93 | 0.06 | 6.87 | 0.07 |
| VGGT- [3] | 43.40 | 0.22 | 12.03 | 0.06 | 9.11 | 0.08 |
| DVGT [4] | 9.39 | 0.20 | 6.86 | 0.11 | 8.05 | 0.09 |
| DriveWeave | 6.81 | 0.17 | 2.90 | 0.05 | 1.70 | 0.05 |
[1] Wang et al. Continuous 3d perception model with persistent state. CVPR 2025.
[2] Lin et al. Depth anything 3: recovering the visual space from any views. arXiv preprint 2025.
[3] Wang et al. VGGT-Ω. CVPR 2026.
[4] Zuo et al. Dvgt: Driving visual geometry transformer. CVPR 2026.
We sincerely appreciate your valuable feedback and time. Should any further concerns or questions arise during the discussion phase, we are happy to provide additional clarifications. Thank you once more for your constructive input, and we welcome any additional comments you may have.
We sincerely thank the reviewer for the thorough and constructive feedback. Below, we provide detailed clarifications, mathematical justifications, and additional quantitative evaluations addressing moving-object temporal consistency, rendering setups, optical flow accuracy, register-based latency optimization, and trajectory metric evaluations.
Response to Major Concern 1: Handling of Independently Moving Objects in FDC
Reviewer Concern: I’m not sure how the depth consistency loss handles moving objects. Suppose the camera is stationary and a car moves toward it... Would Eqs. (1)–(2) penalize this? Is there a mask for moving objects or some additional motion compensation?
Our Response: We appreciate this sharp observation. To clarify, FDC does not assume a static background scene or rely solely on rigid camera ego-motion warping; rather, it is anchored entirely by the predicted optical flow field Ft→t+1.
Response to Major Concern 2: Rendering Setup and Baseline Parity in Table 4
Reviewer Concern: Which images does the model see, and which ones does it render for evaluation? Are the target images excluded from the inputs? Do baselines use the same images and camera information?
Our Response: We apologize for the initial ambiguity and clarify the rendering protocols for Table 4 as follows:
Response to Major Concern 3: Optical Flow Accuracy Evaluation
Reviewer Concern: Since both consistency losses rely on optical flow, I’d like to know how accurate the flow predictions are. Do you have any quantitative results for this?
Our Response: To validate the reliability of our flow backbone, we evaluated optical flow accuracy using standard metrics: AEPE (Average End-Point Error, ↓) and Fl-all (percentage of outlier pixels, ↓). As shown in Table below, DriveWeave consistently outperforms representative dynamic geometry models (such as Flow3R [1]) across both KITTI and SimBEV benchmarks.
[1] Zhongxiao Cong, Qitao Zhao, Minsik Jeon, and Shubham Tulsiani. Flow3r: Factored flow prediction for scalable visual geometry learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.438–447, June 2026.