- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes DrivENVS, a framework for novel-view synthesis from calibrated monocular driving videos. It reconstructs a motion-aware point cloud using estimated depth and object tracks, then renders coarse views along a target camera trajectory. A four-step image diffusion enhancer refines these renderings autoregressively using previously generated frames and geometrically selected source references. The highest-ranked reference is processed through joint attention, while additional references are incorporated through cross-attention with relative camera and time conditioning. Experiments on nuScenes, Waymo, and Argoverse 2 show improvements over image-based baselines in video quality and several reconstruction metrics, with lower reported inference memory than NeoVerse.
Critical strengths
- The proposed method is simple and efficient, the experiments demonstrate its effectiveness compared to baseline methods.
- The paper is well-written and easy to understand.
Critical weaknesses
- The paper emphasizes its advantages over video-based enhancement, including in the abstract, but the comparison with video models is not sufficiently controlled. The comparison uses a 2B image model and a 14B NeoVerse model with different resolutions and training settings, while Table 11 shows lower memory usage but also lower throughput. I would recommend additional comparisons with relevant video-based methods such as ReCamDriving and ReCamMaster, including smaller models, reporting generation quality, inference time, and peak GPU memory under comparable settings.
- The method relies on estimated depth, object tracking, and a fixed motion threshold for both coarse rendering and reference selection, but its robustness to errors in these components is not sufficiently evaluated. I would recommend that the authors provide controlled ablations and moving-object evaluations to clarify how reconstruction errors affect object appearance, position, and motion in the generated videos.
Questions for authors
- Could the authors provide a comparison with a smaller video-based NVS model at matched resolution and sequence length, reporting quality, throughput, and peak GPU memory?
- How sensitive is the method to depth errors, tracking errors, and the motion threshold, particularly for moving vehicles and pedestrians?
4 comments
Response to Reviewer s5TR (3/3)
C3. Robustness to reference selection needs more evaluation
The result of ablating the reference selection is reported in Tables 8–9 in the main paper. We also ablated the number of reference frames. To summarize, increasing the reference count from one to eight reduces FVD from 173.20 to 150.52. Our pose-aware strategy achieves FVD = 150.52, which outperforms a simpler selection strategy, which has FVD = 170.64.
Follow-up response to Reviewer s5TR
Dear Reviewer s5TR,
Thank you again for your helpful suggestions.
We have added comparisons with smaller video models at matched resolution and sequence length (Table 5), together with motion-threshold and alternative depth/segmentation/tracking evaluations (Tables 7 and 15).
Before the discussion period ends, we would appreciate hearing your assessment of these additions and whether any specific concerns remain. We would be happy to discuss them further.
The Authors
Hello, we are approaching the discussion deadline. Can you please provide your thoughts and comments?
Response to Reviewer s5TR (1-2/3)
C1. Request to compare with a smaller video model at matched resolution and sequence length.
Please see our response to a similar question to Reviewer QiWQ C2 and C4 and the new results we have reported in Tables A and B above. We re-did the baseline comparison based on your feedback and added two new small-video models (StreetCrafter, ReCamMaster -- as the reviewer suggested) and compared to NeoVerse at matched resolution and sequence length.
ReCamDriving. We tested ReCamDriving using GaussianStorm for feed-forward driving reconstruction. We found that the model can only generate videos at eight predefined lateral camera shifts: 1, 2, 3, and 4 m to left and right. The model does not support vertical camera shift or camera rotation. As the released models/code does not generalize to arbitrary camera shifts, we excluded it in the comparison.
C2. Sensitivity to depth errors, motion threshold, etc.
Please see our response to Reviewer QiWQ's C5 question about depth estimation sensitivity, etc. We have reported the new results in Tables C and D above.