- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Drive$R^3$, a method for the feed-forward reconstruction of dynamic Gaussian scenes using multi-view images and LiDAR contextual observations. Building upon image-aligned Gaussians, the approach refines shared geometry through spatio-temporal LiDAR–Gaussian association, predicts distinct opacities for cameras and LiDAR, and estimates LiDAR intensity by fusing various contextual returns. Experiments conducted on the nuScenes dataset demonstrate that this method offers improvements over STORM.
Critical strengths
- The issue is important, and the system objectives are clear.
- The experimental results, are excellent, especially the LiDAR results.
- Better computational power-saving efficiency than existing methods.
Critical weaknesses
- The paper attributes the performance gains to the improvement of shared geometry through cross-modal supervision; however, since Drive$R^3$ also introduces changes to feature fusion, geometric refinement, prediction heads, and the training regimen, the observed performance delta cannot be ascribed solely to LiDAR supervision or spatiotemporal correlation.
- A key design of Drive$R^3$ is the static setting of voxel size, neighborhood radius and time window, this setting may lead to failing in rapid moving scenes or occlusion happens.
Questions for authors
-
How do motion factors influence the results?
-
How does performance change when LiDAR geometric supervision or spatiotemporal correlation is added individually?
9 comments
Response to Reviewer r1Je (Part 1/4): Weakness 1
Weakness 1. The paper attributes the performance gains to the improvement of shared geometry through cross-modal supervision. However, since DriveR³ also introduces changes to feature fusion, geometric refinement, prediction heads, and the training regimen, the observed performance delta cannot be ascribed solely to LiDAR supervision or spatiotemporal correlation.
Thank you for this helpful comment. We provide a retrained ablation and a separate experiment with fixed model weights to clarify the roles of LiDAR geometry supervision and input spatiotemporal association.
(1) LiDAR geometry supervision and spatiotemporal association. We compare the original STORM model, a model trained with RGB and LiDAR geometry supervision but without input LiDAR association, and full DriveR³ on the same nuScenes validation scenes.
As shown in Table 1, the supervised model without association reduces range MAE from 1.79 m to 1.65 m and range RMSE from 4.91 m to 4.28 m compared with STORM. Depth L2 and squared CD also decrease. These results indicate that training with LiDAR geometry supervision helps improve the accuracy of geometry predicted from visual inputs.
Compared with the supervised model without association, full DriveR³ further improves all four geometry metrics. Range MAE decreases from 1.65 m to 1.46 m, range RMSE from 4.28 m to 4.05 m, Depth L2 from 0.12 m² to 0.08 m², and squared CD from 4.00 m² to 2.90 m². These results support combining LiDAR geometry supervision with spatiotemporal association, using geometric constraints during training and input observations during scene construction.
Table 1: Ablation of LiDAR geometry supervision and spatiotemporal association on nuScenes validation scenes
| Method | LiDAR Geometry Supervision | Spatiotemporal Association | Range MAE (m) ↓ | Range RMSE (m) ↓ | Depth L2 (m²) ↓ | Squared CD (m²) ↓ |
|---|---|---|---|---|---|---|
| Baseline (STORM) | ✗ | ✗ | 1.79 | 4.91 | 0.14 | 4.72 |
| Trained without association | ✓ | ✗ | 1.65 | 4.28 | 0.12 | 4.00 |
| DriveR³ | ✓ | ✓ | 1.46 | 4.05 | 0.08 | 2.90 |
(2) Association with a fixed model. To further assess the role of input LiDAR association, we evaluate the same trained model on nuScenes validation scenes with association enabled or disabled at inference. Model weights, feature fusion, prediction heads, and the geometry update mechanism remain unchanged.
As shown in Table 2, enabling association reduces range MAE from 2.79 m to 1.96 m and Depth L2 from 1.15 m² to 0.23 m², with improvements across all four geometry metrics. These results show that the trained model uses current LiDAR observations to improve geometric reconstruction. This inference-time experiment evaluates reliance on the association pathway, while the retrained comparison in Table 1 evaluates the benefit of adding association under geometry supervision.
Table 2: Input LiDAR association enabled or disabled at inference on nuScenes validation scenes
| Input LiDAR Association | Range MAE (m) ↓ | Range RMSE (m) ↓ | Depth L2 (m²) ↓ | Squared CD (m²) ↓ |
|---|---|---|---|---|
| Disabled | 2.79 | 5.72 | 1.15 | 5.01 |
| Enabled | 1.96 | 4.91 | 0.23 | 4.11 |
Tables 1 and 2 use different evaluation samples. Their absolute metrics therefore should be interpreted within each experiment.
Response to Reviewer r1Je (Part 2/4): Weakness 2
Weakness 2. A key design of DriveR³ is the static setting of voxel size, neighborhood radius and time window. This setting may lead to failure in rapidly moving scenes or when occlusion occurs.
Thank you for raising this concern. We analyze the effects of motion speed and visibility and compare the fixed configuration with adaptive alternatives.
(1) Effect of motion speed. We group vehicles by speed on nuScenes validation scenes and compare DriveR³ with STORM, our visual reconstruction baseline. The mean number of associated voxel centroids is computed over Gaussians inside the source vehicle 3D boxes.
As shown in Table 3, this number decreases from 66.91 in the low-speed group to 17.01 in the high-speed group, while the PSNR gain decreases from 0.275 dB to 0.061 dB. Although vehicle-region PSNR decreases with increasing speed for both methods, DriveR³ maintains a positive mean gain over STORM in every speed group under the current fixed settings. These results indicate that rapid motion reduces local association support and the resulting reconstruction gain.
Table 3: Vehicle-region PSNR gains over STORM and association counts for low- and high-speed groups
| Vehicle Speed (m/s) | PSNR Gain over STORM (dB) | Mean Number of Associated Voxel Centroids |
|---|---|---|
| ≤0.5 | 0.275 | 66.91 |
| >10 | 0.061 | 17.01 |
(2) Effect of visibility. We further group the results by keyframe visibility labels to examine association support under occlusion. As shown in Table 4, association support is 0.923 in the high-visibility group and 0.487 in the low-visibility group. These results indicate that geometric refinement receives weaker support from local measurements under low visibility.
Table 4: Association support across keyframe visibility groups
| Keyframe Visibility (%) | Association Support |
|---|---|
| 80–100 | 0.923 |
| 0–40 | 0.487 |
(3) Fixed versus adaptive settings. We keep the model weights fixed and separately evaluate density-adaptive voxelization and a speed-adaptive temporal kernel. Relative to the original fixed configuration, each alternative changes the mean PSNR in dynamic regions by less than 0.01 dB. Neither adaptive setting provides a clear improvement under the current model and evaluation conditions.
These analyses show that motion and occlusion affect the available association support. The fixed configuration retains positive reconstruction gains in the evaluated speed groups, while rapid motion and low visibility remain challenging conditions.
Response to Reviewer r1Je: General Response
Thank you for your careful review and for recognizing the research value of efficient dynamic scene reconstruction and the clarity of our system objectives. We particularly appreciate your recognition of DriveR³’s reconstruction results, especially its LiDAR reconstruction performance, and its computational efficiency advantages over existing methods. Below, we address your concerns about the sources of the performance gains and the effectiveness of fixed association settings under rapid motion and occlusion through additional experiments and analyses.