acceptodds
Under review as a conference paper at ICLR 2027

DriveR: Feed-Forward Gaussian Reconstruction for Camera-LiDAR Simulation in Autonomous Driving

Abstract

Cameras and LiDAR provide complementary appearance and geometric information for autonomous driving, making their joint simulation essential for realistic evaluation of multi-sensor perception systems. However, existing joint reconstruction methods typically rely on time-consuming per-scene optimization, while efficient feed-forward reconstruction approaches primarily target visual geometry recovery and image synthesis, leaving joint camera-LiDAR simulation underexplored. To address this gap, we introduce DriveR, a feed-forward framework that directly reconstructs a unified dynamic Gaussian scene representation from sparse camera and LiDAR observations. Our key design establishes spatiotemporal LiDAR-Gaussian associations, enabling shared dynamic Gaussian geometry to be jointly learned from complementary image reconstruction and LiDAR range supervision. DriveR further predicts Gaussian-level LiDAR intensity by adaptively fusing multi-source intensity cues. Experiments on nuScenes demonstrate that DriveR enables feed-forward joint camera-LiDAR reconstruction while also improving visual reconstruction quality, highlighting the benefit of complementary sensor information. Compared with the per-scene optimization method SplatAD, DriveR achieves better results on multiple LiDAR reconstruction metrics and an approximately 10x speedup in scene reconstruction.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Feed-forward 3DGS, Autonomous driving

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_r1JeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Drive$R^3$, a method for the feed-forward reconstruction of dynamic Gaussian scenes using multi-view images and LiDAR contextual observations. Building upon image-aligned Gaussians, the approach refines shared geometry through spatio-temporal LiDAR–Gaussian association, predicts distinct opacities for cameras and LiDAR, and estimates LiDAR intensity by fusing various contextual returns. Experiments conducted on the nuScenes dataset demonstrate that this method offers improvements over STORM.

Critical strengths
  1. The issue is important, and the system objectives are clear.
  2. The experimental results, are excellent, especially the LiDAR results.
  3. Better computational power-saving efficiency than existing methods.
Critical weaknesses
  1. The paper attributes the performance gains to the improvement of shared geometry through cross-modal supervision; however, since Drive$R^3$ also introduces changes to feature fusion, geometric refinement, prediction heads, and the training regimen, the observed performance delta cannot be ascribed solely to LiDAR supervision or spatiotemporal correlation.
  2. A key design of Drive$R^3$ is the static setting of voxel size, neighborhood radius and time window, this setting may lead to failing in rapid moving scenes or occlusion happens.
Questions for authors
  1. How do motion factors influence the results?

  2. How does performance change when LiDAR geometric supervision or spatiotemporal correlation is added individually?

9 comments
AuthorsOpenReview ↗

Response to Reviewer r1Je: General Response

Thank you for your careful review and for recognizing the research value of efficient dynamic scene reconstruction and the clarity of our system objectives. We particularly appreciate your recognition of DriveR³’s reconstruction results, especially its LiDAR reconstruction performance, and its computational efficiency advantages over existing methods. Below, we address your concerns about the sources of the performance gains and the effectiveness of fixed association settings under rapid motion and occlusion through additional experiments and analyses.

AuthorsOpenReview ↗

Response to Reviewer r1Je (Part 1/4): Weakness 1

Weakness 1. The paper attributes the performance gains to the improvement of shared geometry through cross-modal supervision. However, since DriveR³ also introduces changes to feature fusion, geometric refinement, prediction heads, and the training regimen, the observed performance delta cannot be ascribed solely to LiDAR supervision or spatiotemporal correlation.

Thank you for this helpful comment. We provide a retrained ablation and a separate experiment with fixed model weights to clarify the roles of LiDAR geometry supervision and input spatiotemporal association.

(1) LiDAR geometry supervision and spatiotemporal association. We compare the original STORM model, a model trained with RGB and LiDAR geometry supervision but without input LiDAR association, and full DriveR³ on the same nuScenes validation scenes.

As shown in Table 1, the supervised model without association reduces range MAE from 1.79 m to 1.65 m and range RMSE from 4.91 m to 4.28 m compared with STORM. Depth L2 and squared CD also decrease. These results indicate that training with LiDAR geometry supervision helps improve the accuracy of geometry predicted from visual inputs.

Compared with the supervised model without association, full DriveR³ further improves all four geometry metrics. Range MAE decreases from 1.65 m to 1.46 m, range RMSE from 4.28 m to 4.05 m, Depth L2 from 0.12 m² to 0.08 m², and squared CD from 4.00 m² to 2.90 m². These results support combining LiDAR geometry supervision with spatiotemporal association, using geometric constraints during training and input observations during scene construction.

Table 1: Ablation of LiDAR geometry supervision and spatiotemporal association on nuScenes validation scenes

MethodLiDAR Geometry SupervisionSpatiotemporal AssociationRange MAE (m) ↓Range RMSE (m) ↓Depth L2 (m²) ↓Squared CD (m²) ↓
Baseline (STORM)✗✗1.794.910.144.72
Trained without association✓✗1.654.280.124.00
DriveR³✓✓1.464.050.082.90

(2) Association with a fixed model. To further assess the role of input LiDAR association, we evaluate the same trained model on nuScenes validation scenes with association enabled or disabled at inference. Model weights, feature fusion, prediction heads, and the geometry update mechanism remain unchanged.

As shown in Table 2, enabling association reduces range MAE from 2.79 m to 1.96 m and Depth L2 from 1.15 m² to 0.23 m², with improvements across all four geometry metrics. These results show that the trained model uses current LiDAR observations to improve geometric reconstruction. This inference-time experiment evaluates reliance on the association pathway, while the retrained comparison in Table 1 evaluates the benefit of adding association under geometry supervision.

Table 2: Input LiDAR association enabled or disabled at inference on nuScenes validation scenes

Input LiDAR AssociationRange MAE (m) ↓Range RMSE (m) ↓Depth L2 (m²) ↓Squared CD (m²) ↓
Disabled2.795.721.155.01
Enabled1.964.910.234.11

Tables 1 and 2 use different evaluation samples. Their absolute metrics therefore should be interpreted within each experiment.

AuthorsOpenReview ↗

Response to Reviewer r1Je (Part 2/4): Weakness 2

Weakness 2. A key design of DriveR³ is the static setting of voxel size, neighborhood radius and time window. This setting may lead to failure in rapidly moving scenes or when occlusion occurs.

Thank you for raising this concern. We analyze the effects of motion speed and visibility and compare the fixed configuration with adaptive alternatives.

(1) Effect of motion speed. We group vehicles by speed on nuScenes validation scenes and compare DriveR³ with STORM, our visual reconstruction baseline. The mean number of associated voxel centroids is computed over Gaussians inside the source vehicle 3D boxes.

As shown in Table 3, this number decreases from 66.91 in the low-speed group to 17.01 in the high-speed group, while the PSNR gain decreases from 0.275 dB to 0.061 dB. Although vehicle-region PSNR decreases with increasing speed for both methods, DriveR³ maintains a positive mean gain over STORM in every speed group under the current fixed settings. These results indicate that rapid motion reduces local association support and the resulting reconstruction gain.

Table 3: Vehicle-region PSNR gains over STORM and association counts for low- and high-speed groups

Vehicle Speed (m/s)PSNR Gain over STORM (dB)Mean Number of Associated Voxel Centroids
≤0.50.27566.91
>100.06117.01

(2) Effect of visibility. We further group the results by keyframe visibility labels to examine association support under occlusion. As shown in Table 4, association support is 0.923 in the high-visibility group and 0.487 in the low-visibility group. These results indicate that geometric refinement receives weaker support from local measurements under low visibility.

Table 4: Association support across keyframe visibility groups

Keyframe Visibility (%)Association Support
80–1000.923
0–400.487

(3) Fixed versus adaptive settings. We keep the model weights fixed and separately evaluate density-adaptive voxelization and a speed-adaptive temporal kernel. Relative to the original fixed configuration, each alternative changes the mean PSNR in dynamic regions by less than 0.01 dB. Neither adaptive setting provides a clear improvement under the current model and evaluation conditions.

These analyses show that motion and occlusion affect the available association support. The fixed configuration retains positive reconstruction gains in the evaluated speed groups, while rapid motion and low visibility remain challenging conditions.

Reviewer_1TocOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents DriveR³, a feed-forward framework that constructs a shared dynamic Gaussian scene from multi-view images and LiDAR observations to render camera images, LiDAR ranges, and intensities. The method has clear practical value and demonstrates faster scene construction and improvements in several LiDAR reconstruction metrics. However, some central claims require stronger experimental support.

Critical strengths
  1. Well-motivated representation. Shared geometry with sensor-specific responses provides a reasonable foundation for joint camera–LiDAR reconstruction.
  2. Efficient scene construction. The method avoids per-scene optimization at test time and substantially reduces construction time relative to the evaluated optimization-based methods.
  3. Complementary sensor outputs. Supporting camera images, LiDAR ranges, and intensities makes the framework relevant to multi-sensor simulation.
  4. Promising LiDAR reconstruction results. Improvements in several range, intensity, and point-cloud metrics demonstrate useful reconstruction quality alongside the efficiency gains.
Critical weaknesses
  1. The association mechanism is not sufficiently validated against the dynamic correspondence problem that motivates it. The central motivation is to associate sparse, asynchronous LiDAR measurements with dynamic visual Gaussians. However, Eq. (2) weights observations by spatial proximity and acquisition-time difference without explicitly compensating for object motion before association. Although velocity enters the subsequent descriptor, it does not enter the neighborhood search or association weights. Moreover, the 1 m association radius and support-gated center correction may limit recovery when motion or initial visual geometry errors place the correct evidence outside the neighborhood. These observations do not establish that the method fails, but they leave its effectiveness on moving objects insufficiently justified. Please report results by motion state, speed, and temporal offset, and compare retrained spatial-only, spatiotemporal, and motion-compensated association variants to clarify what the proposed mechanism resolves.
  2. The independent value of shared geometry and explicit association remains unclear. The experiments support the complete system, but do not adequately separate the benefits of additional LiDAR input and supervision from those of explicit association and shared Gaussian geometry. Sharing positions, covariances, and motion imposes a useful structural constraint, yet sensor-specific opacities can still make the two renderers rely on different subsets of primitives. The opacity anchor constrains this flexibility, but cross-sensor consistency is not directly evaluated. In addition, the association and opacity ablations modify a trained model only at inference, so they measure reliance on those pathways rather than their advantage over alternatives trained for the same task. Matched-input, matched-supervision retrained controls for simple fusion and separate versus shared geometry, together with alignment and occlusion-consistency evaluation on mutually visible surfaces, would better establish the distinctive contribution.
  3. The evaluation does not yet establish complete sensor simulation or unambiguously attribute range gains to geometry. Appendix B.1 primarily evaluates temporal interpolation along recorded observations, which is a valid reconstruction setting but does not establish robustness to the novel sensor poses emphasized by the simulation motivation. Appendix B.2 further evaluates observed valid returns and computes Chamfer distance from point clouds back-projected along common target-return directions. This protocol does not fully assess false returns, missed returns, or scan completeness. A related concern is that geometry training uses a soft opacity-crossing range, whereas evaluation uses unnormalized accumulated depth. The latter depends on both primitive locations and total compositing opacity, so lower range error alone does not unambiguously demonstrate more accurate surface geometry. Please explain this readout choice and baseline comparability, report opacity accumulation and alternative surface-oriented readouts, and evaluate complete scans and controlled sensor-pose perturbations. These checks are important for interpreting the reported geometric gains and the scope of the simulation claims.
Questions for authors
  1. How does association performance vary with object motion, temporal offset, and initial geometry error, and how does it compare with retrained spatial-only and motion-compensated alternatives?
  2. Under matched inputs and supervision, what do explicit association and shared geometry contribute beyond simple fusion, and what evidence demonstrates cross-sensor consistency?
  3. Why does evaluation use a different range readout from training, and do the conclusions hold with alternative readouts, complete-scan evaluation, and perturbed sensor poses?
8 comments
AuthorsOpenReview ↗

Response to Reviewer 1Toc: General Response

Thank you for your careful review and for recognizing the practical value of DriveR³ for efficient joint camera–LiDAR reconstruction. We particularly appreciate your recognition of the representation design that combines shared geometry with sensor-specific responses, as well as the method’s ability to provide complementary camera and LiDAR outputs without per-scene optimization while achieving good LiDAR reconstruction quality. Below, we address your concerns about the effectiveness of dynamic association, the independent contributions of explicit association and shared geometry, and range evaluation and sensor-simulation capability through additional experiments and analyses.

AuthorsOpenReview ↗

Response to Reviewer 1Toc (Part 1/6): Weakness 1

Weakness 1. The association mechanism is not sufficiently validated against the dynamic correspondence problem that motivates it. Please report results by motion state, speed, and temporal offset, and compare retrained spatial-only, spatiotemporal, and motion-compensated association variants to clarify what the proposed mechanism resolves.

Thank you for this valuable suggestion. We provide additional analyses of object motion, acquisition-time differences, and initial geometry errors, together with comparisons against separately retrained association alternatives.

(1) Effect of object motion. Even after ego-pose alignment, object motion can cause spatially nearby observations captured at different times to correspond to different surface states. On the original 50 nuScenes validation scenes, DriveR³ improves PSNR in moving-object regions from STORM’s 19.5 dB to 20.3 dB, with positive mean gains in every speed group.

As shown in Table 1, the high-speed group has fewer associated voxel centroids and a smaller PSNR gain. These results indicate that rapid motion reduces the available local association support, while DriveR³ retains a positive mean gain over STORM in both reported groups.

Table 1: Vehicle-region PSNR gains and association counts for low- and high-speed groups

Vehicle speed (m/s)PSNR gain over STORM (dB)Mean number of associated voxel centroids
≤0.50.27566.9
>100.06117.0

(2) Effect of acquisition-time differences. The model downweights observations with larger time offsets to reduce association mismatches caused by motion or changes in occlusion. As shown in Table 2, DriveR³ achieves higher mean PSNR in vehicle regions than STORM across all four effective association time-offset groups.

Table 2: Vehicle-region PSNR gains across effective association time-offset groups

Effective association time offset (s)Mean PSNR gain over STORM (dB) ↑
≤0.150.856
(0.15, 0.30]0.277
(0.30, 0.60]0.331
>0.600.150

(3) Effect of initial geometry errors. Initial geometry errors can shift the query center away from the true surface, affecting which local observations are selected and how they are weighted. The model applies a bounded correction to the Gaussian center based on observation support and retains its initial position when no associated observations are available. This mechanism limits unsupported geometric updates, although it cannot guarantee recovery when the correct evidence lies outside the local neighborhood.

(4) Comparison of association schemes. Using the same STORM visual backbone, we separately retrain the spatial-only and motion-compensated variants while retaining LiDAR geometry supervision and the subsequent fusion and prediction architecture.

As shown in Table 3, full spatiotemporal association achieves the best joint RGB and LiDAR reconstruction results among the three variants. It reduces range RMSE by approximately 5.8% and LPIPS by approximately 8.5% compared with both alternatives. CD decreases by approximately 15.0% and 16.2% relative to spatial-only and motion-compensated association, respectively. These results demonstrate the benefit of the full association scheme under the compared training conditions.

Table 3: Reconstruction results for three association schemes

AssociationPSNR ↑SSIM ↑LPIPS ↓Range MAE (m) ↓Range RMSE (m) ↓Depth L2 (m²) ↓CD (m²) ↓
Spatial-only22.30.6700.2701.604.300.0973.41
Motion-compensated22.30.6700.2701.604.300.0973.46
Full spatiotemporal22.70.6820.2471.464.050.0802.90
AuthorsOpenReview ↗

Response to Reviewer 1Toc (Part 2/6): Weakness 2

Weakness 2. The independent value of shared geometry and explicit association remains unclear. Matched-input, matched-supervision retrained controls for simple fusion and separate versus shared geometry, together with alignment and occlusion-consistency evaluation on mutually visible surfaces, would better establish the distinctive contribution.

Thank you for this helpful suggestion. We provide separately trained controls and directly evaluate cross-sensor consistency to clarify the contributions of shared geometry and explicit association.

(1) Role of shared geometry. Shared geometry allows information from associated LiDAR observations to support both camera and LiDAR reconstruction. We compare DriveR³ with an independent-geometry variant that uses separate Gaussian representations for cameras and LiDAR. DriveR³ improves PSNR by approximately 0.8 dB and reduces range RMSE and intensity RMSE by 49.4% and 32.0%, respectively.

We further evaluate cross-sensor consistency on mutually visible static surfaces. We back-project normalized expected depths from both rendering branches into a common 3D coordinate system. We measure the discrepancy between corresponding surfaces and the camera-surface error relative to ground truth.

As shown in Table 4, DriveR³ reduces cross-sensor surface discrepancy from 1.88 m to 0.95 m, a reduction of approximately 49.6%, and increases joint correctness by 12.2 percentage points. Both metrics improve in all 50 scenes. Joint support also increases, and camera-surface error relative to ground truth decreases from 2.29 m to 2.01 m. These results provide direct evidence of improved cross-sensor geometric consistency on the evaluated mutually visible static surfaces.

Table 4: Cross-sensor consistency on mutually visible static surfaces

MethodCross-sensor Surface Discrepancy (m) ↓Joint Correctness ↑Joint Support ↑Camera-surface GT Error (m) ↓
Independent geometry1.8837.4%93.6%2.29
DriveR³0.9549.6%99.8%2.01

(2) Role of explicit association. We independently train a control model that retains RGB and LiDAR geometry supervision but excludes input LiDAR association. As shown in Table 5, adding spatiotemporal association reduces all four reported LiDAR geometry errors. Range MAE decreases from 1.65 m to 1.46 m, range RMSE from 4.28 m to 4.05 m, Depth L2 from 0.12 m² to 0.08 m², and CD from 4.00 m² to 2.90 m².

This comparison shows that input LiDAR association provides additional geometric reconstruction gains under the same geometry supervision. Together with the retrained association comparisons in Table 3 and the consistency evaluation in Table 4, these results clarify the contributions of explicit association and shared geometry.

Table 5: Effect of spatiotemporal association under LiDAR geometry supervision

MethodRange MAE (m) ↓Range RMSE (m) ↓Depth L2 (m²) ↓CD (m²) ↓
LiDAR geometry supervision1.654.280.124.00
LiDAR geometry supervision + spatiotemporal association1.464.050.082.90
Reviewer_SMsrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents DriveR³, a feed-forward framework for reconstructing dynamic Gaussian scenes that support both camera and LiDAR rendering. Built on top of STORM, where sparse LiDAR measurements are linked to Gaussians aligned with images, geometry is learned with supervision from cameras and LiDARs, and per-sensor opacity is preserved. An independent intensity model takes into account spatial and temporal information and weighted intensity estimation. The ability to perform reconstruction without per-scene optimization helps to solve an interesting problem in a novel manner by trading off quality for speed.

Critical strengths
  1. This work shows a good speed-accuracy trade-off by reconstructing scenes 10× faster than SplatAD while producing fewer errors on all the reported LiDAR-based metrics.
  2. The opacity ablation study presents an explainable insight that camera opacity sharing impairs the LiDAR reconstruction and vice versa, hence supporting the geometry sharing along with individual sensor opacity.
  3. The Chamfer-distance improvement over SplatAD is large.
Critical weaknesses
  1. The work is limited to one dataset called nuScenes, while its applicability to other types of driving scenarios remains unknown.
  2. The improvement of the DriveR³ method compared to Instant NuRec is not well justified. A comparative study or baseline with independent camera and LiDAR parameters could better quantify the effect of a common representation used in DriveR³.
  3. The evaluation includes observations of LiDAR responses for recorded trajectories but does not test for false responses, ray drops, or responses to off-trajectory positions. It is necessary to conduct full scan evaluation, shifting the ego-pose position in order to prove simulation assertions.
Questions for authors
  1. Could you compare against Instant NuRec or an equivalent baseline with a different camera and LiDAR geometry to measure the impact of the common representation?
  2. How does the time distribution of 23.5 seconds per window look like?
  3. Would it be possible for the authors to examine the impact of false alarms, ray drops, and ego pose control on their simulation arguments?
8 comments
AuthorsOpenReview ↗

Response to Reviewer SMsr: General Response

Thank you for your careful review and for recognizing our new approach to feed-forward joint camera–LiDAR reconstruction. We particularly appreciate your recognition of the method’s ability to balance scene-construction efficiency and LiDAR reconstruction quality without per-scene optimization, as well as the rationale for sharing geometry while modeling sensor-specific opacities. Below, we address your concerns about applicability across datasets, the contribution of the shared representation, scene-construction runtime, and sensor-simulation evaluation through additional experiments and analyses.

AuthorsOpenReview ↗

Response to Reviewer SMsr (Part 1/6): Weakness 1

Weakness 1. The work is limited to one dataset called nuScenes, while its applicability to other types of driving scenarios remains unknown.

Thank you for this valuable suggestion. To evaluate the applicability of DriveR³ beyond nuScenes, we additionally trained and evaluated our method on Waymo and report joint RGB and LiDAR reconstruction results on validation scenes.

As shown in Table 1, DriveR³ improves PSNR from 24.5 dB to 25.2 dB and reduces LPIPS from 0.199 to 0.162 compared with STORM, our visual reconstruction baseline. These correspond to an approximately 0.7 dB PSNR gain and an 18.6% reduction in LPIPS, indicating improved pixel-level fidelity and perceptual quality.

Compared with SplatAD, which relies on per-scene optimization, DriveR³ achieves lower average errors on all four reported LiDAR metrics while constructing the scene in a single feed-forward pass. These results extend our evaluation to a second driving dataset and further support the applicability of the proposed joint reconstruction framework.

Table 1: Reconstruction quality on Waymo validation scenes

MethodFeed-forwardJoint ReconstructionPSNR ↑SSIM ↑LPIPS ↓Range RMSE (m) ↓Depth L2 (m²) ↓Squared CD (m²) ↓Intensity RMSE ↓
STORM✓✗24.50.7660.199––––
SplatAD✗✓30.90.8960.0892.213.112.930.013
DriveR³✓✓25.20.7570.1622.150.240.650.009

Feed-forward indicates whether scene construction is performed in a single forward pass. Joint Reconstruction denotes joint camera–LiDAR reconstruction within a unified framework, including methods that use separate sensor representations, such as SimULi.

AuthorsOpenReview ↗

Response to Reviewer SMsr (Part 2/6): Weakness 2

Weakness 2. The improvement of the DriveR³ method compared to Instant NuRec is not well justified. A comparative study or baseline with independent camera and LiDAR parameters could better quantify the effect of a common representation used in DriveR³.

Thank you for this helpful suggestion. We clarify the comparison setup and provide an independent-geometry control to quantify the contribution of the shared representation.

(1) Comparison setup. DriveR³ takes images from six surround-view cameras and LiDAR scans collected at multiple time steps as input. It constructs a scene representation in a single forward pass and supports rendering RGB images, LiDAR ranges, and return intensities.

Existing multisensor Gaussian reconstruction methods, such as SimULi (ICLR 2026), use separate Gaussian representations for cameras and LiDAR and rely on per-scene optimization. Instant NuRec, the concurrent work most closely related to our objective, also explores LiDAR reconstruction. It evaluates a variant that takes LiDAR scans as input on internal Pandar128 data. Because the model weights and reconstruction code for this variant have not been released, the public version cannot be used directly for LiDAR range and return intensity comparisons on nuScenes.

To quantify the contribution of shared geometry, we construct an independent-geometry control using the same STORM visual backbone. This control uses STORM’s camera Gaussians to render RGB images and, following the LiDAR reconstruction approach described in Instant NuRec, separately predicts another set of Gaussians to render LiDAR ranges and return intensities. We additionally compare reconstruction quality with SimULi.

(2) Reconstruction quality. As shown in Table 2, compared with the independent-geometry control, DriveR³ improves PSNR from 21.9 dB to 22.7 dB, an approximately 0.8 dB gain. It also reduces range RMSE and intensity RMSE by 49.4% and 32.0%, respectively. Compared with SimULi, DriveR³ achieves lower LPIPS and reduces range RMSE and CD by approximately 33.2% and 15.2%, respectively, while constructing the scene in a single forward pass. These results support the benefit of shared geometry for joint RGB and LiDAR reconstruction.

Table 2: Joint RGB and LiDAR reconstruction quality

MethodFeed-forwardJoint ReconstructionPSNR ↑SSIM ↑LPIPS ↓Depth L2 (m²) ↓Range MAE (m) ↓Range RMSE (m) ↓Intensity RMSE ↓CD (m²) ↓
Independent geometry✓✗21.90.6650.3203.204.078.010.068718.44
SimULi✗✓23.30.7070.4410.322.056.060.04753.42
DriveR³✓✓22.70.6820.2470.081.464.050.04672.90

Feed-forward indicates whether scene construction is performed in a single forward pass. Joint Reconstruction denotes joint camera–LiDAR reconstruction within a unified framework, including methods that use separate sensor representations, such as SimULi.

(3) Cross-sensor consistency. We further compare the shared representation with the independent-geometry control on mutually visible static surfaces. We use LiDAR scans to establish correspondences and back-project normalized expected depths from both rendering branches into a common 3D coordinate system. We then measure the discrepancy between corresponding surfaces and the camera-surface error relative to ground truth.

As shown in Table 3, DriveR³ reduces cross-sensor surface discrepancy from 1.88 m to 0.95 m, increases joint correctness from 37.4% to 49.6%, and increases joint support from 93.6% to 99.8%. The camera-surface error relative to ground truth also decreases from 2.29 m to 2.01 m. These results show that shared geometry improves consistency between the two rendering branches on the evaluated mutually visible static surfaces.

Table 3: Cross-sensor consistency on mutually visible static surfaces

MethodCross-sensor surface discrepancy (m) ↓Joint correctness ↑Joint support ↑Camera-surface GT error (m) ↓
Independent geometry1.8837.4%93.6%2.29
DriveR³0.9549.6%99.8%2.01

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised Manuscript Uploaded

Dear Reviewers and Area Chair,

Thank you for your constructive feedback. We have uploaded a revised manuscript incorporating the clarifications and additional experimental results discussed in our responses. Key revisions are highlighted in blue for ease of reference, with supporting details provided in the main text and appendix.

We appreciate your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.