LiGRe: Grounding Feed-Forward Gaussian Reconstruction in LiDAR Geometry for Joint RGB and LiDAR Rendering

Reconstructing real driving logs into controllable 3D scenes provides a practical basis for autonomous driving simulation. These scenes must support consistent camera and LiDAR rendering beyond recorded trajectories, where maneuvers such as lane shifts expose previously occluded regions. While feed-forward methods offer an efficient alternative to costly per-scene optimization, they primarily focus on visual reconstruction and remain limited by the narrow viewpoint coverage of driving logs. The resulting incomplete geometry can degrade both RGB and LiDAR rendering at shifted viewpoints. We present LiGRe, a LiDAR-guided generative framework that reconstructs a shared dynamic Gaussian scene from sparse RGB and LiDAR observations in a single forward pass. Our approach aggregates cross-sensor evidence while accounting for scene motion and integrates LiDAR geometry with learned generative priors. This combination supports the joint reconstruction of scene geometry and dynamics, using observed evidence to guide the completion of poorly observed regions. The resulting representation supports joint RGB and LiDAR rendering and controllable actor editing without per-scene optimization. Experiments on large-scale driving benchmarks demonstrate strong RGB and LiDAR reconstruction at both on- and off-trajectory viewpoints, alongside effective scene editing and efficient feed-forward inference.