LUMEN-GS: Towards Light-Disentangled and Intrinsic-Aware Gaussian Splatting for Feed-forward Sparse-View Street Reconstruction
Abstract
Feed-forward reconstruction models have recently advanced spatial intelligence and simulation but mainly predict color radiance fields, treating scenes as non-interactive assets without exploring intrinsic properties and lighting representations for controllable editing. Bridging this gap requires a feed-forward paradigm that jointly represents intrinsic properties and outdoor illumination, instead of the hours of per-scene optimization required by iterative inverse rendering. Therefore, we present Lumen-GS, to our knowledge the first feed-forward reconstruction system that integrates a physically based rendering (PBR) pipeline for street scenes. In a single forward pass, it jointly predicts intrinsic primitives, sunlight and global environment radiance within a sunlight-integrated representation from sparse-view images at a single timestamp, supporting reconstruction, relighting, and harmonization. Lumen-GS aggregates multi-view features via a Hybrid Ring representation, with sunlight direction from GPS/IMU metadata. It further introduces a Physically-Aligned Decoder (PAD), whose intrinsic prediction heads are jointly supervised by teacher pseudo-labels and Physically-Based Rendering to improve disentanglement. Results on nuScenes show that Lumen-GS remains competitive with the strongest feed-forward baselines in novel-view synthesis and with iterative inverse-rendering methods in PBR quality, while supporting object insertion, harmonization, and data augmentation for downstream perception. These results demonstrate the potential of Lumen-GS for lighting-controllable street-scene simulation.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.