InvSplat: Inverse Feed-Forward Scene Splatting

Inverse rendering recovers geometry and material properties from images and is a prerequisite for relighting and material editing. Recent learning-based methods predict materials from images, but despite exploiting multi-view information, their image-space predictions can remain inconsistent across viewpoints. Meanwhile, per-scene optimization methods are computationally expensive and, in our experiments, achieve lower material accuracy than learned approaches despite using many views and can bake illumination into material maps. We therefore predict materials directly in 3D, as part of a feed-forward Gaussian splatting reconstruction. We observe that a direct combination of a 2D material estimator with a feed-forward reconstruction model is insufficient: the two networks must share features, and surface normals derived from the reconstructed geometry absorb texture, which degrades relighting. Based on these observations, we present InvSplat, which reconstructs 3D Gaussians with albedo, metallic, roughness and normals from posed images in 1.5 seconds. InvSplat predicts normals from appearance features and uses a point-light relighting loss that jointly constrains materials and normals without introducing errors from reconstructed geometry into the shading supervision. From two input views, InvSplat achieves higher relighting PSNR than the evaluated 2D baselines and produces more temporally consistent materials than methods processing all 32 frames of a video sequence, while maintaining comparable per-view material accuracy. Its explicit 3D representation further enables novel-view rendering of materials, relighting, and material editing.