RefracSurfel: Novel View Synthesis of Refractive Scenes with Enclosed Objects via Gaussian Surfel Ray Tracing

Recent novel view synthesis methods can reconstruct refractive scenes but still struggle with those containing enclosed objects: they either omit enclosed objects or rely on simplified optical models that limit rendering fidelity. To address this, our key idea is to use separate representations for the refractive interface and opaque surfaces: a neural signed distance field and ray-traceable Gaussian surfels, respectively. Our optimization proceeds in three stages: the first stage fits the surfels to the input images and masks while learning a per-surfel probability to separate the foreground from the background; the second stage initializes a signed distance field from the foreground surfels and refines it through differentiable ray tracing; the final stage optimizes surfels representing the enclosed objects using the recovered refracted ray paths. The effectiveness of our approach is demonstrated on various datasets of refractive scenes, comparing favorably against state-of-the-art techniques in terms of rendering quality.