Store Sparse, Render Dense: Neural Gaussian Mixture Splatting

We introduce Neural Gaussian Mixture Splatting (NGMS), a compact, unified representation for static and dynamic scenes. Our core insight is that dense geometry, view-dependent appearance, and motion can be generated from a sparse set of stored Gaussians through a shared residual field. NGMS stores $N$ complete base Gaussians and expands each into $K$ local components through learned residuals in position, rotation, scale, opacity, and spherical-harmonic (SH) appearance. A single nonlinear field query per base jointly predicts all $K$ components, reducing nonlinear evaluations by a factor of $K$ relative to independent per-component prediction. Only diffuse base Gaussians and shared network weights are stored, avoiding independent storage of every rendered primitive and its higher-order SH coefficients. Factorized view and time branches capture appearance and deformation within the same formulation, while the generated components retain compatibility with standard Gaussian rasterization. Across four static and dynamic benchmarks, NGMS achieves the smallest serialized models among the evaluated methods, with competitive static reconstruction quality and a $6.5$ dB PSNR gain over 4DGS on D-NeRF. Dataset-average storage is reduced by $9$–$28\times$ relative to scaffold methods and by $31$–$303\times$ relative to explicit 3D/4DGS, reaching two orders of magnitude on dynamic scenes while maintaining real-time rendering.