FactorGrad-GS: Exact-Primal Monte Carlo Structural Backpropagation for Efficient 3D Gaussian Splatting

Efficient 3D Gaussian Splatting (3DGS) methods reduce Gaussian populations, rasterization work, or the number of optimizer updates, but retained updates commonly evaluate full red–green–blue (RGB) structural derivatives. FactorGrad-GS separates the exact structural dissimilarity (D-SSIM) value from the derivative used for optimization. It retains dense \(\ell_1\) supervision and samples D-SSIM derivatives across color channels and iterations, with inverse-probability weighting. A densification-aware schedule uses channel-only sampling during growth, restores full RGB after densification, and then enables temporal sampling. A selective fused-SSIM autograd path reuses one render and preserves the scalar objective supplied to a host gate. Matched SkipGS experiments show lower training wall time with small mean quality decreases. Additional-host comparisons identify schedule-dependent gains, while profiling and gradient measurements locate the saving in loss-side backward and document changes in optimization. These results support structural-derivative sampling as a complementary training acceleration mechanism. Project page: https://factorgrad-gs.github.io/.