NPM-GS: 3D-Native PatchMatch for Sparse-View Gaussian Splatting

Gaussian Splatting achieves high-quality novel-view synthesis with dense observations but degrades severely in sparse-view settings, producing floaters, distorted geometry, and strong train-view overfitting. Existing depth priors, regularization, and feed-forward initialization alleviate these failures, yet residual geometric errors can still survive subsequent Gaussian optimization. We identify a key cause: under sparse observations, geometry can co-adapt with appearance, opacity, scale, and densification, allowing different 3D configurations to explain the same training images. In contrast, cross-view matching directly tests whether a geometric hypothesis is supported by multiple observed views. We further find that this signal is highly geometry-sensitive but effective only within a finite capture range, motivating a bounded discrete search rather than unconstrained continuous optimization. Based on these observations, we propose NPM-GS, a 3D-native hypothesis search and verification framework that directly refines Gaussian centers using discrete cross-view matching. A learned geometric prior defines the search region, while multi-view and spatial agreement identify reliable Gaussians whose geometry is propagated to ambiguous regions through plane-snap and superpixel-based propagation. A subsequent reliability-aware training stage refines appearance while preserving the verified structure. NPM-GS consistently improves sparse-view reconstruction on DTU and Mip-NeRF 360 across PSNR, SSIM, and LPIPS.