PriorGuard-GS: Taming Overfitting in Sparse-View Reconstruction with Guidance from Pretrained Priors

Reconstructing scenes from sparse views requires fitting observed images while generalizing to novel views. We present PriorGuard-GS (PGGS), which uses pretrained feed-forward priors to tame overfitting in sparse-view reconstruction. Prior-Guided Latent Optimization (PLO) retains the pretrained prior during adaptation, implicitly regularizing scene updates through learned relationships among Gaussian attributes. Conditional Gaussian Decoding (CGD) generates a variable set of rendering Gaussians, allowing the representation to evolve during optimization. Dynamic Event Awareness (DEA) preserves state across densification, pruning, and opacity resets during optimization. This formulation provides a common interface for multiple feed-forward architectures. We evaluate PGGS across four datasets and five backbones. Across backbones, PGGS improves overall novel-view PSNR by 1.48–4.10 dB over direct Gaussian optimization from the same initialization. With DepthSplat, it outperforms all six optimization-based baselines in all three aggregate metrics. With its fastest backbone, PGGS requires only 30.2 s in the optimization loop on average, compared with 89.1–786.8 s for optimization-based baselines. Qualitative comparisons show reduced ghosting and fewer fragmented structures, supporting the role of continued prior guidance in limiting sparse-view overfitting.