Learning Which Regions to Trust: Support-Aware Risk Ranking for Selective Novel View Synthesis"

Novel-view synthesis (NVS) produces spatially heterogeneous errors, making it important to identify which regions of a rendered view can be trusted. We study spatially selective NVS, where regions within each view are ranked to minimise error at a prescribed coverage under a declared loss. Cross-view consistency provides strong spatial evidence when source observations are available, but becomes incomplete under occlusion, extrapolation, and limited view overlap. Moreover, the desired reliability ordering depends on the deployment objective: pixel, structural, and perceptual losses induce different regional rankings. We derive a pairwise decomposition of selective ranking error by source-view support, identifying where cross-view evidence is informative and where complementary evidence is needed. Guided by this analysis, we introduce L2R-GS, a target-conditioned learning-to-rank framework that combines external cross-view evidence, internal Gaussian-state evidence, and observability cues according to source support and the declared error objective. Across 11 scenes from the Mip-NeRF 360, Tanks and Temples, and Deep Blending datasets, L2R-GS lowers L1, DSSIM, and LPIPS AURC by 4.9%, 9.8%, and 15.8% relative to the strongest published methods on the same held-out views. On held-out views, pairs involving unsupported regions account for more of the ranking gain as source support weakens. A separate study with human annotations shows that training for perceived artifacts improves the selection of reliable regions.