UG-Diff: Improving Neural Rendering in Distractor-Corrupted Scenes via Uncertainty-Guided Diffusion

Neural rendering in real-world scenes is often compromised by transient distractors, leaving noticeable artifacts even after distractor suppression by existing distractor-free methods. Diffusion-based generative restoration can further improve rendering quality, but typically requires clean, distractor-free supervision, which is unavailable in distractor-corrupted scenes; directly applying frozen generative priors avoids this requirement but limits adaptation to scene-specific corruptions. We propose UG-Diff, an uncertainty-guided diffusion framework for robust neural rendering restoration. ur key insight is to employ uncertainty as a conditioning signal, enabling the model to simultaneously learn static structure restoration and transient distractor fitting under distractor-corrupted supervision. Specifically, we design an Uncertainty Mixing Strategy that modulates the diffusion input with spatially varying uncertainty, allowing the model to learn an uncertainty-conditioned graded restoration behavior directly from distractor-corrupted supervision: in low-uncertainty regions, it preserves more reliable static structures and performs restoration; whereas as uncertainty increases, it progressively strengthens distractor-dependent fitting. We further introduce Multi-view Condition Injection, which aggregates complementary cues from reference views to alleviate distractor-induced information loss and improve structural restoration accuracy. At inference, setting uncertainty to zero reduces the learned behavior to static structure restoration. Experiments on diverse distractor-corrupted datasets demonstrate that UG-Diff is applicable to multiple diffusion-based restoration methods and consistently outperforms existing baselines.