acceptodds
Under review as a conference paper at ICLR 2027

UG-Diff: Improving Neural Rendering in Distractor-Corrupted Scenes via Uncertainty-Guided Diffusion

Abstract

Neural rendering in real-world scenes is often compromised by transient distractors, leaving noticeable artifacts even after distractor suppression by existing distractor-free methods. Diffusion-based generative restoration can further improve rendering quality, but typically requires clean, distractor-free supervision, which is unavailable in distractor-corrupted scenes; directly applying frozen generative priors avoids this requirement but limits adaptation to scene-specific corruptions. We propose UG-Diff, an uncertainty-guided diffusion framework for robust neural rendering restoration. ur key insight is to employ uncertainty as a conditioning signal, enabling the model to simultaneously learn static structure restoration and transient distractor fitting under distractor-corrupted supervision. Specifically, we design an Uncertainty Mixing Strategy that modulates the diffusion input with spatially varying uncertainty, allowing the model to learn an uncertainty-conditioned graded restoration behavior directly from distractor-corrupted supervision: in low-uncertainty regions, it preserves more reliable static structures and performs restoration; whereas as uncertainty increases, it progressively strengthens distractor-dependent fitting. We further introduce Multi-view Condition Injection, which aggregates complementary cues from reference views to alleviate distractor-induced information loss and improve structural restoration accuracy. At inference, setting uncertainty to zero reduces the learned behavior to static structure restoration. Experiments on diverse distractor-corrupted datasets demonstrate that UG-Diff is applicable to multiple diffusion-based restoration methods and consistently outperforms existing baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
diffusion-based generative restoration, Neural rendering, Uncertainty modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RAqwOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary
  1. Background and Problem: Neural rendering methods (e.g., NeRF and 3DGS) often suffer from artifacts and missing textures in dynamic scenes. Existing diffusion-based methods rely on clean supervision or generic priors, limiting their adaptability to scene-specific degradation.

  2. Key Contributions: The paper proposes UG-Diff, an uncertainty-guided diffusion framework that eliminates the need for clean supervision. It introduces uncertainty mixing for adaptive feature restoration and multi-view condition injection to leverage complementary geometric and texture information across views.

  3. Experimental Evaluation: Experiments on DF3DV, On-the-go, and RobustNeRF compare UG-Diff with existing methods, including Difix3D, GenFusion, and GeoQuery, demonstrating its reconstruction capabilities and potential as a general-purpose plug-in.

Critical strengths
  1. Unified and Elegant Weakly Supervised Modeling: The method treats uncertainty as a continuous conditioning variable, enabling adaptive modeling of static structures and dynamic occluders during training, while removing occluders at inference. This design avoids hard-threshold segmentation and additional clean data, offering an effective and streamlined solution.

  2. Strong Plug-and-Play Capability and Generalizability: The approach works across different 3D representations, including NeRF and 3DGS, and can be integrated into existing diffusion methods such as Difix3D and GenFusion, demonstrating its practical value and versatility.

Critical weaknesses
  1. Strong Dependence on Upstream Uncertainty Estimation: The framework relies heavily on the accuracy of residual uncertainty estimates from the rendering backbone. In scenes with extensive occlusion, weak textures, or reflective surfaces, inaccurate uncertainty maps may lead to hallucinated content or residual artifacts.

  2. Sensitivity to Reference View Selection and Cross-View Alignment: Ablation studies show that restoration quality is highly sensitive to reference-view overlap. The reliance on 2D cross-attention without explicit geometric constraints, such as epipolar or depth-reprojection constraints, may cause performance degradation or introduce incorrect textures under large viewpoint changes or limited shared visibility.

Questions for authors
  1. Reference View Selection: In the main experiments, are reference views selected automatically based on camera-pose overlap or manually using heuristics? How robust is the method when the camera follows a one-way trajectory with minimal view overlap?

  2. Inference Latency and Computational Overhead: How much additional GPU memory and inference time are required by multi-view feature concatenation and cross-attention? How does UG-Diff's end-to-end throughput compare with the single-step baseline Difix3D?

  3. Theoretical Motivation for the Perturbation Type: Checkerboard noise significantly outperforms Gaussian and uniform noise in the ablation studies. Could the authors provide further explanations from geometric or frequency-domain perspectives?

4 comments
AuthorsOpenReview ↗

Response RAqw (Part 1/2)

We thank the reviewer for the thoughtful feedback and are particularly pleased that you found our weakly supervised formulation elegant. Your concerns help us clarify how the method handles imperfect uncertainty estimates and reference information. Below, we provide supporting evidence, report computational costs, and discuss the intuition behind our perturbation design.

W1. Strong Dependence on Upstream Uncertainty Estimation

Thank you for this insight. The upstream backbone can indeed produce inaccurate uncertainty maps in scenes with extensive occlusions, weak textures, or reflective surfaces. Our core contribution is to use distractor-aware uncertainty as a continuous condition for weakly supervised learning, without requiring it to serve as a pixel-accurate distractor mask.

The last two rows of Fig. 5 show that the model reproduces the supervised umbrella/person and bus during distractor fitting even though the uncertainty responses do not precisely cover these objects. Conversely, Fig. 8 shows that high uncertainty responses on static structures do not automatically induce hallucinated distractors. These examples indicate that the learned behavior accommodates local mismatches between uncertainty responses and object regions. Fig. 3 further illustrates the conditional mapping itself: varying uncertainty for the same rendered input progressively shifts the output from static restoration toward distractor fitting. Together, these observations support the model’s ability to learn coherent conditional behavior from imperfect uncertainty estimates. Table 4 further shows that continuous uncertainty outperforms binary masking across all reported metrics, supporting the value of preserving graded uncertainty information for restoration. These results support the effectiveness of retaining graded uncertainty information for learning restoration under the evaluated backbone estimates.

W2 & Q1. Reference View Selection

Thank you for raising these concerns about reference-view selection and alignment, which we also considered in our work. Explicit cross-view alignment can provide more precise restoration guidance, but it relies on multi-view geometric consistency, which is not always guaranteed in distractor-corrupted observations. We reuse the source views selected by MU-GeNeRF to reconstruct the target rendering. Selection is automatic, using a heuristic score based on camera rotations and translations (R,T)(R,T).

We use reference views to provide scene-level appearance and complementary restoration cues without enforcing explicit geometric correspondences (which can be unreliable in the presence of transient distractors). To preserve restoration capability when these reference cues are unreliable or unavailable,we retain the per-view attention output XsvX_{\mathrm{sv}} alongside the multi-view output XmvX_{\mathrm{mv}}. This blending preserves a dedicated pathway for the target view’s own features, allowing restoration to draw on both the target rendering and reference information.

Our evaluations on DF3DV, On-the-go, and RobustNeRF use sparse, discrete scene observations rather than continuous one-way camera trajectories. To address the concern about minimal overlap, we evaluate with references from unrelated scenes as a proxy for the extreme case where no useful cross-view overlap is available(As shown in A.7). Even under these conditions, UG-Diff better preserves the target structure than Difix-style Attn., supporting the role of XsvX_{\mathrm{sv}} in maintaining restoration based on the target rendering itself. These results characterize the restoration capability retained without useful reference evidence, with qualitative comparisons provided in the appendix.

AuthorsOpenReview ↗

Response RAqw (Part 2/2)

Q2. Inference Latency and Computational Overhead

We report model latency, peak GPU memory, and end-to-end performance for Difix3D-ref and UG-Diff with four references. With the additional reference views, UG-Diff takes 228.3 ms per image for model inference, compared with 186.2 ms for Difix3D, while peak GPU memory remains comparable (4.58 vs. 4.63 GiB(ours)). End-to-end latency is 442.1 vs. 347.1 ms per image, corresponding to throughputs of 2.26 and 2.88 images/s, respectively. These measurements quantify the overall computational cost of the two reference configurations.

Q3. Theoretical Motivation for the Perturbation Type

Thank you for raising this interpretability question. Our ablations empirically show that checkerboard perturbations outperform Gaussian and uniform noise. We hypothesize that this relates to the diffusion prior: single-step models such as Difix3D+ treat artifact-corrupted renderings as intermediate noisy states in the denoising process. Gaussian or uniform perturbations may therefore be less distinguishable from the corruption already present in the rendering. Checkerboard perturbations instead introduce a structured, high-frequency periodic signal, which may help the model distinguish rendering artifacts from distractor-related variations. This provides a possible frequency-domain explanation for the observed advantage.

Thank you to the authors for the response and the computational overhead benchmarks.

While the latency/memory numbers and frequency-domain explanation address my questions on efficiency and perturbation design, my core concerns regarding robustness (W1, W2/Q1) remain insufficiently resolved:

  1. The rebuttal relies heavily on selective qualitative visualizations rather than systematic quantitative stress-tests on challenging subsets (e.g., low-texture, reflective, or wide-baseline scenes). A few selected examples are prone to cherry-picking and do not convincingly establish overall reliability, especially given the failure modes already acknowledged in Section 6 and Appendix A.5.
  2. Using completely unrelated scenes does not accurately reflect low-overlap camera trajectories, where similar local textures in non-overlapping regions can mislead unconstrained 2D cross-attention into copying wrong textures.

Given the lack of rigorous quantitative validation on these failure-prone regimes, I find the evidence insufficient to raise my score and will maintain my rating of Weak Reject.

AuthorsOpenReview ↗

Response RAqw

Thank you for the follow-up and for acknowledging our responses on efficiency and perturbation design.

Regarding W1, we respectfully disagree with characterizing the evidence as cherry-picked. The cited visualizations illustrate imperfect uncertainty estimates already present in our evaluations, while the quantitative results cover the full test sets under the standard evaluation protocol. Together, they support the practical effectiveness of uncertainty-conditioned learning. We acknowledge that dedicated challenging-subset evaluations would further characterize its robustness.

Constructing reliable low-texture, reflective, and wide-baseline subsets, together with controlled same-scene low-overlap reference sets, requires additional curation and validation that we cannot complete within the rebuttal period. We also recognize the distinction you raise between unrelated-scene references and same-scene low-overlap references. Our experiment examines restoration when useful cross-view reference information is unavailable.

We appreciate your careful assessment and the specific directions for further evaluation.

Reviewer_cxTPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors suggest a diffusion-based post-fix method that considers uncertainty to handle distractors in the scene. The method is trained using an uncertainty mixing strategy that inserts noise based on the uncertainty of the reconstruction. It is validated on DF3DV-41, On-the-go, and RobustNeRF, and also shows generalization across multiple diffusion-based restoration methods

Critical strengths
  • The paper is mostly well-written and easy to follow.
  • The main idea is straightforward and easy to understand.
  • The suggested method looks practical, with impressive generalization and both qualitative and quantitative performance.
Critical weaknesses
  • Missing relevant related works - The authors should include recent and highly relevant literature, both in the related works (explaining the differences between the methods) and mainly in the comparison section. Specifically, FreeFix[1] - a Difix3D variant that uses uncertainty to guide generation, and HAD[2], which measures hallucinations for reconstructions. Also, recent Difix3D variants like ArtiFixer[3] and GSFixer[4] should be included in the comparisons.

  • Multi-view condition injection - the suggested approach is very similar to DifiX3D+. The authors should explicitly state this and the difference between the methods. Specifically, scaling the MV and single-view features using an alpha parameter. Nonetheless, as this simple change is at the heart of the method, the authors should ablate whether it makes a substantial difference.

[1] Zhou, Hongyu, et al. "FreeFix: Boosting 3D Gaussian Splatting via Fine-Tuning-Free Diffusion Models." 2026 International Conference on 3D Vision (3DV). IEEE, 2026.

[2] Liu, Xi, et al. "HAD: Hallucination-Aware Diffusion Priors for 3D Reconstruction." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

[3] De Lutio, Riccardo, et al. "ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models." Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers. 2026.

[4] Yin, Xingyilang, et al. "Gsfixer: Improving 3d gaussian splatting with reference-guided video diffusion priors." arXiv preprint arXiv:2508.09667 (2025).

Questions for authors

See weaknesses

2 comments
AuthorsOpenReview ↗

Response cxTP

We thank the reviewer for the comments. We have added experiments on Artifixer, GSFixer, and the attention design to the appendix, and revised the Introduction and Related Work sections in the main paper.

Our core insight is to use distractor-aware uncertainty to parameterize a continuum of behaviors from static restoration to distractor fitting, enabling the diffusion model to learn restoration directly from distractor-corrupted supervision.

This unified weakly supervised training paradigm uses the (not always perfect) distractor-aware uncertainty provided by the distractor-free backbone, without requiring hard-threshold segmentation, additional annotations, or clean data. Our definition of uncertainty and our task objective differ fundamentally from those of FreeFix and HAD, as we discuss in detail below.

W1. Missing relevant related works

Thank you for pointing out these related works. We first distinguish their definitions of uncertainty from ours: our distractor-aware uncertainty follows the modeling approach used in distractor-free reconstruction methods (NeRF-W, NeRF On-the-go, WildGaussians, and MU-GeNeRF), characterizing observation residuals that cannot be explained by geometric consistency of a static scene. FreeFix’s rendering confidence measures local rendering reliability to guide diffusion refinement, while HAD assesses hallucinations in diffusion-generated content to evaluate its reliability for reconstruction. Neither explicitly addresses learning static restoration from distractor-corrupted supervision, which is the central role of our uncertainty conditioning. We will cite and distinguish these methods in Related Work to clarify these differences.

Artifixer and GSFixer use video diffusion models to restore rendered images. Although neither explicitly addresses distractor-corrupted supervision, comparisons with them provide a broader evaluation of UG-Diff’s applicability to our task. We have added comparisons with Artifixer and GSFixer to the Appendix A.7, including experimental settings, quantitative results, and qualitative comparisons. For Artifixer, we fine-tune the released S1 teacher (~60 GPU-hours; without subsequent distillation) and also evaluate its frozen counterpart. For GSFixer, we evaluate the frozen pretrained model because its official training code is unavailable. When the input views follow a discrete camera sequence rather than a continuous camera trajectory, we observe viewpoint shifts in GSFixer’s outputs, resulting in misalignment with the input renderings. UG-Diff outperforms these baselines across all three datasets on all metrics, supporting the robustness and consistency of its weakly supervised restoration in distractor-corrupted scenes. Moreover, Artifixer (w/ uncer) consistently improves over Artifixer across all three datasets on all metrics, further supporting the generality of our Uncertainty Mixing Strategy and its potential for integration into a range of existing rendering restoration methods.

W2. Multi-view Condition Injection

Thank you for pointing out this connection. Our cross-view interaction follows a joint-attention approach similar to Difix3D+. The distinction is that, alongside the multi-view attention output XmvX_{\mathrm{mv}}, we retain the per-view attention output XsvX_{\mathrm{sv}}, computed independently for each view. Blending these outputs preserves a dedicated pathway for the target view’s own features while incorporating complementary information from reference views.

When sufficient useful reference information is available, both designs can obtain restoration cues through multi-view interaction, making the additional benefit of XsvX_{\mathrm{sv}} less apparent. We therefore examine unreliable references and changes in reference count. Both experiments retain multi-view interaction and compare UG-Diff with Difix-style Attn using identical reference inputs:

  • Varying reference counts: Both models are trained with one reference view and evaluated with 1, 2, 4, and 8 reference views. As the reference count increases, UG-Diff maintains relatively stable restoration performance, whereas Difix-style Attn degrades at larger reference counts. This supports the ability of our blending design to accommodate changes in reference count.(Appendix A.8)
  • Unreliable references: With reference images from unrelated scenes, UG-Diff better preserves the target structure, supporting the role of XsvX_{\mathrm{sv}} when useful reference information is unavailable.(Appendix A.9)

Together, these experiments suggest that XsvX_{\mathrm{sv}} provides an attention pathway for restoration based on the target rendering itself, improving robustness to unreliable references and changes in reference count.

Thank you for the detailed responses and additional experiments. I will re-evaluate my score based on this answer and the other reviews (and discussion).

Reviewer_McBEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes UG-Diff, an uncertainty-guided diffusion framework for neural rendering in distractor-corrupted scenes. It introduces two components, Uncertainty Mixing Strategy and Multi-view Condition Injection, to alleviate distractor-induced information loss and improve structural restoration accuracy.

Critical strengths
  • The paper addresses an interesting and practically meaningful problem. Achieving high-quality reconstruction in unconstrained environments with transient distractors is challenging and has important real-world applications.

  • The authors conduct adequate experiments across multiple datasets and compare their method with several existing baselines. The overall results are promising.

Critical weaknesses
  • During training, regions with high uncertainty are supervised using images containing strong distractors. However, it is unclear why simply setting (\beta=0) at inference would enable the model to recover distractor-free static structures. The paper does not sufficiently explain how uncertainty-dependent input perturbations lead to the disentanglement of static structure restoration and distractor fitting. Further justification and experimental evidence are needed.
  • Prior works such as FaithFusion and FreeFix have already explored uncertainty-guided diffusion for spatially adaptive 2D rendering refinement. The paper does not sufficiently distinguish its contributions from these closely related approaches. Moreover, existing image and video diffusion-based refiners, such as GSFixer and ArtiFixer, can also restore rendering artifacts. It remains unclear what advantages UG-Diff offers over directly applying these methods to imperfect renderings.
  • Line 195, the authors refer to optimizing Eq. (2), which describes the forward diffusion process rather than an optimization objective. Presumably, the intended reference is Eq. (3).
  • The Introduction Section is not good, which lack a comprehensive literature survey and novelty. Plus, it not clear to read and understand for readers: what is the task? why it is important? what is the motivation.
Questions for authors

No

4 comments
AuthorsOpenReview ↗

Response McBE (Part 1/2)

We thank the reviewer for the comments. We have added experiments on Artifixer, GSFixer, and the attention design to the appendix, and revised the Introduction and Related Work sections in the main paper.

Our core insight is to use distractor-aware uncertainty to parameterize a continuum of behaviors from static restoration to distractor fitting, enabling the diffusion model to learn restoration directly from distractor-corrupted supervision without overfitting to the distractors it contains.

This unified weakly supervised training paradigm uses the (not always perfect) distractor-aware uncertainty provided by the distractor-free backbone, without requiring hard-threshold segmentation, additional annotations, or clean data. Our definition of uncertainty and our task objective differ fundamentally from those of FreeFix and FaithFusion, as we discuss in detail below.

W1. Why β=0 At Inference?

Thank you for raising this concern. Setting β=0\beta=\mathbf{0} selects the static-restoration endpoint of the uncertainty-conditioned behavior learned during training.,This is the core idea behind our method and experimental design. Below, we explain this choice through the training mechanism and experimental results.

Both static restoration and distractor fitting are trained to match the corresponding target regions (whether or not they contain distractors). However, we couple distractor fitting with stronger noise modulation of the rendered features (Eq. 5): regions with unreliable supervision receive stronger noise perturbations. This forces the model to rely more on generative priors to fit distractor-corrupted targets, whereas in low-uncertainty regions, it relies on the existing rendered features for restoration.

Because uncertainty takes continuous values in [0, 1] (rather than binary values of 0/1), different spatial locations provide training conditions with varying noise levels. We believe this enables the model to learn a continuum of behaviors from static restoration to distractor fitting within the unified framework in Fig. 1. At β=0\beta=\mathbf{0}, the rendered features undergo no noise modulation, and the model restores the rendering based on these features while avoiding the introduction of distractors. The Eq. (9) ensures that β=0\beta=\mathbf{0} is not an entirely OOD condition and Table 4 demonstrates that continuous uncertainty values offer greater robustness than a binarized mask.

Fig. 3 further shows that increasing uncertainty for the same rendered region gradually shifts the output from static restoration toward distractor fitting. This supports the interpretation that the model learns continuous conditional behavior, rather than merely applying fixed treatments to different regions.

W2. Baseline & Prior Works

Thank you for pointing out these related works. We first distinguish their definitions of uncertainty from ours: our distractor-aware uncertainty follows the modeling approach used in distractor-free reconstruction methods (NeRF-W, NeRF On-the-go, WildGaussians, and MU-GeNeRF), characterizing observation residuals that cannot be explained by geometrically consistency of a static scene. FreeFix’s rendering confidence measures local rendering reliability to guide diffusion refinement, while FaithFusion’s EIG measures the expected reduction in 3D model uncertainty to guide generation and reconstruction updates. Neither explicitly addresses learning static restoration from distractor-corrupted supervision, which is the central role of our uncertainty conditioning. We will cite and distinguish these methods in Related Work to clarify these differences.

Artifixer and GSFixer use video diffusion models to restore rendered images. Although neither explicitly addresses distractor-corrupted supervision, comparisons with them provide a broader evaluation of UG-Diff’s applicability to our task. We have added comparisons with Artifixer and GSFixer to the Appendix A.7, including experimental settings, quantitative results, and qualitative comparisons. For Artifixer, we fine-tune the released S1 teacher (~60 GPU-hours; without subsequent distillation) and also evaluate its frozen counterpart. For GSFixer, we evaluate the frozen pretrained model because its official training code is unavailable. When the input views follow a discrete camera sequence rather than a continuous camera trajectory, we observe viewpoint shifts in GSFixer’s outputs, resulting in misalignment with the input renderings. UG-Diff outperforms these baselines across all three datasets on all metrics, supporting the robustness and consistency of its weakly supervised restoration in distractor-corrupted scenes. Moreover, Artifixer (w/ uncer) consistently improves over Artifixer across all three datasets on all metrics, further supporting the generality of our Uncertainty Mixing Strategy and its potential for integration into a range of existing rendering restoration methods.

AuthorsOpenReview ↗

Response McBE (Part 2/2)

W3. Reference Typo

Thank you for pointing this out. We apologize for the typo in our paper. As you noted, the optimization objective on line 195 refers to Eq. (3), not Eq. (2).

W4. Task, Motivation, and Significance

Thank you for this feedback. We have revised the Introduction to provide a more comprehensive discussion of related work and clarify our novelty. We address a practical challenge in diffusion-based restoration of reconstructions from observations containing transient distractors: paired clean training targets are unavailable, and the available supervision consists of the original distractor-corrupted observations. The restorer must repair static structures while avoiding reintroducing transient content (e.g., pedestrians and vehicles) from the supervision into its outputs.

Our contribution is to use distractor-aware uncertainty, already produced during reconstruction optimization, as a diffusion condition within a unified weakly supervised framework. This enables the restorer to learn static restoration directly from corrupted observations, making diffusion-based restoration trainable on real-world captures without requiring paired clean targets.

I thank the authors for their detailed response and additional experiments. However, prior works such as FaithFusion have already explored uncertainty-guided diffusion for restoring unreliable regions, while ArtiFixer have demonstrated effective rendering refinement using powerful video diffusion. Although the results are better, it remains unclear to what extent the improvements come from the proposed technical innovations rather than dataset-specific adaptation. A fairer comparison would require training the competing methods on the same datasets under comparable settings. Therefore, I still find the technical contribution relatively limited and will maintain weak reject.

AuthorsOpenReview ↗

Response McBE

We are deeply disappointed that the reviewer maintains the original score on the grounds of “conceptual overlap,” while the follow-up response provides no specific explanation of which mechanisms overlap with which prior works.

Our rebuttal clarified the distinctions from the cited works (Sec1 & Sec2.2) and provided additional experiments (Appendix A.7), yet the follow-up response addresses neither. We reaffirm that the central contribution of this work is to establish a weakly supervised training paradigm for learning directly from distractor-corrupted observations, transferable across diffusion backbones and integrable into existing image restoration frameworks. Specifically,

  • Learning from Corrupted Supervision. We enable diffusion-based static restoration to be learned directly from observations containing transient distractors, without clean training targets.
  • Joint Modeling and Decoupling. We introduce continuous uncertainty-guided noise modulation to jointly model static structures and transient distractors, disentangling static restoration from distractor fitting within a single unified framework.
  • Broad Applicability. Our framework supports both NeRF and 3DGS (Appendix A.4) and integrates into existing diffusion restorers, including Difix3D , GenFusion and Artifixer (Sec 5.5 & Appendix A.7) , with experiments validating its effectiveness across representations and restoration models.

We therefore respectfully disagree with the assessment of “conceptual overlap” and ask the reviewer to identify the specific mechanisms and prior works underlying this judgment, and explain how that assessment accounts for the methodological distinctions and additional evidence presented in our rebuttal.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

General Response

Thank you for the feedback

We sincerely thank all reviewers for their thoughtful feedback and constructive suggestions. Their comments have helped us clarify UG-Diff’s central contribution and strengthen its experimental evaluation.

We are pleased that the reviewers recognize the practical significance of restoring neural renderings in distractor-corrupted scenes:

“The paper addresses an interesting and practically meaningful problem.” — McBE

We particularly appreciate the recognition of our unified weakly supervised formulation, which uses continuous uncertainty conditioning to learn static restoration directly from distractor-corrupted supervision:

“Unified and Elegant Weakly Supervised Modeling” — RAqw

“This design avoids hard-threshold segmentation and additional clean data, offering an effective and streamlined solution.” — RAqw

We are also encouraged by the reviewers’ positive assessments of the method’s generalizability and practical effectiveness:

“The approach works across different 3D representations, including NeRF and 3DGS, and can be integrated into existing diffusion methods such as Difix3D and GenFusion, demonstrating its practical value and versatility.” — RAqw

“The suggested method looks practical, with impressive generalization and both qualitative and quantitative performance.” — cxTP

Updates and new experiments

Guided by the reviewers’ comments, we have added new experiments to the appendix and revised the Introduction and Related Work. We summarize the updates incorporated into the revised paper below.

  1. Additional comparisons with Artifixer and GSFixer.(Appendix A.7) We add quantitative and qualitative comparisons on DF3DV-41, On-the-go, and RobustNeRF. These include fine-tuned and frozen Artifixer variants and the frozen pretrained GSFixer model. UG-Diff outperforms these baselines on all three datasets across PSNR, SSIM, and LPIPS.

  2. Further validation of the generality of Uncertainty Mixing.(Appendix A.7) We integrate our Uncertainty Mixing Strategy into Artifixer. It consistently improves Artifixer on all three datasets across all reported metrics, extending the evidence that our training strategy can benefit different diffusion-based restoration methods under distractor-corrupted supervision.

  3. New ablations on the attention design and reference robustness.(Appendix A.8 &A.9) We compare UG-Diff with Difix-style attention using identical reference inputs. With references from unrelated scenes, UG-Diff better preserves the target structure. When both models are trained with one reference and evaluated with 1, 2, 4, and 8 references, UG-Diff maintains relatively stable restoration performance. These experiments support the role of blending per-view and multi-view attention outputs in preserving restoration based on the target rendering itself.

  4. Revised Introduction and Related Work. We strengthen the task motivation and clarify our central contribution: learning static restoration directly from training targets containing transient distractors through a unified weakly supervised formulation. We expand the discussion of related methods, including FreeFix, FaithFusion, HAD, Artifixer, and GSFixer, and clarify how their objectives and reliability measures differ from our use of distractor-aware uncertainty to learn from corrupted supervision.

We sincerely thank all reviewers again for helping us improve the paper, and we welcome further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.