acceptodds
Under review as a conference paper at ICLR 2027

GAD-Splat: Geometry-Appearance Decoupled Reconstruction for Distractor-Free 3D Gaussian Splatting

19 views

Abstract

Transient objects disrupt multi-view consistency and limit observations of static scenes, leading to ghosting and blurring artifacts in 3D Gaussian Splatting (3DGS) reconstructions. Existing approaches primarily suppress unreliable supervision or separate static and transient scene components. However, minimizing photometric residuals can still favor incorrect Gaussian configurations that fit observed colors but misrepresent static surfaces, particularly where unmasked observations are limited. We introduce GAD-Splat, which estimates masking probabilities from motion cues and rendering residuals, then selects candidate masks from a segmentation model according to the likelihood of their union. Cross-view Depth Regularization (CDR) fuses filtered depth predictions into a shared truncated signed distance field (TSDF). Reference depths from this volume constrain rendered expected depth through surface and free-space losses. With geometry fixed, Cross-view Appearance Transfer (CAT) transfers color residuals computed from unmasked source observations and running averages of their renderings. These residuals complement color estimates for the target viewing direction to provide auxiliary supervision for masked pixels. This staged optimization aims to remove transient artifacts while preserving static scene content. Experiments on public benchmarks and self-captured datasets show effective transient-object removal and improved rendered image quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Transient Object Removal, TSDF, Appearance Transfer

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_afs1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GAD-Splat addresses transient-object removal in 3D Gaussian Splatting (3DGS) reconstructions of casually captured scenes. The core observation is that accurate masking alone is insufficient: when surfaces behind distractors are observed in few views, photometric optimization over remaining pixels can settle on geometries that fit colors but misrepresent static surfaces, and joint geometry/appearance optimization corrupts the cross-view correspondences needed to supervise masked regions. The paper proposes a three-stage pipeline with masks fixed throughout. Validation spans six NeRF On-the-go scenes and nine self-captured scenes, reporting best mean PSNR/SSIM (23.68 dB / 0.836 on NeRF On-the-go) and strong LPIPS, plus ablations isolating each stage and qualitative RGB/depth comparisons.

Critical strengths

The key contribution, explicitly decoupling geometry from appearance optimization so that cross-view correspondences used for masked-region supervision remain fixed, is well motivated and directly supported by the ablations. The empirical evaluation is broad and the margins are consistent, not isolated to a single scene or metric. GAD-Splat is best on mean PSNR/SSIM in every occlusion group on NeRF On-the-go and on all three means for self-captured data, with per-scene tables showing it leads all metrics in several self-captured scenes even where DeSplat wins others.

Critical weaknesses

(1) The method's dependence on external, largely off-the-shelf components (GeoMotion motion priors, DINOv3 features, SAM2 whole-mask candidates, Depth Anything 3, TSDF fusion) is heavy, yet the paper provides no cost, runtime, memory, or sensitivity analysis for these dependencies. This is consequential because the pipeline's practicality and reproducibility are unclear: TSDF fusion plus dense depth prediction at training-view resolution plus SAM2 prompting is plausibly a significant overhead relative to baseline 3DGS training, and the paper's own limitations section concedes failures under insufficient unmasked observations or erroneous fixed geometry, which are precisely the regimes where the external priors would be most stressed.

(2) The quantitative ablation compares only configurations with/without CDR and CAT (Table 3); it does not isolate a "masks only, no geometry/appearance decoupling" baseline against which the decoupling principle itself is tested, nor does it report geometry-specific metrics that would substantiate the claim that geometry is actually better rather than merely producing better renders. Since the fixed-geometry assumption is the paper's core assumption, the absence of direct geometric evaluation weakens the central argument.

Questions for authors

(1) What is the memory overhead of the full pipeline (motion prior, DINOv3 features, SAM2 prompting/selection, Depth Anything 3 inference, TSDF construction and raycasting) relative to baseline 3DGS training, and how does it scale with the number of views?

(2) How sensitive are results to the external priors, e.g., what happens if GeoMotion or Depth Anything 3 fails or is replaced by a weaker model? Is there an ablation of motion-prior quality or depth-prediction quality?

(3) Can you report a geometry-specific metric (e.g., rendered-depth error against held-out reference depths, or surface accuracy against TSDF/LiDAR where available) comparing CDR against a masks-only baseline, to directly support the claim that decoupling improves geometry rather than only appearance?

(4) For the constrained appearance update, how much does the gradient projection matter versus simply applying the auxiliary losses with tuned weights? An ablation removing the projection (unconstrained auxiliary gradients, zero-order SH only) would clarify whether the constraint or the SH restriction drives the gains.

(5) The limitations note failures when a background surface is hidden in all source views. What fraction of masked pixels typically lack valid source observations in your datasets (the ρ_t = 0 case), and how does CAT behave in those regions, does it default to the rendering average?

1 comment

Thanks for the efforts. I carefully reviewed all the comments and authors' responses. I think the key contributions are still limited and not fully validated, so I will keep my score.

Reviewer_1uLtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper reconstructs static scenes from casually captured images containing transient objects. Its motivation is that masking transient observations does not provide supervision for the occluded background, while joint photometric optimization can alter geometry to compensate for appearance errors and destabilize the correspondences used for appearance transfer. The proposed approach therefore refines geometry before fixing it for appearance refinement.

The method has three stages. First, object-level transient filtering combines a motion prior with RGB residuals and DINOv3 feature differences, shares information through descriptor-based coefficients, and selects whole SAM2 candidate masks. These masks are then fixed. Second, Cross-view Depth Regularization (CDR) filters aligned Depth Anything 3 predictions using masks and cross-view consistency, fuses retained samples into a shared TSDF, and uses raycast reference depths for surface and one-sided free-space losses. Third, Cross-view Appearance Transfer (CAT) fixes geometry, opacity, and the Gaussian set, then builds color targets from unmasked source observations through directional color fitting and attention-weighted residual transfer. Auxiliary gradients act on zero-order SH coefficients and are constrained relative to the unmasked RGB gradient. Evaluation covers six NeRF On-the-go scenes and nine self-captured scenes using PSNR, SSIM, and LPIPS.

Critical strengths

1. The overall workflow addresses a well-motivated limitation of masking. Removing transient observations does not supply supervision for the static surfaces behind them. Stabilizing geometry with CDR before transferring appearance with CAT is a reasonable response to this gap. Under the same masks and training budget, Table 3 improves from 22.60 dB without either component to 23.68 dB with the full method, supporting the practical value of the added supervision.

2. Depth priors are used cautiously, with improvements demonstrated on two datasets. CDR removes masked and cross-view-inconsistent samples, constructs reference depths from a shared TSDF, and distinguishes surface constraints from one-sided free-space constraints. Evaluation on six public and nine self-captured scenes provides useful breadth. The method reaches mean PSNR/SSIM of 23.68 / 0.836 on the public benchmark, and the qualitative depth comparisons show clearer boundaries and smoother surfaces.

Critical weaknesses

1. The central mechanism is not directly validated, and the contributions of the components remain difficult to separate. Table 3 uses the same masks and training budget and demonstrates the benefit of adding CDR/CAT. However, it does not keep CAT supervision fixed while changing only whether geometry is frozen. Consequently, it does not directly test the central claim that geometry updates cause correspondence drift and that freezing geometry prevents it. CDR and CAT alone achieve 23.53 and 23.48 dB, respectively, leaving marginal gains of 0.15–0.20 dB when both are present. Overlapping gains are not inherently problematic, but make it more important to distinguish the mechanisms involved. Figure 8 provides qualitative depth and appearance ablations without a quantitative assessment of geometry or correspondence changes.

The masks fixed after the first stage also determine the valid depth samples for CDR and usable source observations for CAT, yet Figure 7 compares masking variants only qualitatively. The experiments do not sufficiently separate the contributions of the new masking mechanism, depth regularization, and appearance transfer, or characterize the propagation of early filtering errors. Given the mean PSNR improvement of 0.26 dB over the strongest reported baseline on the public benchmark, this evidence matters when assessing whether the additional design complexity is justified.

2. The coupling of numerous external components with reconstruction iterations is insufficiently specified, affecting clarity, reproducibility, and cost assessment. The pipeline uses GeoMotion motion priors, DINOv3 features, SAM2 masks, and Depth Anything 3 depth, together with auxiliary Gaussian reconstruction, probability fitting and whole-mask selection, TSDF fusion, directional color fitting, EMA-based residual attention, and gradient projection. Using external components is reasonable, but the paper needs a clear account of when each module runs, which outputs are cached or refreshed, and what the proposed mechanisms contribute beyond the underlying priors.

The text establishes the mask–geometry–appearance order, and Figure 1 marks 0/10k/20k/30k iterations. What remains unclear is whether auxiliary reconstruction is included in this timeline, whether its Gaussians are inherited by later stages, how mask probabilities and residuals are updated relative to Gaussian iterations, and when TSDF construction and CAT precomputation occur at stage transitions. Knowing the stage order is insufficient to reconstruct the training procedure. Despite detailed local rules in the appendix, I could not locate the actual values of λ_surf, λ_free, and λ_src. The complete runtime and memory cost, including prior inference, auxiliary reconstruction, and preprocessing, is also not reported. This makes the tradeoff between performance and system complexity difficult to assess.

Questions for authors
  1. Targeted validation of the central mechanism. With the same masks, training budget, and CAT supervision, what happens if geometry is allowed to update during CAT? Please relate the image metrics to changes in depth or correspondences to establish the effect of freezing. The qualitative evidence in Figure 8 is useful, but this intervention should be distinguished from enabling or disabling CDR/CAT.

  2. Clarification of the training procedure and iteration schedule. Please provide pseudocode or a schedule corresponding to Figure 1's 0–10k, 10k–20k, and 20k–30k intervals. It should identify auxiliary reconstruction and parameter inheritance, the Gaussian parameters updated in each stage, mask-probability/residual refresh frequency, when TSDF and CAT targets are constructed, and the update order for attention, EMA, and Gaussian optimization. Please specify λ_surf, λ_free, and λ_src and whether they are fixed across scenes, and report end-to-end time and peak memory including external priors and preprocessing. The motion-prior naming in Figure 2 and the text should also be reconciled to identify the actual component used.

  3. Targeted ablation of external priors and proposed modules. With the subsequent CDR/CAT stages and budget fixed, please compare final reconstruction quality using the full masking scheme versus a simplified motion-prior/SAM2 filtering scheme, and report the fraction of depth samples retained by CDR. A quantitative ablation of residual transfer or gradient projection in CAT would further clarify which parts of this complex pipeline are essential. Existing mask annotations could help explain errors, but a new large annotation effort is not necessary.

  4. Clarification of baseline and metric comparability. Why is AsymGS included in qualitative comparisons but not quantitative tables? Please clarify whether its quantitative results can be added and explain the baseline selection for the self-captured dataset. Appendix B.1 distinguishes AlexNet and VGG LPIPS settings; this distinction should also be explicit in the main table. Please clarify whether the cited results use matching test splits and image preprocessing, so that the relatively modest performance margins can be interpreted appropriately.

Reviewer_1gxDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes GAD-Splat, a geometry-appearance decoupled framework for distractor-free 3DGS. It combines CDR, and CAT to improve geometry and appearance reconstruction. Experiments demonstrate improved rendering quality and transient artifact removal.

Critical strengths

S1. Well-motivated framework. The paper proposes a geometry-appearance decoupled framework to address transient-object interference in 3DGS reconstruction, with a clear motivation and reasonable design.

S2. Effective reconstruction improvements. The proposed method achieves improvements over several existing baselines in novel-view synthesis quality and transient artifact removal.

Critical weaknesses

W1. Lack of Quantitative Evaluation of Geometric Accuracy. The paper presents CDR as a core contribution and claims that it improves Gaussian geometry. However, the experiments primarily report PSNR, SSIM, LPIPS, and qualitative comparisons of rendered depth maps. These evaluations do not directly demonstrate improvements in 3D geometric accuracy.

W2 Limited experimental diversity. The evaluation is limited to two datasets, including only six scenes from the public NeRF On-the-go benchmark and nine self-captured scenes. More evaluations on diverse public benchmarks are needed to better demonstrate the generalization and robustness of the proposed method.

W3.Insufficient analysis of computational overhead. The method relies on multiple pretrained models, including DINOv3, SAM2, and RAFT, DepthAnything. However, the additional computational overhead is not sufficiently analyzed. End-to-end runtime, GPU memory consumption, and the cost of individual components should be reported.

Questions for authors

Q1. Geometry Reconstruction Accuracy. CDR is presented as a key contribution for improving Gaussian geometry. Could the authors provide quantitative geometric evaluations, such as depth errors or surface reconstruction metrics, rather than relying primarily on rendering metrics and qualitative depth comparisons?

Q2. Computational Overhead. The method relies on multiple pretrained models, including DINOv3, SAM2, and Depth Anything 3. Could the authors report the end-to-end runtime, GPU memory consumption, and computational overhead of individual components?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.