acceptodds
Under review as a conference paper at ICLR 2027

DiFSplat: Extrapolative Feed-Forward 3D Gaussian Splatting with Diffusion Feature

Abstract

Feed-forward 3D Gaussian Splatting (3DGS) models enable efficient novel view synthesis but struggle with extreme view extrapolation. Their image-based features (IBFT) rely on source observations and become unavailable in unseen regions, leading to geometric holes and blurry artifacts. We propose DiFSplat, an extrapolative feed-forward 3DGS framework that combines deterministic IBFT with generative diffusion features (DiFT) from a pre-trained 3D-aware diffusion model. We project the initial geometry into novel views and initialize uncovered depths through nearest-neighbor dilation. A frozen diffusion U-Net, conditioned on point renderings, provides features for an auxiliary depth predictor that refines these depths by matching across source and novel views in DiFT space. The refined depths are unprojected to instantiate auxiliary voxels as anchors for unseen geometry. We then fuse IBFT and DiFT on the unified voxel set to predict Gaussian parameters. Experiments show improved extrapolation and zero-shot cross-dataset generalization while maintaining high-fidelity interpolation, demonstrating the benefit of using diffusion priors for both geometry completion and feature fusion.

Primary area:
generative models
Keywords:
Feed-forward 3DGS, Diffusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bV3jOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a feed-forward 3D Gaussian Splatting framework for better view extrapolation. Specifically, DiFSplat uses intermediate diffusion features from pretrained diffusion models to guide unseen regions. By applying extrapolative fine-tuning, DiFSplat shows relatively higher fidelity in extrapolated areas.

Critical strengths

While many methods use diffusion generated images as prior, DiFSplat extracts intermediate diffusion features to guide unobserved areas, which can be considered a novel idea.

Critical weaknesses

The claimed problem does not seem to be solved. The proposed method is still struggling to generate unseen areas with high fidelity as shown in the qualitative results. It is expected that after explicit extrapolative training the model can provide more reasonable novel parts instead of large artifacts, and feed-forwardly fix that would be valuable. However, the current reults are far from applicable with still severe artifacts. For the mild extrapolation in DiFSplat's experiment setting, simply run a SOTA camera conditioned video generation model and then a feed-forward recontruction model can already provide high quality 3DGS results covering unseen areas in the input views. Thus this method does not provide clear advantage.

Questions for authors

The authors are welcomed to address my concerns raised in the Weaknesses section by clarifying the advantage of DiFSplat.

Reviewer_38mAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DiFSplat uses diffusion features to predict auxiliary geometry and enrich voxel features for sparse-view Gaussian reconstruction. It reports improved interpolation, extrapolation, and cross-dataset results.

Critical strengths

The goal is clear, and jointly adding geometric support and appearance features is sensible. The reported gains are consistent, and the video shows less stretching in newly exposed regions.

Critical weaknesses

Critical Weaknesses

  1. The comparisons do not establish an advantage over existing diffusion-assisted feed-forward methods. MVSplat360 is directly relevant and has publicly available code and pretrained models, but it is missing from the comparisons. The VolSplat+See3D baseline performs substantially worse than VolSplat itself, so beating this simple combination does not demonstrate an advantage over a well-developed generative approach. Evaluation is also limited to RE10K and ACID. The ACID transfer results are useful, but a comparison on wider viewpoint changes, such as the DL3DV setting studied by MVSplat360, would better test the claimed extrapolation capability. Input views and evaluation conditions should be matched.

  2. The ablations do not separate unseen-region completion from geometric densification. The key missing control is to use the dilation-initialized depths directly while keeping DiFT feature fusion and the rest of the pipeline unchanged. Without this comparison, it is unclear how much DiFT depth matching contributes beyond adding geometric support. The uncovered mask is also based on projected point coverage, which may include sampling gaps on visible surfaces rather than only genuinely unobserved regions. Since the masked evaluation uses the same type of coverage definition, it does not independently rule out this explanation. The full system improves rendering quality, but the evidence does not yet show whether the gains come from recovering unseen geometry or simply filling gaps in the initial reconstruction.

Questions for authors
  1. How does DiFSplat compare with MVSplat360 under matched input views and evaluation settings, particularly on wide-baseline DL3DV scenes?

  2. How much does DiFT depth refinement improve over directly using the dilation-initialized depths, with feature fusion and all other components unchanged?

  3. Does the uncovered mask include sampling gaps on visible surfaces? If these gaps are excluded, do the gains remain in genuinely unobserved regions?

Reviewer_DbLEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a feed-forward gs method for novel view synthesis. This pipeline is built on VoltSplat and applies DiFT to predict geometry in unseen regions. The main contribution of this paper is the combination of image-based features and diffusion features.

Critical strengths
  1. Combining DiFT features with explicit 3D geometry is interesting, though it sounds more like an engineering trick.

  2. The method surpasses other baselines on two datasets. And the ablations examine the efficiency of the proposed modules.

Critical weaknesses
  1. Insufficient pipeline. This paper doesn't compare with some common baselines, eg, MVSplat and pixelSplat. Camera-conditioned video generation is also a relevant track, but this paper lacks discussion and comparison with those methods.

  2. Unclear pipeline. Fig 2 does not clearly introduce how dift feature is integrated into the pipeline. DiFT module is an additional module rather than a carefully designed framework for geometry reconstruction.

  3. The motivation is weak. This paper does not clearly explain why adding DiFT would reduce gaussian artifacts. The artifacts of VolSplat shown in Fig 1 are also uncommon.

  4. The efficiency is not clear. This paper introduces additional diffusion models and it would lead significant computional cost.

  5. It is actually not clear what the strength of this paper is from visual comparisons, both images and video in supp.

Questions for authors
  1. Why adding DiFT would reduce GS artifact intuitively? Can authors show the improvement come from DIFT rather than adding auxilary geometry?

  2. How can Dift extracted from incomplete renderings provide reliable depth info for unseen region?

  3. Can authors explain why camera-conditioned video generation methods are not included in this paper?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.