acceptodds
Withdrawn from ICLR 2027

FinerSplat: Distilling Diffusion Priors for Feed-Forward Gaussian Refinement

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Feed-forward 3D Gaussian Splatting (3DGS) enables efficient reconstruction from sparse views, yet its single-pass prediction must determine both Gaussian attributes and primitive allocation without rendering feedback, often producing inaccurate and redundant primitives in under-constrained regions. Although generative priors can improve rendered images, transferring them into 3D Gaussian representations often requires costly per-scene optimization. We present FinerSplat, a feed-forward framework that casts refinement and compaction as coupled primitive-level decisions without per-scene optimization. It restores target-view renderings with a frozen diffusion model and lifts the resulting residuals into Gaussian-level observations through differentiable rendering contributions. An entropy-aware primitive-level Transformer predicts recurrent attribute updates and primitive-wise keep scores from these observations. Experiments on DL3DV and Waymo demonstrate that FinerSplat performs feed-forward refinement in 0.6 s and removes 30% of the initialized Gaussians while preserving rendering quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-Forward 3D Gaussian Splatting, Diffusion Model, Recurrent Network, Novel View Synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_G5mJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FinerSplat, a feed-forward framework for refining and compacting sparse-view 3D Gaussian Splatting representations without per-scene optimization. The method uses a frozen diffusion model to restore renderings from under-constrained target views, lifts the resulting image-space residuals into Gaussian-level feedback through differentiable rendering contributions, and employs an entropy-aware KNN Transformer to iteratively update Gaussian attributes while predicting primitive keep scores for pruning. 16873 The approach is evaluated on DL3DV and Waymo against feed-forward, optimization-based, and diffusion-based baselines. The reported results show improved rendering quality and efficient reconstruction, while pruning about 30% of the initialized Gaussians with little degradation in rendering performance.

Critical strengths
  1. FinerSplat introduces a meaningful diffusion-guided refinement strategy for feed-forward 3DGS. It lifts target-view restoration residuals to Gaussian-level feedback, helping improve under-constrained regions without per-scene optimization.

  2. The method achieves a strong quality–efficiency trade-off. It improves rendering quality with 0.61 s reconstruction time while pruning about 30% of the initialized Gaussians, with ablations supporting the effectiveness of its main components.

Critical weaknesses

Critical Weaknesses

  1. The reported ablation results contain a clear internal inconsistency that weakens confidence in the empirical evidence. Table 3 reports the Full Model as 23.12 / 0.717 / 0.202 (PSNR/SSIM/LPIPS), whereas Table 4 reports 22.71 / 0.717 / 0.243 for the default pruning ratio ρ=30%\rho=30\%. Since the default pruning configuration should correspond to the reported full model, these values should be consistent, yet both PSNR and LPIPS differ substantially and the manuscript provides no explanation. This directly affects the reliability of the ablation and pruning conclusions.

  2. Several central components of the proposed refinement pipeline are insufficiently specified, making both the method and its efficiency claims difficult to verify. Eq. (4) concatenates a ResNet-derived feature residual with an H×WH\times W RGB residual without defining how their spatial resolutions are aligned; Eq. (9) introduces git+1=git⊕Δgitg_i^{t+1}=g_i^t\oplus\Delta g_i^t without specifying how ⊕\oplus operates on the different constrained Gaussian parameters; and Eq. (12) does not clearly define the keep-score range or the deletion prior. In addition, the four target views used for diffusion-based refinement are not selected by a clearly specified protocol, even though their poses directly determine the information available to the method. Finally, the reported 0.61 s runtime is presented without a breakdown despite the additional rendering, diffusion restoration, residual propagation, and recurrent refinement stages. Together, these omissions make it difficult to reproduce the method precisely or determine whether the reported efficiency is directly comparable to feed-forward baselines.

Questions for authors
  1. The reported ablation results contain a clear internal inconsistency. Table 3 reports the Full Model as 23.12 / 0.717 / 0.202 (PSNR/SSIM/LPIPS), while Table 4 reports 22.71 / 0.717 / 0.243 for the default pruning ratio ρ=30%\rho=30\%. These values do not match, and the discrepancy is not addressed in the manuscript. This raises concerns about the reliability of the reported ablation and pruning results.

  2. Several core operations of the method are mathematically under-specified. In Eq. (4), the ResNet-derived feature residual is concatenated with an H×WH \times W RGB residual without defining how their spatial resolutions are aligned. Eq. (9) introduces the update git+1=git⊕Δgitg_i^{t+1} = g_i^t \oplus \Delta g_i^t, but the operator ⊕\oplus is not specified for different Gaussian parameters. Eq. (12) also does not explicitly define the range of the keep score sis_i or the computation of the deletion prior pip_i. These omissions concern central parts of the proposed refinement and pruning mechanism and make the formulation difficult to verify precisely.

  3. The target-view refinement protocol is insufficiently specified. The method uses four additional target views for diffusion-based restoration after initializing the scene from four context images, but the paper does not clearly define how these target camera poses are selected. Since the proposed refinement explicitly depends on feedback from these additional viewpoints, their selection directly affects the amount and difficulty of information available to the method, making the experimental protocol difficult to assess and reproduce.

  4. The reported efficiency is insufficiently substantiated given the additional diffusion-based processing required by the method. FinerSplat reports a reconstruction time of 0.61 s, comparable to conventional feed-forward baselines, despite additionally rendering target views, running a diffusion restoration model, propagating residuals to Gaussians, and performing recurrent refinement. The paper does not provide a runtime breakdown for these components, making it difficult to assess whether the reported timing is directly comparable to the baselines.

Reviewer_JPLfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed FinerSplat, which refines and compresses the results of feedforward Gaussian reconstruction by aggregating the diffusion-recovered rendering residuals into Gaussian-level features. An entropy-based local Transformer model is employed to predict recursive attribute updates and pruning scores. Experiments on the DL3DV and Waymo datasets demonstrate that this method not only improves rendering quality with running time of 0.61 seconds, but also achieves a 30% Gaussians removal rate with minimal quality loss. However, the study does not provide comparison results against approaches that use only initializers and simple pruning strategies, and details regarding runtime measurements remain unclear, which limits the effective validation of the proposed quality-efficiency trade-off scheme.

Critical strengths

A beneficial trade-off between rendering quality and compactness. Removing 30% of the Gaussian distribution largely preserves rendering quality, which indicates a significant reduction in redundancy.

The reconstruction performance is significantly improved. This method improves the PSNR value of DL3DV by 1.37 dB compared to ReSplat and integrates a diffusion-guided mechanism without the need for gradient optimization during testing.

Critical weaknesses
  1. The validation of the contribution of central compression remains insufficient. Pruning ratio analysis indicates that primitives can be removed, but it does not prove that the entropy-guided learning scoring method outperforms simpler pruning strategies. Furthermore, local color variance is not a direct measure of geometric redundancy. In the absence of a budget-matched control group, the necessity of this design remains unclear.

  2. The existing evidence is insufficient to fully support the claimed advantages of the proposed method. Although DepthSplat is an initializer, the absence of standalone results makes it difficult to isolate and analyze the optimization gains it provides. Compared to DepthSplat + DIFIX3D+, PSNR improved by only 0.25 dB, while LPIPS actually worse.

Questions for authors
  1. Given the same Gaussian budget, how does entropy-guided pruning perform compared to pruning based on opacity or rendering contribution? Does it preserve fine details and geometrically important regions with uniform colors?

  2. Please clearly specify the timing hardware and resolution used, and whether the 0.61 seconds reported in the paper includes initialization, diffusion recovery, feature extraction, and all refinement operations?

  3. Under the same protocol, how does image quality improve compared to using DepthSplat alone?

Reviewer_GVNhOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FinerSplat to address Gaussian inaccuracies and redundancies in sparse-view feed-forward 3DGS. The proposed method aims to overcome the limitations of existing approaches that struggle to recover sufficient information in unobserved regions or require substantial computational time for per-scene optimization.

The main contributions are as follows. First, residuals are constructed using images restored by a pretrained diffusion model as pseudo targets. Second, Local Color Entropy and a KNN Transformer are employed to aggregate information from neighboring Gaussians and iteratively refine Gaussian attributes. Third, primitive-wise keep scores are predicted to prune redundant Gaussians.

Critical strengths

This paper propagates residuals from diffusion-restored images to 3D Gaussians for feed-forward refinement. A key strength of the proposed method is its ability to jointly refine Gaussian primitives and remove redundant Gaussians within a unified framework.

Critical weaknesses
  • The key design choices (color entropy, Euclidean KNN grouping, and contribution-based pruning) are not sufficiently justified or validated.
  • The experimental protocol is unclear. The full-model PSNR differs across Tab. 1, Tab. 3, Tab. 4, and Fig. 6, and generalization is evaluated only in-domain.
  • Detailed questions are provided in the "Questions for Authors" section.
Questions for authors
  1. The choice of Color Entropy as an explicit measure of local information richness raises concerns. In addition to color, 3DGS primitives have other attributes such as opacity and scale that are also related to Gaussian overlap and spatial density. Since the proposed method relies solely on color and lacks experimental validation of this design choice, stronger justification is needed to support its effectiveness.
  2. The proposed method identifies neighboring Gaussians based on the Euclidean distance between their centers. However, spatial proximity in 3D space does not necessarily guarantee that neighboring Gaussians belong to the same surface. This issue may arise when objects are in close contact or when the front and back surfaces of thin objects are spatially close. In such cases, the proposed approach may incorrectly group Gaussians from different surfaces into the same neighborhood. Given the absence of ablation experiments investigating this issue, further clarification and experimental validation are needed.
  3. The paper assumes that Gaussians with low rendering contributions are more likely to be redundant and removes the lowest-ranked Gaussians according to a predefined pruning ratio. However, some Gaussians may still be necessary despite their low rendering contributions. For example, Gaussians that are invisible from the observed camera views may represent important surfaces from novel viewpoints. Similarly, Gaussians with low contributions may need to be preserved to maintain fine geometric details. This raises concerns about whether the proposed method can distinguish genuinely redundant Gaussians from those that simply have low rendering contributions.
  4. The paper acknowledges diffusion hallucination as a limitation. While recognizing this limitation is appropriate, failure case experiments should have been included to demonstrate the extent to which diffusion hallucination affects 3D fidelity.
  5. The full-model PSNR differs across Tab. 1 (21.49 dB), Tab. 3 (23.12 dB), and Tab. 4 (22.71 dB, ρ=30%\rho = 30\%). Notably, the value in Tab. 3 appears to match iteration 0 of the 4-target-view curve in Fig. 6, rather than the default T=2 (about 23.7 dB). The evaluation split, number of scenes, image resolution (cf. “native-resolution PSNR” in Supp. Fig. 1), and number of refinement stages should be clarified for each result.
  6. Removing entropy degrades all metrics in Tab. 3, but since i is fed into both the keep-score head (Eq. 10) and the residual decoder (Eq. 9), it is unclear whether entropy helps pruning, as claimed in L281, or attribute refinement. Tab. 4 without entropy and visualizations of the entropy map and pruned Gaussians should be provided.
  7. FinerSplat is evaluated only in-domain (DL3DV and five Waymo scenes). Zero-shot results of the DL3DV-trained model on other benchmarks, such as RealEstate10K (SIGGRAPH 2018) and Mip-NeRF 360 (CVPR 2022), should be provided. Results with another initializer would also support the initializer-agnostic claim (L178).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.