acceptodds
Withdrawn from ICLR 2027

From Uncertainty to Stability and Fidelity: Guiding Sparse-View 3D Gaussian Splatting with Fisher Information

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

3D Gaussian Splatting (3DGS) achieves high-quality novel view synthesis but requires dense input views and overfits in sparse-view settings, producing noticeable artifacts and degraded rendering quality. Previous methods mitigate this issue by introducing additional geometric priors or regularization techniques (e.g. Dropout). However, these methods lack principled guidance: prior-based augmentation randomly samples novel viewpoints, while Dropout-based regularization randomly removes Gaussians, which introduces uncertainty and instability. In this paper, we propose a Fisher Information-guided method that quantitatively guides both strategies. First, Fisher Information-guided Stereo Augmentation actively select most informative supporting views and use depth priors to curate reliable pseudo ground truths, which improves stability and rendering fidelity. Second, Uncertainty-aware regularization measures the uncertainty of each Gaussian using Fisher Information, and adaptively adjusts the removal probability, leading to more stable and effective regularization. With these two components, our method effectively mitigates overfitting and improves the stability of optimization in sparse-view 3DGS, resulting in superior rendering fidelity. Extensive experiments on sparse-view novel view synthesis show that our method achieves state-of-the-art rendering quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Sparse-view Novel View Synthesis, Fisher Information

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wNbSOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces two components for sparse-view 3D Gaussian Splatting. First, Fisher Information-guided Stereo Augmentation selects supporting training views for a sampled novel camera and uses Depth Anything 3 depth estimates to unproject support views and construct pseudo-ground-truth images through soft z-buffer rendering. Second, Uncertainty-aware Regularization estimates a per-Gaussian uncertainty score from a Fisher-information submatrix and assigns higher dropout probabilities to Gaussians with lower estimated uncertainty. The paper also proposes soft-scale dropout, which reduces Gaussian opacity instead of setting it to zero. The method is evaluated on LLFF, DTU, and Mip-NeRF 360 under sparse-view settings, with reported improvements in rendering quality and run-to-run stability.

Critical strengths
  1. The paper addresses two relevant sparse-view problems in one framework. It attempts to improve both pseudo-view supervision and the instability caused by indiscriminate Gaussian dropout, giving the method a coherent practical motivation.

  2. Soft-scale dropout is a reasonable rendering-aware design. Retaining a reduced opacity can preserve part of the occlusion relationship along a ray and may avoid some abrupt compositing changes caused by completely removing a Gaussian.

Critical weaknesses
  1. The Fisher derivation conflates expected and observed curvature in the support-view selection argument. For a nonlinear renderer, the observed Hessian includes residual-dependent second-derivative terms in addition to the Jacobian Gram term. Equality therefore requires a Gauss-Newton or local-linearization assumption. The derivation does not specify this assumption.

  2. The view-selection rule is an indirect proxy for visual similarity. The paper minimizes a trace upper bound on information gain and interprets lower values as indicating closer views, but this does not by itself establish that the selected views yield more reliable depth-warped targets.

  3. The determinant-based uncertainty score is parameterization sensitive. Rescaling heterogeneous Gaussian parameters changes the determinant, and a small value may reflect low visibility or degeneracy rather than over-optimization.

  4. The pseudo-ground-truth construction relies on Depth Anything 3. Its depth errors, occlusions, or incomplete projected coverage can enter the pseudo-target, while Fisher selection does not directly assess depth reliability. The paper does not establish how dependent the augmentation is on this particular depth prior.

  5. Soft opacity scaling has a nonlinear compositing effect. Reducing one Gaussian changes the relative contributions of other Gaussians along the ray, so one decay factor does not uniformly represent removal across visibility configurations.

  6. The incremental novelty of the dropout component is not fully established. DropGaussian and DropoutGS already use dropout-based sparse-view regularization. The paper does not clearly distinguish its Fisher-conditioned selection and soft scaling as a sufficiently distinct conceptual advance.

Questions for authors
  1. For support-view selection, does the implementation use the observed Hessian, expected Fisher information, or a Gauss-Newton/Jacobian-Gram approximation? Please clarify this distinction and align the derivation with the implemented quantity.

  2. How sensitive is the pseudo-view augmentation to Depth Anything 3's depth quality? Could another depth estimator be used with the same view-selection and unprojection pipeline?

  3. How does the score distinguish over-optimization from poor visibility, degeneracy, or parameter scaling? Is it normalized across Gaussian parameter blocks?

Reviewer_A8JfOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a Fisher-information-guided framework for sparse-view 3D Gaussian Splatting. It selects support images to construct pseudo training views through depth-based warping, and uses per-Gaussian uncertainty to guide dropout, with opacity attenuation instead of complete removal. Experiments on LLFF, DTU, and Mip-NeRF360 report improved rendering quality, and ablations on 3-view LLFF show gains from both components and lower PSNR variance across runs.

Critical strengths
  • The method presents a systematic way to use Fisher information to help with sparse-view 3D gaussian splatting with support-view selection and Gaussian dropout.
  • The experiments show improvements across 3 benchmarks, and the ablation studies support the benefits of both FIGSA and UAR.
Critical weaknesses
  • The motivation does not fully align with the proposed method: The paper identifies random sampling of pseudo-training-view camera poses as a source of instability. However, it still generates these poses by randomly perturbing the input training cameras. Fisher information is only used to select the input images for warping and the pseudo ground truth are still rendered with perturbed camera poses. The paper needs to explain how this addresses the instability attributed to random camera sampling.

  • The ablations are limited to the 3-view LLFF setting and are insufficient to isolate the contribution of the Fisher-based selection criterion: It is unclear if the improvement with the proposed FIGSA and UAR hold with more input views, where support-view selection becomes more important. The current comparisons in Table 6 vary both the selection criterion and the number of support images, making it difficult to isolate the benefit of Fisher-based selection. Nearest-k and random-k baselines, with k matched to the number of images selected by FIGSA for each pseudo training view, would provide a more controlled comparison.

  • Several relevant baselines are missing, and the use of geometric priors needs clarification: D²GS (ICLR 2026) and SE-GS (ICCV 2025) directly address adaptive regularization and stability, and report better results on some metrics under the same evaluation datasets and number of views. MAtCha Gaussians (CVPR 2025) is also an important comparison using strong geometric priors. For fair comparison, the authors should also clarify if baselines and ablations involving warping, point-cloud initialization, or depth priors also use DA3.

  • The gains over the evaluated baselines are modest in several settings. For example, the PSNR improvements over DropGaussian are only 0.06 dB on LLFF with 9 views and 0.11 dB on Mip-NeRF360 with 24 views. The supplementary videos also show limited qualitative improvement over the baselines. In particular, the proposed method still produces noticeable floaters in the tables in fortress and room scenes.

Questions for authors
  • How does Fisher-based support-view selection mitigate the instability attributed to random sampling of pseudo-training-view camera poses, given that these poses are still randomly sampled?

  • Do the gains from FIGSA and UAR hold beyond the 3-view LLFF setting? Does FIGSA outperform nearest-k and random-k selection when, for each pseudo training view, k is set to the number of support images selected by FIGSA?

  • Do the warping, point-cloud initialization, and depth-prior baselines use the same DA3 outputs as your method? Does the method outperforms SOTA methods for sparse-view gaussian splatting, including D²GS, SE-GS, and MAtCha Gaussians, under matched evaluation conditions?

  • How are “over-optimized” and “under-optimized” Gaussians defined? Beyond the rendering improvements in Table 8, is there more direct evidence that the Fisher-derived uncertainty score reliably identify these two type of gaussian?

I would consider raising my score if the authors address these questions with clear explanations and the necessary supporting experiments.

Reviewer_ed47OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Fisher-information-guided framework for sparse-view 3D Gaussian Splatting. It uses a Fisher-based criterion to select support images for depth-guided warping into randomly sampled pseudo viewpoints. It also derives per-Gaussian uncertainty scores from Fisher information and preferentially applies dropout to Gaussians with lower estimated uncertainty. A soft dropout mechanism attenuates opacity rather than removing selected Gaussians completely. Experiments on LLFF, DTU, and Mip-NeRF 360 report improved novel-view rendering quality, while LLFF ablations report reduced variation across runs.

Critical strengths
  1. The framework addresses two practical aspects of sparse-view reconstruction within a coherent pipeline. Selecting support images can improve the coverage of warped pseudo supervision, while softened dropout provides a reasonable way to moderate the disruption caused by complete Gaussian removal. The component ablations show that both augmentation and regularization contribute to the final rendering results.
  2. Evaluation covers complementary public benchmarks and multiple view counts. The reported improvements on forward-facing LLFF scenes, object-centric DTU scenes, and Mip-NeRF 360 provide broader empirical support than a single-dataset evaluation. The explicit examination of variation across runs is also relevant to the paper’s stability objective, although its statistical protocol needs clarification.
Critical weaknesses
  1. The Fisher-based theoretical interpretation is insufficiently justified. Equation 7 equates Fisher information with the observed negative-log-likelihood Hessian because Fisher information is independent of the observation. For a nonlinear renderer under Gaussian noise, the observed Hessian additionally contains residual-weighted second derivatives; the equality generally requires an expectation or an explicit Gauss–Newton approximation. Furthermore, high Fisher information indicates local parameter sensitivity or constraint under the assumed model, but does not by itself establish that a Gaussian is over-optimized or overfitted. Consequently, the mathematical framing does not yet substantiate the proposed interpretation of the dropout policy.
  2. The experiments do not sufficiently isolate the value of Fisher guidance from simpler selection and regularization effects. The augmentation comparison changes both the support-selection rule and the number of contributing views, without a matched-support-count geometric selection baseline. Likewise, Fisher-ranked dropout is not compared with rankings based on visibility or rendering contribution. The comparison against direct DA3 Gaussian predictions does not isolate the benefit of Fisher guidance when using the same depth priors. These omissions are consequential because the proposed guidance adds substantial training cost, while simpler depth-based augmentation and soft dropout already explain part of the improvement. The stability claim is also difficult to assess without a clearly specified number of runs and aggregation protocol.
Questions for authors
  1. Clarification of the Fisher computation: Is Equation 7 intended as a Gauss–Newton approximation? Which Gaussian parameters are included in the Fisher matrices, over which views are they accumulated, and what matrix approximation and damping are used for inversion and log-determinant computation?
  2. Targeted validation of Fisher guidance: With the same DA3 depths, pseudo viewpoints, support-view count, and training schedule, how does Fisher-based support selection compare with geometric-neighbor or overlap-based selection? For dropout, is there evidence that Fisher ranking outperforms visibility- or contribution-based ranking under the same dropout and opacity-attenuation settings?
  3. Clarification of stability and computational trade-offs: How many independent runs underlie the reported σPSNR\sigma_{\mathrm{PSNR}}, which sources of randomness vary, and how is the statistic aggregated across scenes? Does the quality advantage persist under an equal training-time budget, particularly on Mip-NeRF 360 where the reported runtime more than doubles?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.