acceptodds
Under review as a conference paper at ICLR 2027

DIGS: Diffusion Priors Guided by Information Gain for Extrapolative Gaussian Splatting

Abstract

Recent advances in neural radiance fields and 3D Gaussian Splatting (3DGS) have enabled photorealistic, real-time novel view synthesis. However, reconstruction quality often degrades severely at extrapolated views that deviate from the training views. Existing methods improve extrapolative reconstruction by generating pseudo ground truths at pseudo views with diffusion models. Yet they overlook either the information gain at pseudo views or the reliability of the generated images, both of which are pivotal to effectively integrating diffusion priors. In this work, we introduce DIGS, a framework that integrates Diffusion priors guided by Information gain for extrapolative Gaussian Splatting. We first propose a Group-Block-Diagonal (GBD) approximation of the Fisher information matrix to accurately and efficiently estimate the information gain of pseudo views. We further propose Reliable Information Gain (RIG), which accounts for the reliability of the diffusion-generated pseudo ground truths. Building on RIG, DIGS selects informative and reliable pseudo views and optimizes the 3DGS model in an uncertainty-aware manner. Experiments on three challenging benchmarks show that DIGS consistently outperforms existing methods, enabling a more principled and effective integration of diffusion priors into 3DGS.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Novel View Synthesis, 3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_8PiaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes DIGS, a 3D Gaussian Splatting approach guided by diffusion priors. DIGS consists of four major components: (1) Group-Block-Diagonal (GBD) approximates Fisher information using blocks containing neighboring Gaussians, preserving some inter-Gaussian correlations. (2) Reliable Information Gain (RIG) measures the reliability of the diffusion-generated pseudo ground truths using Depth Anything 3 and computes the gain of reliable Fisher information. (3) Greedy view selection strategy progressively selects the pseudo view with the highest RIG. (4) Uncertainty-aware optimization emphasizes pseudo supervision where rendering uncertainty and estimated reliability are high. The experiments show that DIGS consistently outperforms the baselines qualitatively and quantitatively. Ablation studies also verify the individual effectiveness of GBD and RIG.

Critical strengths
  1. The proposed DIGS demonstrates consistent improvements over the NerfBusters, WildExplore, DL3DV benchmarks.
  2. The supplementary material includes high-quality rendered videos with plausible 3D consistency.
  3. The proposed GBD and RIG are compelling and mathematically sound.
Critical weaknesses
  1. Missing DA3 details. The paper should explicitly explain how point-wise confidence is computed by DA3. Explaining the full pipeline of DA3 is not necessary. However, since the reliability estimation is a core component of the proposed DIGS, the paper should at least provide intuitive explanation about how the geometry consistency is measured and used to guide the reliability estimation in DA3. Equation 4 along does not contain much information.
  2. Critical implementation details missing. The paper misses some implementation details, such as the input views in Table 2 and the input view selection protocol in Table 3. These details are critical for the evaluation.
  3. Insufficient baseline coverage. Diffusion-guided 3DGS is often used under sparse-view scenarios to fill the unseen parts. I believe these baselines are highly related to the paper and should be included for comparison. Examples include Deceptive-3DGS [1], RI3D [2], GSFixer [3], and many more. More critically, FreeFix also uses Fisher information to guide the diffusion denoising process. The paper includes the discussion with FreeFix but does not include the comparison.
  4. Insufficient evidence of extrapolation. The method is motivated by boosting the 3D consistency of pseudo extrapolated views. However, the paper does not provide sufficient evidence to show that DIGS improves the quality of pseudo views and rendered views with large viewpoint extrapolation. The paper has to at least include an additional figure with examples of the training views, selected extrapolated pseudo views, and rendered views at the same camera poses.

Overall, I find the method technically sound and novel. However, some important implementation details are missing, and the evaluation lacks key comparisons and analyses. Therefore, I recommend weak reject in its current form. I am open to raise my score if the above concerns are well-addressed during the rebuttal period.

  • [1] Liu et al., “Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction”, ECCV 2024.
  • [2] Paliwal et al. “RI3d: Few-shot gaussian splatting with repair and inpainting diffusion priors”, ICCV 2025.
  • [3] Yin et al. “GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors”, ICML 2026.
Questions for authors
  1. How many input views are used in Table 2? Are you using the full training dataset? If so, is there a reason to compare feed-forward methods under the sparse-view setting, while comparing with the single-scene methods only with the full datasets? Can you conduct the same limited-input experiments in Table 2?
  2. How are the input views selected in Table 3?
7 comments

We appreciate Reviewer 8Pia for their positive comments and constructive suggestions. We address the reviewer's concerns below.

W1 Details on the use of DA3. We provide an intuitive explanation of how the geometric consistency is measured with DA3 and how it is used.

  • Measuring the geometric consistency. We feed the initial training images and the generated pseudo images, together with their camera poses, into the DA3 backbone. The encoded multi-view features are passed to the dense prediction transformer (DPT) head, which jointly predicts multi-view consistent depth and ray maps. The DPT head has an additional channel that decodes a per-view confidence map, which is trained to be low where the geometry prediction is unreliable. Since the posed training images provide consistent geometry, when the content of a generated image conflicts with the training views, DA3 predicts low confidence in these regions. We therefore use the confidence map as a measure of geometric consistency. As shown in Figures 5 and 18 (Figure 16 in the original submission), hallucinated regions tend to have lower confidence than regions consistent with the training images.
  • Using the geometric consistency as reliability. The goal of measuring the geometric consistency with DA3 is to identify unreliable regions in the generated pseudo images. We normalize the confidence map to [0, 1] as the pixel-wise reliability map, which is used both in view selection via RIG and in optimization via Eq. 7.

We will add these details on DA3 and the geometric consistency in the revised version.

W2 Implementation details.

We clarify the experimental settings and the reasons below.

  • End-to-end comparison in Table 2. We use the full training views of each dataset and evaluate all methods on the distinct test trajectories, as shown in Figure 12 (Figure 10 in the original submission). This aligns with the evaluation protocol of prior works [1, 2]. The numbers of training and test views for each dataset are listed in Q1.
  • Comparison with feed-forward methods in Table 3. We uniformly sample a subset of views from the training views used in Table 2, and evaluate all methods on the same test trajectories. This is because feed-forward methods such as AnySplat [3] are trained with 2 to 24 input views, and their evaluation covers up to 64 views (Table 1 in AnySplat [3]). Directly applying them to the full training trajectories (around 180 views per scene) falls outside the input range they are designed for.

We will add these details to further clarify our experimental settings.

[1] Wu, Jay Zhangjie, et al. "Difix3d+: Improving 3d reconstructions with single-step diffusion models."

[2] Kim, Minsu, et al. "Exploregs: Explorable 3d scene reconstruction with virtual camera samplings and diffusion priors."

[3] Jiang, Lihan, et al. "Anysplat: Feed-forward 3d gaussian splatting from unconstrained views."

W3 Comparison with more related works.

We provide more quantitative comparisons with related works in the table below, following the evaluation setting of Table 2 in our manuscript. While Deceptive-3DGS, RI3D and GSFixer also use diffusion models to improve 3DGS, they mainly focus on the sparse-view setting, where the test views are typically in the same distribution as the training views. In contrast, we consider a more challenging setting, where the test views are from distinct trajectories and out of the distribution of the training views. Moreover, these methods do not jointly consider the informativeness or reliability of the generated pseudo views. While FreeFix uses a Fisher information-based confidence map to guide the denoising process of the diffusion model, it does not consider selecting the most effective pseudo views. In contrast, DIGS selects pseudo views that are both informative and reliable.

MethodNerfBustersWildExploreDL3DV
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
3DGS14.510.4170.45415.810.4870.43416.190.5370.363
Deceptive-3DGS14.820.4330.44116.110.5090.42616.760.5740.351
RI3D15.270.4540.43716.200.5150.41817.090.5800.344
GSFixer15.930.4710.44116.780.5220.42817.460.5930.337
FreeFix16.010.4820.42416.910.5290.40717.530.5840.315
Difix3D+15.780.4680.43316.560.5140.42317.330.5880.324
ExploreGS16.290.4790.43416.960.5340.41317.650.5920.319
Ours17.150.5230.38117.780.5860.34918.140.6230.279

As shown in the above table, Deceptive-3DGS, RI3D, GSFixer and FreeFix improve the overall reconstruction quality compared to 3DGS. Our method consistently outperforms all baselines, validating the importance of informative and reliable view selection when incorporating diffusion priors.

W4. Evidence of view extrapolation. We provide more evidence of view extrapolation in Figure 11 on page 10 of the updated PDF, which shows the training views, the selected extrapolated pseudo views, and the rendered and diffusion-generated images at the same camera poses. The selected extrapolated pseudo views expand the coverage of the training views, and the diffusion-generated images complete scene boundaries and remove rendering artifacts, improving reconstruction quality at extrapolated test views. By selecting informative and reliable pseudo views, the diffusion-generated images generally remain consistent with the scene, even under large viewpoint changes.

Q1: Number of views and limited-input experiments in Table 2. In Table 2 of our manuscript, we use the full training views of each dataset by default, and evaluate the reconstruction quality on extrapolative test views from distinct trajectories. The numbers of views for each dataset are listed in the table below.

Dataset# ScenesAvg. # Training ViewsAvg. # Test Views
NerfBusters12181183
WildExplore8186296
DL3DV12172172

As explained in W2, we compare with feed-forward methods under the sparse-view setting in Table 3, since they are mostly trained with limited input views. As suggested by the reviewer, we further conduct the limited-input experiments of Table 2 on NerfBusters with 16, 32, and 64 training views in the table below.

Method16 views32 views64 viewsAll views
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
3DGS13.510.3390.51613.580.3440.50113.980.3740.46914.510.4170.454
Difix3D+13.820.3460.50514.270.3850.49114.730.4060.44215.780.4680.433
ExploreGS14.010.3580.48915.150.4170.46015.690.4420.43116.290.4790.434
AnySplat11.570.2760.59111.930.2910.59012.090.3060.59312.060.3110.596
Ours14.370.3760.46916.290.4670.44316.560.4840.41717.150.5230.381

As shown in the above table, per-scene optimization methods consistently outperform AnySplat across different numbers of input views. The 16-view extrapolative setting is extremely challenging, since the training views only provide limited coverage of the scene, making it difficult to improve the reconstruction quality even with diffusion priors. As the number of input views increases, diffusion-enhanced methods improve more significantly, while AnySplat remains poor. DIGS outperforms all baselines under all limited-input settings. Notably, DIGS with only 32 input views achieves reconstruction quality comparable to ExploreGS with the full training views, further validating the importance of informativeness- and reliability-aware view selection.

Q2: View selection in Table 3. In Table 3, we uniformly sample the input views from the full training views, which is a common practice in the sparse-view evaluation setting. All methods use the same input views for a fair comparison.

We really appreciate the reviewer's valuable time and effort. We hope our responses address the reviewer's concerns, and we will incorporate these discussions and results into the revised version. We would be happy to address any further questions, and we would appreciate it if the reviewer could consider raising the score if the concerns are addressed.

Reviewer_nEdbOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces information gain-based view selection into diffusion-prior-guided 3D reconstruction. The authors further use uncertainty estimated by Depth Anything 3 to evaluate the reliability of diffusion-generated novel views and reweight their supervision during 3DGS optimization.

Critical strengths
  1. The proposed pipeline is interesting. The authors introduce information gain-based view selection into diffusion-prior-guided 3D reconstruction and use uncertainty to reweight the supervision from diffusion-generated novel views. The experimental results demonstrate the effectiveness of this approach.

  2. Using uncertainty estimated by Depth Anything 3 to measure the reliability of diffusion-generated images is an interesting idea, and the experimental results show that it works well.

  3. The ablation studies are comprehensive and convincing, clearly demonstrating the effectiveness of the proposed components.

Critical weaknesses
  1. I have some concerns about whether such a complex information gain mechanism is necessary. Firstly, I think the paper should compare with view selection strategies from active learning-based 3DGS methods (e.g., ActiveGS[1]) and NeRF methods (e.g., ActiveNeRF [2]), since the core ideas are similar. Second, GSFixer [3] also proposes a naive view selection strategy. I think a more detailed comparison and discussion of these methods is necessary.

  2. Comparisons with image quality assessment [4] or diffusion hallucination scoring [4] methods are also necessary. For example, would models specifically designed to detect hallucinations or 3D inconsistencies provide better reliability estimates than the uncertainty used in this paper?

  3. The paper should include more experiments to evaluate whether the proposed framework generalizes to different diffusion models, particularly video diffusion models.

[1] Jin, L., Zhong, X., Pan, Y., Behley, J., Stachniss, C., & Popović, M. (2025). Activegs: Active scene reconstruction using gaussian splatting. IEEE Robotics and Automation Letters, 10(5), 4866-4873.

[2] Pan, X., Lai, Z., Song, S., & Huang, G. (2022, October). Activenerf: Learning where to see with uncertainty estimation. In European Conference on Computer Vision (pp. 230-246). Cham: Springer Nature Switzerland.

[3] Yin, X., Zhang, Q., Chang, J., Feng, Y., Fan, Q., Yang, X., ... & Cun, X. (2025). Gsfixer: Improving 3d gaussian splatting with reference-guided video diffusion priors. arXiv preprint arXiv:2508.09667.

[4] Choi, I., Lee, S., Nam, S. H., & Song, S. (2026). PR-IQA: Partial-Reference Image Quality Assessment for Diffusion-Based Novel View Synthesis. arXiv preprint arXiv:2604.04576

[5] Liu, X., Sun, W., Ren, Z., Broaddus, C., Huang, S., & Guigues, L. (2026). HAD: Hallucination-Aware Diffusion Priors for 3D Reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 29781-29791).

Questions for authors
  1. In Difix3D+, although pseudo views are uniformly sampled, the sampling process gradually moves away from existing training views during optimization. This detail seems to be missing from the paper. Does DIGS follow a similar progressive sampling strategy, or does it directly select the views with the highest information gain, even if they are far from the existing training views?
4 comments

We appreciate Reviewer nEdb for their recognition in our work and constructive feedback. We address the reviewer's concerns below.

W1 Comparison with different view selection strategies.

(1) Comparison with active learning-based methods. We have already compared with FisherRF and POp-GS in the active view selection setting in Table 4, following the setup of POp-GS on the Mip-NeRF 360 dataset, as detailed in Appendix B. As suggested by the reviewer, we further include ActiveNeRF and ActiveGS in the table below. By capturing the inter-Gaussian parameter correlations, our method achieves more accurate information gain estimation, and consistently outperforms all baselines. This indicates that accurate information gain estimation is necessary for effective view selection.

MethodPSNR ↑SSIM ↑LPIPS ↓
ActiveNeRF16.020.4510.488
ActiveGS18.230.5540.396
FisherRF16.810.4930.445
POp-GS18.150.5480.401
Ours19.630.5880.382

(2) Comparison with naive view selection strategies. We provide a quantitative comparison with GSFixer in the table below. GSFixer mainly focuses on the in-distribution sparse-view setting, where 3DGS is trained with 3, 6, or 9 views and evaluated on test views in the same distribution as the training views. GSFixer outperforms Difix3D+ in PSNR and SSIM, which might result from conditioning the diffusion model on VGGT features, enhancing the geometric consistency of the generated images. However, it does not consider the informativeness or reliability of the pseudo views. Our method consistently outperforms GSFixer across all three datasets, which validates the effectiveness of the proposed view selection.

MethodNerfBustersWildExploreDL3DV
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
3DGS14.510.4170.45415.810.4870.43416.190.5370.363
Difix3D+15.780.4680.43316.560.5140.42317.330.5880.324
ExploreGS16.290.4790.43416.960.5340.41317.650.5920.319
GSFixer15.930.4710.44116.780.5220.42817.460.5930.337
Ours17.150.5230.38117.780.5860.34918.140.6230.279

W2 Comparison with PR-IQA and HAD. DIGS can naturally adapt to different reliability criteria. We replace the pixel-wise reliability map from DA3 confidence with the pixel-wise quality map from PR-IQA and the hallucination score map from HAD, and report the results on WildExplore in the table below. The baseline uses no reliability signal, i.e., it selects views by GBD and weights the loss with only the rendering uncertainty map.

Reliability signalPSNR ↑SSIM ↑LPIPS ↓
None (baseline)16.930.5410.398
PR-IQA17.590.5720.355
HAD17.950.5980.326
DA3 confidence17.780.5860.349

All reliability signals improve the reconstruction quality over the baseline. PR-IQA performs slightly worse than DA3 confidence, while HAD yields the best performance. This may be because PR-IQA assesses the quality of a generated image against a single reference view, while HAD fuses the hallucination scores from multiple sampled reference views, and DA3 jointly reasons over all training views and the generated image, better capturing multi-view consistency. As noted in Section 7 of HAD, hallucinations in generated images mainly arise from inconsistent 3D structure, which explains why multi-view geometric signals provide more reliable estimates. We note that HAD requires training a dedicated hallucination scoring network, while DA3 is used off-the-shelf without additional training. These results show that DIGS is complementary to dedicated hallucination detectors and can directly benefit from more accurate reliability estimates.

W3 Generalization to different diffusion models. We have ablated different diffusion models in Appendix H of our manuscript. In the table below, we further evaluate DIGS with five diffusion models on NerfBusters, covering both fine-tuned and training-free image and video diffusion models. For image models, the baselines follow the view selection of Difix3D+, using SD-Turbo fine-tuned by Difix3D+ or the pretrained SDXL. For video models, the baselines follow the view selection of ExploreGS, using DynamiCrafter fine-tuned by ExploreGS, CogVideoX-5B-I2V fine-tuned by GSFixer, or the pretrained SVD.

Diffusion modelTypePSNR ↑SSIM ↑LPIPS ↓
SD-Turbo baselineImage, fine-tuned15.780.4680.433
SD-Turbo + DIGS16.730.4880.401
SDXL baselineImage, training-free14.760.4390.449
SDXL + DIGS15.110.4540.445
SVD baselineVideo, training-free15.250.4520.441
SVD + DIGS15.890.4710.429
DynamiCrafter baselineVideo, fine-tuned16.290.4790.434
DynamiCrafter + DIGS17.150.5230.381
CogVideoX-5B-I2V baselineVideo, fine-tuned16.510.4930.419
CogVideoX-5B-I2V + DIGS17.320.5410.364

The performance gains from DIGS generalize to different diffusion models. Models fine-tuned on 3DGS renderings outperform training-free ones, and also benefit more from DIGS. Since RIG selects views that are both informative and reliable, a stronger generator provides more such candidate views. DIGS is therefore complementary to advances in diffusion models. In our main experiments in the manuscript, we use the fine-tuned DynamiCrafter from ExploreGS for a fair comparison with the baselines. We find that using a stronger model such as CogVideoX-5B-I2V further improves the performance.

Q1 Progressive optimization in Difix3D+. In DIGS, we use progressive multi-stage training schedule is similar to prior works [1, 2], where pseudo views gradually move away from current training views. In Appendix B, we describe the setup of Difix3D+ and our training schedule. We further clarify the details below. We train all models for 45,000 iterations. After the first 30,000 iterations of standard 3DGS training, we invoke Algorithm 1 every 5,000 iterations, i.e., three stages of diffusion-enhanced fine-tuning. At each stage, we sample candidate pseudo views around the current training views and select the most informative and reliable ones with RIG. This gradually shifts the pseudo views away from the existing observations. Moreover, DIGS does not simply select the views with the highest information gain. Views too far from the training views often have high information gain but unreliable generated images, and are thus down-weighted by the reliability term in RIG. We will clarify the progressive training schedule in the revised version.

We really appreciate the reviewer's valuable time and effort. We hope our responses address the reviewer's concerns, and we will incorporate these discussions and results into the revised version. We would be happy to address any further questions.

[1] Wu, Jay Zhangjie, et al. "Difix3d+: Improving 3d reconstructions with single-step diffusion models."

[2] Ni, Junfeng, et al. "G4splat: Geometry-guided gaussian splatting with generative prior."

Thanks for the additional experiments and detailed responses. These have addressed my concerns and strengthened the paper. I am satisfied with the rebuttal and will maintain my positive score.

AuthorsOpenReview ↗

Thank you for your positive feedback

We sincerely appreciate the reviewer's timely feedback, and we are glad to know that our responses have addressed the reviewer's concerns. We thank the reviewer for the valuable suggestions, and we will incorporate them into the revised version.

Reviewer_cv3mOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses extrapolative novel view synthesis with 3D Gaussian Splatting, where diffusion-generated pseudo ground truths at pseudo views are used to supervise under-constrained regions. It proposes DIGS, which estimates the information gain of candidate pseudo views with a group-block-diagonal approximation of the Fisher information matrix that keeps correlations among spatially adjacent Gaussians. It weights this information gain by a pixel-wise reliability map obtained from the confidence of a geometry foundation model. Pseudo views are selected greedily under the resulting log-determinant objective, and the pseudo-view loss is weighted by rendering uncertainty and reliability. Experiments on NerfBusters, WildExplore, DL3DV and LLFF compare against depth-regularized and diffusion-enhanced baselines.

Critical strengths

The motivation is clear and well supported. Fig. 2(c) shows that the most informative pseudo views become less useful when supervised by generated images rather than real captures, which justifies considering informativeness and reliability jointly. This matters because it identifies a concrete failure mode of existing view selection for diffusion-enhanced reconstruction. The empirical gains are consistent. DIGS improves over ExploreGS by about 0.5 to 0.9 dB PSNR on all three main benchmarks and on LLFF. The per-scene results in Table 12 improve on all eight WildExplore scenes, while the additional training time stays moderate.

Critical weaknesses

The novelty is limited relative to closely related prior work that is not discussed. OracleGS, Grounding Generative Priors for Sparse-View Gaussian Splatting (Topaloglu et al., WACV 2026), already validates diffusion-generated views with a 3D geometric model and uses the resulting uncertainty in an uncertainty-weighted loss on the synthetic views. This is essentially the role of the reliability map and the weighting in Eq. 7, so the claim that RIG is the first to incorporate the reliability of generated pseudo ground truths needs to be qualified. PriSplat, Propagating Reliable Multi-view Information for Distractor-Free 3DGS (Yang et al., ECCV 2026), also combines a generative novel-view prior with Fisher-based view scoring for 3DGS. It is therefore directly relevant to information-gain-guided integration of generative priors. Meanwhile, GBD gives only 0.19 dB over the block-diagonal approximation on WildExplore, and the diagonal variant is even below uniform sampling (Table 5). The remaining contribution thus narrows to inserting the reliability weight into the Fisher-based selection criterion. This is a partly fundamental issue for the paper’s positioning.

The central reliability component is not validated quantitatively. The paper never measures how well the DA3 confidence predicts the actual error of the pseudo ground truth, even though clean ground truth is available in the setting of Fig. 2 and Fig. 16, and evidence is limited to qualitative examples. The derivation also treats the L1 loss as maximum likelihood but uses a squared-error likelihood for the Fisher matrix. The SSIM term in Eq. 7 is not reliability-weighted. FreeFix, which also uses Fisher information to guide diffusion priors, is excluded from the comparison. No variance over seeds is reported. These issues are fixable.

Questions for authors

1.Discuss and, if possible, compare with OracleGS, for example by replacing the DA3 reliability map with an OracleGS-style multi-view uncertainty inside DIGS. This would show whether the gain comes from the reliability signal itself or from its integration into view selection.

2.Please report a quantitative evaluation of the reliability map against the actual pseudo ground truth error, for example AUSE or rank correlation on views where clean ground truth exists.

3.Explain the use of a Gaussian likelihood for the Fisher matrix when 3DGS is trained with an L1 loss, and why the SSIM term in Eq. 7 is left unweighted. Please also clarify the number of NerfBusters scenes, stated as 12 in App. B while Fig. 10 shows eight.

6 comments

We appreciate Reviewer cv3m for their constructive feedback. We address the reviewer's concerns below.

W1 Novelty compared to prior works.

We thank the reviewer for pointing out these related works. While OracleGS and PriSplat also use generative models to improve 3DGS, together with reliability estimates or Fisher-based scores, they address different settings. Both methods use predefined pseudo views for image generation and evaluate the reconstruction on in-distribution test views. They do not consider the informativeness and reliability of pseudo views in the extrapolative setting, which is key in our work.

  • Comparison with OracleGS. (1) OracleGS evaluates on held-out views that are in the distribution of the training views. It primarily focuses on the sparse-view setting, where the unused training camera poses are used as the pseudo poses to generate synthetic images, without considering their informativeness or reliability. In contrast, DIGS selects pseudo views based on both informativeness and reliability for extrapolative novel view synthesis. (2) While the uncertainty-weighted loss of OracleGS is similar to our Eq. 7, it only considers the reliability of the generated images, but not the uncertainty of the 3DGS rendering. Eq. 7 also accounts for the rendering uncertainty, giving more weight to reliable generated content in under-reconstructed regions and reducing its influence in regions that are already well reconstructed.

  • Comparison with PriSplat. (1) PriSplat addresses a different problem, i.e., removing distractors from the training views. To restore the masked region of a training view, it selects support views from the other training frames using a Fisher-based score, and uses an NVS model to generate a distractor-free image at the same training pose. Since PriSplat only restores existing training views, it does not consider the information gain of novel views. (2) PriSplat's evaluation also assumes that test views are in the distribution of the training views. (3) Its Fisher-based score is only used to select support views before generation, and the generated images are used as dense supervision without considering their reliability. (4) The Fisher-based score in PriSplat directly follows the diagonal approximation in FisherRF[1], which ignores the inter-Gaussian parameter correlations.

  • Performance gains. In Table 4, GBD significantly improves information gain quantification, reducing AUSE from 0.24 to 0.19 and improving PSNR by 1.48 dB over the block-diagonal approximation. In the extrapolative setting in Table 5, GBD still outperforms the block-diagonal baseline by 0.19 dB, but the gain becomes smaller. This aligns well with our analysis that considering only the information gain is not enough when the pseudo ground truths are unreliable. By jointly considering the informativeness and reliability of candidate views, GBD with RIG outperforms the block-diagonal baseline by 0.64 dB. The diagonal approximation in FisherRF has also been reported to perform similarly to or worse than uniform sampling in prior work (Tables 2 and 4 in POp-GS[2]). This is because it discards all cross-parameter correlations when quantifying the information gain, leading to unstable estimates in practice.

To sum up, OracleGS and PriSplat generate images at pre-defined camera poses and are evaluated on in-distribution test views. In contrast, we find that selecting informative and reliable pseudo views is key to integrating diffusion priors into 3DGS in the extrapolative setting. We therefore propose GBD to accurately quantify the information gain of candidate views, and RIG to incorporate the geometric consistency of generated images into view selection. During optimization, we further jointly consider the rendering uncertainty of 3DGS and the reliability of the generated images. Since OracleGS and PriSplat rely on pre-defined pseudo views, they do not directly address these challenges.

We note that our claim in the introduction is that RIG is the first to incorporate the reliability of generated pseudo ground truths into information gain estimation, rather than the first to consider their reliability in general (e.g., through loss weighting as in OracleGS). We will further clarify our claims and incorporate a more detailed discussion of OracleGS and PriSplat to better highlight our contribution.

[1] Jiang, Wen, Boshu Lei, and Kostas Daniilidis. "Fisherrf: Active view selection and mapping with radiance fields using fisher information."

[2] Wilson, Joey, et al. "Pop-gs: Next best view in 3d-gaussian splatting with p-optimality."

Q1 Comparison with OracleGS.

In response to the Reviewer's question, we replace the DA3 reliability map with an OracleGS-style multi-view reliability signal, the attention maps from VGGT, and report the results on the WildExplore dataset in the table below. The baseline uses the GBD selection without any reliability signal, and weights the loss with only the rendering uncertainty map. Upon this baseline, we use each reliability signal to weight the loss, to guide view selection, or both.

Reliability signalLoss weighting onlyView selection onlyLoss weighting + view selection
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
None (baseline)16.930.5410.398——————
OracleGS (VGGT attention)17.010.5450.39117.320.5490.38217.610.5710.357
DA3 confidence17.120.5460.38717.430.5540.37417.780.5860.349

The results show that how the reliability signal is integrated matters more than the signal itself. For both signals, incorporating reliability into view selection is more effective than using it only to weight the loss, and combining the two yields further gains. Notably, using the OracleGS-style signal only for loss weighting, as in OracleGS, improves the baseline by only 0.08 dB, whereas integrating the same signal into DIGS improves it by 0.68 dB. The DA3-based reliability signal consistently yields better reconstruction quality than the attention-based signal in OracleGS. DIGS therefore remains effective with an alternative reliability signal.

W2/Q2 Quantitative evaluation of the reliability map. We evaluate the AUSE on the test views of WildExplore, where clean ground-truth images are available. We generate images at these views and compute the AUSE of each reliability map against the actual pseudo-ground-truth error. As shown in the table below, our DA3-based reliability map reduces the AUSE by 30% compared to the OracleGS-style attention map, which is consistent with our ablation in Q1. We note that OracleGS validates its reliability maps only qualitatively.

Reliability mapAUSE ↓
OracleGS (VGGT attention)0.568
DA3 confidence0.393

W2/Q3 The use of Gaussian likelihood and more ablations.

(1) The use of Gaussian likelihood. 3DGS is optimized with an L1 loss, which corresponds to MLE under Laplace noise. However, the Laplace likelihood is not twice differentiable. We therefore follow FisherRF, POp-GS and PUP 3D-GS, which also train 3DGS with L1 but derive the Fisher information under a Gaussian likelihood. This choice does not affect our view selection, because the Fisher information under the Laplace likelihood differs from the Gaussian one only by a constant factor, which cancels in the log-determinant ratio of RIG. Therefore, both likelihoods lead to the same view selection. We will detail this formulation in the revised version for better clarity.

(2) The SSIM term is not reliability-weighted. In Eq. 7, we follow 3DGS-Enhancer [3] and ExploreGS [4] to weight only the L1 term with the uncertainty map, and additionally incorporate the reliability map. In contrast, OracleGS weights both the L1 and SSIM terms. We ablate the effect of weighting the SSIM term on the WildExplore dataset.

Loss weightingPSNR ↑SSIM ↑LPIPS ↓
L1 only17.780.5860.349
L1 + SSIM17.750.5790.348

As shown in the above table, additionally weighting the SSIM term slightly decreases PSNR and SSIM in the extrapolative setting, with comparable LPIPS. This may be because the weighting further suppresses the structural supervision from the generated images in under-reconstructed regions, where such supervision is most needed. We note that OracleGS targets in-distribution evaluation, where this trade-off may differ.

[3] Liu, Xi, Chaoyi Zhou, and Siyu Huang. "3dgs-enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors."

[4] Kim, Minsu, et al. "Exploregs: Explorable 3d scene reconstruction with virtual camera samplings and diffusion priors."

(3) Comparison with FreeFix. We provide a quantitative comparison with FreeFix in the table below. FreeFix uses a Fisher information-based confidence map to guide the diffusion model during image generation. However, it does not consider the information gain or reliability of the added pseudo views. Our method consistently outperforms FreeFix across all three datasets in the extrapolative setting.

MethodNerfBustersWildExploreDL3DV
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
3DGS14.510.4170.45415.810.4870.43416.190.5370.363
Difix3D+15.780.4680.43316.560.5140.42317.330.5880.324
ExploreGS16.290.4790.43416.960.5340.41317.650.5920.319
FreeFix16.010.4820.42416.910.5290.40717.530.5840.315
Ours17.150.5230.38117.780.5860.34918.140.6230.279

(4) Variance over seeds. We report the results of three independent runs with different random seeds in the table below. The performance of our method is stable across all three datasets and metrics.

RunNerfBustersWildExploreDL3DV
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
Seed017.1510.52340.381317.7830.58620.349118.1440.62310.2792
Seed117.1870.52250.384717.8250.58180.349518.1090.62200.2804
Seed217.0980.52240.385817.7510.58410.348418.1930.62500.2783
Std3.66e-24.50e-41.92e-33.03e-21.80e-34.55e-43.45e-21.24e-38.60e-4
Var1.34e-32.02e-73.67e-69.18e-43.23e-62.07e-71.19e-31.54e-67.40e-7

(5) The number of NerfBusters scenes. The NerfBusters dataset contains 12 scenes: aloe, art, car, century, flowers, garbage, picnic, pikachu, pipe, plant, roses, and table. In our initial manuscript, we visualize eight scenes for each dataset to illustrate the distinct training and extrapolated test trajectories. We visualize the remaining four NerfBusters scenes in Figure 10 on page 10 of the updated PDF.

We really appreciate the reviewer's valuable time and effort. We hope our responses address the reviewer's concerns, especially regarding the novelty compared to OracleGS and PriSplat, and we will incorporate these discussions and results into the revised version. We would be happy to address any further questions, and we would appreciate it if the reviewer could reconsider the evaluation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their valuable time and constructive feedback. The reviewers recognized several key strengths of our work, including:

  • Technical novelty and soundness. Reviewer 8Pia found our method "technically sound and novel", and GBD and RIG "compelling and mathematically sound". Reviewer nEdb found the proposed pipeline interesting, and the use of DA3 to measure the reliability of generated images an interesting idea that works well.
  • Clear and well-supported motivation. Reviewer cv3m found the motivation clear and well supported, as it identifies "a concrete failure mode" of existing view selection for diffusion-enhanced reconstruction.
  • Consistent improvements and comprehensive evaluation. Reviewers cv3m and 8Pia noted the consistent improvements on the NerfBusters, WildExplore, and DL3DV benchmarks, and Reviewer nEdb found our ablations "comprehensive and convincing".

In the initial reviews, the reviewers raised questions about the discussion and comparison with related works, including (1) diffusion-enhanced 3DGS methods, such as OracleGS, Deceptive-3DGS, RI3D, and GSFixer, which mainly focus on the sparse-view setting with in-distribution test views, and FreeFix, which uses Fisher information to guide diffusion denoising at given views; (2) PriSplat, which uses Fisher-based scores to filter support views, and removes the distractors in training views; and (3) active view selection methods, such as ActiveNeRF and ActiveGS, which select real captures. We note that these works mostly do not target the extrapolative setting, where the test views are out of the distribution of the training views. More importantly, they do not jointly consider the informativeness and the reliability when selecting pseudo views, which is central to integrating diffusion priors.

After our responses, Reviewers nEdb and 8Pia confirmed that all their concerns in the initial reviews have been addressed by our detailed analyses and additional experimental results. The discussion also helped us clarify several key points beyond the comparison with related works:

  • Reliability-guided view selection (Reviewers cv3m and nEdb). DIGS is compatible with alternative reliability signals, including the OracleGS-style VGGT attention map, PR-IQA, and HAD. We show that incorporating reliability into pseudo-view selection is more effective than only using it to weight the loss.
  • Generalization to different diffusion models (Reviewer nEdb). We show that the gains of DIGS are consistent across different image and video diffusion models.
  • Robustness of the evaluation (Reviewers cv3m and 8Pia). The variance over training seeds and the mean ± std over multiple sampled input-view subsets validate the robustness of our evaluation.
  • Implementation and evaluation details (Reviewers cv3m, nEdb, and 8Pia). We clarified the details raised by the reviewers, including the use of the Gaussian likelihood in the Fisher information, the loss weighting, the progressive training schedule, the use of DA3, and the evaluation protocols.

We are pleased that our responses in the discussion phase addressed the reviewers' concerns. We sincerely thank all reviewers for their valuable feedback and efforts, and Reviewers nEdb and 8Pia for engaging in the discussion and for their support. We also thank Reviewer 8Pia for raising the rating to weak acceptance and for noting that our responses to Reviewers cv3m and nEdb helped resolve their concerns. We will incorporate these discussions and results into the revised version for better clarity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.