- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces NPM-GS, an approach that investigates sparse-view 3D Gaussian Splatting (3DGS) from a geometric perspective. The authors argue that under sparse-view conditions, the various parameters of 3DGS can become entangled, resulting in similar RGB fitting but incorrect geometry. The method employs DA3 to generate a dense initialization and samples candidate points for GS centers along each ray; it then assesses the reliability of these candidates using NCC calculated via reference-to-target view back-projection. Finally, a reliability-aware GS optimization strategy is implemented, incorporating depth supervision, multi-view color regularization, and a dropout mechanism to enhance accuracy.
Critical strengths
1.The paper provides a compelling diagnosis of the problem and employs multiple experiments to demonstrate the issue of geometric perturbation in few-view 3DGS reconstruction.
Critical weaknesses
- The distinctions and boundaries of innovation regarding existing work have not been sufficiently articulated: using VFM for initialization is already common practice in prior studies (e.g., DNGaussian, NexusGS); the "drop" mechanism originates from DropGaussian; and supervision based on depth estimation is also frequently employed (e.g., FatesGS, Sparse2DGS). The authors need to clearly delineate the differences between this paper and these existing works to establish the paper's specific innovations and contributions.
- Insufficient experimentation: Although the primary issue addressed in this paper is geometric error, the final experimental evaluation lacks an assessment of geometric accuracy. This is a significant shortcoming; metrics such as surface reconstruction accuracy (e.g., Chamfer Distance/CD, commonly used with the DTU dataset) or the accuracy of depth maps rendered by the Gaussian Splatting model (common in RGB-D datasets) should be included.
Questions for authors
- The paper employs DA3 for dense initialization; are the camera parameters encoded into the network? Furthermore, since DA3 is essentially a feed-forward network for unconstrained images, could directly using the predicted depth maps (even with official alignment) and the actual camera parameters result in a failure to accurately align the multi-view point clouds? The authors could compare rendered images from the test views (using only the initialization) against the actual images to check for misalignment.
- The DTU images in Figure 3 do not have the background removed, yet the DTU dataset provides background masks. Were the backgrounds removed for the evaluation data presented in the tables? If so, please update the DTU images in Figure 3; if not, the comparison may be unfair due to background discrepancies.
3 comments
Results Update
We thank Reviewer GXCT again. Following up on our previous response, we have uploaded the revised manuscript with all promised experiments, visualizations, and clarifications:
1. Chamfer Distance Evaluation on DTU (W2) We evaluated 3D geometric accuracy on DTU using Chamfer Distance (mm ) via TSDF fusion of rendered depths and the official DTU script, and added the results to Sec. 5.2 and Appendix G (Table 9):
| Method | 3 Views | 6 Views | 9 Views |
|---|---|---|---|
| SparseGS | 4.2214 | 4.5669 | 4.3511 |
| DNGaussian | 4.2620 | 3.5568 | 3.1225 |
| CA-3DGS | 3.1029 | 2.2064 | 2.4766 |
| NexusGS | 1.6273 | 1.2937 | 1.4813 |
| NPM-GS (Ours) | 0.9591 | 0.9066 | 0.8201 |
2. Visualizations & Manuscript Clarifications
- DA3 initialization (Q1): Clarified the DA3 camera input in Sec. 4.2 and the
"DA3 Init"baseline in Sec. 5.3 and Table 4, and added visualizations of the misaligned initialized point cloud and test-view renderings in Appendix F. - DTU evaluation masks (Q2): Updated Fig. 3, detailed the visibility mask protocol and unobserved regions in Sec. 5.2 and Appendix C.1, and added mask visualizations in Appendix H.
- Distinctions from prior work (W1): Expanded Sec. 2 to discuss FatesGS and Sparse2DGS and clarify our differences in how priors are used and verified.
We thank the reviewer again for the valuable feedback.
Looking forward to hearing from you
Dear Reviewer GXCT,
We sincerely appreciate your time and effort in reviewing our manuscript and providing valuable feedback.
In response to your insightful comments, we have made the following revisions and clarifications:
- We have expanded Sec. 2 (including FatesGS and Sparse2DGS) to clarify our boundary of innovation: our core contribution lies not in learned-prior initialization or depth supervision, but in the subsequent test-time optimization and multi-view verification.
- We have added 3D geometric accuracy evaluation on DTU using Chamfer Distance across 3, 6, and 9 input views in Sec. 5.2 and Appendix G (Table 9), demonstrating sub-millimeter surface accuracy and confirming that strong novel-view RGB metrics under continuous optimization do not guarantee accurate 3D geometry.
- We have clarified in Sec. 4.2 that dataset camera parameters are provided to DA3, detailed the "DA3 Init" baseline in Sec. 5.3 and Table 4, and added Appendix F (Fig. 8) visualizing both the cross-view misalignment in the unprojected DA3 point cloud and the corresponding test-view rendering comparisons.
- We have updated the DTU visualizations with evaluation mask contours in Fig. 3 and Appendix H (Fig. 9), and detailed our shared visibility mask protocol and handling of unobserved regions in Sec. 5.2 and Appendix C.1 to ensure fair comparisons across all methods.
We hope these revisions adequately address your main concerns regarding our boundary of innovation relative to prior work and the evaluation of 3D geometric accuracy.
We look forward to actively participating in the Author-Reviewer Discussion session and welcome any additional feedback you might have on the manuscript or the changes we have made.
Once again, we sincerely thank you for your contributions to improving the quality of this work.
Best regards, The Authors of Submission 25601
We thank the reviewer for recognizing our diagnosis of geometric perturbation under sparse views.
W1: Differences from prior work
We differ from these works not in using priors, but in how the final geometry is decided. Dense priors, depth supervision, and dropout are common, and we do not claim them as contributions: DA3 only initializes and bounds the search (Sec. 4.2), our depth loss uses depth corrected by reliable Gaussians, not raw DA3 depth (Eq. 4), and for dropout we already cite DropGaussian.
DNGaussian regularizes with monocular depth, NexusGS initializes from optical flow and epipolar geometry, FatesGS regularizes with monocular depth ranking and multi-view feature consistency, and Sparse2DGS initializes from learning-based MVS with geometry-prioritized constraints. In all of them, the prior is an initial value or a constraint target, and the final geometry is still set by continuous joint optimization. In Tables 1–2, NPM-GS leads DNGaussian/NexusGS by 7.61/3.96 dB PSNR on DTU (3 views) and 4.82/1.54 dB on Mip-NeRF 360.
Sec. 3 and App. E show why continuous optimization cannot reliably recover geometry from sparse views: (1) depth supervision pulls geometry toward the prior without checking it against the images, so prior errors persist (Fig. 1(b)); (2) optimizing geometry before appearance stays well below NPM-GS (19.86, or 19.97 dB with depth supervision, vs. 23.19 dB; DTU scan 105, 4 views; Table 7); (3) the rendering loss barely reacts to geometric perturbation, while cross-view NCC reacts strongly but flattens beyond about 13% of scene scale, giving no gradient toward distant correct positions (Figs. 1(c), 1(d), 7). NPM-GS thus uses the prior only to bound the search and accepts only discrete candidates supported by multiple views.
Our contributions are (i) the analysis of geometry–appearance coupling (Sec. 3), (ii) discrete cross-view verification on the final Gaussians (Sec. 4.3), and (iii) reliability-guided propagation and training (Secs. 4.4–4.5). The revision will discuss FatesGS and Sparse2DGS and add a table comparing these methods by prior source, prior use, and whether geometry is checked against the input images.
W2: Geometric accuracy
Our task is sparse-view novel view synthesis, so we only report its standard metrics (PSNR, SSIM, LPIPS), as do DNGaussian and NexusGS. Here, geometry is the means rather than the target: NPM-GS fixes geometry by discrete search to resolve the coupling in Sec. 3, which improves rendering. Sec. 3 and App. A also measure geometry directly in controlled tests, as the displacement of perturbed geometry from a reference reconstruction.
As suggested, we are computing Chamfer Distance on DTU for all methods with one pipeline (TSDF fusion of rendered depths as in 2DGS, then the official DTU script) and will add the results to the revision.
Q1: DA3 initialization alignment
Yes, DA3 takes the dataset cameras as input, and its depths are unprojected with the same cameras (Sec. 4.2). As the reviewer expects, the fused depths are still clearly misaligned across views, forming layers especially in weakly textured planar regions. This is the "dense initialization with misalignment" in Fig. 2, a key problem NPM-GS addresses, and why we treat planar regions separately (Sec. 4.4).
The suggested check matches the "DA3 Init" row of Table 3, which uses the DA3 initialization without our NPM optimization: it gets only 19.77 dB (19.91 dB with DA3 depth supervision) vs. 21.62 dB for the full NPM-GS. This shows that the misalignment has a large impact. The revision will clarify this row and its setting in Sec. 5.3 and the Table 3 caption, and add its test-view renderings to the appendix.
Q2: DTU background
The background was neither removed nor kept in full. As stated in Sec. 5.1, we evaluate pixels visible from the training views: we reproject reference depths of the 3-view training set into each test view (App. C.1) and keep pixels seen by at least one of them, plus the DTU object mask. This keeps visible background such as the table, where sparse-view failures like floaters often occur and an object mask would hide them. In our supplementary videos, the compared methods show many floaters in these regions. PSNR uses only pixels in the mask; for SSIM and LPIPS, pixels outside it are filled with ground truth and add no error for any method. All methods and the 3/6/9-view settings share one mask, so the background causes no unfairness.
The black regions in Fig. 3 are such unseen, excluded pixels, which NPM-GS does not fill (see our reply to Reviewer rN7k, W3). As our description was too brief, the revision will state in the main text that pixels outside the mask are excluded, show the masks, update Fig. 3, add these details to App. C.1, and release all masks.
We thank the reviewer again; the clarifications and new experiments will be included in the revision.