- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PCAsplat, a geometry-aware regularization framework for Gaussian splatting based on differentiable local principal component analysis (PCA). The main motivation is to address poorly localized Gaussian primitives and floaters that may receive insufficient geometric supervision from rendering-based losses.
The proposed method constructs local neighborhoods of Gaussian centers and introduces an eigenspectrum regularization loss to encourage local planarity and isotropic tangent-plane coverage. An additional normal alignment loss encourages consistency between Gaussian orientations and PCA-estimated surface normals. The framework is implemented on top of an existing Gaussian splatting pipeline.
The authors evaluate their method on DTU, Tanks and Temples, NeRF Synthetic, and ScanNet++, reporting improvements in point-cloud surface accuracy, instance segmentation, and Poisson surface reconstruction, while maintaining competitive novel-view synthesis and mesh reconstruction performance.
Critical strengths
-
Effective geometric regularization with convincing improvements in point-cloud quality. The paper addresses a meaningful limitation of rendering-based Gaussian Splatting methods: insufficient geometric supervision for Gaussians with weak rendering contributions. Directly regularizing local Gaussian neighborhoods is a reasonable approach that can complement existing rendering objectives. The quantitative results are encouraging, particularly the improvement in mean Chamfer Distance on DTU from 0.662 mm for RaDe-GS to 0.519 mm. The qualitative comparisons also suggest better surface localization and fewer floating artifacts.
-
Useful downstream evaluations beyond conventional rendering metrics. I appreciate that the paper evaluates Gaussian centers as geometric surface samples rather than focusing exclusively on novel view synthesis. The segmentation and Poisson reconstruction experiments help demonstrate the potential practical value of improved Gaussian localization. In particular, the reported improvements in ScanNet++ segmentation are substantial. The additional experiments integrating PCA regularization into different Gaussian Splatting methods also provide useful evidence of its potential generality.
Critical weaknesses
- The technical novelty and the individual contributions of the proposed regularization terms are not sufficiently established. While the overall formulation is reasonable, local PCA and covariance eigenspectrum analysis are well-established techniques in point-cloud processing. The proposed method appears to be a relatively straightforward application of these classical geometric priors to Gaussian centers. I would like to see stronger evidence explaining what makes the specific formulation particularly effective compared with simpler local planarity or neighborhood regularization alternatives.
More importantly, the current ablation study does not fully isolate the contributions of the different terms. In Table 4, the PCA loss accounts for almost all the improvement, whereas adding the normal alignment loss only changes the Chamfer Distance from 0.520 to 0.519. The planarity and tangent-plane isotropy terms are also not evaluated separately. Consequently, it remains unclear which components are essential and whether the full proposed formulation is justified by the experimental evidence.
- The practical limitations and trade-offs of the local PCA regularization require more thorough analysis. The method encourages locally planar Gaussian neighborhoods, but local planarity alone does not necessarily imply alignment with the correct scene surface. For example, isolated clusters of floating Gaussians could potentially form locally coherent structures while remaining geometrically incorrect. The paper acknowledges that an additional depth-based step is required for distant floaters, but its precise role and impact are not sufficiently explained. This makes it difficult to determine how much of the improvement can be attributed to the proposed PCA regularization itself.
Furthermore, the computational overhead is non-negligible. On Tanks and Temples, the reported training time increases from 18.3 minutes for RaDe-GS to 53.0 minutes for PCAsplat. The novel view synthesis results in Table 8 also indicate some degradation compared with RaDe-GS, particularly on outdoor scenes. A more explicit analysis of these trade-offs would help establish the practical benefits and applicability of the method.
Questions for authors
-
Contribution of individual loss terms. Could the authors provide separate ablations for the planarity term, tangent-plane isotropy term, and normal alignment term? Given the small additional improvement from normal alignment in Table 4, it would be useful to clarify its importance, particularly for downstream tasks relying on surface normals. A comparison with a simpler neighborhood planarity regularizer would also help establish the necessity of the proposed formulation.
-
Handling distant floaters and neighborhood sensitivity. The conclusion mentions a depth-based step for distant floaters. Could the authors clarify whether this step is part of training or post-processing, and quantify its contribution? Additionally, how sensitive is the method to the number of nearest neighbors (K), especially near sharp boundaries, thin structures, or regions with nonuniform Gaussian density?
-
Computational cost and rendering trade-offs. Could the authors provide more detailed analysis of the runtime overhead introduced by neighborhood construction and differentiable PCA? It would also be helpful to examine the trade-off between surface reconstruction accuracy and novel view synthesis quality as the PCA regularization strength varies.
-
Downstream segmentation evaluation. For the ScanNet++ experiments, how are differences in point-cloud size and sampling density handled before feeding the reconstructed points into Mask3D? Since these factors can affect segmentation performance, additional clarification would help establish how much of the reported improvement comes specifically from better surface alignment.
Official Comment
Thank you for the feedback. We added simpler baselines, loss ablations, neighborhood-size experiments, same-GPU timings, and segmentation comparisons with equal point counts.
Classical PCA-based simplification operates on an existing point cloud (Pauly et al., 2002). Our contribution is using differentiable PCA to encourage local planarity while learning a cleaner Gaussian cloud from images. Optimized jointly with rendering losses, it directly constrains Gaussian neighborhoods, including Gaussians with weak rendering contributions.
We added centroid smoothing as a simpler neighborhood baseline, with scheduled weights and detached neighborhood means (Appendix E.2):
$\mathcal{L}_{c} = \frac{1}{N}\sum_i \lVert x_i - \bar{\mu}_i \rVert^2$.
On 15 DTU scenes, PCAsplat achieves 0.519 mm CD versus 0.523 mm for centroid smoothing (Tab. 12). Their training times on RTX PRO 4500 are 4.98 and approximately 10 min, respectively. PCA provides tangent-plane isotropy and local normals, with differentiable statistics supporting future work on geometric regularizers.
(W2 / Q1) “Could the authors provide separate ablations for the planarity term, tangent-plane isotropy term, and normal alignment term?”
Tab. 11 presents the ablation:
Regarding the isotropy-only experiment, Gaussians spread without sufficient geometric constraints, and optimization did not converge. Planarity loss accounts for most of the CD improvement; isotropy promotes splat coverage and isotropic neighborhoods (Fig. 3). The normal term aligns PCA normals with the Gaussian normals; however, Gaussian-center CD does not directly measure normal orientation. Fig. 19 shows more coherent normal directions in the DTU apple example. We use the centers and normals for Poisson reconstruction (Fig. 9).
Tab. 13 reports DTU results:
Increasing K from 20 to 40 improves the Gaussian CD but increases the TSDF mesh CD. K=20 remains our default; the results suggest choosing K according to the target application. These results illustrate the trade-off across the tested settings but do not isolate behavior near sharp boundaries, thin structures, or regions with nonuniform density.
New same-GPU DTU measurements, with GPU memory capped at 24 GB, quantify the training overhead (Appendix E.4, Tab. 14):
The RTX 4500 figures measure the training loop (without logging). The overhead varies across the hardware. And we leave for future to optimize our KNN based batched PCA regularization.
We could not reproduce RaDe-GS’s published scores. See Tab. 9 and Reviewer RxEG
answer W5 which reports the published results and our reproduction showing a gap of only 0.14 PSNR in outdoor scenes and 0.31 in indoor scenes and similar LPIPS and SSIM. Fig. 16 show rendering comparison over two indoor and two outdoor scenes, visually showin almost no difference.
All methods use the same ScanNet++ scenes, frozen Mask3D, and evaluation protocol. Tab. 5 reports results using the original point counts. To control point count, we added comparisons of PCAsplat and RaDe-GS centers, depth-derived points and TSDF mesh samples, using 732k points per cloud, the same eight ScanNet++ scenes, and the same frozen Mask3D model. The version show additional results in Tab. 10 which are discussed both in the paper and in Reviewer RxEG (W1). Mesh and point samples are distributed uniformly over the extracted surfaces which are later passed trough Mask3D original inference chunking for equal density. The advantage of PCAsplat's centers persists after matching point count or changing media.