- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes GS-Mesh, a method that leverages differentiable mesh optimization within Gaussian-Splatting reconstruction to improve continuous surface reconstruction while maintaining high-fidelity novel-view synthesis (NVS). The contributions include: (1) bridging discrete Gaussians to continuous mesh surfaces through implicit moving least squares (IMLS) surface instantiation from Gaussian-native attributes; (2) IMLS-based densification; and (3) coarse-to-fine surface optimization. The paper conducts experiments on three datasets to validate surface reconstruction and NVS quality.
Critical strengths
This is a well-motivated paper working on a highly significant problem. Reconstructing a continuous surface from discrete 3D Gaussians through mesh extraction is an important problem with direct applications. Involvement of mesh optimization in GS reconstruction is also very intuitive. The paper is also easy-to-follow with clear definition of the claimed novelties. The authors promise code release upon publication, enabling the reproduction of the work.
Critical weaknesses
[Claimed Contributions and Novelties]
• (Major) To my best understanding, the first novelty is the integration of several ideas already introduced/explored by prior works. Specifically, (1) the use of Gaussian-Native parameters to obtain an auxiliary surface is explored by GSRec [1]; (2) IMLS surface representation is explored by IMLS-Splatting; and (3) Mesh involvement in optimization is explored by IMLS-Splatting, SuGaR, and MILo. The paper needs to do a better job of distinguishing itself from these works. Also, regarding the third novelty: Coarse-to-fine surface optimization appears in the baseline work IMLS-Splatting and is an established training paradigm in the literature. To be considered as a meaningful novelty, it should involve some form of special adaptation to IMLS representation or GS-Mesh methodology in general.
• (Minor) The paper needs to cite and discuss a closely related prior work named GSRec [1] (ECCV '24). Moreover, this prior work has overlapping ideas with the proposed GS-Mesh in terms of constructing an IMLS surface directly from Gaussians. The authors are encouraged to elaborate the distinction between GS-Mesh and this prior work.
[Methodology]
• (Minor) To address under-sampled geometry issue, the paper identifies geometrically sparse regions on the IMLS surface and adds new Gaussians there. However, this approach can unnecessarily overpopulate some local regions and increase Gaussian counts potentially without any benefit. Perhaps the more intuitive and straightforward way of identifying under-sampled regions is to use some sort of supervision from normal and depth targets?
[Experiments and Ablations]
• (Major) The experiments do not seem to establish the superiority of GS-Mesh over baselines: GS-Mesh achieves similar results to the baseline on DTU dataset (with higher compute time) and achieves second-best result on the BlendedMVS dataset. Also, the benefit of Gaussian-Native parameterization to induce a bridge needs to be explored. What is the benefit of using these attributes as compared to SDF-based parametrization of baseline work MILo? or other potential parametrizations?
• (Minor) While the ablation establishes the benefits of IMLS-guided densification, it does not compare this densification strategy with other potential approaches including using direct supervision for targeted densification as discussed above.
[1] Wu, Qianyi, Jianmin Zheng, and Jianfei Cai. "Surface reconstruction from 3d gaussian splatting via local structural hints." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
Questions for authors
I invite the authros to focus on addressing the major points above in their rebuttal and consider minor points for future works or further improvements. This work needs improvement in terms of major weaknesses mentioned above, and so I lean towards rejection. I thank the authors for putting effort into the presentation of their work both through the paper and the supplementary demo video.