- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates appearance-induced surface bias in Gaussian splatting and proposes Score Separation to decouple regularized appearance estimation from geometry selection. The method uses either the data-fitting error (DATA) or a complexity-corrected score based on generalized cross-validation (GCV) to select geometry. The authors provide a theoretical analysis of the proposed bias and evaluate the method through controlled experiments and scene-level reconstruction studies.
Critical strengths
-
The paper studies an important issue in Gaussian-based surface reconstruction: improving image fitting does not necessarily improve geometric accuracy.
-
The theoretical analysis provides a useful perspective on how appearance residuals and regularization can affect geometry selection.
-
Controlled experiments and analyses on reconstructed scenes provide initial evidence supporting the proposed explanation.
Critical weaknesses
-
Limited novelty. The coupling between appearance fitting and geometric accuracy is a recognized issue. Although the paper provides a more formal analysis, the practical contribution of Score Separation appears incremental, and its advantage over existing appearance–geometry decoupling and model-selection approaches is not sufficiently established.
-
Limited experimental coverage. The scene-level experiments use only seven DTU scenes, and no results are reported on Tanks and Temples. Broader evaluation would strengthen the evidence that the findings generalize beyond the tested settings.
-
Insufficient comparisons with recent methods. Although several optimization strategies are evaluated, the paper lacks direct experimental comparisons with relevant recent approaches, including methods that explicitly decouple appearance and geometry. This makes it difficult to assess the practical advantage of the proposed method.
-
Unclear practical significance. While the controlled experiments show relative improvements in geometric error, the absolute errors remain notable. More evidence is needed to establish whether the proposed approach offers a substantial practical advantage over simpler alternatives.
Questions for authors
- What is the key methodological novelty of Score Separation beyond excluding the regularization penalty from geometry selection? How does it differ from existing model-selection and appearance–geometry decoupling approaches?
- Why is the scene-level evaluation limited to seven DTU scenes, and can the authors provide results on additional datasets such as Tanks and Temples?
- Can the authors compare Score Separation with relevant recent appearance–geometry decoupling methods or explain more clearly why these methods are not directly comparable?
- How should the absolute geometric errors reported in the controlled experiments be interpreted in terms of practical reconstruction quality?
8 comments
Response to Reviewer tGwZ (2/2)
W3 / Q3 (decoupling methods). Can the authors compare Score Separation with relevant recent appearance–geometry decoupling methods or explain more clearly why these methods are not directly comparable?
The system run. Following your suggestion, we ran GStex with its own code and DTU recipe, starting from our 2DGS models, on all 15 DTU scans. Its images improve while its surface worsens (§4.3, Table 3a).
| GStex texture stage, 15 DTU scans | Change | Scans |
|---|---|---|
| Test SSIM | +0.061 | 15/15 better |
| Test LPIPS | −0.017 | 14/15 better |
| Test PSNR | +0.46 dB | 12/15 better |
| Chamfer distance | +0.28 mm (p = 0.0004) | 14/15 worse |
| Accuracy | +0.52 mm | 14/15 worse |
The direct comparison, where geometry is decided. Table 1 adapts the core mechanism of each family under identical observations, start, geometry family, estimator and budget.
| Mechanism (Table 1) | Height RMSE (mm) |
|---|---|
| Joint fitting (PROFILE) | 6.09 |
| Closed-form color (Fang et al.) | 6.33 |
| Finer shared texture grid, 49², after GStex | 5.65 |
| PGSR multi-view NCC | 5.35 |
| Camera-PGP / Pixel-CV | 5.03 / 4.75 |
| Richer spectral appearance | 4.87 |
| DATA / GCV (ours) | 4.75 / 4.32 |
GCV has the best mean, P90 and rank, and its paired advantage over every comparator has an interval that excludes zero. GStex's mechanism, adapted to the shared estimator as a finer texture grid, ends at 5.65 mm against 4.32 mm for GCV.
Why complete systems are not directly comparable (Appendix M):
- They are built for a different quantity. GStex, A²TG, Textured Gaussians and AuGS target novel-view synthesis and report image metrics, with no geometry-accuracy metric. Image metrics cannot rank their surfaces, as the GStex run above shows.
- They act on a different layer. These methods change what appearance can represent, and Score Separation changes the rule that ranks geometry, so the two compose rather than compete (§3.4).
- They bring their own representation, rasterizer and schedule. A benchmark of systems therefore measures those choices together. Published ablations, reported for their own purposes, show the same trade (PGSR Table IV, Ref-NeRF Table 1, Appendix M).
A decoupled representation alone therefore does not protect the surface, and image metrics hide the loss of surface accuracy.
W4 / Q4 (practical significance). More evidence is needed to establish whether the proposed approach offers a substantial practical advantage over simpler alternatives. … How should the absolute geometric errors reported in the controlled experiments be interpreted in terms of practical reconstruction quality?
- Against simpler alternatives. The simplest alternative is joint fitting with λ chosen from the training images. At that weight, GCV ends at half the error of this alternative, and even below joint fitting tuned against the reference geometry (3.96 against 4.44 mm, Appendix J). After tuning against the reference, closed-form color and multi-view NCC fall behind joint fitting, while GCV lowers the mean error by 13.7 % and improves every unseen material (Table 2). On reconstructed scenes where the penalty dominates, DATA removes 85 % of the drift of joint fitting (Table 5). DATA adds negligible cost, and GCV 3.9 % per step against 61 % for Camera-PGP (Appendix Table A8).
- The controlled benchmark is sub-pixel by design. Every method starts below one pixel footprint (11.8 mm on the surface), and GCV ends near a third of one. The error of every surface in the benchmark, the starting one included, shifts the image by less than a quarter pixel between training cameras. The quantity compared is therefore the paired difference under identical conditions (Appendix N). At this scale every score in Table 2 reaches the same test PSNR within 0.1 dB, so only a geometry score can tell these surfaces apart.
- On real scenes the bias is large in practical terms. GStex's texture stage costs more surface accuracy on our models than Unbiased Depth gains on 2DGS (0.28 against 0.17 mm). Under SH0, the cost on PGSR models exceeds the gap between the published DTU results of 2DGS and PGSR (0.43 against 0.27 mm, Appendix M).
The practical advantage is therefore largest where a user must choose λ without a scan, as on every real scene.
Thank you again. Do the 15 DTU scans with two published systems, the six Tanks and Temples scenes and the GStex run answer your questions on coverage and comparison? We would also welcome your view on novelty and practical significance.
Follow-up to Reviewer tGwZ
Dear Reviewer tGwZ,
Thank you again for your detailed review. We would welcome your view on these additions to the revision.
Coverage (W2, W3). The expanded evaluation uses host models trained with their official recipes. Under SH0, test PSNR improves while surfaces worsen on all 15 DTU scans for both 2DGS and PGSR, and on all six Tanks and Temples scenes for 2DGS. GStex, run using the authors' implementation, improves test SSIM in every scan while worsening the surface in 14 of 15 (Table 3).
Novelty and practical significance (W1, W4). Four variants in PGSR's published ablation have higher PSNR and lower F1 than the full method (Table IV, arXiv v2). PGSR links this trade-off to an incomplete image-rendering model.
We test whether the objective itself favors worse surfaces by comparing optimal appearance fits for each candidate geometry. Our certificates establish such preferences in 19 of 23 reconstructed scenes (§4.4). Table 6 gives the steps for identifying the capacity or penalty route before any geometry update.
Score Separation changes how geometry is ranked while retaining regularized appearance fitting. It can combine with methods that decouple appearance and geometry at the representation level. In practice, with λ chosen from the training images, GCV halves the error of joint fitting on six unseen materials (Appendix J).
Do these additions address your concerns on coverage and novelty? We would also welcome your view on the practical gain from choosing λ without a reference scan.
Response to Reviewer tGwZ (1/2)
TL;DR.
Thank you for the detailed review, and for finding the issue important and the theoretical analysis "a useful perspective".
Earlier work observed false surfaces in specific settings. Unbiased Depth traces false 2DGS surfaces on glossy regions to spherical harmonics that cannot follow sharp changes of radiance. NeRO, Ref-NeRF, GOF and PGSR address the coupling with richer appearance or additional evidence (§1, §2, Appendix M). Relative to them, the paper adds three things.
A certified explanation. The certificates bound the optimal appearance fit of each frozen reconstruction from both sides. They therefore test whether the objective itself prefers a worse surface, whichever optimizer is used. Of 508 comparisons, 505 are decided by the bounds, and strict preferences for physically worse surfaces appear in 19 of 23 reconstructed scenes (§4.4). This contradicts the account in which limited capacity protects the true surface. A causal test agrees, since reversing the aligned residual flips the preferred surface in all 36 contexts tested.
A separate geometry decision. The penalty that stabilizes the appearance fit overturns the data term in nearly a third of the signed comparisons (§3.3). The fit must keep it, and the ranking of surfaces must not. Score Separation therefore fits appearance once per candidate surface, with the penalty, and ranks it by a separate score. Because that score differs from the fitting objective, its gradient must include how the fitted appearance moves with the surface, a term that vanishes in joint fitting (§3.4, Eq. 4). Cross validation and GCV usually choose a weight or a model for fixed data. Here they rank surfaces, and each surface changes the rendering operator A(g) itself. Used directly as geometry scores, the cross-validation comparators recover worse surfaces than GCV at higher cost (Table 1, Table A8).
Representation-level decoupling changes what appearance can represent, while Score Separation changes the rule that ranks geometry, so the two compose.
A rule of use with measured benefit. One λ serves every score, chosen by the leave-one-camera-out error of the appearance fit. At that λ, GCV ends at half the error of joint fitting (3.96 against 7.82 mm), and DATA and GCV both improve on it in every unseen material (Appendix J). The initial penalty share tells beforehand where the remedy applies, and at a 65 % share DATA removes 85 % of the drift of joint fitting (§4.5, Table 6).
Score Separation is simple because the analysis locates where the bias enters. That simplicity makes it a plug-in for any pipeline that fits appearance with a penalty, such as nvdiffrec and NeILF++, which train material smoothness priors jointly with geometry (Appendix M).
Your requests for more scenes, for Tanks and Temples and for a decoupling method produced the strongest additions of the revision (§4.3, Table 3, Appendix I).