acceptodds
Under review as a conference paper at ICLR 2027

APPEARANCE-INDUCED SURFACE BIAS AND SCORE SEPARATION IN GAUSSIAN SPLATTING

Abstract

Gaussian splatting reconstructs surfaces by fitting geometry and a finite appearance model jointly to multi view images, a coupling that the prevailing account interprets as an ambiguity in which several shapes explain the images equally well. We show that finite appearance induces a strict preference for physically worse surfaces, a phenomenon we term appearance induced surface bias. On seven DTU scenes, allowing the geometry of 2D Gaussian splatting to move raises test PSNR by 2.9 dB while the reconstructed surface degrades in every scene. We propose Score Separation, a geometry inference framework that retains a regularized estimator for appearance and selects geometry with an independent score. Two mechanisms underlie the bias. Near the true surface, an appearance residual aligned with the image change of a surface displacement shifts the preferred surface, and the penalty that regularizes appearance also enters the ranking of candidate surfaces. Variable projection characterizes the first mechanism, and primal and dual certificates establish strict preferences for worse surfaces in 19 of 23 reconstructed scenes independently of the optimizer. Score Separation reduces controlled recovery error from 6.09 to 4.75 mm, lowers error on unseen materials by 13.7% through its complexity corrected score, and improves reconstructed surfaces where the penalty dominates the descent. As a secondary diagnostic, the contribution depth error reveals surface deterioration that image quality and mean depth conceal

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian splatting, surface reconstruction, inverse rendering, appearance modeling, shape–radiance ambiguity, geometric identifiability, model misspecification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_tGwZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates appearance-induced surface bias in Gaussian splatting and proposes Score Separation to decouple regularized appearance estimation from geometry selection. The method uses either the data-fitting error (DATA) or a complexity-corrected score based on generalized cross-validation (GCV) to select geometry. The authors provide a theoretical analysis of the proposed bias and evaluate the method through controlled experiments and scene-level reconstruction studies.

Critical strengths
  1. The paper studies an important issue in Gaussian-based surface reconstruction: improving image fitting does not necessarily improve geometric accuracy.

  2. The theoretical analysis provides a useful perspective on how appearance residuals and regularization can affect geometry selection.

  3. Controlled experiments and analyses on reconstructed scenes provide initial evidence supporting the proposed explanation.

Critical weaknesses
  1. Limited novelty. The coupling between appearance fitting and geometric accuracy is a recognized issue. Although the paper provides a more formal analysis, the practical contribution of Score Separation appears incremental, and its advantage over existing appearance–geometry decoupling and model-selection approaches is not sufficiently established.

  2. Limited experimental coverage. The scene-level experiments use only seven DTU scenes, and no results are reported on Tanks and Temples. Broader evaluation would strengthen the evidence that the findings generalize beyond the tested settings.

  3. Insufficient comparisons with recent methods. Although several optimization strategies are evaluated, the paper lacks direct experimental comparisons with relevant recent approaches, including methods that explicitly decouple appearance and geometry. This makes it difficult to assess the practical advantage of the proposed method.

  4. Unclear practical significance. While the controlled experiments show relative improvements in geometric error, the absolute errors remain notable. More evidence is needed to establish whether the proposed approach offers a substantial practical advantage over simpler alternatives.

Questions for authors
  1. What is the key methodological novelty of Score Separation beyond excluding the regularization penalty from geometry selection? How does it differ from existing model-selection and appearance–geometry decoupling approaches?
  2. Why is the scene-level evaluation limited to seven DTU scenes, and can the authors provide results on additional datasets such as Tanks and Temples?
  3. Can the authors compare Score Separation with relevant recent appearance–geometry decoupling methods or explain more clearly why these methods are not directly comparable?
  4. How should the absolute geometric errors reported in the controlled experiments be interpreted in terms of practical reconstruction quality?
8 comments
AuthorsOpenReview ↗

Response to Reviewer tGwZ (1/2)

TL;DR.

  • W2, W3. The findings generalize. New runs on all 15 DTU scans with 2DGS and PGSR at their published accuracy, on the six Tanks and Temples scenes and with GStex's own code all show better test images with worse surfaces (15/15 and 6/6 under SH0, 14/15 for GStex).
  • W1, W4. The contribution goes beyond excluding the penalty. The paper certifies that the fitting objective itself prefers worse surfaces (19 of 23 scenes) and tells before the update where separation applies. With λ from the training images, GCV ends at half the error of joint fitting (PROFILE).

Thank you for the detailed review, and for finding the issue important and the theoretical analysis "a useful perspective".

W1 / Q1 (novelty). What is the key methodological novelty of Score Separation beyond excluding the regularization penalty from geometry selection? How does it differ from existing model-selection and appearance–geometry decoupling approaches?

Earlier work observed false surfaces in specific settings. Unbiased Depth traces false 2DGS surfaces on glossy regions to spherical harmonics that cannot follow sharp changes of radiance. NeRO, Ref-NeRF, GOF and PGSR address the coupling with richer appearance or additional evidence (§1, §2, Appendix M). Relative to them, the paper adds three things.

A certified explanation. The certificates bound the optimal appearance fit of each frozen reconstruction from both sides. They therefore test whether the objective itself prefers a worse surface, whichever optimizer is used. Of 508 comparisons, 505 are decided by the bounds, and strict preferences for physically worse surfaces appear in 19 of 23 reconstructed scenes (§4.4). This contradicts the account in which limited capacity protects the true surface. A causal test agrees, since reversing the aligned residual flips the preferred surface in all 36 contexts tested.

A separate geometry decision. The penalty that stabilizes the appearance fit overturns the data term in nearly a third of the signed comparisons (§3.3). The fit must keep it, and the ranking of surfaces must not. Score Separation therefore fits appearance once per candidate surface, with the penalty, and ranks it by a separate score. Because that score differs from the fitting objective, its gradient must include how the fitted appearance moves with the surface, a term that vanishes in joint fitting (§3.4, Eq. 4). Cross validation and GCV usually choose a weight or a model for fixed data. Here they rank surfaces, and each surface changes the rendering operator A(g) itself. Used directly as geometry scores, the cross-validation comparators recover worse surfaces than GCV at higher cost (Table 1, Table A8).

Geometry scoreHeight RMSE (mm)Extra cost per step
Camera-PGP5.03+61.0 %
Pixel-CV4.75+62.4 %
GCV (ours)4.32+3.9 %

Representation-level decoupling changes what appearance can represent, while Score Separation changes the rule that ranks geometry, so the two compose.

A rule of use with measured benefit. One λ serves every score, chosen by the leave-one-camera-out error of the appearance fit. At that λ, GCV ends at half the error of joint fitting (3.96 against 7.82 mm), and DATA and GCV both improve on it in every unseen material (Appendix J). The initial penalty share tells beforehand where the remedy applies, and at a 65 % share DATA removes 85 % of the drift of joint fitting (§4.5, Table 6).

Score Separation is simple because the analysis locates where the bias enters. That simplicity makes it a plug-in for any pipeline that fits appearance with a penalty, such as nvdiffrec and NeILF++, which train material smoothness priors jointly with geometry (Appendix M).


W2 / Q2 (coverage). Why is the scene-level evaluation limited to seven DTU scenes, and can the authors provide results on additional datasets such as Tanks and Temples?

Your requests for more scenes, for Tanks and Temples and for a decoupling method produced the strongest additions of the revision (§4.3, Table 3, Appendix I).

  • DTU. All 15 standard scans, at the official recipes of 2DGS and PGSR. Freeing the geometry under SH0 raises test PSNR and worsens the Chamfer distance in every scan for both methods. The eight added scans form a separate replication, which holds in all eight.
  • Tanks and Temples. The six scenes of the 2DGS paper, with the official recipe and evaluator. Under SH0, test PSNR rises and the F-score falls in every scene, by 0.061 on average, nearly a fifth of the host F-score. At the default SH3 it still falls in every scene.
  • Faithful models. Both methods reach their published DTU accuracy (2DGS 0.78 against 0.80 mm, PGSR 0.53 against 0.53 mm), and the Tanks and Temples models match the F-scores published with the 2DGS code.
  • Certificates. They cover all 15 DTU scans and the 8 Replica scenes.
AuthorsOpenReview ↗

Response to Reviewer tGwZ (2/2)

W3 / Q3 (decoupling methods). Can the authors compare Score Separation with relevant recent appearance–geometry decoupling methods or explain more clearly why these methods are not directly comparable?

The system run. Following your suggestion, we ran GStex with its own code and DTU recipe, starting from our 2DGS models, on all 15 DTU scans. Its images improve while its surface worsens (§4.3, Table 3a).

GStex texture stage, 15 DTU scansChangeScans
Test SSIM+0.06115/15 better
Test LPIPS−0.01714/15 better
Test PSNR+0.46 dB12/15 better
Chamfer distance+0.28 mm (p = 0.0004)14/15 worse
Accuracy+0.52 mm14/15 worse

The direct comparison, where geometry is decided. Table 1 adapts the core mechanism of each family under identical observations, start, geometry family, estimator and budget.

Mechanism (Table 1)Height RMSE (mm)
Joint fitting (PROFILE)6.09
Closed-form color (Fang et al.)6.33
Finer shared texture grid, 49², after GStex5.65
PGSR multi-view NCC5.35
Camera-PGP / Pixel-CV5.03 / 4.75
Richer spectral appearance4.87
DATA / GCV (ours)4.75 / 4.32

GCV has the best mean, P90 and rank, and its paired advantage over every comparator has an interval that excludes zero. GStex's mechanism, adapted to the shared estimator as a finer texture grid, ends at 5.65 mm against 4.32 mm for GCV.

Why complete systems are not directly comparable (Appendix M):

  1. They are built for a different quantity. GStex, A²TG, Textured Gaussians and AuGS target novel-view synthesis and report image metrics, with no geometry-accuracy metric. Image metrics cannot rank their surfaces, as the GStex run above shows.
  2. They act on a different layer. These methods change what appearance can represent, and Score Separation changes the rule that ranks geometry, so the two compose rather than compete (§3.4).
  3. They bring their own representation, rasterizer and schedule. A benchmark of systems therefore measures those choices together. Published ablations, reported for their own purposes, show the same trade (PGSR Table IV, Ref-NeRF Table 1, Appendix M).

A decoupled representation alone therefore does not protect the surface, and image metrics hide the loss of surface accuracy.


W4 / Q4 (practical significance). More evidence is needed to establish whether the proposed approach offers a substantial practical advantage over simpler alternatives. … How should the absolute geometric errors reported in the controlled experiments be interpreted in terms of practical reconstruction quality?

  • Against simpler alternatives. The simplest alternative is joint fitting with λ chosen from the training images. At that weight, GCV ends at half the error of this alternative, and even below joint fitting tuned against the reference geometry (3.96 against 4.44 mm, Appendix J). After tuning against the reference, closed-form color and multi-view NCC fall behind joint fitting, while GCV lowers the mean error by 13.7 % and improves every unseen material (Table 2). On reconstructed scenes where the penalty dominates, DATA removes 85 % of the drift of joint fitting (Table 5). DATA adds negligible cost, and GCV 3.9 % per step against 61 % for Camera-PGP (Appendix Table A8).
  • The controlled benchmark is sub-pixel by design. Every method starts below one pixel footprint (11.8 mm on the surface), and GCV ends near a third of one. The error of every surface in the benchmark, the starting one included, shifts the image by less than a quarter pixel between training cameras. The quantity compared is therefore the paired difference under identical conditions (Appendix N). At this scale every score in Table 2 reaches the same test PSNR within 0.1 dB, so only a geometry score can tell these surfaces apart.
  • On real scenes the bias is large in practical terms. GStex's texture stage costs more surface accuracy on our models than Unbiased Depth gains on 2DGS (0.28 against 0.17 mm). Under SH0, the cost on PGSR models exceeds the gap between the published DTU results of 2DGS and PGSR (0.43 against 0.27 mm, Appendix M).

The practical advantage is therefore largest where a user must choose λ without a scan, as on every real scene.

Thank you again. Do the 15 DTU scans with two published systems, the six Tanks and Temples scenes and the GStex run answer your questions on coverage and comparison? We would also welcome your view on novelty and practical significance.

AuthorsOpenReview ↗

Follow-up to Reviewer tGwZ

Dear Reviewer tGwZ,

Thank you again for your detailed review. We would welcome your view on these additions to the revision.

Coverage (W2, W3). The expanded evaluation uses host models trained with their official recipes. Under SH0, test PSNR improves while surfaces worsen on all 15 DTU scans for both 2DGS and PGSR, and on all six Tanks and Temples scenes for 2DGS. GStex, run using the authors' implementation, improves test SSIM in every scan while worsening the surface in 14 of 15 (Table 3).

Novelty and practical significance (W1, W4). Four variants in PGSR's published ablation have higher PSNR and lower F1 than the full method (Table IV, arXiv v2). PGSR links this trade-off to an incomplete image-rendering model.

We test whether the objective itself favors worse surfaces by comparing optimal appearance fits for each candidate geometry. Our certificates establish such preferences in 19 of 23 reconstructed scenes (§4.4). Table 6 gives the steps for identifying the capacity or penalty route before any geometry update.

Score Separation changes how geometry is ranked while retaining regularized appearance fitting. It can combine with methods that decouple appearance and geometry at the representation level. In practice, with λ chosen from the training images, GCV halves the error of joint fitting on six unseen materials (Appendix J).

Do these additions address your concerns on coverage and novelty? We would also welcome your view on the practical gain from choosing λ without a reference scan.

Reviewer_PzNROpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This work identifies an important problem in Gaussian splatting. Improved image quality does not necessarily imply more accurate reconstructed geometry. In some cases, the training objective may even drive the reconstructed surface toward an incorrect location. Through several experiments, the authors show that this is not merely an occasional optimization failure but may instead reflect an inherent limitation of the current optimization objective. The paper also proposes a potential remedy, although its effectiveness is not consistent across datasets.

Critical strengths
  1. The problem identified is important because many works use image metrics such as PSNR and SSIM to judge the ability of a model.
  2. The paper goes beyond documenting the phenomenon of good image quality but poor geometry and investigates its possible causes. One cause is insufficient capacity in the appearance model; another is that including the appearance regularization term in geometry selection may assign a lower training loss to an incorrect surface.
  3. The paper supports this claim with complementary experiments, including controlled surface and material studies and reconstructions on DTU scenes.
Critical weaknesses
  1. The proposed remedy is inconsistent. Score Separation uses DATA or GCV to select geometry updates. On the first Replica group, where the penalty accounts for 65% of the initial objective, DATA is 0.764 mm closer to the reference than PROFILE. Neither DTU evaluation shows this gain.
  2. The paper offers three scores, PROFILE, DATA, and GCV, but without a scanned 3D reference it is unclear how a user should choose among them beforehand.
Questions for authors
  1. Table 5 shows that DATA outperforms PROFILE only when the penalty dominates the objective. In the additional DTU scenes, the penalty accounts for only 6%, and DATA brings almost no gain. Does this mean that the dominant error there is insufficient appearance capacity, rather than the regularization bias that DATA targets?
  2. How do you decide to use PROFILE, DATA, or GCV when you don't have real 3D geometry to refer to? Can we detect whether the current error is mainly from insufficient expressive power of the appearance model or from the appearance regularization term before the geometry update?
3 comments
AuthorsOpenReview ↗

Response to Reviewer PzNR (1/2)

TL;DR.

  • Q1, W1. Yes, DTU is capacity-dominated, as you read it. The gain of DATA tracks the penalty share, with 85 % of PROFILE's drift removed at a 65 % share, while on DTU the surface cost falls with SH degree in all 15 scans.
  • W2/Q2 (2/2). Yes. The training images tell the cause before the geometry update, and Table 6 makes every choice in four steps, without a scan.

Thank you for the careful review. Your summary names the two causes exactly, "insufficient capacity in the appearance model" and "including the appearance regularization term in geometry selection". The revision calls them the capacity route and the penalty route (§1).

W1 (consistency of the remedy). The proposed remedy is inconsistent. … Neither DTU evaluation shows this gain.

The remedy is targeted, which is why its gain differs between datasets. It follows the route, not the dataset.

  • Where it acts. DATA differs from PROFILE only by leaving the penalty out of the ranking, so it changes the outcome where the penalty drives the descent. At a 65 % share it removes 85 % of PROFILE's drift away from the reference (Table 5).
  • The share predicts the gain, within one dataset and within one scene. Replica regions span initial shares from 0.06 to 0.99, and within Replica alone DATA's advantage grows with the share (ρ = −0.49 over 40 regions). Within a scene, regions with a share above one half favor DATA more than the scene's other regions, in 5 of the 6 scene cohorts where both kinds occur (Appendix K). The recorded DTU regions stay below a share of 0.3, so none is penalty-dominated. The revised Table 5 orders the cohorts by share.
  • Both DTU evaluations carry the capacity signature instead. At DTU's small share, DATA matches PROFILE, so DATA is a safe default. In both DTU cohorts, the native SH appearance, the remedy of the capacity route, ends closest to the reference (Table 5, Table A9).
  • Pipelines on the penalty route. A pipeline is on this route whenever the loss that moves its geometry also penalizes appearance, as in nvdiffrec, NeILF++ and GS-W (Appendix M). Score Separation applies there directly.

The remedy is therefore consistent with its mechanism. It acts where the penalty drives the descent, and the training images say where that is.


Q1 (capacity on DTU). Does this mean that the dominant error there is insufficient appearance capacity, rather than the regularization bias that DATA targets?

Yes. Your reading is the one the revision adopts, and the new runs support it.

  • Capacity sets the cost on DTU. We varied appearance capacity on all 15 DTU scans under the official 2DGS recipe. The surface cost of free geometry falls with SH degree in every scan and vanishes at SH3 (§4.3, Table 3, Fig. 1).
  • Evidence that bypasses appearance removes it too. Under the official PGSR recipe, the full loss keeps most of the image gain and leaves the surface unchanged across the 15 scans (Table 3a). Its multi-view NCC term compares observed patches without the fitted appearance, and the 160 px ablation attributes the protection to this term (Table A5).
  • Real scenes need more than SH3. SH3 closes the deficit on DTU's lab objects, but on Tanks and Temples freeing the geometry at SH3 still lowers the F-score in every scene (Table 3b).

Capacity, not the penalty, sets the surface cost on DTU, and the remedies of the capacity route remove that cost.

AuthorsOpenReview ↗

Response to Reviewer PzNR (2/2)

W2 / Q2 (choosing a score without a scan). How do you decide to use PROFILE, DATA, or GCV when you don't have real 3D geometry to refer to? Can we detect whether the current error is mainly from insufficient expressive power of the appearance model or from the appearance regularization term before the geometry update?

Yes. Every choice comes from the training images and the setup, in the order of Table 6, and the route is read before the geometry moves (§4.5). PROFILE needs no rule of its own, since DATA matches it where the share is small and improves on it where the share is large. Three terms need a definition. The initial penalty share is the penalty's part of the objective at the start. The capacity gap is the training loss of an appearance model minus that of a richer one, both refitted on the initial geometry. The material support is the set of surface points that carry the material.

StepInformation available before the updateAction
1. λLeave-one-camera-out error of the appearance fitOne λ for every score
2. RouteInitial penalty shareDATA in place of PROFILE, then step 3 if the share is small
3. CapacityCapacity gap on the initial geometryRicher appearance, or evidence that bypasses appearance (PGSR's full loss)
4. ScoreWhether the setup fixes the material supportGCV if fixed, DATA if the support moves

A large initial share means that the penalty drives the ranking. A small share leaves the error in the data term, which only richer appearance or evidence that bypasses appearance can reduce.

What each step buys.

  • λ. GCV ends at half the error of PROFILE (3.96 against 7.82 mm), and DATA and GCV both beat PROFILE in all six unseen materials (Appendix J).
  • Route. DATA removes 85 % of the drift at a 65 % share, and within Replica alone the share tracks DATA's advantage (ρ = −0.49 over 40 regions, Appendix K).
  • Capacity. SH3 removes the cost on DTU (−0.01 mm), and PGSR's full loss keeps 1.94 dB of image gain with a Chamfer distance change of at most 0.03 mm (Table 3a). Within every scan, the gap orders the cost of the SH degrees (15/15, Appendix L).
  • Score. On DTU, where the support moves, GCV's endpoints lose 23.9 effective degrees of freedom against 2.4 for PROFILE, so GCV there rewards simpler fits rather than better ones (Appendix G). DATA adds negligible cost, and GCV 3.9 % per step.

Worked example. The first Replica cohort has a 65 % share, so step 2 flags the penalty route and DATA removes most of the drift. The DTU confirmation cohort has a 6 % share, so the error is capacity-dominated, as you read it. DATA matches PROFILE there, and step 3 points to richer appearance, which removes the cost on DTU.

A user therefore needs no scan at any step.


The value, as you put it. "Many works use image metrics such as PSNR and SSIM to judge the ability of a model." Image metrics cannot choose among the scores, since every score in Table 2 reaches the same test PSNR within 0.1 dB. Under its own code and recipe, GStex, a published decoupling method, improves test SSIM and LPIPS while the official evaluation finds a worse surface in 14 of 15 DTU scans.

"An inherent limitation of the current optimization objective". The certificates establish this. They compare optimal fits over a convex class of native SH3 appearance, so the strict preference for worse surfaces in 19 of 23 reconstructed scenes lies in the objective, whatever the optimizer. More than half of these preferences also lower both training and test image error.

In short. Your two questions now have a procedure a user can follow without a scan, and none of its choices can be made from image metrics, by which many works judge a model.

Thank you again for the questions, which shaped §4.5 and Table 6. Does the procedure of Table 6 answer your Q2, choosing a score and telling the cause before the update without a scan? We would also welcome your view on the consistency of the remedy.

AuthorsOpenReview ↗

Follow-up to Reviewer PzNR

Dear Reviewer PzNR,

Thank you again for your review. Your two questions directly shaped the procedure in Table 6.

Q1. Why the gain differs across scenes. The revised DTU results support your capacity explanation. Geometric degradation decreases as the SH degree increases in every scan (Table 3). Within Replica, DATA's gain grows with the initial penalty share (Appendix K).

Q2. Choosing a score before the update. Table 6 uses the training images and the problem setup, without a reference scan.

  • On reconstructed scenes, use DATA in place of PROFILE.
  • When the initial penalty share is small, compare the appearance fit with a richer one on the same initial geometry. This comparison guides the choice between increasing appearance capacity and adding constraints that bypass fitted appearance.
  • When the material support stays fixed, use GCV, as in the controlled benchmark. The support is the set of points carrying the material.

Do these two answers resolve your concerns on the consistency of the remedy and on choosing a score? If anything remains open, we would be glad to answer it right away.

Reviewer_oe3XOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper provides an analysis of the 'appearance induced surface bias' when fitting the geometry from multi-view images using Gaussian splatting. The paper proposes Score Separation (DATA and GCV) for inferring geometry separately from regularized appearance fitting. It also proposes the contribution depth error metric to detect surface deterioration that image quality and mean depth often miss.

Critical strengths
  1. The experimental analysis provides helpful insights and evidence to analyze why such bias exists. It also shows that under SH0 setting, freeing the geometry leads to better PSNR but worse Chamfer distance.
  2. The authors conduct controlled experiments to demonstrate that GCV lowers the mean error by 13.7% for different materials.
Critical weaknesses
  1. The proposed method does not show reliable reconstruction gains after proper tuning of both DATA and Camera-PGP (as mentioned in line 361-362). In the appendix, results also show that GCV gives worse geometry estimation than PROFILE in all 7 DTU scenes (line 1173 to 1174, where average error difference is +1.161 mm).
  2. The main experiments are conducted on a restricted setting (SH0 rather than SH3, and images with only 160-pixel width), so the method only optimize over diffuse color. However, in the real world, most materials' rendering is not governed by the diffuse term alone, so the analysis itself is only in a narrow setting and the practical relevance is unclear.
Questions for authors

Please see the weakness section. My question is mainly related to the Weakness 2. Q1. Since the paper mainly considers SH0 representation and, how does the 'appearance induced surface bias' change with richer SH3 appearance, and with bigger image resolution?

5 comments
AuthorsOpenReview ↗

Response to Reviewer oe3X (1/2)

TL;DR.

  • W2/Q1. The bias holds beyond SH0 and 160 px. At the official recipes and resolution, freeing the geometry under SH0 leads to better PSNR but worse Chamfer distance in all 15 DTU scans. The cost shrinks as SH degree rises, yet at the default SH3 the F-score still falls in all six Tanks and Temples scenes.
  • W1 (2/2). Reliable gains without a scan. With λ from the training images, GCV ends at half the error of joint fitting (PROFILE) and even below joint fitting tuned against the reference geometry. For the DTU refinements you cite, DATA is the score to use, and (2/2) explains why.

Thank you for finding that the analysis provides "helpful insights and evidence to analyze why such bias exists". Your question on SH3 and resolution led us to rerun the study with the official training recipes of 2DGS and PGSR, and its answer is now the lead result of the revised paper.

W2 / Q1 (richer appearance and resolution). … the analysis itself is only in a narrow setting and the practical relevance is unclear. … how does the 'appearance induced surface bias' change with richer SH3 appearance, and with bigger image resolution?

We should have made clearer that SH0 is the lowest color setting of one experiment, which continues trained 2DGS and PGSR models. The certificates (§4.4), which test the objective itself, use full SH3 color, and the table that now opens §4 lists the appearance model of every experiment.

Richer appearance. The bias scales with the appearance deficit, the part of the appearance that the color model cannot represent. We continued 2DGS models, trained with the official recipe on all 15 DTU scans, for 2,500 steps with their color restricted to SH degree 0 to 3 and the geometry either frozen or free. The table gives free minus frozen.

Appearance (DTU, 15 scans)Δ test PSNR (dB)Δ Chamfer distance (mm)PSNR betterSurface worse
SH0+2.18+0.2815/1515/15
SH1+1.22+0.1515/1513/15
SH2+0.39+0.0615/1511/15
SH3+0.12−0.0114/157/15

The surface cost of free geometry falls with SH degree in every scan. On DTU's lab objects, SH3 removes most of the deficit, so the mean cost nearly vanishes (Appendix I). The bias itself remains wherever the color model falls short, as three results show.

  • Tanks and Temples at SH3. Freeing the geometry brings no image gain (test PSNR −0.04 dB), yet the F-score falls in all six scenes of the 2DGS benchmark, through precision and recall alike. The mean drop of 0.024 is about 7 % of the host F-score (0.33). Real scenes like these are the case you describe, where "most materials' rendering is not governed by the diffuse term alone".
  • GStex. It gives every Gaussian its own texture and still worsens the surface in 14 of 15 DTU scans under its own recipe (Table 3a).
  • The objective itself. With the best possible SH3 color fitted to each surface, the objective still prefers a physically worse surface in 19 of 23 reconstructed scenes (§4.4). More than half of these worse surfaces also lower training and test image error.

Bigger resolution. The trade between better images and worse surfaces keeps its sign and size when the pixel count grows more than 23-fold. The first two columns compare the seven scans of the submission's 160 px experiment, and the last extends the official recipe to all 15. Each cell gives Δ test PSNR / Δ Chamfer distance and the scans that show both changes.

Model, SH0, photometric loss160 px, 7 scansOfficial recipe (-r 2), same 7 scansOfficial recipe, all 15 scans
2DGS+2.87 dB / +0.33 mm (7/7)+2.71 dB / +0.26 mm (7/7)+2.18 dB / +0.28 mm (15/15)
PGSR+3.33 dB / +0.39 mm (7/7)+3.45 dB / +0.42 mm (7/7)+2.71 dB / +0.43 mm (15/15)

At the official recipe, both models match their published accuracy (§4.3, Table 3, Fig. 1, Appendix I).

Practical relevance. Dense SLAM, compression and mobile rendering, and feed-forward reconstruction restrict color capacity by design (nine systems in Appendix M). At the lowest setting, a 2DGS model trained from scratch with SH0 color under the official recipe has a worse surface than the SH3 model in all 15 DTU scans (1.23 against 0.78 mm, Appendix I). This gap of 0.45 mm exceeds the 0.27 mm between the published DTU results of 2DGS and PGSR.

The bias therefore follows the appearance deficit, at the official resolution and on real scenes, and it costs surface accuracy wherever color capacity is restricted.

AuthorsOpenReview ↗

Response to Reviewer oe3X (2/2)

W1 (reliable gains). The proposed method does not show reliable reconstruction gains after proper tuning of both DATA and Camera-PGP (as mentioned in line 361-362). … GCV gives worse geometry estimation than PROFILE in all 7 DTU scenes (line 1173 to 1174, where average error difference is +1.161 mm).

With the penalty weight λ chosen from the training images, as a user without a scan must choose it, GCV ends at half the error of joint fitting (PROFILE) on six unseen materials (3.96 against 7.82 mm). It even ends below joint fitting tuned against the reference geometry (4.44 mm, Appendix J), in five of the six materials. The rule of use is DATA on reconstructed scenes and GCV where the material support (the surface points that carry the material) is fixed, as in the controlled benchmark (§4.2, §4.5, Table 6).

Why DATA and joint fitting end on par after tuning. You are right that the two end on par after this tuning. The tuning in lines 361–362 of the submission (now §4.2) selects λ against the reference geometry. It gives joint fitting a penalty weight ten times smaller than the one the training images select (10⁻⁶ against 10⁻⁵). The penalty that DATA leaves out then weighs ten times less, so the two end on par, as the mechanism predicts. A user without a scan cannot choose that weight.

Reliable gains without a scan. The training images select λ by the leave-one-camera-out error of the appearance fit, one value for every score. At that λ, each of the three scores that leave the penalty out of the ranking improves on joint fitting in all six unseen materials, with an interval that excludes zero. GCV does best, at 3.9 % extra cost per step against 61 % for camera cross validation (Appendix J, Table A8).

ScoreDifference from joint fitting (mm), 95 % CIMaterials improved
Camera-PGP (camera cross validation)−3.44 [−4.98, −2.12]6/6
DATA−2.87 [−4.22, −1.75]6/6
GCV−3.86 [−5.41, −2.62]6/6

Compared with its error at the tuned λ, GCV's error rises by only 3 %, while that of joint fitting rises by 76 %. As you note, GCV also lowers the mean error by 13.7 % under fair tuning and improves all six unseen materials (Table 2).

Why the DTU result selects DATA. GCV divides the image error by a complexity factor. This ratio estimates prediction error only while the material support stays fixed, as noted in §4.5 of the submission. In the controlled benchmark the support is fixed, and GCV has the best mean, P90 and rank of all ten methods (Table 1). In the DTU region refinements behind the +1.161 mm, the support moves with each trial surface. A surface can then lower GCV by making the fitted material simpler instead of fitting the images better. GCV's endpoints there lose 23.9 effective degrees of freedom against 2.4 for joint fitting, and the more an endpoint loses, the farther it ends from the reference (Spearman ρ = 0.43, Appendix G). Since the setup tells before any update whether the support moves, the rule of use selects DATA there (Table 6, step 4).

DATA on reconstructed scenes. Where the penalty dominates, DATA removes 85 % of the drift of joint fitting away from the reference. Where the penalty is small, as in the DTU scenes you cite, DATA matches joint fitting (Table 5).


Presentation. Thank you for flagging it. The abstract and introduction now lead with the official-recipe evidence, and §4 opens with a table that gives the claim, data, appearance model and route of every experiment. The contribution depth error noted in your summary exposes deterioration that images hide. Among DTU endpoints with better RGB, it flags 43 as deteriorated, against 27 for mean depth (§4.6).

Thank you again. Do the 15-scan SH0 to SH3 table and the resolution table answer your Q1? We would also welcome your view on the results for reliable use.

AuthorsOpenReview ↗

Follow-up to Reviewer oe3X

Dear Reviewer oe3X,

Thank you again for your careful review. We would welcome your view on the revised evidence addressing your two main concerns.

Q1. The bias persists with SH3 at the official resolution. On Tanks and Temples, photometric geometry updates with SH3 lower the F-score in all six scenes. The mean drop is about 7% of the host F-score, while test PSNR is nearly unchanged (Table 3b).

With SH3 appearance fitted optimally to each candidate surface, the objective still prefers physically worse surfaces in 19 of 23 reconstructed scenes, as our certificates establish (§4.4). On DTU, geometric degradation decreases as the SH degree increases.

W1. Reliable gains without a reference scan. With λ chosen from the training images, GCV halves the error of joint fitting on six unseen materials (Appendix J). On reconstructed scenes where the appearance penalty dominates, DATA removes 85% of the drift of joint fitting (Table 5).

Table 6 gives the steps for choosing the score before any geometry update. When appearance capacity is insufficient, PGSR's full loss also protects the surface using evidence that bypasses fitted appearance (Table 3a).

Does the evidence at native SH3 and the official resolution address your concern about the scope of the effect? We would also welcome your view on the gains obtained without a reference scan.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.