acceptodds
Under review as a conference paper at ICLR 2027

Beyond Angular Error: Global and Local Geometry of Failure in Monocular Surface-Normal Models

Abstract

Per-pixel angular error measures how much a surface-normal prediction is wrong, but it hides the geometric form of that error. We show that failures of modern monocular surface-normal estimators exhibit reproducible structure at both global and local scales. Globally, we use continuous generalized bas-relief (GBR) fitting as an interpretable coordinate system for prediction–reference discrepancies. Across four architectures, matched pairs concentrate in a shared low-shear regime, while correspondence-destroying controls move the fitted transforms toward higher shear. The remaining relief coordinate also transfers beyond the pixels used to estimate it: on 100 randomly sampled NYUv2 validation images, cross-fitted correlates with angular error on disjoint held-out pixels across all four models (). Locally, we derive a prediction-only curl score from a single estimated normal field. On 500 NYUv2 images per model, it identifies the worst 10% of predictions with AUROC 0.77–0.91, outperforming generic normal roughness and improving selective risk–coverage. Together, these results show that normal-estimation errors are organized by reusable geometric structure, providing both a diagnostic view of model failure and a practical signal for test-time reliability.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
monocular surface normals, monocular geometry, geometric foundation models, generalized bas-relief

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_q4kGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the geometric structure of errors in monocular surface-normal estimation beyond conventional angular-error metrics. It introduces two complementary analyses: continuous generalized bas-relief (GBR) fitting to characterize global prediction–reference discrepancies, and a prediction-only curl score to assess local geometric inconsistency.

Experiments across four normal estimators show that matched predictions tend to exhibit low-shear GBR structure, while the fitted relief parameter correlates with held-out angular error. The curl score achieves AUROC values of 0.772–0.913 for identifying the worst 10% of predictions on NYUv2, outperforming normal roughness. Additional experiments include correspondence-breaking controls, selective prediction, and cross-dataset evaluation.

Critical strengths
  1. The paper addresses an important limitation of angular-error metrics by examining how prediction errors are geometrically structured. Its evaluation across four different architectures, together with correspondence-breaking controls and pixel cross-fitting, provides evidence that the observed patterns are not purely incidental.

  2. The curl score requires neither retraining nor additional network inference. Its consistent improvements over roughness, together with supplementary TTA comparisons and iBims-1 evaluation, demonstrate promising practical value for identifying unreliable predictions.

Critical weaknesses

Although matched predictions exhibit low fitted shear, this does not establish that GBR explains a dominant portion of their errors. Substantial angular errors remain after full GBR fitting (e.g., 35.19° to 32.71° for Marigold and 33.74° to 31.41° for MoGe-2 in Table 1).

Furthermore, GBR is not compared with alternative low-dimensional transformations of comparable complexity. The random pixel cross-fitting protocol also does not establish transferability across spatially distinct regions. The NYUv2–GSO comparison changes both projection geometry and scene content, making the interpretation of the lower shear on GSO less conclusive.

Consequently, the evidence supports GBR as a useful diagnostic representation but does not yet fully establish it as a distinctive, general failure structure.

The curl formulation follows integrability conditions under an orthographic height-field model, whereas the primary experiments involve perspective images. Although Appendix K mentions perspective-correct alternatives, quantitative evidence is limited.

Moreover, an integrable normal field can still exhibit substantial systematic angular error. Thus, curl measures local consistency rather than accuracy itself. Stronger controls are needed to establish whether its predictive advantage specifically reflects integrability violations rather than boundary complexity, surface orientations, or numerical effects.

The novelty discussion should also better distinguish this work from prior integrability-based normal estimation (e.g., Yoon et al., ECCV 2016) and surface-normal uncertainty estimation (e.g., Bae et al., ICCV 2021).

Questions for authors

1. Can the authors quantitatively compare the proposed curl score with perspective-correct formulations and boundary-masked variants on the full reliability evaluation set? Does its advantage persist after controlling for surface-orientation and boundary effects?

2. How much of the original angular error is explained by GBR compared with alternative global transformations of similar complexity? Does GBR fitting remain effective when evaluated on spatially contiguous held-out regions rather than randomly sampled pixels?

3. Can the authors clarify the reference-normal generation, curl implementation, and evaluation protocols, and provide the main experimental code or per-image results? The supplied supplementary code primarily reproduces a GBR visualization rather than the paper's principal quantitative experiments.

Reviewer_Ln7FOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript studies the geometric patterns in monocular surface-normal estimation errors. It characterizes global prediction–reference discrepancies using continuous generalized bas-relief (GBR) fitting, revealing a low-shear regime dominated by relief scaling across modern models. The GBR analysis is conducted on perspective NYUv2 images and orthographically rendered GSO scenes, while the fitted relief parameter correlates with angular error on held-out pixels. The manuscript also presents a prediction-only score, curl, derived under an orthographic height-field assumption, to measure local non-integrability and assess test-time reliability. Experiments on perspective NYUv2 images show that this score outperforms generic normal roughness in identifying high-error predictions and supporting selective prediction.

Critical strengths
  1. Practical reliability assessment. The curl-based score identifies high-error predictions without additional training or reference geometry.
  2. Extensive experiments. Evaluations across four models, multiple controls, and datasets support the findings.
Critical weaknesses
  1. Orthographic projection assumption. Both GBR fitting and the curl-based consistency measure rely on orthographic geometry, while the main experiments use perspective NYUv2 images. Their geometric validity under perspective projection needs clarification.
  2. Limited novelty of the low-shear finding. Surface-normal estimation inherently suffers from ambiguity in surface relief, closely related to the ambiguity in relative depth variations in monocular depth estimation. Therefore, relief-dominated errors are not surprising, and the current experiments do not convincingly establish that the observed low-shear regime is a non-trivial finding.
  3. Limited practical significance of the global GBR analysis. Although GBR fitting can reduce angular error, it requires ground-truth normals. The practical utility of the low-shear finding remains unclear.
  4. The proposed curl score is insensitive to integrable distortions, including GBR shear. It remains unclear whether curl can detect more challenging errors outside the low-shear regime.
  5. Limited experimental validation. The experiments focus primarily on NYUv2, with limited evaluation on other datasets. Moreover, the qualitative analysis only presents rendered examples.
Questions for authors

1.How are the GBR fitting and curl-based consistency measures geometrically justified under perspective projection? 2.Can the proposed curl score detect errors outside the low-shear regime, given its insensitivity to integrable distortions? 3.Is the relief ambiguity in monocular normal estimation related to the ambiguity in relative depth estimation, as both arise from uncertainty in scene geometry along the viewing direction? Is there any fundamental difference between the two?

Reviewer_rbhHOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies distortions of surface normal map from monocular surface normal estimation using two post-hoc geometric measures: continuous GBR fitting for global prediction–reference discrepancies and an image-space curl statistic for prediction-only reliability. Experiments on four pretrained models report low-shear GBR fits, correlations between fitted relief parameters and held-out angular errors, and curl-based failure detection on NYUv2. Additional experiments include correspondence controls, orthographic GSO renders, and iBims-1 transfer.

Critical strengths
  1. The motivation is quite interesting and well grounded. Using GBR to characterize geometric distortions beyond angular error offers an interpretable perspective on monocular normal-estimation failures. The distinction between reference-dependent global diagnostics and prediction-only reliability is also meaningful.

  2. The evaluation is well structured across four pretrained models, including correspondence-destroying controls, pixel cross-fitting, and controlled orthographic rendering. The authors also carefully separate fitting from evaluation and aggregate results at the image level. The reported AUROC improvements, selective-prediction results, supplementary TTA/repeat-inference comparisons, and iBims-1 transfer provide useful empirical evidence.

Critical weaknesses
  1. The geometric interpretation of the curl score under perspective imaging remains insufficiently established. Section 3.2 derives Eqs. (5)–(6) under an orthographic height-field model, whereas the main reliability evaluation uses perspective NYUv2 images. For a smooth perspective surface parameterized as X(x,y)=Z(x,y)(x,y,1)⊤\mathbf{X}(x,y)=Z(x,y)(x,y,1)^\top, using normalized image coordinates and camera-axis depth Z>0Z>0, the log-depth gradients satisfy $ \partial_x\log Z=-\frac{n_x}{D},\qquad \partial_y\log Z=-\frac{n_y}{D}, \qquad D=xn_x+yn_y+n_z\neq 0. $ These differ from Eq. (5), so Eq. (6) can be nonzero even for exact normals of a smooth perspective surface. Appendix K mentions perspective-correct and projection-based variants on a 30-image cache but provides neither their formulations nor numerical results. A related limitation, inherent to integrability-based measures, arises even under the orthographic model. For spatially constant GBR parameters, Eq. (2) implies p′=λp+μp'=\lambda p+\mu and q′=λq+νq'=\lambda q+\nu, hence $ \partial_x q'-\partial_y p' =\lambda(\partial_x q-\partial_y p). $ Thus, GBR transformations preserve zero curl despite potentially substantial angular errors, while nonzero curl is scaled by λ\lambda. This limitation is compatible with the proposed complementary global/local analyses and does not invalidate the empirical reliability results, but clarifies the failure types that curl cannot detect. The reported AUROC, roughness comparisons, and selective-prediction results support its empirical usefulness, but do not resolve the perspective-model discrepancy. Clarifying these distinctions would strengthen the geometric interpretation of the local score.

  2. The global experiments do not sufficiently distinguish GBR-specific failure structure from alternative explanations or establish the full scope of transfer. Table 2 shows that wrong-image pairing increases fitted shear by factors of 1.74–2.61, with confidence intervals excluding one for all four models. This supports correspondence dependence, but the controls do not hold baseline angular error fixed. It therefore remains unclear whether the low-shear pattern provides information beyond overall prediction–reference agreement. Moreover, the comparisons remain within the GBR family; a comparison with a similarly simple global transformation, such as a three-parameter normal rotation, would help assess the distinctiveness of the observed structure. The cross-fitting experiment uses five random 50/50 pixel partitions per image, evaluated in both directions, with coordinate selection and fitting restricted to the fitting subset and results aggregated at the image level. This properly establishes transfer to unused pixels within the same image, but not to contiguous spatially held-out regions. Additionally, the correlation between fitted log⁡λ\log\lambda and held-out angular error does not establish incremental information beyond the fitting subset’s baseline angular error, since no conditional analysis is reported. These results support reproducible and transferable GBR coordinates, while further targeted analysis would strengthen the interpretation of their distinctiveness and the extent of the claimed spatial generalization.

In summary, I lean toward weak rejection. The prediction-only reliability results are empirically promising, and the experimental controls support the reported observations. However, I am not yet convinced that the global GBR results establish a distinctive and reusable geometric structure of model failures beyond simpler explanations. The perspective-model discrepancy further limits the geometric interpretation of the local score, although it does not invalidate its empirical utility. Targeted additional analysis could change my assessment.

Questions for authors
  1. Could authors provide the formulations and numerical results for the perspective-correct and projection-based variants mentioned in Appendix K, and clarify the interpretation of Eq. (6) for perspective inputs?

  2. Could authors explicitly discuss the limitation that coherent GBR-induced normal errors can preserve zero curl, and clarify which failure types the proposed reliability score can and cannot detect?

  3. Could authors compare GBR fitting with a three-parameter global normal rotation under the same protocol? Could you also assess whether the matched-versus-control shear difference persists after accounting for baseline angular error?

  4. Could authors clarify the term "spatial generalization" in relation to random pixel splitting? Does log⁡λ\log\lambda remain informative after accounting for the fitting subset’s baseline angular error? If available, could you also report results using contiguous spatial holdouts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.