acceptodds
Under review as a conference paper at ICLR 2027

Surface Reconstruction from Range Scans Under Visibility Constraints

Abstract

Surface reconstruction from range scans is challenging in the presence of noise and outliers, as fidelity to observations must be balanced against sensitivity to measurement errors. Yet, most neural approaches discard the sensor positions and corresponding visibility information, compensating for this loss with geometric priors or regularization terms. We instead formulate surface reconstruction as robust regression of the predicted first surface intersection along each acquisition ray against its measured depth, which penalizes predicted surface intersections within the segment between the sensor and the observed point. We represent the surface as the zero set of a 1-Lipschitz multi-layer perceptron, allowing the use of linearly convergent sphere tracing to compute the first hit, and use a robust Huber loss for regression. Our formulation requires neither oriented normals, sampled off-surface points, nor an eikonal loss. Comparisons with the state of the art on synthetic and real data with varying levels of noise and outlier contamination demonstrate competitive performance on clean data and much improved fidelity and robustness under even moderate corruption. Our analysis further confirms a trade-off between fidelity and robustness: greater fitting capacity improves data fidelity in the clean case but increases sensitivity to the measurement errors inherent in real acquisitions, with a sweet spot typically emerging at intermediate capacity.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Surface reconstruction, first-hit depth regression, visibility-aware reconstruction, neural implicit surfaces, sphere tracing, 1-Lipschitz multi-layer perceptron

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LdH7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a visibility-aware approach to surface reconstruction from range scans. Instead of directly fitting an implicit surface to observed point samples, the method formulates reconstruction as robust regression of the first surface intersection along each acquisition ray against the measured depth. The surface is represented by a Lipschitz-constrained neural implicit function, and sphere tracing is used to locate the predicted first intersections. The optimization combines an incidence-weighted depth residual with a Huber loss to improve robustness to measurement noise and outliers.

The authors evaluate the method on synthetic datasets, including SRB and ShapeNet, as well as real-world LiDAR data from ARKitScenes and KITTI. The results demonstrate competitive reconstruction quality on clean data and improved robustness under noisy and outlier-contaminated observations. The paper also investigates the trade-off between reconstruction fidelity and robustness through different positional encoding levels.

Critical strengths
  1. Well-motivated formulation that makes effective use of visibility information. A key strength is the formulation of surface reconstruction as first-hit depth regression. By explicitly using acquisition rays, the approach incorporates free-space information that is often discarded by point-based neural implicit reconstruction methods. The resulting objective is conceptually simple and avoids requiring oriented normals, off-surface samples, or additional geometric regularization losses. I find this formulation meaningful, particularly for range scans where sensor positions are available.

  2. Convincing robustness results across different corruption settings. The experimental results provide useful evidence of the method's effectiveness under noise and outliers. The comparisons on SRB and ShapeNet cover multiple corruption settings, while the ARKitScenes experiments further demonstrate its potential on real LiDAR measurements. The ablation studies and the analysis of positional encoding levels also help explain the trade-off between reconstruction accuracy and robustness. These results support the practical value of the proposed approach.

Critical weaknesses
  1. The theoretical properties of the practical variants could be better clarified. The theoretical motivation for using a 1-Lipschitz network is reasonable, as it supports reliable first-hit computation through sphere tracing. However, as discussed in Section 3.5, the empirical rescaling and positional encoding used in the practical variants no longer provide a formal 1-Lipschitz guarantee. Although the paper acknowledges this limitation and provides empirical evidence, it would be useful to better understand how the relaxation affects the reliability of first-hit tracing in practice. This is particularly relevant because first-hit computation is central to the proposed formulation.

  2. Computational cost and the scope of real-world validation remain limitations. The method requires substantially more training time than the compared baselines, as acknowledged in Section 5. While the robustness improvements are encouraging, this overhead may limit its applicability to larger-scale reconstruction tasks. In addition, the quantitative real-world evaluation is mainly conducted on one ARKitScenes scene, while the KITTI evaluation is qualitative. A clearer discussion of the practical trade-off between reconstruction quality, computational cost, and scene complexity would strengthen the experimental evaluation.

Questions for authors

Q1. Since the practical variants are not formally guaranteed to be 1-Lipschitz, could the authors clarify how this affects sphere-tracing reliability? In particular, is there empirical evidence that the first-hit computation remains stable across different positional encoding levels and challenging geometries?

Q2. The paper reports a substantial increase in training time compared with the baselines. Could the authors further clarify the quality-efficiency trade-off, particularly whether the robustness advantages remain significant when comparing methods under similar computational budgets?

1 comment

1. Since the practical variants are not formally guaranteed to be 1-Lipschitz, could the authors clarify how this affects sphere-tracing reliability? In particular, is there empirical evidence that the first-hit computation remains stable across different positional encoding levels and challenging geometries?

We estimate the Lipschitz constant from sampled gradient norms and rescale the field accordingly, validating the resulting sphere tracing against certified reference intersections (Table 6). Across all tested PE levels (L=0,2,4,6L=0,2,4,6), we observe no overshoots and over 99% first-hit accuracy. These results support reliable tracing on the geometries represented in standard reconstruction benchmarks. Whether this extends to more challenging geometries remains to be systematically evaluated.

2. The paper reports a substantial increase in computation time compared with the baselines. Could the authors further clarify the quality-efficiency trade-off, particularly whether the robustness advantages remain significant when comparing methods under similar computational budgets?

We reduced computation time per shape from approximately 2h45 to 45 minutes on an NVIDIA RTX 6000 (L=0L=0) through CUDA graph replay and kernel fusion, achieving a 3.7× speedup with unchanged reconstruction results. Updated runtime comparisons are provided in Table 15. Furthermore, Table 16 reports reconstruction accuracy at intermediate training budgets on SRB, using the architecture with the lowest final CD1\mathrm{CD}_1 for each corruption setting. After 15k steps (approximately 27 minutes), our method already achieves the lowest mean CD1\mathrm{CD}_1 among all compared methods under both scanner noise (λ=1\lambda=1 with L=0L=0) and outlier contamination (α=0.2\alpha=0.2 with L=2L=2), demonstrating that its robustness advantages persist at reduced computational budgets.

We have also expanded quantitative real-world validation from one to four distinct ARKitScenes acquisitions, reconstructing surfaces from iPad LiDAR measurements and evaluating against held-out FARO reference depths. Our method achieves the lowest depth MAE on all four scenes (Table 4).

Reviewer_PD5yOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an algorithm to reconstruct surfaces from range scans using scan positioning data. Like other reconstruction methods, it represents the surface implicitly with a coordinate network; however, unlike other work, it uses sphere tracing and exact differentiation to encourage the reconstruction to satisfy empty space constraints. Its results are competitive when compared to some prior work and are particularly good in the presence of noise and outliers.

Critical strengths

The paper's main strength is in the direct supervision of each sensor ray with sphere tracing. While this is something that had been attempted before, this paper elegantly avoids differentiating through the many steps of sphere tracing (which could lead to gradient explosion and non-robustness) with implicit differentiation. I think this is a powerful contribution to the field of implicit reconstruction: while this paper places it in competition with other previously existing methods, II think it can be seen as complementary and I am excited for the future work it may inspire.

Critical weaknesses

The paper's main weakness is in its comparisons with prior work. A wide array of methods for this task have been published in the past three years that not compared against in this paper, including some which purport to be the state of the art. For example,

  • Methods that orient normals followed by PSR, like "Fast and Globally Consistent Normal Orientation..." or "iterative PSR"
  • All the methods deriving from applications of the Gauss formula; e.g., "Anisotropic Gauss Reconstruction for Unoriented Point Clouds"
  • Heat-based methods like "HotSpot: Signed Distance Function Optimization with an Asymptotically Sufficient Condition"
  • "Neural-Singular-Hessian" and its followups
  • "StEik: Stabilizing the Optimization of Neural Signed Distance Functions and Finer Shape Representation"

All or almost all of these purport to outperform DiGS, which seems competitive with the method proposed in this paper; therefore, I am wondering what this paper's performance would be if compared to these more modern baselines for this task.

Questions for authors

I would love to understand the reasoning behind the chosen comparisons, and whether the authors plan to include any more modern state of the art methods in their revised manuscript.

Also, I wonder: given the filtering operation in L210 is not differentiated through, and that the sole supervision is on the sensor rays, how does this method ensure that there is no "garbage geometry" away from the areas covered by the sensor rays?

Additionally, I have minor questions (for example, about the accuracy of saying that this paper is "regress[ing] the first zero [...] directly" when it is really a part of a loss function like it is in other methods, or about the lack of an absolute value in Eq (2)), but these will not materially affect my assessment.

3 comments

1. I would love to understand the reasoning behind the chosen comparisons, and whether the authors plan to include any more modern state of the art methods in their revised manuscript.

Our original baselines cover representative neural implicit and classical surface reconstruction approaches. As suggested, we extended our comparisons with four additional neural methods (StEik [58], HotSpot [55], NSH [56], and Neural RGB-D [6]) and two classical methods (AGR [36] and iPSR [25]), and updated Tables 1–3 accordingly. Our method achieves the lowest Chamfer distance on SRB under scanner noise and outliers, confirming that our robustness advantage persists against these additional baselines.

2. Also, I wonder: given the filtering operation in L210 is not differentiated through, and that the sole supervision is on the sensor rays, how does this method ensure that there is no "garbage geometry" away from the areas covered by the sensor rays?

Our objective focuses on the observed geometry rather than explicitly regularizing unobserved regions, where spurious surfaces remain possible. As established in Appendix B.4, for a 1-Lipschitz field, sphere-tracing iterates define certified surface-free balls whose union extends these constraints beyond the rays. For our empirically rescaled fields (L=2L=2), the corresponding balls cover 85–94% of reference free-space voxels, with only 0.01–0.18% overlap with occupied voxels (Table 7).

3. Additionally, I have minor questions (for example, about the accuracy of saying that this paper is "regress[ing] the first zero [...] directly" when it is really a part of a loss function like it is in other methods, or about the lack of an absolute value in Eq (2)).

We agree that "direct regression" is imprecise. We regress the predicted first-intersection depth against the measured depth. Our distinction is using this as the sole fitting objective, without auxiliary losses. We have clarified this distinction in the revised manuscript.

We also agree that Eq. (2) implicitly assumes an even penalty ρ\rho, as is the case for Huber. We have added the absolute value to make the formulation valid for general robust penalties.

Thank you! Glad to see these changes

We sincerely thank you for your constructive feedback and positive evaluation. We are glad that you found the revisions helpful.

Reviewer_NodDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a surface reconstruction method for 3D scanning and reports better reconstruction quality than competing methods, especially when the scans contain outliers. The paper combines existing techniques like visibility constraints from the sensor positions, sphere tracing, 1-Lipschitz neural networks and the Huber loss into a single formulation. A 1-Lipschitz MLP represents the surface as an SDF, and the loss compares the depth where each sensor ray first hits this surface with the measured depth. The method is evaluated on synthetic data with noise and outliers and on real LiDAR scans. The experimental results show that the method is much more robust to outliers than the neural baselines and performs comparably on clean data.

Critical strengths

The paper combines a lot of these existing reconstruction techniques into a clean loss formulation. A single depth loss along each sensor ray replaces several inputs and loss terms that other neural methods rely on. Because the loss uses the first point where the ray hits the predicted surface, any predicted surface between the sensor and the measured point would make the predicted depth too short. This keeps that space empty without the separate free-space term that other similar works use. The 1-Lipschitz network is designed so that sphere tracing finds this first hit without stepping past it. This is the paper’s main contribution.

Critical weaknesses

It is not clear how much of the outlier robustness comes from the proposed formulation and how much comes from simply using the sensor positions. Keeping the space between the sensor and each measured point empty is standard in depth-based reconstruction. Most of the baselines in the paper don’t take into account the position of the sensor, so they have no way to tell an outlier from a real point. The only baseline that uses the sensor positions like the paper is Labatut, and the results show that it comes close to the proposed method. The only neural baseline that uses visibility is NDF-vis, which is the authors’ own adaptation and performs poorly even on clean data. Closer neural methods that also use depth and sensor poses, such as [1], are not compared. The paper’s main contribution is the formulation itself. Without these comparisons, the experiments mainly show that using the sensor positions helps, which was already known.

The paper recommends using L = 2, which adds positional encoding, and all reported variants rescale the network by a slope estimated from sampled points. The paper acknowledges that both mean the network is no longer guaranteed to be 1-Lipschitz, but it doesn’t measure whether sphere tracing actually overshoots the first hit as a result. Since the 1-Lipschitz network is the main justification for first-hit tracing being reliable, this should be verified.

[1] Azinović, Dejan, et al. “Neural RGB-D surface reconstruction.” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.

Questions for authors
  1. Can the authors compare against a neural method that also uses depth and sensor poses, such as a depth-only version of [1]?
  2. How often does sphere tracing overshoot the first hit for the rescaled and positional encoding variants?
  3. Was NDF-vis tuned for the visibility-based labels, or run with the original NDF settings?
  4. If there is a scan with unknown noise and outliers, how should L and the Huber thresh be chosen?
3 comments

1. Can the authors compare against a neural method that also uses depth and sensor poses, such as a depth-only version of [1]?

Our comparisons already include the visibility-aware method of Labatut et al. [32]. We additionally evaluate Neural RGB-D [6], which imposes free-space constraints at sampled points along depth rays. Since it requires depth maps, for SRB we derive sparse ones from the input point clouds and use the LiDAR depth maps for ARKitScenes. Our method achieves lower Chamfer distance on SRB, both on clean scans and under noise and outliers (Tables 1 and 3), and lower depth MAE on all four ARKitScenes scans (Table 4), with approximately 45 minutes of fitting versus 12 hours for Neural RGB-D on an RTX 6000. These results indicate that access to sensor information alone does not explain our method's advantages.

2. How often does sphere tracing overshoot the first hit for the rescaled and positional encoding variants?

Across all tested PE levels (L=0,2,4,6L=0,2,4,6), we observe no overshoots and over 99% first-hit accuracy against certified reference intersections (Table 6).

3. Was NDF-vis tuned for the visibility-based labels, or run with the original NDF settings?

We thank the reviewer for raising this point. Our initial NDF-vis experiments used 10K samples, following NDF's noisy-data configuration [14]. Hyperparameter tuning had limited impact, while increasing to 100K samples improved performance without altering our conclusions. Compared with NDF's original labels, visibility-derived labels reduced CD1\mathrm{CD}_1 from 1.370 to 0.187 on clean scans and from 1.702 to 0.593 under scanner noise. We have updated Tables 1–4 and added NDF-vis to Figure 4. Under scanner noise, NDF-vis retains high normal consistency despite relatively large Chamfer errors, consistent with the zero-set localization uncertainty associated with the hKR margin and the low-frequency bias discussed by Coiffier and Béthune (2024).

4. If there is a scan with unknown noise and outliers, how should L and the Huber thresh be chosen?

In the original submission (lines 447–449), we suggested selecting LL based on the scanner's known noise distribution. We have clarified that noise alone is insufficient to determine LL, since the appropriate fitting capacity also depends on geometric complexity and sampling density. We therefore recommend L=2L=2, based on its performance across the datasets and noise levels considered in our experiments.

We recommend choosing the Huber threshold δ\delta on the scale of the scanner's measurement noise, obtained from its calibrated noise model or estimated through repeated measurements of planar targets at representative distances and incidence angles, following established scanner-calibration procedures. Without calibration data, robust local plane fitting provides an approximate alternative.

Dear Reviewer NodD,

Thank you again for your careful evaluation and constructive feedback. As the discussion deadline approaches, we wanted to briefly follow up on our rebuttal, including the new Neural RGB-D comparison and certified first-hit validation.

We would be grateful to hear whether these additions address your concerns or if any issues remain.

Best regards,

The Authors

Thank you for the clarification and revisions. The Neural RGB-D comparison and the first-hit analysis in Table 6 addressed my main concerns.

Comments on the paper

2 comments

We thank the reviewers for their constructive feedback.

Summary of revisions. We have uploaded the revised manuscript that includes:

  • Additional baselines: Four neural and two classical methods (Tables 1–4).
  • Expanded real-world evaluation: Three additional ARKitScenes acquisitions (Table 4).
  • Updated NDF-vis evaluation: 100K samples per shape instead of 10K (Tables 1–3; Figure 4).
  • Sphere-tracing reliability: Diagnostics against certified first-hit reference intersections (Table 6).
  • Computational efficiency: Updated runtime comparisons following a 3.7× speedup (Table 15 and 16).
  • Free-space analysis: Quantitative evaluation of free-space coverage (Table 7).

Unless otherwise stated, all table, figure, section, and appendix references refer to the revised manuscript. Citation numbers refer to its bibliography, numbered in the order listed.

Area_Chair_hyWJOpenReview ↗

Looking for reviewers' reply !!!

Dear Reviewers,

The authors have posted their rebuttal. Please read it and reply in the discussion, covering:

Whether each of your questions and concerns has been addressed. Whether your rating and recommendation change, and why. Any remaining concern that must be settled before the meta-review. Best.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.