acceptodds
Under review as a conference paper at ICLR 2027

A Histogram Is Worth a Thousand Pixels: Single-Photon Metric Anchoring for Monocular Depth Estimation

Abstract

Single-pixel SPAD measurements provide accurate metric range information but lack explicit spatial correspondence with image pixels, making their fusion with dense monocular depth predictions inherently ambiguous. We introduce Hist2Depth, a global-to-local framework that uses monocular depth itself to associate spatially unresolved SPAD measurements with pixel-wise depth hypotheses. Global D2 first aligns the monocular prediction with the measured depth distribution through an order-preserving metric calibration. MS-GRU then performs depth-conditioned recurrent refinement, dynamically querying SPAD evidence according to the evolving pixel-wise depth estimates and integrating it with spatial image context. On NYU-Depth-v2, Hist2Depth with AdaBins achieves 0.0649 AbsRel and 0.2750 m RMSE, and consistently improves RGB-only predictions across six monocular backbones. When transferred directly from synthetic training to captured RGB-SPAD measurements without fine-tuning, Hist2Depth also retains strong metric accuracy. Under photon thinning, performance remains nearly unchanged at a 60 s equivalent acquisition, using only 7.4% of the full photon count. These results demonstrate that global SPAD measurements can effectively guide dense metric depth reconstruction through depth-conditioned global-to-local fusion.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
single-pixel SPAD, metric depth estimation, RGB-SPAD fusion, multimodal sensor fusion, single-photon imaging

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_vj7XOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Hist2Depth, a method that uses a global SPAD histogram to improve monocular depth estimation. It first calibrates the overall depth prediction, then uses the current depth estimates to query SPAD information and refine local depth with image features. Experiments on NYU-Depth-v2 and captured RGB–SPAD data show improvements across several monocular depth models.

Critical strengths
  1. The method is well motivated. Global D2 uses the SPAD distribution to correct the overall depth prediction, while MS-GRU combines depth-based queries with image context for local refinement. The two stages make sense given that the SPAD measurement provides distance information but no explicit pixel correspondence.

  2. The results are encouraging. Hist2Depth improves the RGB-only predictions across all six tested backbones. With the same AdaBins backbone, it also outperforms SSD and PhoMoNet. The transfer from synthetic training to captured RGB–SPAD measurements without fine-tuning is a useful result.

  3. The ablation studies are fairly comprehensive. Table 3 covers alternative fusion methods, the two stages, calibration order, and the number of refinement steps. The photon-budget experiments and qualitative examples also help show how the method behaves.

Critical weaknesses
  1. The method's applicability to open road scenes remains unclear. Most experiments use NYU-Depth-v2, with depths evaluated up to 10 m (Section 4.1). Although the captured RGB–SPAD results are useful, they do not establish how well the method would work in open driving scenes with a much wider depth range. Since autonomous driving is an important application of depth estimation, some discussion of the limitations and adaptations needed for this setting would be helpful.

  2. The role of updating the query locations could be explained more clearly. Table 3 shows that MS-GRU and additional refinement steps help, but it is less clear how much the changing query locations contribute to those gains. Some explanation based on the existing results would be useful. This is a smaller concern, since the ablations already support the overall module.

Questions for authors

Please see the weaknesses above.

Reviewer_kLHKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper presents Hist2Depth, a framework combining monocular depth and SPAD measurements. By addressing RGB-SPAD mismatch with incorporating global range measurements into dense monocular predictions through metric calibration and local refinement, the method achieves effectiveness and robustness across models and fusion strategies. Specifically, Global D2 first estimates a bounded, order-preserving quadratic calibration from the discrepancy between the predicted and SPAD-derived depth distributions, after which MS-GRU uses each pixel’s evolving depth estimate to query depth-conditioned SPAD evidence and refines local depth through multi-scale recurrent updates. On NYU-Depth-v2, Hist2Depth outperforms baselines with low loss and high robustness. It also transfers zero-shot from synthetic training to captured RGB–SPAD data, where accuracy stays nearly unchanged even when only 7.4% of the photons are retained.

Critical strengths

The contribution is clear, with some novelty. The authors uses the distribution discrepancy only to estimate compact global calibration parameters, resulting in Low-DoF metric calibration and geometric preservation, providing robustness and flexibility under limited range. Depth-conditioned refinement is another main contribution, which is a neat adaptation of RAFT-style correlation sampling to the depth axis, turning a spatially unresolved transient into per-pixel queryable evidence that is rebuilt as the prediction evolves, which is a more principled link between pixel hypotheses and SPAD evidence than PhoMoNet’s implicit feature fusion. Despite most of these design can be linked with previous works, the incremental contribution is clear.

Critical weaknesses

The evaluation is somewhat narrow. The author only evaluates their works on NYU-Depth-v2, a single datasets containing indoor testcases. More evaluation should be done with other datasets. In addition, the author test with Signal-to-background-ratio=100. However, this is a very ideal setting, which cannot even achieve with normal indoor settings. The author should present quantitative results at lower SBR to show the method's robustness.

Questions for authors
  1. How do the difference in effective SBR of the captured RGB–SPAD measurements affect your results?
  2. Is a learned regressor for Global D2 justified? Given that Table 3(c) shows nearly identical results for polynomial orders 0–2, can the authors compare the learned Global D2 regressor against a training-free closed-form alternative and clarify whether the calibration effectively reduces to a global offset?
Reviewer_paFcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Hist2Depth for refining monocular metric depth using a spatially unresolved SPAD transient. Global D2 predicts a bounded, monotonic calibration from depth-distribution discrepancies. MS-GRU then queries histogram evidence around the evolving pixel-wise depth estimates and predicts spatial corrections. The method improves six monocular backbones on NYU-Depth-v2 and transfers to captured RGB–SPAD data without fine-tuning.

Critical strengths

1.The problem is well motivated, and the distinction between global range information and spatial image structure is clearly explained. 2.Improvements extend across six backbones, including recent metric-depth models. The captured-data results provide useful evidence beyond synthetic evaluation. 3.The bounded calibration preserves depth ordering by construction. Component, calibration-order, and iteration studies help characterize the system, while the appendix documents the architecture and training objective in reasonable detail.

Critical weaknesses

1.The contribution of SPAD guidance is not separated from additional supervised refinement. The RGB-only baseline is the frozen backbone, whereas Hist2Depth adds a trained correction network with access to decoder features and dense depth supervision. Table 3 removes individual modules but never removes SPAD information from the complete refinement architecture. A matched RGB/depth-only refinement baseline is needed to establish how much improvement comes from the measurement. Within MS-GRU, comparisons against broadcast global evidence and fixed queries would further isolate the benefit of dynamic depth-conditioned lookup. 2. The interpretation and novelty of the lookup mechanism need sharper boundaries. Pixels with the same current depth query the same global evidence neighborhood; the measurement itself still cannot distinguish their spatial identities. Different corrections come from image context and learned priors. The method therefore provides depth-conditioned measurement guidance, but does not resolve correspondence ambiguity in an identifiable sense. Moreover, estimate-conditioned lookup, multiscale GRU updates, and convex upsampling are established in RAFT-Stereo. The contribution should be positioned around their adaptation to evolving SPAD distribution evidence, with direct validation of that adaptation.

Questions for authors

What accuracy remains with the same refinement architecture and supervision but without SPAD input?

What causes the gap between the reproduced and published PhoMoNet results, particularly on captured data?

Does dynamic lookup outperform fixed-query or broadcast-evidence alternatives under comparable capacity?

How stable are the results under lower SBR, measurement mismatch, and repeated photon-thinning seeds?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.