acceptodds
Withdrawn from ICLR 2027

BRIDGE: Building Reward Optimized Depth-to-Image Data Generation Engine for Monocular Depth Estimation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Monocular Depth Estimation (MDE) recovers scene geometry from a single RGB image, yet what a model can learn is capped less by its architecture than by the depth supervision available: sensor-captured RGB-D is metrically accurate but sparse, rendered synthetic RGB-D is dense but domain-shifted, and web-scale pseudo-labels cover broad appearance but inherit teacher boundary noise. We propose BRIDGE, a reward-optimized depth-to-image (D2I) engine that synthesizes over 20M RGB images from diverse source depth maps, each paired with the source depth map used to condition it. We then train a depth model on this data with similarity-guided hybrid supervision: source depth supervises regions where the generated image preserved geometry, and teacher pseudo-labels supervise where it drifted. BRIDGE achieves the strongest zero-shot relative-depth results among compared methods on NYUv2, ScanNet, ETH3D, and Sintel, using 20M generated images against Depth Anything V2's 62M.

Primary area:
generative models
Keywords:
Depth Estimation, Reward Optimized, Data Synthesis, Data Generation Engine

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aDUMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BRIDGE, a reward-optimized depth-to-image data engine for monocular depth estimation. It generates RGB images from existing synthetic depth maps, using RL to backpropagate a geometric-consistency reward through diffusion sampling, producing ~20M RGB-D pairs. A similarity-guided hybrid supervision uses source depth where an SSIM mask indicates preserved geometry and teacher pseudo-labels elsewhere. Model trained on this data achieves strong zero-shot results on NYUv2, ScanNet, ETH3D, and Sintel with 20M images versus Depth Anything V2's 62M.

The top half of Table 2 compares a student trained on 1M standard synthetic data against one trained on 1M BRIDGE-generated data, concluding that generated data "significantly boosts performance." However, the paper does not specify whether the two 1M sets are drawn from the same source depth maps or cover the same scene distribution, so the comparison may partly reflect a difference in data composition rather than generation quality. Moreover, the statistical properties of the generated data, such as its diversity, are never quantitatively assessed.

The baseline comparison in Table 1 omits recent strong monocular depth methods such as MoGe-2 [1], InfiniDepth [2], and FoundationGeo [3].

Critical strengths

The paper recognizes that dataset diversity matters for monocular depth estimation and proposes to obtain it through depth-to-image generation, synthesizing diverse appearance onto existing synthetic depth maps with known geometry. Zero-shot results on NYUv2, ScanNet, ETH3D, and Sintel are strong, and Table 9 shows monotonic improvement with the number of generated appearances per depth map, supporting the premise that varying appearance over fixed geometry helps.

Critical weaknesses
  1. The top half of Table 2 compares a student trained on 1M standard synthetic data against one trained on 1M BRIDGE-generated data, concluding that generated data "significantly boosts performance." However, the paper does not specify whether the two 1M sets are drawn from the same source depth maps or cover the same scene distribution, so the comparison may partly reflect a difference in data composition rather than generation quality. Moreover, the statistical properties of the generated data, such as its diversity, are never quantitatively assessed.
  2. The baseline comparison in Table 1 omits recent strong monocular depth methods such as MoGe-2 [1], InfiniDepth [2], and FoundationGeo [3]. Besides, the evaluation datasets do not sufficiently cover driving scenes (KITTI), a key real-world application domain. As a result, the model's generalization to this domain remains unverified.

[1] Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, Jiaolong Yang. MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details. NeurIPS 2025.

[2] Hao Yu, Haotong Lin, Jiawei Wang, Jiaxin Li, Yida Wang, Xueyang Zhang, Yue Wang, Xiaowei Zhou, Ruizhen Hu, Sida Peng. InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields. CVPR 2026.

[3] Muxin Liu, Xiaoyang Lyu, Tianhe Ren, Peng Dai, Xiaoshan Wu, Zhiyue Zhang, Jiaqi Zhang, Jiehong Lin, Shaoshuai Shi, Xiaojuan Qi. FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry. ECCV 2026.

Questions for authors

The reward is defined as a cosine loss between a frozen depth estimator's prediction and the conditioning depth, while the student is trained with MiDaS's scale-and-shift invariant loss. Why use a cosine reward rather than the same scale-and-shift invariant objective (LssiL_{ssi})?

Reviewer_JzJbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To overcome the lack of ground-truth depth data in training monocular depth models, the paper reuses existing image-depth pairs by regenerating the image while preserving the depth. And a heuristic masking is used to reduce the inconsistency between original depth and new generated image.

Critical strengths

It is a clever way for generating more data and their paper have some heuristic improvement on the naive design.

Critical weaknesses

There are several key issues that make this paper unconvincing

  • Table 1 does not compare with some sota models, e.g., MoGe, which has better results on NYUv2. It would be better to pick the current sota model and apply the technique in the paper to further improve the performance.
  • As I compare with the numbers in the DepthAnything V2 paper, this paper used the ViT-L version, while the model in this paper used the Giant version. So the comparison is unfair.
  • Why does Table 1 not contain the KITTI and DIODE dataset?
  • Also, the paper does not use the same evaluation set for ScanNet and Sintel without explaining a good reason. And The DepthAnything V2 paper has a much better score on Sintel.
  • In the "Efficiency of hybrid supervision" of Table 2, it would be more interesting to see unmasked source only vs hybrid, because the masking is proposed as a remedy solution to the original data generation pipeline.
  • Why is the "0.95M pseudo label and 0.05M masked source"?

From a conceptual view, even the pipeline with this hybrid supervision has fundamental issue that the depth can only come from the original depth data (or outputted by a teacher who learns from such data). There is never a mechanism that can produces more variants to the depth data. I think one drawback of synthetic depth data is the lack of details. It is better to create such details from some intrinsic mechanism like better microgeometry generator.

Questions for authors

Answer to my concerns about comparison to sota models could help improve my score.

Reviewer_gyoqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BRIDGE, a Depth-to-Image (D2I) data generation framework for monocular depth estimation. It combines reward-optimized image generation with similarity-guided hybrid supervision to construct 20M RGB-D training samples, achieving competitive zero-shot depth estimation performance.

Critical strengths

The paper presents a practical framework for generating large-scale RGB-D training data with diverse appearances. It achieves strong zero-shot depth estimation performance across multiple benchmarks and includes several ablation studies to evaluate the proposed components.

Critical weaknesses

The methodological novelty is limited, as the proposed modules mainly combine existing techniques. The advantages of D2I-generated data over real RGB images with teacher pseudo-labels remain unclear. Moreover, the geometric consistency reward and SSIM-based supervision lack sufficient justification and direct validation. Comparisons with recent methods, particularly Depth Anything 3, are missing, and the ablation studies are insufficient to isolate the contributions of individual components.

Questions for authors
  1. The authors propose using Depth-to-Image (D2I) generation to address the scarcity of high-quality depth annotations. However, the advantages of this approach over existing paradigms, such as Depth Anything V2's large-scale real RGB images with teacher pseudo-labels, remain unclear. Since the proposed method still relies heavily on teacher-generated pseudo-labels and uses GT depth only in selected regions, the authors should clarify whether D2I generation effectively reduces pseudo-label noise and offers advantages over simply scaling up real RGB training data.

  2. The two main modules, Reward-Optimized D2I Generation and Similarity-Guided Hybrid Supervision, largely build upon existing techniques. The former adopts Flux.1 Depth and VADER's reward-gradient optimization with geometric consistency and aesthetic rewards, while the latter combines conventional ORB registration, SSIM filtering, and GT-depth fine-tuning. Therefore, the method appears more like an integration of existing techniques into a data generation pipeline than a fundamentally new algorithm. The authors should better clarify the core technical contributions.

  3. The proposed reward measures cosine similarity between the source depth and predictions from a frozen depth estimator. However, the estimator itself may produce errors, particularly around transparent objects, thin structures, and complex boundaries. Thus, high similarity does not necessarily indicate correct geometric preservation. Moreover, cosine similarity primarily captures global depth agreement and may overlook local geometric details. The authors should provide direct quantitative evaluations of geometric consistency and demonstrate whether the reward improves fine-grained geometry.

  4. The authors use SSIM between original and generated RGB images to identify geometrically consistent regions. However, SSIM measures appearance similarity rather than geometric consistency. Changes in texture, lighting, or style may reduce SSIM even when geometry is preserved, while high SSIM does not guarantee accurate depth alignment. The authors should justify this design and compare it with depth-consistency or feature-based selection strategies. Additionally, although the method is described as hybrid supervision, it actually adopts pseudo-label pretraining followed by masked GT-depth fine-tuning. The advantages of this two-stage strategy over directly mixing both supervision sources should be clarified.

  5. The comparisons mainly include Depth Anything V2, Depth Pro, and Metric3D V2, but omit more recent visual geometry foundation models. In particular, Depth Anything 3 has reported stronger monocular depth estimation performance than Depth Anything V2. A comparison under the same zero-shot monocular depth evaluation protocol is necessary to support the claimed SOTA performance.

  6. Several key design choices lack adequate validation. First, Table 9 simultaneously changes the number of generated appearances per depth map and the total training data size, making it difficult to distinguish the benefits of appearance diversity from data scaling. Second, the reward optimization ablation only reports downstream depth estimation performance, without directly measuring geometric consistency in generated images. Third, the hybrid supervision ablation lacks comparisons between different mask construction strategies, such as direct SSIM, ORB registration with SSIM, and their combination. More controlled experiments are needed to validate these components.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.