Reformulating Uncalibrated Photometric Stereo as a Joint Depth–Normal Estimation Task

Uncalibrated Photometric Stereo (UPS) is traditionally formulated as a surface normal estimation problem, with post-integration to reconstruct depth. This decoupled strategy ignores the intrinsic coupling between local shape (normal) and global geometry (depth), leading to inconsistent reconstructions and error accumulation. To address this, we reformulate UPS as a joint depth–normal estimation problem that enforces mutual geometric constraints. Our framework leverages CLIP as a stable semantic cues to resolve illumination ambiguity by providing pseudo-lighting cues. A plane-sweeping-like module first recovers a coarse depth map, which is then jointly refined with camera intrinsics. The refined depth and intrinsics are used to derive geometry-consistent surface normals, enabling a consistency-driven supervision that supports semi-supervised learning with limited synthetic data and bridges photometric and geometric reconstruction. Extensive experiments demonstrate that our method outperforms state-of-the-art UPS techniques and is competitive with calibrated and monocular depth estimation methods. Ablation studies confirm that CLIP provides robust semantic cues that effectively stabilize the reconstruction process.