Model-Free 6D Pose Estimation via 3D Registration with Distance-Color Fields

We address model-free 6D pose estimation, which recovers the relative pose, i.e., 3D translation and orientation, of an arbitrary object from a pair of reference and query RGB-D images, without requiring a pre-built CAD model. Unlike existing approaches, we tackle this problem directly in 3D space through two steps: (i) reconstructing a 3D mesh of the reference object using a generative reconstruction framework, and (ii) registering the query instance to the reconstructed mesh via 3D registration based on distance–color fields. Our amodal generator effectively reconstructs a complete 3D mesh aligned with the occluded reference view. Our Distance-Color Field Sample Consensus (DCF-SAC) efficiently performs 3D registration using distance-color fields of the reference mesh as fast lookup tables for sample-consensus scoring. On six benchmarks, the proposed method matches or outperforms reconstruction-based and reconstruction-free methods while running about four times faster than the leading reconstruction-based method.