UniReg: Joint Multi-Source Learning for Universal Image-to-Point-Cloud Registration

Image-to-point-cloud registration recovers the 6-DoF pose of a query RGB image relative to a reference point cloud. While recent learning-based methods perform well on individual benchmarks, they remain largely tied to a fixed sensing process: outdoor sparse LiDAR and indoor RGB-D reconstructions differ substantially in density, scanning pattern, metric scale, completeness, and image–map overlap. As a result, models trained under one acquisition setup transfer poorly to another. We analyze this failure mode and attribute it to representations that entangle acquisition cues with scene geometry, pose pathways that are sensitive to density and scale, and the limited effectiveness of naively mixing heterogeneous datasets. We present UniReg, a single feed-forward network for I2P registration across indoor and outdoor scenes, LiDAR and RGB-D inputs, and fragment- to scene-level geometry. UniReg separates acquisition context from geometric reasoning through token-space conditioning, predicts sensor-robust ray-centric multi-geometry, and aggregates these predictions via confidence-aware fusion. To support this factorization, we introduce a scene-level ScanNet I2P dataset and a CARLA corpus with diverse camera–LiDAR configurations, and train jointly with standard benchmarks. With one set of weights, UniReg matches or surpasses specialist models across established indoor and outdoor evaluations.