Human-Robot Alignment for Dexterous Hand Recognition with Geometry-aligned Synthetic Pairs

Anthropomorphic robot hands imitate human hand kinematics, but the appearance gap degrades the performance of SOTA human-hand detectors and pose estimators, while the scarcity of labeled robot-hand images limits finetuning and reduces accuracy for human hands. We present SynDex, which adapts pretrained human-hand models to robot hands without any real robot training images. SynDex renders depth and normal maps from annotated human hands and generates paired human-robot images from the same geometry. A vision-language model checks each candidate against its real reference, yielding 47K pixel-aligned human–robot pairs. A frozen human-domain teacher then supervises a student. Student features on each robot image regress teacher features on the paired human image, while a preservation term on real human images limits forgetting. SynDex improves the WiLoR's robot-hand detection AP${}_{50}$ from 9.55 to 38.45 on RealDex and from 2.93 to 60.66 on HRDexDB, and lowers the robot-hand PA-MPJPE of HaMeR and WiLoR by up to 1.85 mm, while keeping human-hand pose error within 0.4 mm of the original models on FreiHAND and HO3Dv2.