STEREOHOI: RECOVER METRIC HAND-OBJECT IN- TERACTION FROM EGOCENTRIC STEREO VISION

Reconstructing 3D hand-object interaction (HOI) in physical metric space is impor- tant for embodied AI and robotic manipulation. Existing HOI approaches mostly rely on RGB observations from monocular images or video streams, where the lack of binocular metric scale makes physical hand-object alignment challenging, often necessitating expensive test-time optimization. While stereo cameras provide binocular disparity cues to resolve metric scale, existing stereo networks focus primarily on dense surface depth estimation, which cannot directly predict hand keypoints or infer their corresponding metric depths. In this work, we explore stereo vision for recovering 3D hand-object interaction and present StereoHOI. Rather than recovering metric hand scale through additional fitting to stereo depth estimates, StereoHOI directly recovers metric hands by fusing monocular hand priors extracted from a frozen hand foundation model within an alternating cross- view attention framework. By encoding the monocular hand priors and camera parameters into conditioned special tokens, StereoHOI implicitly captures binoc- ular disparity cues to predict metric wrist translations and pose residuals. This enables placing hands and metric object meshes into a shared metric coordinate frame, simplifying hand-object alignment and supporting video-based interaction analysis. Code will be released to benefit the research community.