StreamHaR: Streaming World-Grounded Hand Reconstruction from Egocentric Videos

Augmented reality, teleoperation and robot learning from demonstration require the wearer's hands reconstructed in a metric world frame as the egocentric video arrives. Existing world-space methods are offline, relying on future frames and global optimization, limiting their interactive applicability. To address this, we present StreamHaR, a fully online method that emits a metric, world-space hand at every frame with a state of constant size, built to be causal, faithful, general and efficient. StreamHaR splits the world-space hand into three problems with three estimators: the wrist is measured by a per-frame Perspective-n-Point solve on projection-consistent keypoints, the articulation is recognized by an encoder, and the camera comes from a causal odometer that anchors its metric scale from monocular depth, so the trajectory is metric the moment it is emitted, with no bundle adjustment and no calibration. With zero look-ahead and its own detector, StreamHaR reaches the world-space accuracy of the offline state of the art at a lower articulation error and transfers zero-shot to unseen headsets.