- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes EgoH4RT, a feed-forward model that jointly estimates bimanual MANO parameters and a metric camera trajectory from egocentric RGB video. Within each 16-frame window, hand queries built from WiLoR features attend to Pi3X video-geometry features to decode both hands, while Pi3X predicts the metric camera poses. For long videos, windows sampled at several frame strides provide relative camera-pose constraints. These are fused by pose graph optimization (PGO), and the MANO parameters are then refined with the camera trajectory fixed. The model is trained on about 2,600 hours of video from 14 egocentric datasets with heterogeneous annotations. On HOT3D, OakInk2, and Xperience, it outperforms or matches baselines such as HaWoR, MINT, and HandFlow. This holds for camera-space hand reconstruction, short-window camera estimation, and long-horizon world-space reconstruction.
Critical strengths
- Clear goal and unified design. The paper estimates bimanual MANO parameters and a metric camera trajectory with a single feed-forward model, instead of running a separate hand estimator and a SLAM system. This avoids the error accumulation of multi-stage pipelines. The problem also matters for learning from egocentric human video.
- Strong empirical results, especially for long videos. On long-horizon HOT3D sequences (Table 4), world-space hand error (W-MPJPE) drops to 44.5 mm, compared with 108.1 mm for HaWoR and 61.1 mm for HandFlow. Camera ATE drops to 5.3 mm, compared with 21.2 mm for HandFlow. The method also outperforms or matches the baselines in camera-space hand reconstruction on HOT3D and OakInk2 (Table 2), and in short-window camera estimation against VGGT- and Pi3-based models (Table 3).
Critical weaknesses
-
Limited technical novelty The short-window model mainly combines two pretrained models, Pi3X and WiLoR. The long-video aggregation uses standard PGO. PGO is common in visual SLAM and has already been used to build long-sequence reconstructions from feed-forward geometry models, e.g., MonST3R (Zhang et al., 2025b) and VGGT-Long (Deng et al., 2025; not cited).
-
Unclear positioning against the closet work MINT (Zhu et al., 2026) and ReViV (Lyu et al., 2026) also estimate camera and hand motion jointly, but the paper does not clearly explain how its short-window pipeline differs from them. The relevant parts of the Related Work section (L119–123 and L127–131) are hard to follow. As a result, it is hard to identify the paper’s own technical contribution.
-
Insufficient experimental validation
- The PGO and hand refinement could likely also be applied to existing short-window methods such as MINT and ReViV. Without that comparison in Sec. 4.2.3, it is unclear whether the large gains in Table 4 come from the proposed window model or from the aggregation step.
- The ablation in Table 7 shows only modest gains from these components.
- Most of the evaluation, including all world-space hand results, uses HOT3D only. Other common benchmarks such as ARCTIC are not used.
Questions for authors
- How does the short-window pipeline differ from MINT and ReViV in architecture, supervision, and training data? Which of these differences explain the improvements? (clarification)
- Can the PGO and hand refinement be applied to the outputs of MINT or ReViV? If so, how do these combinations compare with EgoH4RT in the setting of Sec. 4.2.3? (could change my assessment)
- What is new in the multi-stride PGO compared with existing global optimization over pairwise or chunk predictions, such as MonST3R and VGGT-Long? (clarification)
- Can the authors report results on another benchmark, e.g., a held-out split of ARCTIC, especially for world-space hand reconstruction? (could change my assessment)
- In Eq. (3), the shape coefficients β are indexed by frame n. Shouldn’t hand shape be constant over a sequence? Appendix B.3 already shares a shape offset within each run. (clarification)
- Please cite the baseline methods again in the Experiments section. This would make the tables easier to follow. (suggestion)
Score updating after rebuttal
The author rebuttal addressed all my concerns so I am happy to keep my rating as weak accept.