- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Different from previous world-space hand motion reconstruction method that relies on separate models for hand localization, MANO parameter prediction, and camera trajectory estimation, the paper tries to build a unified framework that jointly estimates MANO parameters, camera trajectories, and hand locations directly from uncalibrated egocentric video. The proposed framework is pretrained on a larger set of data by aggregating existing public egocentric datasets with MANO or 3D keypoint annotations. The framework is shown to achieve SOTA accuracy in worldspace hand motion estimation with remarkable efficiency, and strong generalization on in-the-wild videos.
Critical strengths
The paper addresses an important and timely problem: estimating world-space hand motion from egocentric video. This direction is highly relevant to embodied AI, robot learning, dexterous manipulation, and large-scale hand-motion data generation. Compared with methods that only estimate camera-space hand pose, world-space hand motion estimation is more directly useful for downstream robotics and human-object interaction applications.
Besides, the paper aggregates and cleans existing public egocentric datasets into a standardized, high-quality corpus, which could facilitate the community to build a stronger model for egocentric hand motion reconstruction.
Critical weaknesses
Despite the importance of the problem and the potential of the system, I have several major concerns about the current submission.
-
The baseline comparison does not appear fully fair. The paper emphasizes that InfiniHand is trained with around 5,000 hours of egocentric data, while the compared methods do not seem to use training data at the same scale. Under this setting, it is difficult to determine whether the reported improvements come from the proposed architecture or mainly from much larger-scale pretraining. The authors should provide more controlled comparisons, such as comparing InfiniHand and strong baselines under the same training-data scale, or reporting an ablation where InfiniHand is trained with a smaller amount of data.
-
On ARCTIC and HOT3D, some metrics of the proposed method are worse than the baselines. This weakens the overall value of the contribution. If the proposed method still underperforms existing methods on some important benchmarks or metrics, the authors should analyze these cases more explicitly rather than only emphasizing favorable results. The current presentation is not sufficient to convincingly show that the method has a stable advantage in world-space hand motion estimation.
-
Important implementation details are missing or under-specified. The paper should more clearly describe the network architecture of each module, the input and output dimensions, the feature-fusion strategy, the organization of different prediction heads, and the information flow between modules. Without these details, it is difficult to assess the technical soundness of the method and to reproduce the results.
-
The claim that the method is end-to-end is questionable. The proposed framework relies on a set of models/tools for camera-space reconstruction and world-space reconstruction. The limitation section further mentions that an additional post-processing alignment model is needed to handle metric scale. If the final world-space output depends on such cascading steps and an external alignment step, then it is unclear in what sense the method can still be called end-to-end. Thus, I don't see intrinsic difference of this paper from previous world-space hand reconstruction baseline of HaWoR which also relies on cascading components.
Questions for authors
-
The comparison of parameter count and efficiency is not sufficiently detailed. What is the parameter count of the full system? Please provide a detailed efficiency table including parameter count, hardware setup, and inference latency.
-
Does the reported 11.19 FPS include the full pipeline, including camera estimation, bundle adjustment, metric-scale alignment, and other post-processing steps? If not, please report the runtime of each stage separately.
-
How are the large-scale training corpus and the test sets of ARCTIC, HOT3D, EgoDex, and other benchmarks separated? Are test videos, scenes, objects, and annotations strictly excluded from all training and data-cleaning stages?
-
Please clarify whether the post-processing module is part of the proposed method, how it is trained, whether it is used during evaluation, whether the same treatment is applied to all baselines, and how much it affects the final results.