- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents GroundHand, a capture-and-annotation system for obtaining absolute-scale hand-state annotations from egocentric dexterous manipulation data. The system combines synchronized egocentric and wrist-mounted cameras with a calibrated multi-view RGB-D setup. On top of this hardware system, the paper proposes a multi-source hand reconstruction model that combines camera-aware image features, fused point-cloud features, a subject-specific MANO shape prior, and temporal decoders for hand pose and global translation. The authors further introduce an iterative human-in-the-loop refinement procedure in which sparse 2D corrections are used to optimize MANO annotations and fine-tune the reconstruction model.
The method is evaluated on DexYCB and on a newly captured evaluation subset. The reported results include MPJPE and MPVPE on DexYCB, point-cloud/mesh consistency metrics on the captured data, and reductions in human correction ratio and annotation time after iterative refinement. The paper argues that separating egocentric observation collection from exocentric geometric annotation can provide more accurate hand states at a substantially lower hardware cost than marker-based motion-capture systems.
Critical strengths
The paper addresses an important practical bottleneck in learning from egocentric dexterous manipulation videos: obtaining hand states with consistent global scale under severe occlusion. The combination of calibrated multi-view RGB-D observations, wrist-mounted views, point-cloud features, and MANO-based reconstruction is a reasonable system design for this problem.
The paper also evaluates the system from multiple perspectives, including benchmark hand reconstruction, geometric consistency on captured data, component ablations, and the reduction of manual correction effort through iterative refinement. This provides a useful initial assessment of the proposed data-collection pipeline and its practical potential.
Critical weaknesses
1. The methodological contribution is not sufficiently differentiated from existing techniques.
GroundHand combines several established components, including calibrated multi-view reconstruction, RGB-D point-cloud features, Plücker-ray embeddings, MANO shape priors, temporal attention, and decoupled prediction heads. The paper does not clearly identify a new learning objective, reconstruction principle, or architectural mechanism that constitutes the central algorithmic contribution. As a result, the main novelty appears to lie primarily in the integration of known components into a particular capture system. The results on DexYCB show that GroundHand is competitive with existing methods, but the improvement in absolute MPJPE and MPVPE over the strongest listed multi-view baseline is relatively limited. On the captured data, the larger gains may also be substantially influenced by the use of seven calibrated exocentric RGB-D cameras and two wrist-mounted cameras. The paper therefore needs stronger evidence to separate improvements caused by the sensing setup from improvements caused by the proposed reconstruction model. In its current form, the contribution is closer to a system integration study than to a sufficiently distinct learning contribution for ICLR.
2. The experimental evidence does not fully support the claims of high-precision absolute-scale annotation.
The captured-data evaluation primarily uses PC2Mesh and Mesh2PC distances between reconstructed MANO meshes and visible RGB-D points. These metrics are useful for measuring geometric agreement, but they do not directly establish the accuracy of the full hand pose and global translation, especially for occluded surfaces. The paper also compares against a human-corrected optimization result, which is useful as a practical reference but does not provide an independent ground-truth measurement of the hand state. In addition, the compared pipelines receive substantially different inputs. GroundHand uses calibrated multi-view RGB-D observations and wrist-mounted cameras, whereas the ego-view baselines mainly use monocular images and the device-based baselines rely on their own tracking systems. This makes the comparison informative at the pipeline level, but it does not isolate the contribution of the proposed model. The experimental section would also benefit from clearer information about the number of subjects, sequences, tasks, and frames; the train/test split; the evaluation protocol for the pretrained and refined variants; and more quantitative ablations of the major components and camera configurations. Without these additions, the current experiments do not fully justify the broad claims regarding precision, scalability, and generality.
Questions for authors
1、How are the “GroundHand (pretrained)” and “GroundHand (ours)” variants trained and evaluated on DexYCB? In particular, does the latter use data from the newly captured sequences before evaluation on DexYCB, and are subjects and sequences strictly separated across training and testing?
2、For the captured-data evaluation, what is the reference annotation used for each metric? Are the reported metrics computed on held-out sequences that were not used for manual correction or model refinement? Can the authors provide a direct evaluation of global translation and hand pose against an independent reference?
3、Can the authors provide controlled comparisons that disentangle the effects of the sensing setup from the effects of the reconstruction model, such as matched-input baselines or systematic ablations of exocentric views, wrist views, point-cloud features, the MANO shape prior, and temporal decoding? Reporting results across subjects and manipulation tasks would also help assess the scope of the claimed improvements.