- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Geo2Dex learns dexterous human-to-robot retargeting from paired demonstrations. A scene-grounded hand reconstruction stage uses 4DVGGT features to recover human hand motion and latent interaction features. A spatiotemporal encoder and embodiment-conditioned slot decoder then predict wrist and joint trajectories for hands with different action dimensions. Training uses temporally aligned human-robot trajectories and recursive rollout fine-tuning. Experiments cover hand reconstruction, trajectory fidelity and simulation replay on HRDexDB, an additional DexCap evaluation, and downstream VLA fine-tuning in simulation and on a real robot.
Critical strengths
-
Substantial reconstruction and replay gains. The combined pipeline improves scene-relative hand localization and retargeting across two robotic hands. On HRDexDB, absolute hand-joint error falls from 17.58 mm for the strongest listed reconstruction baseline to 8.21 mm (Table 1). Average replay success rises from the best baseline's 48.94% to 69.15% (Table 4), a 20.21 percentage-point improvement. These results go beyond pose reconstruction alone and support the practical value of the combined pipeline.
-
Downstream utility and component evidence. Table 5 compares Geo2Dex and Dex-Retargeting under matched rendering, sampling, instructions, and policy training settings (Appendix G), with consistent improvements across both hands and all three tasks. The DINOv3 scene-feature substitution and object-context ablations (Tables 7-8) support the importance of the interaction representation. The real-robot results and supplementary demonstrations provide additional, although small-scale, evidence of feasibility.
Critical weaknesses
-
Test-time state and split protocol are underspecified. Equation (1) conditions on robot-state history through t while predicting a chunk starting at t; Section 3.6 and Appendix E.2 mention recursive generation and qpos-only rollout fine-tuning, but do not specify initialization, wrist-state updates, or whether the reported trajectory errors use recursively generated states or recorded reference states. The 7:1:2 split also lacks a definition of the split unit and how paired demonstrations, overlapping windows, and reconstruction-stage training are separated. These details are needed to determine whether the reported errors measure independent trajectory generation on held-out demonstrations.
-
The structural decoder's contribution is not isolated. The current comparisons do not distinguish the decoder's benefit from that of paired robot supervision. Tables 7-8 examine scene features and object context, but do not compare the slot/attribute mechanism against a matched supervised predictor, such as the same encoder with embodiment-specific heads, or slots without actuator attributes. This matters because scene-conditioned hand reconstruction and paired human-to-robot learning have precedents (e.g., Hand3R and Human2Robot), making the particular integration and multi-embodiment decoder central to the contribution. The real-robot extension is encouraging but cannot resolve this attribution: Appendix H gives 50 paired demonstrations per task, yet does not explain the retargeter's Revo 2 adaptation/training split or the actual mixing ratio. Table 6 contains only ten trials per task (8/10 and 7/10 versus 10/10), so the breadth and reliability of the benefit remain uncertain.
Overall, the reported gains are promising, but the test-time protocol and contribution attribution currently support a weak rejection. A clarified evaluation protocol and convincing evidence for the shared structural decoder could change this assessment.
Questions for authors
-
Evaluation protocol. For Tables 3-4, how are the initial robot state and every subsequent wrist/joint state obtained? Are any ground-truth robot states used after initialization, and which predicted chunk steps are executed? Please also specify the split unit, whether all windows/embodiment pairings from the same underlying human demonstration stay together, and whether Stage 1 sees any Stage 2 test demonstrations. This would determine how to interpret the reported generalization and rollout errors.
-
Decoder contribution and baseline conditions. Under identical paired data, inputs, and training budgets, what evidence distinguishes the structural slot decoder from an embodiment-specific output head or a slot decoder without actuator attributes? A targeted comparison under matched conditions would materially affect the assessment of the decoder's contribution. Please also clarify which reconstruction inputs and refinement stages were used for the retargeting baselines in Tables 3-4.
-
Real-robot training and evaluation. How was the retargeter adapted to Revo 2, which of the 50 paired demonstrations per task trained each component, and what human/robot mixing ratio and robot-data budget were used for Table 6? Are the reported ten trials from a single trained policy per condition? These details would establish what the downstream gains support about using additional human data.