- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DittoHand, a cross-morphology policy for zero-shot grip-force control. Each actuator is represented by a token encoding its kinematic state, the collision surface it moves, local hand-object geometry, and an analytic closing rate. A shared Transformer then produces per-actuator actions for hands with different topologies and shapes. Trained on 24 simulated hands, DittoHand is evaluated on 648 unseen hands and 20 objects, outperforming adapted cross-hand baselines in grasp stability and force tracking. Ablations show the importance of surface information, and a 12-participant VR study reports improved perceived controllability and realism over per-hand ForceGrip policies.
Critical strengths
- A physically motivated, interpretable representation. Binding each actuator to the surface it moves and giving it an analytic closing rate tells the policy how actuator motion changes contact geometry and force transmission, which representations based only on joint topology leave implicit. This is the paper's main conceptual contribution.
- Broad controlled simulation evaluation. The paper evaluates 24 seen hands, 648 unseen hands, topology changes, thickness and length changes, 20 objects, surface/closing-rate/descendant ablations, and a VR user study. The large surface ablation gap supports the importance of the proposed representation within the simulator.
Critical weaknesses
- The method has not yet been evaluated on any existing hand. All 864 hands, including those in the VR study, are virtual edits of the MANO human-hand model. This is reasonable for the VR avatar setting, but the paper also describes collision geometry as "a necessary axis of embodiment" and compares itself with GET-Zero, which was tested on physical LEAP Hand variants. The broader claim would therefore benefit from evidence beyond MANO-derived hands, together with a discussion of how the per-phalanx contact-force, grip-force and object-velocity inputs could be obtained on hardware. This is my main concern; I would be glad to raise my score if the claim were scoped to VR avatars, or if grasping and force tracking are also evaluated on existing robot hands, even in simulation.
- The VR study relies mainly on subjective ratings and only partly tests the central claims. Objective measures such as task success, object drops, completion time and how closely F_grip followed the trigger would strengthen it, and could be reported if they were recorded. The pick-and-place task uses the three training objects and mostly requires holding the object, so it does not test how closely the grip force tracks the command, which is the paper's main quantitative advantage.
Questions for authors
How many training seeds were used? Can you report mean ± std or CIs over seeds for Tables 1–2? What is the wall-clock cost per control step of the surface-point queries and the network on the VR hardware, and how does it scale with object mesh complexity or with two hands?
7 comments
Response to Reviewer fE7i (2/4)
2. Objective measures and force tracking in the VR user study (W2)
As you said, the previous VR user study focused on subjective ratings, and the pick-and-place task alone was not sufficient to verify force-command tracking. In fact, most participants performed the task with the trigger pulled nearly all the way (trigger ≥ 0.95: DittoHand 97%, ForceGrip 91%). To address this, we additionally analyzed objective measures from the previous study and conducted a new user study that directly evaluates force-command tracking.
Objective measures in pick-and-place. Analyzing the records of the original 12 participants, DittoHand had a higher success rate than ForceGrip (98.1% vs. 92.5%), a shorter completion time (11.2 s vs. 12.9 s), and a lower timeout rate (0.9% vs. 5.9%). All three measures were statistically significant (Table R3-2).
Table R3-2. Objective measures in pick-and-place (12 participants, three hands pooled, mean ± standard deviation, Wilcoxon signed-rank test, Holm correction)
| Success rate (%) ↑ | Drops per trial ↓ | Completion time (s) ↓ | Timeouts (%) ↓ | |
|---|---|---|---|---|
| ForceGrip (trained per hand) | 92.5 ± 9.6 | 0.09 | 12.9 ± 3.5 | 5.9 |
| Ours (unseen hands) | 98.1 ± 3.3 | 0.06 | 11.2 ± 3.2 | 0.9 |
| p | 0.035 | 0.473 | 0.002 | 0.035 |
The per-hand results are added to Table 15 in Appendix C of the revision.
Additional Force-Control Experiment. Inspired by ForceGrip's Can Squeeze experiment, we designed a task in which users match target grip forces using the trigger. Unlike the original experiment, we directly specified target forces and evaluated force controllability across different hand structures. The same 12 participants returned and held a capsule object (5 cm diameter, 10 cm length, 0.1 kg mass), matching displayed target forces of 0.8, 1.2, 1.6, and 2.0 kgf for 2 seconds each.
Table R3-3. Force-control task (12 participants. Mean ± standard deviation over participants, each participant averaged over the three hands; paired Wilcoxon signed-rank test. ForceGrip is trained on each hand; our policy saw none of them.)
| Measure | ForceGrip (per hand) | Ours (unseen) | p |
|---|---|---|---|
| Command following (primary) | |||
| Trigger–grip-force r (ForceGrip's method) ↑ | 0.58 ± 0.09 | 0.81 ± 0.05 | < 0.001 |
| Gain → 1 | 0.40 ± 0.09 | 0.78 ± 0.15 | < 0.001 |
| Relative tracking error |F − C| / C ↓ | 0.77 ± 0.26 | 0.29 ± 0.07 | < 0.001 |
| Task measures | |||
| Time to reach the target (s) ↓ | 0.95 ± 0.39 | 0.69 ± 0.22 | 0.021 |
| Hit rate within half a level (%) ↑ | 89.1 ± 5.6 | 94.0 ± 4.0 | 0.016 |
| Drops per block ↓ | 9.1 ± 2.4 | 2.4 ± 1.7 | < 0.001 |
| Questionnaire (−3 to +3) | |||
| Realism Questionnaire total ↑ | −0.04 ± 0.52 | 0.78 ± 0.80 | 0.001 |
| Grip-force items G1–G4 ↑ | 0.09 ± 0.73 | 1.22 ± 0.97 | 0.005 |
Table R3-4. Force-control task per hand (12 participants; mean ± standard deviation over participants)
| Hand | Gain → 1, ForceGrip | Gain → 1, Ours | Drops per block ↓, ForceGrip | Drops per block ↓, Ours |
|---|---|---|---|---|
| Mean MANO (β = 0) | 0.69 ± 0.18 | 0.80 ± 0.26 | 2.4 ± 2.4 | 1.0 ± 1.3 |
| Thickness ×1.15 | 0.40 ± 0.08 | 0.75 ± 0.12 | 6.1 ± 3.4 | 2.7 ± 2.7 |
| Three-fingered | 0.10 ± 0.15 | 0.79 ± 0.21 | 18.8 ± 5.2 | 3.7 ± 2.3 |
- Force-command tracking. The Gain of DittoHand is 0.78, much closer to 1 than that of ForceGrip (0.40), and its error relative to the command is also smaller, 0.29 vs. 0.77 for ForceGrip.
- Differences across hand structures. Even on the mean MANO hand, on which ForceGrip was trained alone, DittoHand has a higher Gain of 0.80 than ForceGrip (0.69). Although ForceGrip was trained separately for each hand, it drops to 0.40 on the hand with changed thickness and to 0.10 on the three-fingered hand. DittoHand never saw any of the three hands but maintains 0.75–0.80 on all three. In the questionnaire as well, it received significantly higher scores on the RQ Total Score (0.78 vs. −0.04) and all three subscores (Figure 5b in the revision), as well as on the grip-force items (1.22 vs. 0.09).
Thus, DittoHand demonstrated advantages not only in stable object transport but also in accurately tracking user force commands. In particular, it maintained high force-tracking performance across different unseen hand structures, supporting our claim of consistent force control across diverse avatar hands. We will add this experiment to Section 4.5 and report the objective metrics in Appendix C.
Response to Reviewer fE7i (3/4)
3. Number of training seeds and mean ± standard deviation (Q1–2)
While the original submission reported results from a single seed for each method, we have now independently trained Ours, DGZ, and IBS with three seeds each, and updated Tables 1 and 3 to report mean ± standard deviation. We also made two changes to the evaluation conditions to better reflect real-world VR environments.
- Joint Transport and Force Tracking: We previously evaluated transport and force tracking separately because objects slip when transported without sufficient grip force. This is also intended behavior in VR, where objects slip if users grip them too weakly. However, the separate evaluations did not sufficiently support our claim of transporting objects while tracking varying force commands, so we added a joint experiment.
- Grasp Criterion: We replaced the previous static grasp test with an evaluation of whether the object remains held for 0.5 seconds after support removal under the actual simulation friction coefficient (0.5).
Table 1 of the revision. Comparison with the baselines (Mean ± standard deviation over three training seeds; Grasp and Success in %, FTE in kgf. Rows below Unseen hands differ from the Seen hands in one aspect; Topology means fewer fingers or phalanges. Bold: best per row and metric.)
(a) Keeping the object
| Hands | Grasp DGZ | Grasp IBS | Grasp Ours | Success DGZ | Success IBS | Success Ours |
|---|---|---|---|---|---|---|
| Seen hands (24) | 79.1 ± 3.5 | 75.1 ± 4.7 | 83.2 ± 3.4 | 53.3 ± 1.4 | 42.4 ± 1.4 | 62.9 ± 2.6 |
| Unseen hands (648) | 61.6 ± 2.2 | 59.6 ± 3.3 | 70.6 ± 4.6 | 33.9 ± 4.4 | 28.0 ± 3.7 | 42.3 ± 4.5 |
| β only (24) | 78.5 ± 3.0 | 74.8 ± 4.2 | 82.3 ± 3.6 | 52.1 ± 1.5 | 41.2 ± 1.9 | 60.4 ± 2.9 |
| Thickness only (96) | 74.8 ± 4.1 | 68.2 ± 4.6 | 78.1 ± 3.9 | 46.6 ± 3.6 | 35.5 ± 1.1 | 54.7 ± 3.5 |
| Length only (48) | 76.6 ± 6.6 | 70.1 ± 4.6 | 79.2 ± 1.4 | 56.4 ± 8.7 | 44.6 ± 6.5 | 64.4 ± 5.0 |
| Topology only (48) | 53.0 ± 0.5 | 50.8 ± 4.4 | 64.8 ± 5.3 | 22.6 ± 3.3 | 18.2 ± 3.5 | 32.4 ± 5.0 |
(b) Tracking the grip command
| Hands | FTE DGZ | FTE IBS | FTE Ours | Gain DGZ | Gain IBS | Gain Ours |
|---|---|---|---|---|---|---|
| Seen hands (24) | 0.57 ± 0.03 | 0.71 ± 0.10 | 0.51 ± 0.07 | 0.48 ± 0.01 | 0.21 ± 0.21 | 0.52 ± 0.13 |
| Unseen hands (648) | 0.76 ± 0.02 | 0.86 ± 0.09 | 0.68 ± 0.07 | 0.42 ± 0.04 | 0.19 ± 0.17 | 0.46 ± 0.12 |
| β only (24) | 0.57 ± 0.03 | 0.71 ± 0.10 | 0.52 ± 0.06 | 0.48 ± 0.02 | 0.23 ± 0.20 | 0.53 ± 0.13 |
| Thickness only (96) | 0.68 ± 0.04 | 0.90 ± 0.09 | 0.64 ± 0.06 | 0.47 ± 0.02 | 0.23 ± 0.22 | 0.49 ± 0.13 |
| Length only (48) | 0.64 ± 0.03 | 0.92 ± 0.19 | 0.68 ± 0.11 | 0.62 ± 0.02 | 0.33 ± 0.27 | 0.64 ± 0.15 |
| Topology only (48) | 0.82 ± 0.02 | 0.88 ± 0.07 | 0.69 ± 0.08 | 0.30 ± 0.02 | 0.12 ± 0.13 | 0.37 ± 0.12 |
Table 3 of the revision. Ablations (Grasp and Success in %, FTE in kgf. Mean ± standard deviation over three training seeds. Bold: best per column.)
| Method | Seen Grasp ↑ | Seen Success ↑ | Seen FTE ↓ | Seen Gain → 1 | Unseen Grasp ↑ | Unseen Success ↑ | Unseen FTE ↓ | Unseen Gain → 1 |
|---|---|---|---|---|---|---|---|---|
| Ours | 83.2 ± 3.4 | 62.9 ± 2.6 | 0.51 ± 0.07 | 0.52 ± 0.13 | 70.6 ± 4.6 | 42.3 ± 4.5 | 0.68 ± 0.07 | 0.46 ± 0.12 |
| −κ | 76.2 ± 1.8 | 54.9 ± 3.4 | 0.65 ± 0.05 | 0.36 ± 0.03 | 60.4 ± 2.1 | 34.3 ± 1.0 | 0.82 ± 0.03 | 0.30 ± 0.01 |
| −desc | 78.5 ± 3.6 | 55.6 ± 5.0 | 0.71 ± 0.11 | 0.16 ± 0.27 | 67.7 ± 4.5 | 38.7 ± 5.9 | 0.85 ± 0.09 | 0.15 ± 0.26 |
| −surf | 69.3 ± 1.5 | 33.5 ± 1.5 | 0.78 ± 0.02 | 0.20 ± 0.04 | 55.6 ± 10.7 | 22.2 ± 7.0 | 0.89 ± 0.07 | 0.20 ± 0.02 |
Thus, Ours maintained its performance advantage across three seeds and under the revised evaluation conditions, supporting our claim that the surface-based representation, including κ and descendant, contributes to generalization to unseen hands.
Response to Reviewer fE7i (1/4)
Thank you for your feedback, which greatly helped improve our paper. Thanks to it, we scoped our claims to VR avatar hands and strengthened them with evaluations on non-MANO avatar hands and a new force-control user study. Below, we respond to each of your questions.
We agree with your point. This study focuses on VR avatar hands with varying geometries, aiming to enable each actuator to regulate grip force by accounting for the geometry of the surface it controls. Transferring the method to physical robotic hands or arbitrary articulated grippers would require additional consideration of actuator dynamics, physical constraints, and the sim-to-real gap, and is therefore beyond the scope of this work.
Accordingly, we will revise "a hand" to "an avatar hand" in the Introduction and modify expressions such as "a necessary axis of embodiment" that could imply a broader generalization scope than our results support. To ensure consistency throughout the paper, we will also revise the relevant statements in the Abstract, the third contribution (Contribution 3), the Conclusion, and the Limitations section. In addition, our comparison with GET-Zero concerned the methodology for representing different hand structures, not performance on physical robotic hands.
However, we believe it is important to demonstrate that our method is not limited to MANO within this scope. We therefore evaluated five publicly available non-MANO VR avatar hands (Rocketbox male and female, Ultraleap, and two SteamVR glove hands) without retraining. We preserved their original meshes, joint positions, sizes, and masses while applying the same 21-DoF configuration as the training hands. We used the same joint evaluation as in Tables 1 and 3 (Section 3 of this response). Across the five avatar hands, Ours achieved 44.8% Success, comparable to unseen MANO hands (42.3%), with FTE and Gain also at similar levels (Table R3-1). The hand shapes are shown in Appendix B.6.
Table R3-1. VR avatar hands not derived from MANO (Ours without retraining, in the same combined experiment as Tables 1 and 3. The first two rows repeat the MANO-derived Seen and Unseen hands. All values are averaged over three training seeds; ± is the standard deviation over seeds for the grouped rows.)