DittoHand: Zero-Shot Grip-Force Control Across Hand Morphologies with Surface-Aware Actuator Tokens

To a typical cross-embodiment policy, a hand is its kinematic structure; two hands that differ only in finger thickness are the same hand. Yet the same actuator command meets the object earlier on the thicker finger, at a different point and along a different normal. Existing cross-hand approaches either leave the hand–object geometry implicit or recover it with a retargeting step trained per hand. A policy that leaves this collision geometry implicit can often find a grasp, but it keeps that grasp less reliably and barely tracks the commanded grip force. We propose DittoHand, which treats a hand as a set of actuators and gives each an actuator token carrying the collision surface it moves and the closing rate at which that surface approaches the object as the actuator turns, so an action means the same thing on every hand. Trained on 24 hands, one policy is tested on them and zero-shot on 648 unseen hands, with 20 objects of which 17 are unseen in training. On both hand sets it leads two cross-hand baselines, most widely in keeping the object (47.5% vs. at most 34.3% on unseen hands) and in tracking the grip command, where it beats each baseline on at least 99.8% of the unseen hands. In a virtual reality (VR) study on three hands DittoHand had never seen, 12 participants rated it significantly more controllable and realistic overall than ForceGrip policies trained for each hand.