- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents PanTact, a system for teaching a humanoid to manipulate with any suitable body surface (forearms, knees, legs), learned from robot-free human demonstrations. Its central idea is to treat contact as a cross-embodiment interface: the human demonstration specifies where and when contact should occur, and the robot decides how to realize it with its own body. The system has three parts. 1. A wearable data-collection rig that adds modular, magnetically tiled FSR contact patches with Vive trackers on the forearms and knees to an egocentric, bimanual UMI setup. 2. A diffusion policy that takes a body-masked egocentric pointmap and wrist images as input and predicts end-effector (EEF) trajectories together with per-sensor contact goals (binary state and 3D position). 3. Three contact-aware whole-body controllers that realize active contact goals on the closest robot surface, using a soft-min distance and a virtual-target offset to generate pressing force: upper-body differential IK, whole-body IK tracked by SONIC, and an RL-trained sparse contact tracker. On a Unitree G1 without body sensors, PanTact succeeds on Grasp-and-Embrace (85% vs. 0% for Humanoid UMI), Sweep-and-Stow (90.6% vs. 0% end-to-end for Humanoid UMI++), and Carry-and-Push (90% vs. 40%). A simulation replay study compares the controllers with and without contact goals.
Critical strengths
-
The paper tackles a well-motivated problem that exploits what makes humanoids distinctive. Most humanoid loco-manipulation work reduces the robot to two grippers on a mobile base. This paper targets manipulation with the forearms, knees, and legs, a capability that only a human-like morphology makes possible and that the hands alone cannot provide. Treating contact location and timing as the quantity that transfers, while leaving body configuration to the robot, is a clean and well-argued way to bridge the human–robot morphology gap. All three tasks genuinely require non-hand contact (embracing a bulky object, pushing a cabinet closed with a knee or foot, load-sharing while pushing with a leg).
-
The system is complete and tested on a real robot, with large gains over pose-based interfaces. The pipeline covers wearable hardware, policy, and controller, and is deployed on a commercial humanoid without adding body sensors. Pose-only baselines fail in characteristic ways: they never establish sufficient arm contact, or they drive the arms into joint limits. The gains are large (0 → 85%, 0 → 90.6%, 40 → 90%). Together with the planned release of the modular patch design, this is a useful contribution to the community.
Critical weaknesses
-
Execution quality is not convincing, and how to realize contact on the robot remains unresolved. The supplementary videos show visibly jittery, non-smooth motion, yet the paper reports only binary success rates and median tracking errors, with no measure of smoothness, jerk, or contact stability. Several design choices could plausibly cause this behavior, and none are analyzed: contact goals are binarized FSR readings, so the controller's contact objective switches on and off abruptly; pressing force is produced open-loop by offsetting a position target into the object (Eq. 4), with no force feedback on the robot; and a 10 Hz diffusion policy is stitched asynchronously into a 50 Hz tracker. The tracking plots (Figs. 4e and 6e) also show noisy EEF and contact traces, and the sparse tracker is reported to have "tracking dead zones." The paper also proposes three controllers without settling on one. Each task uses a different one (Upper-Body IK for Grasp-and-Embrace, WBIK+SONIC for the other two), and Fig. 7 shows a trade-off in which none dominates. Since contact-aware control is presented as a core contribution, the reader should learn which formulation to use and why. As it stands, the robot-side execution looks like a set of options picked per task rather than a solved component, and the jitter makes it unclear how robust the whole-body behaviors are beyond the reported trials.
-
The evaluation is too narrow to isolate the value of contact prediction. The evaluation uses three tasks with 10–20 trials each, in a single lab setting. The baselines also differ from PanTact in more than the use of contact information. For Grasp-and-Embrace, the only baseline (Humanoid UMI) has no information about arm posture at all, so its 0% mainly shows that the arms need some target. The paper omits the most informative ablation: track the forearm patch poses, which are already recorded by the Vive trackers on the patches, continuously and without contact states. Humanoid UMI++ does this only for the knees. There are also no ablations of the policy itself (body masking of the egocentric view, predicting contact goals vs. deriving them from predicted poses, history length), and the recorded contact intensity is never used. The paper therefore does not show which part of "contact as interface" drives the gains. On the learning side, the policy is a standard diffusion policy following HoMMI with an extended action space, so the contribution is mostly at the level of hardware and systems. The evidence should be correspondingly thorough, but it is not.
Questions for authors
Can you quantify motion quality, for example joint/EEF jerk, contact-state switching frequency, or contact-force variance during active contact, for each controller and for the baselines? What causes the jitter visible in the videos: flickering contact goals, the virtual-target offset, the asynchronous chunk scheduling, or the trackers themselves?
Thank you for the constructive feedback and for recognizing the strong motivation for whole-body dexterity, the contact interface for human-to-robot transfer, and our complete real-robot system. We address your concerns about execution quality, controller selection, and the contributions of contact prediction below.
Motion quality
The front-page videos play at 4× speed, which makes small adjustments appear faster and more abrupt. Several non-smooth motions reflect the robot responding to feedback during the task and attempting retrying behaviors during complex, long-horizon tasks. In Sweep-and-Stow, the robot adjusts its arm position relative to the objects as it sweeps them into the bin. In Carry-and-Push, it can press its knee again when the first attempt does not move the floor box sufficiently. The policy also replans as objects become occluded or come back into view. The task videos play at 2x speed, and the additional videos play at 1× speed to show motion quality more clearly.
We quantify motion quality from real-world Grasp-and-Embrace rollouts.
The Sparse Contact Tracker achieves 36% lower arm-joint jerk, while the EEF jerk is similar. Contact transitions contribute to arm reconfiguration as the controller accommodates contact goals alongside wrist targets. The Sparse Contact Tracker smooths target changes through action filtering, softer gains, and action-rate regularization.
Contact stability
We run additional experiments comparing different pressing parameters and applying disturbance forces to objects. The robot maintains stable contacts under the applied disturbances.
Controller selection
We use Upper-Body IK for Grasp-and-Embrace, where the robot can keep its feet fixed, and WBIK+SONIC for the other two tasks, which requires coordinated lower-body manipulation, locomotion, and balance.
We recommend upper-Body IK for fixed-leg upper-body tasks, where it achieves the lowest EEF error. For whole-body tasks, controller selection depends on the task requirements. On Carry-and-Push, WBIK+SONIC achieves 90% success versus 60% for the Sparse Contact Tracker. Its lower EEF error helps maintain the bimanual support configuration during carrying is crucial to this task’s success, while leg contacts tolerate larger tracking errors.
Contact interface ablations
We ran two additional baselines. The first is a continuous tracking baseline that tracks the same contact points continuously using the any-surface objective, without gating by contact states. It achieves 4/10 success (40%), compared with 9/10 (90%) for ours. The failure cases videos show box slips and walking in the wrong direction. The second, a contact-gated fixed-link baseline, retains the same contact positions and states but assigns each goal to a fixed body part. It achieves 5/10 (50%), with box slips and missed leg contacts. These results show that both contact gating and adaptable surface assignment contribute to task success.
Policy ablations
We additionally train policy variants on the same Grasp-and-Embrace dataset and evaluate them on the same 5% held-out validation split. The table reports changes in position error relative to the original policy. Positive values indicate higher error.
Using one history frame increases prediction error. Deriving forearm-contact positions from predicted wrist poses gives comparable validation accuracy to predicting them directly.
Full task completion
For Sweep-and-Stow, we originally used partial credits because the baseline would be a rather 0% success and less informative to analyze failure modes. We rescore the original rollouts and ours fully completes 13/16 rollouts (81.3%), compared with 0% for Humanoid UMI++.