- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes OmniInteract, a framework for transferring human-human interaction demonstrations to collaborative control of dual humanoid robots. The method consists of two stages. First, Surface Contact-Aware Retargeting (SCAR) jointly optimizes the poses of two humanoid robots while preserving motion fidelity and interaction constraints. Second, a partner-aware residual control framework trains task-specific policies using MAPPO on top of a frozen motion tracker, then distills them into a shared generative controller using DAgger and Conditional Flow Matching. Experiments on InterHuman and Inter-X datasets, unseen motions, and real Unitree G1 robots demonstrate improved retargeting performance and successful collaborative interactions.
Critical strengths
- The implementation details are well documented, and the supplementary material and real-world experiments are valuable additions.
- The paper is well written and easy to follow, with a clear structure and well-explained methodology.
Critical weaknesses
- Insufficient comparison with directly related dual-humanoid interaction methods. In particular, It Takes Two[1] presents contact-aware retargeting and interaction-driven control for two humanoids. More importantly, Rhythm[2] proposes a complete dual-humanoid interaction framework that includes jointly constrained interaction retargeting, graph-based interaction rewards, MAPPO, and real-world execution on Unitree G1 robots.
Questions for authors
- (Main Concern) How does SCAR compare quantitatively with Rhythm[2]'s IAMR and a coupled version of OmniRetarget[3] under identical datasets and metrics?
- Could the authors provide a capacity-matched deterministic Transformer or MLP baseline to isolate the contribution of Flow Matching?
- How does the number of flow-matching Euler steps during inference affect performance?
- The motion preprocessing removes finger articulation, but some tasks involve hand contact or grasping. How does this affect interaction fidelity, and can the method extend to humanoids with dexterous hands?
[1] Liu et al., It Takes Two: Learning Interactive Whole-Body Control Between Humanoid Robots, 2025.
[2] H. Chen et al., Rhythm: Learning Interactive Whole-Body Control for Dual Humanoids, Robotics: Science and Systems (RSS), 2026.
[3] L. Yang et al., OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction, IEEE International Conference on Robotics and Automation (ICRA), 2026.
7 comments
Response to Reviewer Hzxs (Part 2 / 2)
Q3: Effect of Euler step count on control performance and inference latency.
A3: Thank you for raising this inference-time trade-off. We evaluate 1, 3, 5, 8, and 10 Euler steps in R3-Table4 using the same checkpoint and evaluation protocol, with batch size 1. Increasing the Euler step count from 1 to 5 improves all three control metrics, raising success from 70.56% to 84.32%. Further increasing to 10 steps yields diminishing returns: success improves by just 0.69% and MPJPE decreases by 1.02 mm, while inference latency nearly doubles—from 9.8 to 18.6 ms on the Orin NX and 4.0 to 7.5 ms on the RTX A6000.
R3-Table4: Effect of Euler step count on OmniInteract’s control performance and inference latency.
| Euler steps | InterHuman† + Inter-X† | Inference policy latency ↓ | |||
|---|---|---|---|---|---|
| Succ. (%) ↑ | Unitree G1 Jetson Orin NX (ms) | NVIDIA RTX A6000 (ms) | |||
| 1 | 56.28 | 8.79 | 70.56 | ≈2.2 | ≈0.9 |
| 3 | 46.78 | 7.54 | 79.51 | ≈6.2 | ≈2.5 |
| 5 (default) | 41.67 | 6.42 | 84.32 | ≈9.8 | ≈4.0 |
| 8 | 40.89 | 6.22 | 84.78 | ≈14.2 | ≈5.7 |
| 10 | 40.65 | 6.15 | 85.01 | ≈18.6 | ≈7.5 |
We therefore retain 5 Euler steps as the default, providing a favorable balance between control accuracy and inference latency.
Q4: Finger simplification and extension to dexterous hands.
A4: Thank you for raising this concern. We conducted a paired ablation on 20 complete Inter-X sequences (2,987 frames), varying source finger articulation while keeping the fixed-hand G1 target and SCAR optimization settings unchanged. As shown in R3-Table5, preserving source finger articulation reduces hand-contact vector error by 32.3% and improves hand-endpoint F1 by 11.28%. These results show that source finger geometry matters for contact preservation even with a fixed-hand target.
R3-Table5: Effect of source finger articulation on hand-contact fidelity with a fixed-hand G1 target.
| Source finger input | Hand-contact vector error (mm) ↓ | Hand-endpoint F1 (%) ↑ |
|---|---|---|
| Neutralized | 70.78 | 26.52 |
| Original articulation | 47.94 | 37.80 |
Both variants are evaluated against the original articulated source using a 20 mm contact threshold.
We also extended our method to G1 with Inspire dexterous hands by retaining source finger poses and binding palm and finger-surface samples to the corresponding robot links. The bodies and fingers of both humanoids are jointly optimized using the same contact-interface and near-field objectives, subject to joint-limit, joint-coupling, velocity, and collision constraints. We have added the visualization of our method with dexterous hands in the revised manuscript (please see Figure 12).
We are happy to see from your comments that you have an overall positive view of our work, and we respectfully ask if you'd be comfortable to adjust your ratings, based on all the new experiments and justification to your questions. Thank you once again for your time and effort. We look forward to incorporating these insightful discussions into the final camera-ready version.
[1] Liu et al., It Takes Two: Learning Interactive Whole-Body Control Between Humanoid Robots, 2025.
[2] H. Chen et al., Rhythm: Learning Interactive Whole-Body Control for Dual Humanoids, Robotics: Science and Systems (RSS), 2026.
[3] L. Yang et al., OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction, IEEE International Conference on Robotics and Automation (ICRA), 2026.
Dear Reviewer,
I hope this message finds you well. As the discussion period is nearing its end, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper!
Response to Reviewer Hzxs (Part 1 / 2)
We really appreciate the reviewer for the detailed and constructive feedback. We believe our response has addressed most of your concerns and we respectfully ask if you're comfortable to re-consider the rating in light of our efforts and dedication. Below, we provide detailed responses to all the comments:
Q1:Comparison with directly related dual-humanoid interaction methods.
A1: Thank you for highlighting these related dual-humanoid interaction methods. To the best of our knowledge, neither It Takes Two [1] nor Rhythm [2] has publicly released its official implementation to date. Nevertheless, we have fully re-implemented their retargeting and control pipelines following the original papers. We further extended OmniRetarget [3] to a coupled dual-agent setting to provide a more comprehensive comparison under the same datasets and evaluation metrics in R3-Table1.
The added quantitative results and qualitative visualizations are highlighted in Table 1 and Figure 3 of the revised manuscript. SCAR achieves the lowest CIE and highest DSR on both datasets, improving DSR over the strongest added baseline by 5.0% and 5.9% on InterHuman and Inter-X, respectively. The complementary controller comparison shows that OmniInteract exceeds Rhythm’s success rate by 13.13% and 6.98% on benchmark and unseen motions, respectively.
These results further support the effectiveness of our design: SCAR explicitly models fine-grained surface contacts and their surrounding geometry, while OmniInteract scalably consolidates partner-aware residual skills over a frozen motion prior for unified dual-humanoid control.
Q2: Capacity-matched experiments (Flow Matching vs deterministic baselines).
Thank you for this question about capacity-matched comparison. We have added deterministic MLP baselines with 30M, 50M, and 80M parameters, as shown in R3-Table3. All variants use the same training data and evaluation protocol.
The MLP performs slightly better at 30M. However, at 50M and 80M, Flow Matching achieves better results across all three metrics, improving success rates by 4.12% and 3.64%, respectively. As capacity increases from 30M to 80M, Flow Matching’s success rate improves by 4.84%, compared with 0.49% for the MLP.
These capacity-controlled results show that OmniInteract’s gains cannot be explained by parameter count alone and support the effectiveness of Flow Matching in utilizing larger model capacity for unified interaction control.