acceptodds
Under review as a conference paper at ICLR 2027

An Interaction Is More Than Two Motions: Unifying Motion Generation and Policy Control For Dyadic Humanoid Interaction

Abstract

Recent advances in physics-based humanoid motion imitation have improved individual robot agility, motivating growing interest in physically coupled humanoid interactions. However, control-oriented approaches often rely on predefined motion templates, while 3D motion generators synthesize diverse interactions but give limited consideration to policy-based execution on physical robots. Our key insight is that generation, embodiment adaptation, and execution should be guided by a shared representation of interaction relations. We present DyadLink, a generation-driven framework that synthesizes a compatible follower motion from a leader's reference motion under multimodal conditioning, such as language or music. DyadLink uses a time-varying, directed Shared Relational Graph (SRG) to maintain interaction relations throughout the pipeline. Its role-asymmetric leader–follower formulation prioritizes fidelity to the leader's prescribed motion while allowing the follower to adapt its motion to preserve the intended interaction. To make it actionable across heterogeneous stages, we design a series of stage-specific, learnable and differentiable relational operators that translate its shared specification into generation guidance, embodiment-aware optimization objectives, and policy-learning signals. Experiments on InterHuman and CoDance show improved contact fidelity and pairwise geometric consistency across generation, retargeting, and policy execution. We further demonstrate the resulting interactions on physical humanoids. Source code will be made available.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Interactive Motion Generation, Humanoid Interaction, Motion Retargeting, Physics-Based Humanoid Control, Graph-Structured Motion Representation.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_K8uxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The submission proposes a unified framework for embodied robot interaction. The input of the system is reference motion and text or music. The output would be a compatible follower motion, and transfers it to actual humanoid robots. The core component is SRG, which represents inter-agent geometric relationships and contact information across different stages. Experiments are conducted on two datasets, InterHuman and CoDance. They also present results on real-world robots.

Critical strengths

The whole framework is engineered together to a very practical (at least in its form) system, covering motion generation, embodiment-aware retargeting, and policy control. it is hard to say any part is completely brand-new, but the whole framework appears to be a useful system-level contribution.

Critical weaknesses

The experiment results need further justification.

First, the execution errors reported in the table are too large that I doubt on the effectiveness of interaction preservation. As shown in Table 4, the executed PRPE / DRGE reach 0.701 m / 0.909 m on InterHuman and 1.055 m / 1.166 m on CoDance. For an embodied humanoid I think an average errors about 1 meter are too large to claim the motion is successfully executed. Furthermore, Table 3 reports extremely large rotation tracking errors like 85.22° and 104.51°. I am not sure if there are any errors in calculation, or I have the wrong interpretation of this metrics.

Considering the large error, the improvements over existing baselines are marginal. For example, the executed DRGE on InterHuman only decreases from 0.941 m (GMR) to 0.909 m, while on CoDance, the executed PRPE decreases from 1.078 m to 1.055 m and DRGE from 1.191 m to 1.166 m. Given these large absolute errors and relatively minor improvements, I am not convinced that the proposed framework can reliably preserve the intended interactions during execution or have significant superiority over existing baselines. The authors should provide further clarifications.

The provided anonymous link provide video results, which is good, but I don't see comparisons with baseline methods. Also, some generation results appears significantly unnatural, such as the six "dance" examples in "part 1: Human motion generation".

Second, I feel that the contribution of this work are not clearly isolated. From my understanding, the central contribution of this paper is SRG, yet some of the existing papers already leveraged similar designs, like [1,2]. Also, in Tables 2 and 5, the model without SRG already substantially outperforms UMF in FID, and even receives better FID than the full model. I wonder what is the exact difference between the proposed model w/o SRG and UMF? This part needs further clarification.

Also, the main retargeting and execution experiments use 15 InterHuman pairs and 8 CoDance pairs, while the ablation experiments uses a different set of 17 InterHuman clips according to Table 10. The full model achieves much better results in the ablation than in the main experiments: PRPE decreases from 0.701 m to 0.1705 m, DRGE from 0.909 m to 0.2466 m. Could you please clarify why choosing different evaluation sets for main experiment and ablation study and why there are substantial performance gap?

[1] Rhythm: Learning interactive whole-body control for dual humanoids [2] InterAgent: Physics-based Multi-agent Command Execution via Diffusion on Interaction Graphs

Questions for authors

Please clarify the issues raised in the "Critical Weaknesses" part.

4 comments
AuthorsOpenReview ↗

Response to Reviewer K8ux

Thank you very much for your careful review and for acknowledging that the whole framework appears to be a useful system-level contribution. Regarding your questions and some misunderstandings, we will provide clarification point by point.

Misconceptions regarding excessive execution errors. Execution errors are accumulated errors of 24 bodies. For example, if the table shows a global position error of 0.84 m and a rotation error of 85 degrees, this translates to an average error of 0.035 m and 3.54 degrees per joint. These are very small, reasonable margins of error that are entirely acceptable in a semantic context. We will include the method of calculation in the paper to avoid any unnecessary misunderstanding:

\frac{1}{TA} \sum_{t=1}^{T}\sum_{a=1}^{A}\sum_{b=1}^{B} ||x_{a,b,t}^{e}-x_{a,b,t}^{r}||_2.$$ $$E_l = \frac{1}{TA} \sum_{t=1}^{T}\sum_{a=1}^{A}\sum_{b=1}^{B} ||(x_{a,b,t}^{e}-x_{a,o,t}^{e})-(x_{a,b,t}^{r}-x_{a,o,t}^{r})||_2.$$ $$E_R = \frac{180}{\pi TA} \sum_{t=1}^{T}\sum_{a=1}^{A}\sum_{b=1}^{B} \arccos( \operatorname{clip}( \frac{\operatorname{tr}((R_{a,b,t}^{r})^T R_{a,b,t}^{e})-1}{2}, -1,1)).$$ Here, $E_g$, $E_l$, and $E_R$ denote global position error, local position error, and rotation error, respectively. T is the number of evaluated frames, A is the number of evaluated agents, and B is the number of tracked bodies per agent. The indices a, b, and t identify the agent, body, and frame, while o identifies the root body. The variables x and R represent world-frame position vectors and rotation matrices, respectively; superscripts e and r denote executed motion and the retargeted reference. The operator tr denotes the matrix trace. **About improvements over existing baselines are marginal.** The DRGE and PRPE metrics you mentioned are evaluation parameters defined in our paper, calculated using Equation (7). Since these metrics are averaged over each semantic edge, they are highly sensitive to interactions. Interaction edges include relationships between agents that span across their respective bodies (edges that do not involve physical contact), which results in a larger base count for these parameters. Furthermore, we believe that the contributions of DyadLink are not limited to these two evaluation metrics. Its effectiveness is demonstrated through the various quantitative results presented in the paper, as well as the visualization results results provided on the anonymous website referenced in the paper. In addition, we provide a set of comparative results covering three distinct stages to demonstrate the effectiveness of DyadLink throughout the entire process, as shown in the table below; the corresponding visualizations can be viewed on an anonymous website. **Whole pipeline comparison on InterHuman.** Each downstream stage takes the output of the preceding stage as its input. Bold indicates the best mean for each metric. | Method | Generation: FID ↓ | Generation: Contact F1 ↑ | Retargeting: PRPE (m) ↓ | Retargeting: DRGE (m) ↓ | Retargeting: Global pos. (m) ↓ | Retargeting: Contact F1 ↑ | Control: Executed PRPE (m) ↓ | Control: Executed DRGE (m) ↓ | Control: Contact Recall ↑ | Control: Contact F1 ↑ | | :--- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | Others | 4.432 ± 0.032 | 0.141 ± 0.012 | 0.65 ± 0.02 | 0.82 ± 0.08 | 0.9 ± 0.08 | 0.322 ± 0.025 | 1.02 ± 0.12 | 1.14 ± 0.11 | 0.362 ± 0.258 | 0.404 ± 0.204 | | **Ours (DyadLink)** | **3.759** ± 0.043 | **0.194** ± 0.004 | **0.15** ± 0.04 | **0.18** ± 0.06 | **0.86** ± 0.12 | **0.415** ± 0.012 | **0.71** ± 0.05 | **0.89** ± 0.06 | **0.698** ± 0.205 | **0.702** ± 0.103 | **Others** combines InterAgent[1] for generation, OmniRetarget[2] for retargeting, and BeyondMimic[3] for control. The retarget and control stage employs six motions sampled from a generative model. Some results in the new table differ from those in the original manuscript because the evaluations follow different input protocols. In the whole comparison, each downstream stage receives the output of the preceding stage. By contrast, the stage-wise evaluations use controlled reference inputs, as described in the manuscript, to isolate the contribution of SRG at each stage. [1]InterAgent: Physics-based multi-agent command execution via diffusion on interaction graphs. [2]OmniRetarget: Interaction-preserving data generation for humanoid whole-body loco-manipulation and scene interaction. [3]BeyondMimic: From motion tracking to versatile humanoid control via guided diffusion.
AuthorsOpenReview ↗

Response to Reviewer K8ux

About video results. Thank you for your positive feedback regarding our video results. We have added several new sets of video results for comparison against the baseline. Regarding the CoDance results, we would like to provide some clarification: two-person dance sequences represent a particularly challenging area in this field, characterized by limited data, data sparsity, and high motion complexity. These factors make the transition from generation to execution quite difficult. We readily acknowledge that our two-person dance results appear unnatural; this work serves merely as an extension of our research and establishes a baseline for future studies. We do not claim the reconstruction of dance sequences as a core contribution of our paper.

Related Research. We have also taken note of relevant research from our peers. In addition to InterAgent [1]—which focuses on establishing interaction constraints during the generation phase—and Rhythm [2]—which addresses the maintenance of interaction relationships during the policy control phase—there is also the work Beyond Mimicry [3], which focuses on preserving two-person relationships during the redirection phase. Rhythm's official project page lists its code as "Coming Soon," and we have not located an official public implementation of Beyond Mimicry. Therefore, we cannot include them in the comparison. These recent studies are all discussed in the "Related Work" section of our paper. We do not claim to be the first to model or constrain interaction relations. Our contribution lies in using a shared relational graph representation to provide consistent relational guidance across motion generation, retargeting, and policy-based control through stage-specific operators.

Clarification regarding experimental data. For InterHuman and CoDance, we constructed two role-ordered examples from each motion pair by exchanging the leader and follower assignments. We will document this role-based data augmentation procedure in detail in the revised manuscript. This preprocessing leads to differences in evaluation scores relative to those reported in these original reference papers. Therefore, it is not appropriate to directly compare data from other papers with the experimental results presented in this paper. We apologize for any misunderstanding caused by missing description of data preprocessing, and we will include further clarification in the paper.

Explanation of the differences between PRPE and DRGE. As clarified in the appendix, Tables 4 and 7 use different motion sets and robot pairings. The main comparison uses G1–G1 pairs, whereas the subsequent ablation uses G1–T1 pairs with a correspondingly updated reinforcement-learning configuration. Different agents and different action sets are the reasons for significant performance differences. This change stems from hardware modifications in the laboratory. Since training multiple sets of new control strategies is extremely time-consuming, we did not rerun the main comparison under this configuration. However, we believe this issue should not affect the assessment of our main contribution.

[1]InterAgent: Physics-based multi-agent command execution via diffusion on interaction graphs.
[2]Rhythm: Learning interactive whole-body control for dual humanoids.
[3]Beyond mimicry: Learning whole-body human-humanoid interaction from human-human demonstrations.

We hope our explanation has cleared up your doubts and misunderstandings. We hope our Instructions allow our contribution to be assessed on the intended basis. We thank the reviewer for your thoughtful feedback.

Dear Reviewer,

Thank you for sharing your concerns. We recognize that our original presentation left room for misunderstanding and have provided detailed explanations in our response, with corresponding clarifications in the manuscript. The revised paper has been resubmitted, and our visualization website has also been updated. We believe the issues raised concern the clarity of our presentation rather than the novelty or substance of our contributions. We would be grateful if you could revisit our response and the revised manuscript and take these clarifications into account when assessing our main contributions.

Dear Reviewers,

We are now in the final stage of the rebuttal period. Thank you for your valuable review of our paper. We have submitted our rebuttal, addressing the concerns raised. Furthermore, we have incorporated additional experiments and clarifications into the paper, following the suggestions provided by you and the other reviewers. We believe that the problems raised stemmed from a misunderstanding of the experimental setup, which should not affect the assessment of our paper's main contribution.

We would be very grateful if you could take some time to review our response and re-evaluate the contributioin of our work. Thank you very much for your time and valuable contributions!

Reviewer_e4H3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents DyadLink, a framework that connects dyadic motion generation, embodiment retargeting, and humanoid control through a Shared Relational Graph (SRG). Given a complete Leader motion and text, with music additionally used for dance, it predicts an interaction plan and generates a compatible Follower motion. The plan provides relational targets for constrained retargeting and supervision for reference-specific PPO policies, alongside an external contact-gated compliance module. Evaluation covers motion generation on InterHuman and CoDance, retargeting and simulated execution, stage-wise ablations, and qualitative hardware demonstrations.

Critical strengths
  1. I can see the motivation. Two plausible motions do not necessarily make a successful interaction, especially after changing the embodiment and executing the references. Carrying an explicit description of the intended relations through these stages is a reasonable way to address this problem.

  2. The evaluation looks beyond motion-distribution metrics and includes contact and pairwise geometry. The control ablation also keeps references, policy inputs, and the compliance module fixed while changing the rewards, which provides useful evidence that relational supervision helps in that setting.

Critical weaknesses
  1. I am less clear about what is actually new in the pipeline. Interaction generation, interaction-aware retargeting, and motion tracking all have close precedents, including pipelines such as PAIR/D-STAR and Rhythm. What does the shared SRG add over putting existing components together? The ablations show that relation and contact constraints help, but do not quite establish whether carrying the predicted plan across stages matters. A strong compositional baseline and a comparison with independently reconstructed stage-wise relations are missing.

  2. I am also not fully convinced by the physical interaction evidence. The real-world examples look fairly simple, and I could not really tell what the compliance module contributes. In some dance examples, the hands do not seem to meet where contact is expected. I am not sure whether that comes from retargeting or tracking, but it seems important given the central claim of preserving interaction. Clearer examples of sustained physical contact, with quantitative contact and force measurements, would make the evidence more convincing.

Questions for authors
  1. How does DyadLink compare with a straightforward combination of an existing interaction generator, interaction-aware retargeting, and the same tracking backend? PAIR/D-STAR and Rhythm seem particularly relevant to the downstream comparison, with appropriate matching of the interaction setting.

  2. What happens if each stage reconstructs its own relations from the motion instead of receiving the predicted SRG plan? Can this comparison keep relation losses, model capacity, and post-processing matched?

  3. Which hardware examples actually use the compliance module? Could you show its response to contact disturbances, with and without the module, and report how the hand forces are measured or estimated?

  4. Are the dance examples showing retargeted references or simulated policy rollouts? Where hand contact is intended but appears to be missed, does the mismatch originate in generation, retargeting, or tracking? Please clarify with stage-wise contact measurements or aligned visualizations.

7 comments
AuthorsOpenReview ↗

Response to Reviewer e4H3

Thank you for your careful reading and review. We appreciate your understanding of the motivation behind our work. We provide the following responses to the questions you raised:

About close precedents. We take note of research similar to ours, such as InterAgent[1], PAIR/D-STAR[2], and Rhythm[3]. However, Rhythm's official project page lists its code as "Coming Soon," and we have not located an official public implementation of PAIR/D-STAR. Therefore, we cannot include them in the comparison. DyadLink differs from existing approaches by integrating shared relational constraints into motion generation, retargeting, and control through stage-specific operators to preserve interaction consistency across the pipeline. The table below presents whole pipline comparisons with existing approaches, with each downstream stage taking the output of the preceding stage as its input. These results support the effectiveness of shared relational guidance throughout the pipeline. We will include this table in the revised manuscript.

Table 1. Whole pipeline comparison on InterHuman. Each downstream stage takes the output of the preceding stage as its input. Bold indicates the best mean for each metric.

MethodGeneration: FID ↓Generation: Contact F1 ↑Retargeting: PRPE (m) ↓Retargeting: DRGE (m) ↓Retargeting: Global pos. (m) ↓Retargeting: Contact F1 ↑Control: Executed PRPE (m) ↓Control: Executed DRGE (m) ↓Control: Contact Recall ↑Control: Contact F1 ↑
Others4.432 ± 0.0320.141 ± 0.0120.65 ± 0.020.82 ± 0.080.9 ± 0.080.322 ± 0.0251.02 ± 0.121.14 ± 0.110.362 ± 0.2580.404 ± 0.204
Ours (DyadLink)3.759 ± 0.0430.194 ± 0.0040.15 ± 0.040.18 ± 0.060.86 ± 0.120.415 ± 0.0120.71 ± 0.050.89 ± 0.060.698 ± 0.2050.702 ± 0.103

Others combines InterAgent[1] for generation, OmniRetarget[4] for retargeting, and BeyondMimic[5] for control. Six sets of data were obtained via controlled sampling. The retarget and control stage employs six motions sampled from a generative model.

Some results in the new table differ from those in the original manuscript because the evaluations follow different input protocols. In the whole comparison, each downstream stage receives the output of the preceding stage. By contrast, the stage-wise evaluations use controlled reference inputs, as described in the manuscript, to isolate the contribution of SRG at each stage.

[1]InterAgent: Physics-based multi-agent command execution via diffusion on interaction graphs.
[2]Beyond mimicry: Learning whole-body human-humanoid interaction from human-human demonstrations.
[3]Rhythm: Learning interactive whole-body control for dual humanoids.
[4]OmniRetarget: Interaction-preserving data generation for humanoid whole-body loco-manipulation and scene interaction.
[5]BeyondMimic: From motion tracking to versatile humanoid control via guided diffusion.

About each stage reconstructs its own relations. Following your suggestions, we attempted to reproduce recent related methods for comparison, however, these methods have not yet been public. To provide an whole pipline comparison, we assembled a baseline pipeline combining recent methods for motion generation, retargeting, and policy-based control. Quantitative results are presented in the table above, while qualitative comparisons are available on the anonymous project website linked in the manuscript.

Use of compliance module. The compliance-related module is used primarily during reinforcement learning in simulation and is not activated during hardware execution. In real-world deployment, force-related inputs are provided by the simulation-trained policy rather than physical force sensors. Our hardware setup therefore does not implement closed-loop force feedback. Consequently, we have not highlighted it as a contribution in our paper.

AuthorsOpenReview ↗

Response to Reviewer e4H3

Dance Results. The dance examples in the manuscript are explicitly labeled as either retargeted reference motions or policy execution results. The anonymous project website follows the same distinction: “Retargeting” presents retargeted references, whereas “Control” shows policy execution results.

Analysis of missed contacts and phased visualization. We have added the suggested stage-by-stage comparison to the anonymous project website. The qualitative results show how interaction errors can arise during generation and accumulate through retargeting and policy execution. DyadLink better preserves the interactions across these stages through shared relational guidance. This supports the core arguments and contributions presented in our paper.

We hope these clarifications address your concerns. Our central contribution is shared relational guidance across the pipeline, and we acknowledge the need for the additional comparisons outlined above to strengthen its empirical evaluation. We believe the issues mentioned above do not affect the assessment of our main contribution. Thank you for your careful assessment and constructive feedback.

Thanks for the clarification. I am still not sure my main question has been answered. I asked what happens when each stage reconstructs its own relations while keeping the relational objectives unchanged. This is an internal ablation of DyadLink; it does not require code from PAIR or Rhythm. The new composed baseline is useful, but it still does not isolate what sharing the plan actually contributes.

One concrete example: Table 4 reports an executed PRPE of 0.701 m on InterHuman. Under Eq. (7), this is already the average relative-root displacement error, so the explanation about summing over 24 bodies does not resolve it. What does preserving an interaction mean here if the partners’ relative root placement is off by about 70 cm on average? I would like to see where this error occurs, especially during intended contact.

I am also puzzled by the hardware clarification. The response says compliance is disabled on hardware, while Appendix E.2 says it operates during deployment. Which configuration do the reported results actually evaluate?

Reviewer_A1MeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed DyadLink, a reaction-generation framework that synthesizes a human reactor's motion from an actor's human motion, conditioned by text and/or music. Then, the motion is retargeted to a humanoid robot. Finally, a reference-tracking policy is trained, first for the leader, then the follower. The method features a Shared Relational Graph (SRG), which predicts the root relation, contact candidates, and joint-level relationships between the actor and the reactor during the motion. A learnable set of differentiable operators translates the SRG's information across motion generation, retargeting, and controller RL training. The method achieved superior performance compared to baselines in interaction generation, motion retargeting, and interaction preservation after execution.

Critical strengths
  1. The problem setting is uniquely positioned: it goes from text/music and leader motion all the way to interaction, humanoid simulation, and real-world robot deployment. Compared to other similar works (Table 1), it combines the fields of human reaction generation and robot motion control.
  2. The use of a unified relation graph (SRG) to guide generation, retargeting, and policy training is insightful and well-documented in the appendix (Eq. 8-15)
Critical weaknesses

Fairness and statistical significance of the evaluations. In the motion generation phase, DyadLink, an offline reaction-generation method, is compared against InterGen (Liang et al. 2024) and InterMask (Javed et al. 2025), which are two-person interaction generation methods. The FID numbers reported (InterMask 5.2796, InterGen 6.0842) are also close to the original papers' two-person generation FID on InterHuman (InterMask 5.154, InterGen 5.918), so it's reasonable to infer they are evaluated under a two-person generation setting, instead of offline reaction generation. When compared fairly in the same offline reaction generation task, InterMask (Javed et al., 2025) achieves 2.850 FID and 0.476 R-Precision top-1; both are superior to DyadLink. RemFlow (Geng et al., 2026) is also superior to DyadLink. ReGenNet (Xu et al., 2024) is cited but not compared, and is superior to DyadLink in FID. Also, there are no confidence intervals for the FID and R-Precision numbers, which are standard in the motion generation, reaction generation, and interaction generation fields.

Questions for authors
  1. Can the authors compare DyadLink with other reaction-generation methods (ReGenNet, ReMFlow) and report InterMask under the reaction-generation setting?
  2. Mismatch in evaluation results. Between Table 4 and Table 7, Full DyadLink achieves 0.701±0.1520.701 \pm 0.152 in Executed PRPE, but 0.1705±0.04180.1705 \pm 0.0418 in PRPE in Table 7. Executed DRGE also doesn't match.
  3. Why is the ground-truth diversity of InterHuman 7.6867, instead of 7.948 in other interaction generation and reaction generation papers, e.g., Liang et al. 2024, Javed et al. 2025?
  4. Table 3 shows an average of 0.84m global position error and an average of 85 degrees of rotation error. How is the interaction semantics preserved under such a large positional and rotational error after retargeting?
6 comments
AuthorsOpenReview ↗

Response to Reviewer A1Me

Thank you for your careful review. We address your questions point by point below.

Fairness and statistical significance of the evaluations. For InterHuman and CoDance, we constructed two role-ordered examples from each motion pair by exchanging the leader and follower assignments. We will document this role-based data augmentation procedure in detail in the revised manuscript. This preprocessing leads to differences in GT Diversity and evaluation scores relative to those reported in these original reference papers. To improve comparability across methods, we evaluated InterMask under the reaction-generation setting and adapted UMF to the same task. Specifically, we duplicate each sample and swap the role labels between the two individuals, ensuring that both role configurations are represented in the resulting dataset.

About Confidence Intervals. We evaluated the same trained model 20 times. We will add approximate 95% confidence intervals for the generation metrics to the revised manuscript, along with the calculation protocol.

Regarding further comparisons. We did not compare ReGenNet and ReMFlow, because they are missing InterHuman generation weights and the corresponding CNN-VAE weights. Following your request, we attempted to reproduce ReGenNet and ReMFlow. However, due to the limited time, we cannot fairly reproduce the results reported in the original papers.

Mismatch in evaluation results. As clarified in the appendix, Tables 4 and 7 use different motion sets and robot pairings. The main comparison uses G1–G1 pairs, whereas the subsequent ablation uses G1–T1 pairs with a correspondingly updated reinforcement-learning configuration. We did not rerun the main comparison under this configuration within the available time. However, we believe this issue should not affect the assessment of our main contribution.

AuthorsOpenReview ↗

Response to Reviewer A1Me

Misconceptions Regarding Error Calculation. Global position, local position, and rotation errors in Table 3 are accumulated errors of 24 bodies. For example, if the table shows a global position error of 0.84 m and a rotation error of 85 degrees, this translates to an average error of 0.035 m and 3.54 degrees per joint. These are very small, reasonable margins of error that are entirely acceptable in a semantic context. We will include the method of calculation in the paper to avoid any unnecessary misunderstanding:

\frac{1}{TA} \sum_{t=1}^{T}\sum_{a=1}^{A}\sum_{b=1}^{B} ||x_{a,b,t}^{e}-x_{a,b,t}^{r}||_2.$$ $$E_l = \frac{1}{TA} \sum_{t=1}^{T}\sum_{a=1}^{A}\sum_{b=1}^{B} ||(x_{a,b,t}^{e}-x_{a,o,t}^{e})-(x_{a,b,t}^{r}-x_{a,o,t}^{r})||_2.$$ $$E_R = \frac{180}{\pi TA} \sum_{t=1}^{T}\sum_{a=1}^{A}\sum_{b=1}^{B} \arccos( \operatorname{clip}( \frac{\operatorname{tr}((R_{a,b,t}^{r})^T R_{a,b,t}^{e})-1}{2}, -1,1)).$$ Here, $E_g$, $E_l$, and $E_R$ denote global position error, local position error, and rotation error, respectively. T is the number of evaluated frames, A is the number of evaluated agents, and B is the number of tracked bodies per agent. The indices a, b, and t identify the agent, body, and frame, while o identifies the root body. The variables x and R represent world-frame position vectors and rotation matrices, respectively; superscripts e and r denote executed motion and the retargeted reference. The operator tr denotes the matrix trace. We hope these clarifications resolve the ambiguities in the evaluation protocol and error reporting. We believe the reviewers already have an understanding of the contributions of our work. We believe that most of the issues mentioned stem from misunderstandings, which do not change the proposed method or its central contribution. We thank the reviewer for your careful assessment and for helping us present the work more clearly.

I appreciate the author's response. However, most of my concerns are not answered satisfactorily.

Regarding the fairness concern, the authors still did not explain why InterMask’s R@1 falls to 0.158 from about 0.45 in the original paper, and did not address my concern about the InterGen numbers. The authors also did not explain their evaluation protocol in detail.

Regarding the error calculation. If the per-body tracking error is 3.5 cm, the executed PRPE should be close to the retargeted 0.13 m. Table 4’s 0.70 m does not fit this, and PRPE is a single root distance, not a sum.

Also, please make sure your responses to the reviewer use the correct spelling and capitalization of terms to facilitate efficient reading.

Thanks again for the authors’ effort. However, I would like to keep my rating of weak rejection.

Comments on the paper

1 comment
Area_Chair_jgtCOpenReview ↗

Rebuttal Discussion

Dear Reviewers,

The authors have now submitted their rebuttal and responded to the concerns raised in the reviews. We would greatly appreciate it if you could take some time to review their responses and participate in the discussion, particularly regarding whether your concerns have been adequately addressed.

Thank you very much for your time and valuable contributions!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.