- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper formulates close-contact two-person interaction (touches, multi-contact hugs) as cooperative decentralized control. Two SMPL virtual humans act from partial egocentric marker observations (a 90 degree head-mounted frustum with normal-facing visibility) and are conditioned on vertex-level contact goals. A single shared actor outputs actions in the latent space of a causal 1D-conv motion VAE trained on single-person HumanML3D data. The same latent space is used to generate synthetic two-person demonstrations through centralized BFGS trajectory optimization of contact/penetration costs, which bootstrap the actor and critic before PPO fine-tuning. Across nine kinematic tasks with randomized body shape and initial pose, BC+PPO raises success over PPO from scratch from 36% to 83% on 2-contact and from 23% to 75% on 4-contact tasks, with ablations on parameter sharing, visibility, initialization factors and success tolerances.
Critical strengths
- Using a single-human motion prior to regularize the policy's action space and to synthesize two-agent demonstrations by differentiable trajectory optimization is an elegant way to sidestep the scarcity of mesh-level close-interaction datasets. The resulting large gains on multi-contact tasks (23% to 75%) are the paper's main evidence that this pipeline works.
- Vertex-pair contact goals, signed-distance penetration penalties and egocentric visibility make a more demanding and more realistic coordination benchmark than skeleton-level HHI generation. The visibility and initialization ablations give useful insight into what makes multi-contact tasks hard.
Critical weaknesses
- The bodies are kinematic SMPL meshes with no forces, balance or dynamics, so "contact-rich" here means geometric proximity without interpenetration. Both agents are the same jointly trained policy, so the agent is never tested against a partner it was not co-trained with: a passive or scripted partner, a recorded human motion, or a differently trained policy. Adapting to a partner whose actions you don't control is the core of the assistive-care motivation, and self-play conventions may not transfer.
- The comparison is limited. No adapted interaction-synthesis baselines (InterControl, Ready-to-React). Results come from 500 evaluation episodes, from a single training seed, and planner demos require manual rejection of failures, so the reliability of the reported gaps is unclear.
Questions for authors
- How does the policy perform when paired with a partner it was not trained with: a frozen or passive partner, a scripted motion, a recorded HumanML3D/Inter-X motion, or a separately trained policy?
- How many training seeds were run, and what is the variance in success for PPO vs. BC+PPO on 2- and 4-contact tasks?
- Why does the independent (non-shared) policy collapse on 4-contact up (0.10) while matching elsewhere? Is this an optimization instability that also affects the unified policy across seeds?
6 comments
Response to reviewer x93b (2/2)
Missing comparisons (W2)
A comparison on exactly the same task is hard for all three methods, because these methods do not share our interface (goals as pairs of body-surface points, random initial states, per-agent partial observations):
-
Text-to-interaction methods (e.g., InterGen): text alone cannot specify which body-surface points are required to make contact, applying this method to our setting would fail entirely.
-
Ready-to-React: no goal input, and its released model covers boxing only. Training it on our tasks would require paired interaction motions; we have not evaluated such an adaptation.
-
InterControl: the closest, but its constraints are between skeleton joints, not surface points, and it generates both motions jointly and offline from the full state.
We still ran InterControl on our nine tasks, using its released model without retraining. Each target pair becomes a distance constraint between the nearest joints, clips start from our initial states, and the generated meshes are scored with our success criterion (500 clips per task).
InterControl is zero-shot, is constrained on joints while success is measured on the surface, and uses a mean body shape (ours: random shapes). Conversely, it is centralized with full state access, an advantage over our decentralized policy.
| 1-contact | 2-contact | 4-contact | |
|---|---|---|---|
| Ours (BC+PPO): success | 0.94 | 0.82 | 0.69 |
| InterControl: success | 0.04 | 0.00 | 0.00 |
| Clips reaching all contacts (within 10 cm) | 0.40 | 0.01 | 0.00 |
| Clips within 2.6 cm penetration | 0.14 | 0.07 | 0.10 |
Success requires both conditions at once: all contacts reached and penetration within 2.6 cm. InterControl fails on both. On 1-contact tasks, the hand reaches its target in 40% of the clips, but mostly by entering the partner's body: only 14% of the clips stay within the penetration limit. On hugs, it almost never reaches all targets (1% of the clips with 2 contacts, none with 4), and penetration remains as frequent. We stress that this comparison is not fair to InterControl: it is evaluated zero-shot on a task it was not designed or trained for.
Thank you for your response
I have few things that I want your clarification:
- InterControl is conditioned on text, which text prompt was used?
- When you say InterControl generates both motions jointly and offline from the full state, the proposed method also uses one shared network for both agents thus I think each agent knows its partners policy. So I think the proposed method is not very different from InterControl other than using skeleton vs surface in essence.
Response to reviewer x93b (1/2)
Dear Reviewer x93b, Thank you for your careful review, and for recognizing the role of the single-human motion prior in demonstration synthesis, the gains on multi-contact tasks, and the design of the benchmark and ablations.
Statistical reporting (W2/Q2)
As also noted by Reviewers 9SXj and p3g7, we have updated all quantitative results in the paper (shown in blue in Tables 1, 2, 3, 4 and Figure 3) to report the mean and standard deviation over 3 training seeds, each evaluated on the same 500 episodes. The main BC+PPO advantage remains. The ordering in Table 4 changes, in the expected direction: orientation-only randomization now gives higher four-contact success than full randomization (0.74 vs. 0.69). A single 100-epoch run takes about 35 hours on one A100 80GB with 32 workers, and the same compute also had to cover the additional experiments requested in the reviews, so running more than three seeds has not been possible yet.
Kinematic-only setting (W1)
We agree, and state it in the introduction (lines 72–80) and in Section 5 (lines 467–468): bodies are kinematic, with no contact forces or balance. Physical assistance is our motivation, not a claim we evaluate. Geometric coordination, i.e. reaching the requested contacts without interpenetration and from partial observations, is a prerequisite for physical interaction, and it is far from trivial: PPO from scratch reaches only 20% on four-contact tasks, against 69% for our method. Physics-based work such as AssistMimic tracks reference demonstrations; our policies reach specified contact goals with no reference motion to track, so the two are complementary.
Planner demo rejection (W2)
As Reviewer p3g7 also asked, we clarify the rejection step. Rejection is an automatic success check, not hand curation: we execute each optimized trajectory and keep it only if it meets the evaluation success criterion, until we have 500 per task. The rejection rate averages about 33%, from 2% (H2H) to 69% (2-contact Low). “Manual rejection” in Section 5 was a poor choice of words, now corrected. Many rejected trajectories are near-successes that methods learning from imperfect demonstrations could use.
Results clarification (Q3)
We thank the reviewer for raising this question and giving us the opportunity to clarify these results.
The low success of the independent policies on 4-contact Low and Cross comes from under-training within our fixed budget of 100 epochs. Each agent has its own network, so each network only receives its own agent’s transitions, which is about half the samples a shared network gets from the same environment steps. As a result, learning a specific task can start late on the harder tasks, and depending on the seed, the model finds a solution more or less quickly.
This is illustrated by two training seeds: training success first exceeds 5% at epochs 95 and 38 for Low, and at epochs 97 and 59 for Cross, compared with epochs 30 and 27 for Up. Thus, the two seeds show substantial variation in convergence speed. The seed reported in the submission only began solving Low and Cross during its final five epochs, explaining the low reported success rates of 0.11 and 0.10. We therefore interpret these results as reflecting the fixed training budget and slower convergence of the independent policies on the harder 4-contact tasks, rather than an inability of the policies to learn these tasks.
Partner adaptation (W1/Q1)
We thank Reviewers x93b and 9SXj for this question. We added a cross-play evaluation (Appendix C.3, Table 9, in blue). We train policies with different seeds and, at test time, control the two agents with policies from two different seeds, so each agent faces a partner never seen during training. We compare with self-play, where a single policy controls both agents (our main method).
A drop is expected: independently trained policies settle on different but equally valid conventions, and a multi-contact task succeeds only when the two are compatible. Cross-play therefore retains 84% on one-contact tasks and falls to 43% and 33% on two- and four-contact tasks. This is consistent with the known zero-shot coordination gap of self-play (Hu et al., ICML 2020). Training against a population of diverse partners improves coordination with unseen partners in other cooperative domains (Strouse et al., NeurIPS 2021; Lupu et al., ICML 2021) and is our next step. Passive, scripted or recorded partners remain untested.