acceptodds
Under review as a conference paper at ICLR 2027

CoWBC: Learning Whole-Body Control for unconstrained Cooperative transport with Humanoid Robots

Abstract

Unconstrained cooperative transport requires humanoid robots to coordinate whole-body actions while maintaining contact with and stable support for a shared object. However, physical coupling through the shared object makes stable whole-body control more challenging than in single-humanoid settings. To address this challenge, we introduce CoWBC, an object-centered framework that learns role-aware cooperative transport policies from real-world human collaboration demonstrations, where roles describe the robots’ complementary contributions to the shared transport task. CoWBC uses a shared base policy to learn common cooperative skills and role-specific residuals to adapt actions to each robot’s role. Specifically, we first reconstructs interactions between human collaborators and a shared object from synchronized multi-view videos and retargets them into coordinated robot reference motions, preserving the collaborators’ relative positions, end-effector–object geometry, and temporal alignment. We then progressively train a centralized teacher with cooperation-aware rewards, then distill its behavior into local student policies. To mitigate the state distribution shift induced by closed-loop student execution, we further augment distillation with lightweight learner-state refinement, where the centralized teacher provides additional supervision on states visited by the current student. This refinement improves closed-loop execution in our evaluations. Extensive experiments show that our dataset achieves a mean valid wrist contact rate 41.10 percentage points higher than that of the evaluated CORE4D subset. CoWBC achieves a safe success rate of 80.18%. Our https://iclr-visualization.pages.dev/project website showcases the workflow and visualization results, making them accessible to the community for future research on humanoid cooperation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Cooperation, Whole Body Control, Humanoid Robot

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YEPkOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whole-body cooperative control for two humanoid robots performing unconstrained shared-object transport. The core claim is: reconstruct human dyadic cooperative motions from multi-view video, retarget them to paired G1 robot references while preserving relative geometry and temporal alignment; train a centralized Teacher with readiness-gated cooperative rewards and a three-stage curriculum; distill into decentralized Student policies that use a shared base policy plus role-specific residuals, augmented with lightweight learner-state refinement to mitigate closed-loop distribution shift. The paper also releases a paired demonstration dataset and achieves an 80.18% success rate in simulation, with a mention of hardware human-robot collaboration.

Critical strengths
  1. Problem is interesting and the data pipeline is solid.​ The paired demonstration construction, coordination-preserving retargeting, and multi-metric evaluation are careful and useful contributions.
  2. Ablations are thorough within the proposed framework.​ The removal of learner-state refinement, role residuals, and staged training all produce significant drops, suggesting these components matter within CoWBC.
Critical weaknesses
  1. The paper fails to justify why the proposed method is necessary.​ The central question is: can a single-robot object-transport tracking policy (trained on single-humanoid box-carrying demonstrations, e.g., SceneBot and OmniContact, and OmniContact seems to be open sourced) be deployed zero-shot on two humanoids, each treating the other as part of the environment? Intuitively, if each robot simply tracks the object's motion while maintaining its own balance, the pair might successfully transport the object without any explicit cooperation mechanism. Without this comparison, readers cannot judge whether CoWBC's complexity is justified.

  2. Hardware validation claim does not match the experimental body. The abstract and conclusion mention "hardware human-robot collaboration," yet all main tables (Tables 2, 3) report only simulation results. The conclusion itself states "dual-robot hardware validation remains future work." This mismatch inflates the perceived impact. Real dual-robot results (even with small sample size) or a downgraded claim are needed. Besides, the authors does not provides video results.

  3. Cooperation metrics are geometric proxies, not physical measures. Grasp Synchronization Error, Dual Support Ratio, and VWCR are all based on wrist-to-anchor distances. The paper acknowledges they do not measure contact force. Yet Safe Success and Cooperation Pass Rate implicitly assume geometric symmetry equals good load sharing. In practice, two robots may appear coordinated while exerting opposing forces that destabilize both. Without wrist torque or contact force estimates, the claimed "cooperation" remains unvalidated.

Questions for authors
  1. Have you evaluated a single-robot tracking policy (e.g., SceneBot, OmniContact) deployed zero-shot on two G1 robots? If yes, what were the Safe Success, Fall Rate, and Drop Rate? If not, why was this obvious baseline omitted?

  2. If you did try it and it failed, what were the dominant failure modes? Does CoWBC's learner-state refinement, role residual, or readiness gating specifically address those failures?

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1)

We thank Reviewer YEPk for the thoughtful review and for highlighting the strengths of our paired demonstration pipeline, coordination-preserving retargeting, and ablation studies. We appreciate your suggestion to replenish a single-robot policy on two robots and we will address this baseline, the scope of our hardware validation, and the interpretation of our cooperation metrics below.

Response to Weakness 1 and Questions 1–2: Can a single-robot transport policy perform the same task when deployed zero-shot on both G1 robots? What does this comparison reveal about the need for CoWBC’s coordination mechanisms?

Thank you for identifying this missing comparison. We have evaluated a frozen single-robot policy deployed independently on two G1 robots. We will include this baseline and the corresponding failure analysis in the paper.

The single-robot policy was trained with only one robot manipulating the object. During dual deployment, the other robot also actively pushes and lifts the same box. This changes both the effects of each robot’s actions and the loads it experiences. Both policy instances still correct errors according to their single-robot training. Without cooperative training, one robot’s corrections conflict with the other robot’s transport and balance requirements.

CoWBC learns cooperation in the physically coupled dual-robot environment. Readiness gating and staged Teacher training establish the prerequisites for joint transport. Role-specific residuals learn local corrections for different postures and contact conditions. Learner-state refinement provides corrective supervision at off-reference states visited by the Student. This improves recovery from execution errors.

The corresponding ablations in Table 3 of our paper also support these designs. Readiness gating, role-specific residuals, and learner-state refinement improve Safe Success by 10.52, 9.32, and 43.78 percentage points, respectively. These results support their task-level effectiveness.

1. Single-robot policy and independent dual deployment

In a representative failure, the two robots exerted opposing vertical forces on the box. One robot then lost contact, leaving the other unable to support the load. The box fell, and the remaining robot lost its balance. A video of this failure is available on our project website rebuttal's part4 https://iclr-visualization.pages.dev/#rebuttal. This case illustrates how independently deployed single-robot policies can fail to maintain stable joint support when coupled through a shared object.

Specifically, to compare a single-robot policy with its extension to two-robot cooperation, we use a PPO-trained single-G1 whole-body box-transport policy. For dual deployment, we apply the same frozen policy separately to each G1. Each robot constructs its input from its local state, assigned motion reference, and the shared object’s state and plan. Each robot then computes its joint targets independently. The two reference clocks are synchronized.

We evaluate a policy trained using the OMOMO dataset in a cooperative transport setting. Each of the three evaluation seeds covers all 19 scenarios. As shown in following table, it reports mean rates across the three seeds. All rates are expressed as percentages and rounded to two decimal places. There is no dual-robot fine-tuning. Neither policy receives teammate states or actions. The robots do not exchange actions. They interact physically through a shared object.

PolicySafe SuccessFall RateDrop Rate
Frozen single-robot policy × 2, object-center adapter0.00%100.00%5.26%
CoWBC84.21%0.00%15.78%
CoWBC without learner-state refinement42.11%36.84%21.05%
AuthorsOpenReview ↗

Rebuttal by Authors (Part 2)

Weakness 1 and Questions 1–2 (continued)

2. Failure mechanism and analysis of a representative episode

The key difficulty is that the shared object couples the robots’ feedback loops, however, each robot still corrects errors independently using its single-robot policy. Each action affects the shared object and the other robot’s contact forces; a common trajectory alone cannot ensure coordinated load sharing or balance. Stability learned in a single-robot setting therefore does not automatically transfer to the coupled dual-robot system.

To examine this process, we selected a representative dual-deployment episode and measured contact forces and motion states, which is shown in rebuttal's part4 https://iclr-visualization.pages.dev/#rebuttal. We also tested a single-robot control with the same checkpoint. It lifted the box in its original task by 1.27 m and finished upright. Dual deployment also initially lifted the box. However, it failed to maintain stable shared support.

During 1.0–1.3 s, the robots applied opposing vertical forces (+17.71 N and −8.96 N), causing one robot’s contact force to counteract the other’s lifting effort. During 1.655–2.375 s, the second robot lost contact with the box, leaving the first robot to support it alone; its mean upward force during 1.8–2.1 s was insufficient (3.63 N versus the 9.81 N weight), and the box accelerated downward. During 2.12–2.28 s, the box hit the floor and the first robot subsequently lost balance, illustrating the coordination and load-sharing challenges that can arise when extending single-robot policies to physically coupled dual-robot transport.

This episode illustrates a possible limitation of independent local tracking under shared-object coupling. It does not imply that all failed episodes follow the same sequence.

CoWBC addresses these challenges through three designs. Readiness gating and staged teacher training establish the conditions for sustained transport. Role-specific residuals learn local corrections for different contact conditions and postures. Learner-state refinement improves recovery from execution deviations.

3. Targeted learning designs and ablation evidence

CoWBC first learns cooperation in a physically coupled dual-robot environment and then distills the Teacher’s skills into Student policies for decentralized execution. Paired references provide consistent relative geometry and timing, while a centralized teacher learns coordinated actions under shared-object contact and robot-balance constraints. The ablations in Table 3 use different Teacher or Student baselines, so their gains should not be added together. The detailed analysis is as follows.

(a) Readiness gating and staged training: organizing objectives around skill prerequisites

Readiness gating and staged training align reward weighting and learning order with the prerequisites for cooperative transport. Gating adjusts reward weights according to body-tracking quality and bilateral readiness, while staged training progresses from stable positioning to transport and posture/robustness refinement, helping both robots establish joint support before learning more demanding skills. As shown in Table 3 in the main paper, adding gating to end-to-end Teacher training increases Safe Success from 60.53% to 71.05%, while reducing Fall/Drop Rates from 38.60%/8.77% to 28.51%/3.51%. Staged training under the same gated objective further improves these metrics to 80.30% Safe Success and 18.00%/1.80% Fall/Drop Rates, supporting the value of cooperative readiness and ordered skill acquisition.

(b) Shared base and role-specific residuals: representing shared skills and role differences

The shared base captures common control skills, while role-specific residuals provide local action corrections, allowing both robots to accommodate different postures and contact conditions. As shown in Table 3 in the main paper, adding role-specific residuals increases Safe Success from 70.86% to 80.18% (+9.32 percentage points) and Cooperation Pass from 63.60% to 82.60%, while reducing Fall/Drop Rates to 18.54%/1.66% and Wrist-to-Anchor Distance Asymmetry from 0.0589 m to 0.0487 m. These results support role-specific adaptation for cooperative transport.

(c) Learner-state refinement: extending supervision to Student-visited states

Learner-state refinement queries the Teacher at Student-visited states to reduce the mismatch between demonstrations and closed-loop execution. Because either robot’s errors alter the shared object’s motion and its partner’s interaction conditions, this corrective supervision helps both robots recover from off-reference states. As shown in Table 3 in the main paper, adding refinement increases Safe Success from 36.40% to 80.18% (+43.78 percentage points) and reduces Fall/Drop Rates from 62.28%/25.88% to 18.54%/1.66%, supporting its effectiveness in improving closed-loop cooperative execution.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 3)

Response to weakness2:What video evidence supports the hardware claims?

  • Thank you to the reviewers for their questions. We have added real-world deployment experiments demonstrating two robots jointly transporting a table while maintaining stable motion under the same fully unconstrained conditions. In response to your inquiry regarding the collaborative task between the two robots, please find attached a video demonstrating the two robots physically moving a table. You can also view the rest of the videos on our project website rebuttal's part5 https://iclr-visualization.pages.dev/ . Our framework supports different forms of cooperation, including human–robot collaboration, collaboration between robots of the same model, and collaboration between robots of different models.
AuthorsOpenReview ↗

Rebuttal by Authors (Part 4)

Response to Weakness 3: What can geometric cooperation metrics establish about load sharing or opposing forces without direct force or torque measurements?

We thank the reviewer for highlighting the distinction between geometric coordination and physical load sharing. We agree that wrist-to-anchor distances do not directly measure contact forces or load sharing.

We also clarify that Cooperation Pass is a geometric coordination diagnostic, not a prerequisite for Safe Success (Appendix B.4). Safe Success evaluates task-stage completion and safety outcomes.

We define adverse interactions in relation to the task and motion phase. These are interaction loads that hinder the robots’ balance, effective object support, or intended motion. Opposing gripping forces and ordinary load-bearing forces are not inherently adverse. We therefore interpret measured loads together with the robots’ execution states.

3.1 Do both robots physically support the load, rather than merely appear geometrically coordinated?

We evaluated the Student policy in 200 simulated episodes across 19 motion conditions, with box masses of 0.95–1.05 kg and friction coefficients of 0.45–0.55. The box followed simulated dynamics throughout execution, and we recorded all robot–box contact forces and Fall/Drop events.

Overall, 162/200 episodes achieved Safe Success. Airborne intervals supported exclusively by robot contacts occurred in 188 episodes, including 162 successful and 26 failed episodes. The remaining 12 episodes contributed only to overall task results.

We computed airborne support statistics per eligible episode and averaged them equally, including failures. Mean signed vertical support was 0.480mg0.480mg for Robot 1 and 0.503mg0.503mg for Robot 2. Both robots simultaneously provided positive vertical support during an average of 99.50% of airborne time. These measurements confirm that both robots physically supported the load, although similar average contributions do not imply equal instantaneous load sharing.

Physical diagnosticResult
Episodes evaluated200
Episodes with airborne intervals supported only by robot contacts188
Mean signed vertical support force: Robot 1 / Robot 20.480mg0.480mg / 0.503mg0.503mg
Fraction of airborne time when both robots simultaneously provide positive vertical support99.50%
Fraction of airborne time when both robots simultaneously provide upward support greater than 0.4mg0.4mg each98.00%

3.2 Can geometric coordination conceal destabilizing interactions?

We next examine interaction loads that may impair the robots’ balance or object support. We assess each robot’s loading alongside its posture, support state, and failure events.

Experimental design

We added a per-robot loading analysis based on simulated contact-force records. We computed the external moment of box contact forces about each robot’s whole-body center of mass (CoM). We examined these moments alongside base-orientation tracking errors, vertical support forces, and Fall/Drop events. The whole-body CoM was calculated from the link CoMs, weighted by their actual masses.

This analysis preserves each robot’s individual loading information. Even when the robots’ moments acting on the box cancel, their individual loads remain observable. These quantities are external contact moments, not wrist-joint torques.

Results and analysis

Physical diagnosticSuccessful episodes (n = 162)Failed episodes (n = 26)
External contact moment about Robot 1’s whole-body CoM (N·m)18.86 ± 6.0127.94 ± 12.59
External contact moment about Robot 2’s whole-body CoM (N·m)19.42 ± 6.3431.90 ± 13.26
Larger base-orientation tracking error across the two robots (°)3.86 ± 1.2923.25 ± 9.91

Failed episodes had higher mean episode-level P95 external moments for both robots (27.94/31.90 N·m, versus 18.86/19.42 N·m in successful episodes), alongside a higher mean episode-level P95 of the larger orientation error (23.25° versus 3.86°).

Within the 0.5 s before the first Fall/Drop event, combined vertical support decreased from 0.972mg0.972mg to 0.740mg0.740mg between the first and second halves of the window, while the larger base-orientation tracking error increased from 7.76° to 27.66°. These statistics are within-episode temporal means averaged equally across failed episodes, rather than P95 values, and show concurrent support loss and growing posture deviations before failure.

Together with the bilateral support measurements above, these results provide physical evidence beyond geometric coordination. They confirm that both robots physically contribute to supporting the object. They also reveal loading patterns associated with weaker support and larger posture deviations. We therefore assess cooperation using contact forces, individual loading, and execution outcomes.

Reviewer_ytR1OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work tackles of the problem of "cooperative transport with humanoids" - e.g., training humanoid policies that perform two-person lifting (and moving) of boxes with a human collaborator. This is challenging since the humanoid must (in principle) watch the human collaborator in real-time and anticipate their actions. The overall pipeline follows a real2sim2real strategy; first by assembling multi-view datasets of real human-to-human two-person lifts, which are then retarged to humanoids to serve as privleged teachers in simulation. These are then distilled to deployable student policies. Importantly, policies can play different roles in the collaboration, such as the leader or the follower. The resulting approach produces policies that keep objects in contact with the humanoid 2X more often that prior art, and produce success rates of 80 percent (compared to prior art of ~70 percent).

Critical strengths
  • To me knowledge, this is the first real-world demonstration of cooperative humanoid-human behaviours such as box-lifting and moving.
  • This was originally a weakess or confusion, but it may in fact be a strength; I had a fundamental confusion about the student policy and what information it has access to. Line 338 states that the student policy "observes neither teammate proprioception or actions nor robot-motion references". Thus it seems that coordination is achieved implicitly via the object state, which the student does have access to. This is an potentially fascinating design choice - firstly, is object state provided via mocap? I hope the authors can elaborate on the how coordination is achieved in their system.
Critical weaknesses
  • The training simulations are always of 2 humanoids of the same size, but it is clear in the deployment that the G1 and humans are quite different in size. This should make collaborative box-lifting much harder - how is this handled?

  • From the videos, it looks as if the human is lifting the box on their own, and the humanoid is lightly touching the box without offerign any support. This also might happen in real-world multi-person lifts; not everyone might be pulling their weight. Can the actual force production of the humanoid be measured, to verify its actually helping?

  • I'm confused by the goal of Table 1; is this evaluating a dataset, or a method? I think this is evaluating a dataset, but the caption uses terms like "downstream performance". Because the real-world data presumably contained contact and no slipping, this table is evaluating the quality of the annotations. But I assume "ours" is not the kinematic annotations, but rather dynamically feasible rollouts fit to kinematic targets. From this perspective, couldn't the same approach be applied to CORE4D's kinematic annotations?

  • It looks like the leader/follower roles do not help much (Table 2).

Questions for authors

Please see questions above.

2 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1)

We thank Reviewer ytR1 for the thoughtful review and for highlighting our human–robot hardware demonstration and the use of shared object state for implicit coordination. Below, we explain how the robot obtains and uses object state, and address your questions about differences in body size, the robot’s contribution to lifting, the evaluation in Table 1, and the benefits of role specialization.

Discussion with strength2:How is the shared object state obtained, and how does the student policy use it to coordinate without directly observing its partner?

Thank you for your comment. Your understanding is correct: each student policy observes its own proprioceptive state and the shared object reference, without access to its teammate’s state or actions.

Our task is formulated as cooperative whole-body control (WBC). Conventional tracking-based WBC relies on time- or phase-aligned reference trajectories specifying desired joint configurations, root poses, and body kinematics. For object interaction, these references may additionally include object poses, contact timing, and contact locations. In our cooperative setting, we use object-level references to communicate the desired transport motion and guide coordination between the multiple robots. Under limited observations, each robot must generate whole-body motions compatible with its teammate while maintaining balance and jointly supporting the object. This need for coordinated execution without directly observing the teammate is a central challenge of the unconstrained cooperative transport setting. A shared object reference provides a practical task interface, allowing the desired transport motion to be specified at the object level.

Such object references can be obtained through three approaches:

  • Motion capture with a high-level planner. Motion capture provides real-time object pose measurements, which a high-level planner, potentially incorporating a large model, can use together with the task goal to generate future reference trajectories. This approach requires a calibrated capture volume.
  • Onboard vision with a high-level planner. Onboard visual observations can support object-state estimation and, together with a task goal, inform a multimodal model or planner that generates future references. This avoids an instrumented workspace, although occlusion, limited field of view, and camera motion can affect estimation reliability.
  • Simulator rollouts. Object trajectories extracted from simulation can serve as references without introducing an additional high-level planning component. This is the approach adopted in our current setup.

We use simulation-derived references to isolate the WBC problem from high-level trajectory generation, such as large-model-based planning, and its potential errors. Specifically, the reference object trajectory in our hardware experiments is supplied as a shared task command within each policy’s observation. Its goal and waypoints describe the desired object motion; they do not provide real-time measurements of the physical box during transport. The physical object may therefore deviate from the reference during execution. To accommodate such discrepancies in hardware deployment, we perturb the reference during training so that the policies learn to operate with imperfect reference guidance.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2)

Respond with weakness1:Human–robot size differences. How does the policy handle size differences between a human and a G1 when it was trained with two identical robots?

  • Thank you for raising this point. To test whether our approach can handle a substantial size mismatch, we trained a policy in simulation with a smaller G1 and a larger H2. As shown in the rebuttal video in rebuttal's part1 https://iclr-visualization.pages.dev/#rebuttal, the two robots use different body postures to jointly lift, carry, and set down the box. At the method level, our formulation does not assume identical robot morphology: each agent uses its morphology-specific kinematic and actuator model, while coordination is learned through shared object-centric objectives and bilateral interaction constraints. This enables the policy to learn asymmetric joint configurations and complementary lifting behaviors rather than requiring the two agents to execute geometrically identical motions. These results demonstrate that our approach can learn coordinated lifting despite a substantial difference in body size. As shown in the video, robots of different heights adjust the elevation of their hands and the degree of flexion in their legs in noticeably different ways to maintain the balance of the box.

Respond with weakness2:Can the robot’s actual force contribution be measured to verify that it helps lift the object in the hardware demonstration?

  • We thank the reviewer for this observation. We agree that the contact observed in the video alone is insufficient to determine whether the robot actually bore the load. To address this, we extracted frame-by-frame robot–box contact forces from the sim2sim simulation.
  • In addition, we have included a video of a real-world scenario showing human-robot collaboration in moving a long box, as a qualitative demonstration of the robot’s role in carrying the load. The video concludes by showing that a single person cannot lift the box on their own; However, the contact forces in this real-world scenario have not yet been directly measured.
  • Simulation-based stress analysis videos and specific videos of the actual machine are available via the links on our project website rebuttal's part3 and part5 https://iclr-visualization.pages.dev/#rebuttal .

Respond with weakness3:What does Table 1 evaluate, and can the same processing pipeline be applied to CORE4D’s kinematic annotations?

  • We thank the reviewer for this question. Table 1 compares datasets rather than methods. We include “downstream performance” to demonstrate that, under the same pipeline, our data achieve a higher retargeting success rate, thereby validating their quality. Since only successfully retargeted data can be used for training, this metric also demonstrates the value of our data for downstream applications.
  • Specifically, for the "physical quality" and "cooperative quality" metrics, we directly evaluate the motion data from both datasets after data collection and before downstream retargeting. To clarify, “Ours” in the physical and cooperative quality columns refers to the kinematic annotations in our dataset, not to dynamically feasible rollouts fit to kinematic targets. We then apply exactly the same retargeting pipeline to both CORE4D and our dataset and compare their retargeting success rates (RSR). I will correct and emphasize this section in the revised version. Thank you for your constructive feedback.

Respond with weakness4:How much benefit do the leader and follower roles provide in the results reported in Table 2?

  • Thank you for this comment. Table 2 compares overall task performance across the adapted baselines, our local Student, and the centralized Teacher, rather than isolating the contribution of role-specific policy components. May we confirm whether “leader/follower roles” refers to the role-specific residual branches?
  • If so, Table 3 provides the relevant controlled comparison. Compared with the Parameter-Matched Shared Residual variant, Full CoWBC improves Safe Success from 68.64% to 80.18% (+11.54 percentage points) and Cooperation Pass from 70.63% to 82.60% (+11.97 percentage points). This comparison supports the benefit of role-specific adaptation under a matched parameter budget.
    Does this address your concern, or were you referring to a different aspect of the role design?
Reviewer_6ua8OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the problem of collaborative object lifting of humanoid robots. The major idea is to leverage reference motion trackign to learn centralized multi-agent policies for task execution then distill them into decentralized student policies, and further refined with MAPPO. The authors show improvement in object-lifting related metrics.

Critical strengths
  • The task of collaborative object lifting for humanoid robots is an important problem study with many real-world application implications.
Critical weaknesses
  • The major concern of this paper is that the major contribution of this paper is implementing the task of object-lifting for humanoid robots. Its learning piplines are largely adopted from existing general motion trackers, and collaborative multi-agent RL for physical characters (which is a slightly easier embodiment). Given this observations, the overall contribution of this paper is more attractive to robotic conferences, and the current design provide limited technical insight for model learning.

  • Compared with works like TeamHOI which discuss the problem of varying number of collaborators, this method mainly tracks the retargeted motions and seem to have limited generalizability to other multi-agent scenarios.

  • The paper left a lot of implementation details (e.g. MAPPO) to the appendix which makes it difficult to have an in-depth understanding given only the main text.

Questions for authors

See the weakness section.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1)

We thank Reviewer 6ua8 for the thoughtful review. Below, we clarify CoWBC’s technical contributions, how its scope differs from TeamHOI’s, and the training details you raised.

Respond with weakness1:Technical contribution. What does CoWBC contribute beyond existing motion tracking and multi-agent reinforcement learning methods?

  • Thank you for raising this concern. We would like to clarify a misunderstanding regarding the scope of our work: CoWBC addresses unconstrained cooperative humanoid-object interaction. To the best of our knowledge, this setting has not been widely explored, despite its importance for enabling humanoids to cooperate in everyday human environments. Reviewer ytR1 also recognized this contribution, noting: “To me knowledge, this is the first real-world demonstration of cooperative humanoid-human behaviours such asbox-lifting and moving.” CoWBC provides a systematic exploration spanning paired demonstration data, cooperation-specific learning, and real-robot deployment. Our contribution goes beyond simply integrating existing techniques. We introduce targeted innovations to address the specific challenges of cooperative manipulation.
  • We study this as a physically coupled whole-body control problem. Cooperative transport is a representative setting because it requires coordinated locomotion and object manipulation while maintaining individual balance and stable support of the shared object. To systematically characterize this setting, we organize our demonstrations by the shared object’s spatial trajectory, distinguishing horizontal translation, yaw rotation, height variation, and their pairwise combinations. This task-space taxonomy covers the motion categories considered in our study and provides a consistent basis for dataset construction and evaluation, capturing both individual motion components and the coordination demands introduced by their combinations. Either robot’s execution errors alter the shared object’s motion and consequently its partner’s interaction conditions, making mutually compatible whole-body actions essential.
  • To our best konwledge, high-quality two-person demonstrations suitable for simulation-based training are scarce in the literature, so we built a reconstruction and retargeting pipeline to generate paired motion references that preserve relative placement, contact geometry, and timing. These references guide policy learning for dynamically stable cooperative transport. The resulting paired cooperative demonstration dataset is itself a core contribution of CoWBC.
  • Our central methodological contribution is object-centered, role-decomposed policy learning for decentralized cooperative whole-body control. The shared base learns common control skills, while history-conditioned, role-specific residuals accommodate complementary grasp geometries and asymmetric interaction requirements. This decomposition allows the two policies to share control structure while producing different whole-body actions appropriate to their roles. MAPPO trains an additional bounded residual with a centralized critic while keeping the distilled Student frozen, enabling cooperative refinement around the learned behavior. Controlled ablations substantiate these contributions (Table 3): replacing role-specific residuals with a parameter-matched shared residual reduces Safe Success from 80.18% to 68.64%, and the final MAPPO refinement improves it from 76.58% to 80.18%. These results demonstrate the importance of preserving role-dependent action adaptation within shared policies and covering the states induced by their coupled execution—concrete learning considerations underlying reliable cooperative whole-body control.
AuthorsOpenReview ↗

Rebuttal by Authors (Part 2)

Respond with weakness2:How does CoWBC differ from TeamHOI in the coordination problem it addresses?

  • Thank you for this important comment. We agree that TeamHOI provides stronger empirical generalization along the team-size axis, since it evaluates a shared policy with varying numbers of collaborators. CoWBC currently focuses on two collaborators and does not claim experimentally validated zero-shot generalization to larger teams. However, the two approaches emphasize different dimensions of generalization: TeamHOI focuses on the number of agents, whereas CoWBC focuses on heterogeneous embodiments, asymmetric collaborative roles, partner types, and diverse shared-object motions.
  • Methodologically, TeamHOI primarily uses a shared policy to support varying numbers of collaborators, whereas CoWBC explicitly models the robots and shared object as a coupled cooperative system. Our controller combines holistic object-level coordination, a shared base policy for common cooperative behaviors, and agent-specific residuals for morphology- and role-dependent adaptation. This decomposition preserves global transport consistency while allowing collaborators to execute complementary, asymmetric motions. For example, during cooperative box rotation, the inner agent follows a smaller-radius trajectory, whereas the outer agent must take larger and faster steps; Robots of different sizes also require different postures and lifting heights. The shared base policy captures the common coordination pattern, while the residuals account for these individual differences. Thus, compared with TeamHOI’s emphasis on team-size generalization, CoWBC is better suited to heterogeneous embodiments and asymmetric cooperative behaviors.
  • Our training distribution covers the full set of shared-object trajectory modes considered in cooperative transport, including multidirectional planar translation, rotation, lifting and lowering, and their temporal combinations. In terms of observation, TeamHOI requires explicit teammate states, including planar positions, headings, and angular positions around the table center. CoWBC instead relies only on each robot’s proprioception and the shared-object state, avoiding direct partner tracking and enabling the same policy interface to collaborate with either robots or humans. In terms of contact modeling, TeamHOI assumes a relatively structured team arrangement and contact configuration around the object. CoWBC does not require preassigned contact regions or fixed collaborator placements, allowing different partners to establish and maintain contact according to their morphology, role, and current state.
  • Together, the broader object-motion distribution, partner-independent observation, and flexible contact formulation make CoWBC better suited to general unconstrained cooperative transport. Although our current experiments focus on two collaborators, extending the shared-base-plus-individual-residual architecture to variable and larger team sizes is an important direction for future work. We also adapted TeamHOI to our cooperative transport task and trained it on this setup. A video showing the resulting policy’s performance is available on our project website rebuttal’s part2:https://iclr-visualization.pages.dev/#rebuttal.
AuthorsOpenReview ↗

Rebuttal by Authors (Part 3)

Respond with weakness3:Training details. Can the main text explain key training details, particularly the use of MAPPO, more clearly?

  • We thank the reviewer for this helpful suggestion. In the revised manuscript, we will add the following description to Section 3.4 to clarify the motivation and implementation of the final MAPPO refinement stage.

  • Residual Cooperative Refinement with MAPPO. Distillation and learner-state refinement transfer the Teacher’s behavior through action matching. To directly optimize the Student’s closed-loop task return, we add a final stage of on-policy cooperative refinement in simulation using MAPPO (Yu et al., 2022), with centralized value estimation and decentralized execution. This stage accounts for the fact that both robots act on the same object, so an action error by one robot can change the conditions under which its partner acts.

  • Let at,Sia_{t,S}^{i} denote the action produced by robot ii’s Student after learner-state refinement. We freeze the entire Student, including its shared base and internal role-specific residual branches. A separate residual actor, shared across both roles, takes the local observation otio_t^i, the object history hto,ih_t^{o,i}, and the one-hot role encoding eie_i. During training, we sample a Gaussian latent variable and compute the executed action as follows:

ηti∼N(μϕ(oti,hto,i,ei),σ2I).\eta_t^i \sim \mathcal{N}\left(\mu_\phi(o_t^i, h_t^{o,i}, e_i), \sigma^2 I\right). ati=clip⁡(at,Si+αtanh⁡(ηti),−1,1).a_t^i = \operatorname{clip}\left(a_{t,S}^{i} + \alpha \tanh(\eta_t^i), -1, 1\right).

  • We set α\alpha to ALPHA_VALUE and fix the exploration standard deviation σ\sigma at SIGMA_VALUE. The scale α\alpha bounds the residual correction in normalized action space, with the frozen Student serving as the nominal controller. We initialize the residual actor’s mean output head to zero, so the initial deterministic policy reproduces the frozen Student’s actions. MAPPO updates only the external residual actor and the centralized critic Vψ(Otteam)V_\psi(O_t^{\mathrm{team}}). The critic estimates cooperative returns from privileged team observations during training.

  • At deterministic execution, we replace ηti\eta_t^i with its predicted mean and apply the same action transformation. The policy runs using local inputs, without the centralized critic or direct teammate observations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.