acceptodds
Under review as a conference paper at ICLR 2027

MASkillBlender: Decentralized Whole-Body Coordination for Multi-Humanoid Loco-Manipulation via Skill Blending

Abstract

Coordinated multi-humanoid loco-manipulation is promising yet challenging due to high-dimensional whole-body control, decentralized decision making, and scalability. While recent reinforcement learning methods have improved single-humanoid whole-body control, extending them to the multi-humanoid setting remains nontrivial and often requires substantial reward engineering or task-specific design. We propose MASkillBlender, a general multi-agent reinforcement learning framework to achieve decentralized multi-humanoid whole-body coordination. By learning a shared decentralized high-level policy over reusable pre-trained single-humanoid skills, MASkillBlender enables coordinated behaviors using only task-level rewards, without requiring task-specific motion references. To improve learning efficiency, we further introduce a permutation-based data augmentation strategy for homogeneous multi-humanoid systems, and theoretically show that the permuted samples preserve the policy-gradient direction of the original samples under the homogeneous Markov game formulation. We evaluate MASkillBlender on multiple multi-humanoid coordination tasks across two humanoid embodiments. Simulation results demonstrate that the proposed framework consistently achieves strong task performance and enables coordinated behaviors across different tasks and humanoid embodiments.

Primary area:
reinforcement learning
Keywords:
multi-agent reinforcement learning, decentralized multi-humanoid coordination, loco-manipulation, skill blending, data augmentation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4NQnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper studies cooperative loco-manipulation with multiple humanoid robots. A shared high-level policy blends frozen skills such as walking, reaching, and squatting, while permutation-based data augmentation exploits symmetry between agents during training. The method is evaluated in simulation on cooperative carrying, pushing, and navigation tasks with H1 and G1 robots.

Critical strengths

The framework effectively reuses pretrained humanoid skills to learn decentralized coordination using task-level rewards, without requiring task-specific reference motions. It addresses the relatively underexplored problem of cooperative loco-manipulation with multiple humanoid robots.

Critical weaknesses

Experimental evidence and attribution of improvements.

In the object-interaction tasks, the gains over the centralized SkillBlender baseline are modest. Moreover, removing permutation augmentation reduces these rates to 6% and 76%, respectively, both below SkillBlender. These results leave the contribution of the decentralized MARL formulation unclear. The comparison with TeamHOI requires further clarification. Its motion-prior-based formulation offers greater behavioral flexibility than a fixed library of pretrained skills, and the original method demonstrates loco-manipulation with up to eight agents. I agree that blending pretrained skills may be better suited to robotic deployment, but this potential advantage remains unverified without real-world experiments.

Furthermore, the very low baseline success rate in the three-agent Move task may reflect difficulties in transferring a character-control method to a robotic embodiment, or differences in basic collision-avoidance behavior, rather than a broader advantage in multi-agent coordination. Further validation is needed to distinguish these explanations, for example by evaluating the adapted baseline on single-agent navigation and multi-agent tasks with and without intersecting paths.

Questions for authors
  1. Can the framework coordinate robots with different embodiments?
  2. Can it scale to larger number of robots?
  3. Could you clarify whether advantages are reused or recomputed for augmented samples, and whether critic-only augmentation or a permutation-invariant critic would yield similar gains?
  4. Have you considered evaluating the method on real robots? A successful demonstration would substantially strengthen the contribution.
Reviewer_qB4GOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MASkillBlender addresses decentralized whole-body coordination for homogeneous humanoid teams. It freezes pre-trained single-humanoid primitive policies, including Walking, Reaching, and Squatting, and learns a shared high-level policy that produces skill goals and joint-wise blending weights. The high-level controller is trained with MAPPO and a centralized critic, while each humanoid executes the shared policy using its local observations. The downstream training uses task-level rewards without task-specific motion references; the primitive skills themselves are pre-trained. The paper also introduces permutation-based data augmentation for homogeneous teams. Under the stated homogeneous Markov game assumptions, it proves invariance of the true value functions and equality of the original and consistently permuted samples' joint-policy surrogate gradients.

Experiments cover two-humanoid Carry and Push and three-humanoid Move tasks in Isaac Gym, with comparisons to a centralized adaptation of SkillBlender and a reference-based TeamHOI adaptation. Additional evidence includes augmentation and primitive-skill ablations, simulated observation noise and delays, scripted long-horizon compositions, H1 box exchange, MuJoCo transfer, and separately trained G1 policies. The method performs well on the tested tasks, with a particularly large advantage on Move. No hardware experiments are reported.

Critical strengths

S1. A useful and reasonably complete system for a challenging control problem. Coordinating multiple full-body humanoids through local policies is practically relevant and substantially more involved than low-dimensional cooperative benchmarks. Reusing frozen single-humanoid skills offers an effective control prior while allowing task-level learning of coordination. The demonstrations span object carrying, pushing, and collision avoidance, and the H1/G1 experiments show that the framework can be instantiated on two embodiments. The noise/delay tests and MuJoCo transfer provide additional evidence beyond nominal Isaac Gym evaluation, although these remain simulation results and the embodiments are trained separately.

S2. Strong performance on Move and informative component ablations. Table 1 reports 100% success on Move compared with 10% for the centralized SkillBlender adaptation and 0% for the adapted TeamHOI. This is a consequential result in favor of the proposed system. The augmentation ablation also reports consistent improvements across tasks, including a Carry success-rate increase from 6% to 100% (Table 4), while primitive-removal experiments support the importance of the skill prior (Figure 5). These results justify taking the approach seriously, even though further controls are needed to explain the augmentation gains and establish their robustness.

Critical weaknesses

W1. The principal methodological contribution is not sufficiently isolated: the theory establishes invariance, but the source of the large augmentation benefit remains unclear. Theorem 2 proves that, under its assumptions and using the true advantage function, an original sample and its consistently relabeled counterpart have exactly the same joint-policy surrogate gradient. This supports the validity of the transformation, but does not establish improved sample efficiency, reduced gradient variance, or better optimization. In the idealized shared-policy objective, the transformed sample repeats the same gradient contribution rather than providing a different one.

The implementation instead uses a learned centralized MLP critic, estimated advantages, and MAPPO (Section 4.3 and Appendix F.4). A plausible source of improvement is enforcing the appropriate symmetry on an otherwise order-sensitive critic, which could improve subsequent advantage estimates. This would be a useful mechanism, but it is not isolated by the current ablation. In particular, the very large Carry gap in Table 4 (6% versus 100% success) calls for a clearer explanation. The paper does not report controls that distinguish symmetry enforcement from repeated sample use or changes in effective optimization budget. Appendix F.4 states that all permutations are used, making those details consequential. A matched-budget duplicate-sample control and a critic-only augmentation or permutation-invariant-critic control would directly test the claimed mechanism. I do not regard this as evidence that the theorem is false or that improvement is impossible; the concern is the gap between what is proved and what explains the empirical contribution.

W2. The comparative evidence is not yet sufficient to establish a stable advantage across tasks. Section 5.2 reports only two independent training trials per task and evaluates the best learned policy of each method over 50 rollouts. The latter assesses execution variability of a selected policy, rather than training-to-training robustness. In Table 1, the success-rate differences against SkillBlender are small on Carry (100% versus 98%) and Push (86% versus 84%): each corresponds to one additional successful rollout out of 50. These results do not establish a robust improvement on those tasks, even though the Move advantage is substantial. Evaluation across independent training seeds, with uncertainty that reflects both training and execution, would materially strengthen the conclusion.

The TeamHOI comparison also requires care in interpretation. Appendix F.4 describes motion retargeting, new or adjusted pushing references, fixed task-wise Full/Masked AMP choices, and removal of explicit teammate-state inputs on Carry and Push. Matching task-level information is a reasonable comparison choice, and the adaptation is disclosed. Nevertheless, these changes mean the result evaluates a particular adapted configuration, leaving uncertainty about sensitivity to reference quality, observation design, and AMP/task-reward balance. This does not invalidate the comparison, but it limits how confidently the reported gap can be attributed to the proposed learning method. Together with W1, this leaves the methodological contribution and the breadth of the performance advantage insufficiently established for acceptance at present.

Questions for authors

Clarification requests

  1. In the actual MAPPO implementation, how are advantages and critic targets handled after permutation? Are the precomputed advantages reused unchanged, or recomputed from the permuted state? Which parts of the implemented actor update are covered by Theorem 2's joint-policy surrogate argument, and which depend on approximation or implementation choices? Please explain the expected mechanism behind the large Carry augmentation gain.
  2. When using all permutations, what changes in minibatch size, number of optimizer steps, sample presentations per update, and wall-clock cost relative to the no-augmentation ablation? Please distinguish environment-sample efficiency from additional optimization or computation.
  3. What criterion selects the “best policy” evaluated in Table 1? Please provide per-training-trial results, clarify what the reported mean and standard deviation summarize, and report uncertainty for success rates. How sensitive is the TeamHOI result to the changes in teammate observations, reference motions, and AMP/task-reward balance described in Appendix F.4?

Focused additional evidence that could change the assessment

  1. On an existing task, especially Carry, compare permutation augmentation with simple duplication of the same samples at a matched optimization budget, and with critic-only augmentation or a permutation-invariant critic. Report the result across additional independent training seeds if feasible. These targeted controls would directly address whether the improvement comes from symmetry-aware value learning, a distinct actor-learning effect, or increased sample reuse. They would be more decisive for my assessment than adding another task demonstration.

Future work, not a requirement for the current recommendation

Hardware deployment, heterogeneous teams, larger team sizes, and broader skill libraries are valuable extensions. I do not require a new real-robot system or a new benchmark suite for rebuttal. Claims about cross-embodiment generalization and long-horizon learning should remain consistent with separately trained embodiment-specific policies and the scripted approach/wait/execute composition used in the present experiments.

1 comment

Thank you for the detailed responses and the additional analyses and experiments. The rebuttal substantially addresses the main concern in my original review about the relationship between the permutation argument and the implemented learning procedure, and it strengthens the empirical evidence for the system.

  1. Theoretical scope and critic learning. The clarification that augmentation is applied only to the centralized critic is important. With stored advantages, agent permutations leave the shared actor’s averaged objective and gradient unchanged. Pairing permuted critic states with the original return targets instead provides a coherent way to exploit value-function symmetry. I therefore consider my main implementation and theoretical-scope questions substantially addressed. Under the stated mean-reduced loss and unchanged optimizer schedule, I also accept that exact duplication within each critic minibatch is algebraically redundant; a separate literal-duplication experiment is unnecessary. Please ensure that the algorithm, contribution statements, and discussion consistently reflect this critic-learning interpretation. The invariance result itself should not be presented as a guarantee of improved learning efficiency.
  2. Interpretation of the augmentation gains. The extended Carry result is particularly informative: the no-augmentation variant reaches 98.2% success at 60,000 iterations, whereas the augmented method reports 99.0% at 45,000 iterations. This changes the interpretation of the original large gap toward faster acquisition of the task under the reported training conditions, rather than a persistent inability of the unaugmented method to solve it. The preliminary PIC comparison on Move also supports the usefulness of exploiting critic symmetry, although it does not establish the precise mechanism behind Carry’s learning dynamics or a reliable advantage over PIC. The clarified budget—equal rollout sizes and eight optimizer steps per iteration, with more critic inputs processed under augmentation—supports distinguishing environment-sample efficiency from computation. Please retain that distinction in the claims.
  3. Evaluation and training variability. Evaluating each final policy over 500 episodes and reporting both training trials and their averages is a meaningful improvement. In particular, the reported two-trial average Move success rates of 97.1% versus 7.7% for the centralized SkillBlender adaptation strengthen the evidence beyond a selected-policy comparison. The remaining limitation is that there are still only two independent training trials. The standard deviation across five evaluation-seed means characterizes evaluation variability, rather than training-to-training uncertainty, and Table 1 still selects the better of two trials. I encourage making the two-trial averages prominent in the main results and explicitly labeling the source of uncertainty. I recognize the training cost and do not regard further independent runs as a prerequisite for this rebuttal, but claims of a robust advantage across tasks should remain appropriately qualified.
  4. Baseline validation and contribution. Restoring teammate conditioning and the attention architecture on Carry, together with successful single-humanoid navigation using the same retargeted references after reward adjustment, materially reduces my concern that the TeamHOI comparison rests on an unvalidated adaptation. The fixed Masked AMP choice and the single additional Carry run still limit the scope of this conclusion. The results support comparison with the evaluated adaptations, without establishing a general limitation of reference-based approaches. I also appreciate the reported scalar-blending control using the same primitive library; it provides useful, albeit single-run, evidence for the utility of joint-wise composition in this setting. Overall, the clearer critic-learning formulation, the targeted controls and baseline validation, and the reported two-trial results shift my assessment in favor of the paper’s practical contribution. The methodological novelty remains incremental and the training statistics remain limited, but my principal original concern has been substantially alleviated. Based on the clarifications and additional results reported in the rebuttal, I am revising my recommendation from 2 (Weak rejection) to 3 (Weak acceptance). My confidence remains 3.
Reviewer_Dy9cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MASkillBlender, a framework for decentralized whole-body coordination among homogeneous humanoid robots without task-specific motion supervision. It extends SkillBlender, a single-humanoid hierarchical control framework, to the multi-agent setting. The policy is trained using MAPPO with a centralized critic and executed independently by each humanoid using local observations. The paper also introduces agent-permutation data augmentation and proves that the corresponding idealized policy-gradient contribution is invariant under the stated homogeneous Markov game assumptions.

The experiments include two-humanoid Carry and Push tasks and a three-humanoid Move task. Comparisons are conducted against a centralized multi-humanoid adaptation of SkillBlender and an adapted TeamHOI baseline, with additional ablations of augmentation and primitive selection. Further evaluations cover H1 and G1 embodiments, localization and delay perturbations, staged long-horizon execution, and transfer from Isaac Gym to MuJoCo.

Critical strengths

The authors present a practical approach to the task of multi-humanoid-robot collaboration and object interaction. Notably, the proposed framework does not require motion references, which enables it to support such interactions with only a limited number of downstream task rewards and without motion-reference supervision; this is a valuable property. Improvements are reported across all three tasks, most notably in the move task. The results for carry and push further indicate that single-agent primitive skills remain applicable within a multi-agent framework, which suggests a promising research direction.

Critical weaknesses

1. The distinctive contribution of the method and its underlying mechanism are not yet sufficiently substantiated.

The parameterization of skill goals, the per-joint blending scheme, and the use of a frozen primitive library are all inherited from SkillBlender [1]. CooHOI [2] has already enabled multiple simulated humanoid characters to learn collaborative manipulation by exploiting object dynamics through a shared policy and MAPPO, although it fine-tunes a motion-prior-based single-agent policy rather than blending frozen RL-pretrained primitives. In addition, MAPush [3] and decPLM [4] are relevant precedents for quadruped teams, and neither is discussed in the manuscript. Both train a shared decentralized policy with MAPPO on top of a frozen locomotion controller to perform collaborative manipulation.

The existing experiments do not sufficiently isolate this contribution. The most evident comparative advantage appears on Move, but this task uses only a single primitive, Walking (Table 2). In this case, every per-joint blending weight is identically 1, so the task does not test non-trivial skill blending. The skill-removal ablation (Fig. 5) shows that the removed primitives (Squatting for Carry, Walking for Push) are important for learning performance. However, it does not establish whether per-joint blending outperforms other hierarchical methods with comparable motor capabilities.

The permutation theorem likewise does not explain the source of the improvement. For a shared local policy, consistently permuting observations, actions, stored advantages, and old-policy log-probabilities merely rearranges the same set of actor loss terms. Under consistent mean reduction and normalization, the averaged actor optimization objective remains unchanged. Algorithm 1 computes advantages before augmentation and applies augmentation within each sampled minibatch, so this issue is directly relevant to the training procedure described in the manuscript. Section 4.3 explicitly trains the centralized critic on original and permuted states with the same return targets, which may improve subsequent advantage estimation; details of the optimization procedure may also affect training. However, Theorem 2 relies on true value and advantage functions and does not establish these practical mechanisms. Consequently, the ablation between the full method and the no-augmentation variant cannot yet determine where the gains originate. The manuscript should also explicitly discuss its relationship to permutation-invariant critic methods such as PIC [5].

2. The current comparative experiments are insufficient to establish a robust advantage over related methods.

Section 5.2 reports only two independent training runs and evaluates the best learned policy with 50 rollouts, but does not define the meaning of “±” or specify the checkpoint selection rule. The Carry success rates correspond to 50/50, 49/50, and 48/50 successful episodes, respectively, while Push corresponds to 43/50, 42/50, and 39/50. These small differences, together with the practice of selecting the best policy for evaluation, are insufficient to demonstrate a reliable advantage across independent training runs. The large gap between the full method and the no-augmentation variant on Carry in Table 4 is noteworthy, but repeated experiments and component-level attribution analyses are particularly needed.

The comparison with TeamHOI [6] is also affected by how the baseline was adapted. For Carry and Push, Appendix F.4 makes two changes. First, it removes explicit teammate information to match MASkillBlender’s observations. Second, it replaces the original distance-dependent switching between Full and Masked AMP with a fixed per-task choice, which the appendix justifies by near-object initialization and empirical training performance. Teammate-conditioned inputs and interaction-dependent masking are core components of the original method, so these changes limit what the Carry and Push comparisons can show about TeamHOI’s capabilities.

On Move, neither change applies: closest-teammate information is retained, and there is no object interaction to mask. Here the policy learns to face the target but does not reach it. This failure more likely reflects other adaptation choices, such as retargeting to H1, the selection of locomotion references, or the equal weighting of task and AMP rewards. Unless the adapted baseline is validated, this result should not be taken as evidence about reference-based methods in general.

In addition, the manuscript does not report the success rate for G1, and the statistical aggregation of the MuJoCo results requires clarification.

[1] Kuang et al. SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blending. arXiv:2506.09366, 2025.

[2] Gao et al. CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics. arXiv:2406.14558, 2024.

[3] Feng et al. Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing. arXiv:2411.07104, 2024.

[4] Pandit, Shrestha, and Fern. Multi-Quadruped Cooperative Object Transport: Learning Decentralized Pinch-Lift-Move. arXiv:2509.14342, 2025.

[5] Liu, Yeh, and Schwing. PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learning. Proceedings of the Conference on Robot Learning, PMLR 100:590–602, 2020.

[6] Lionar and Lee. TeamHOI: Learning a Unified Policy for Cooperative Human-Object Interactions with Any Team Size. CVPR, 2026.

Questions for authors

1. How are advantages handled after permutation-based augmentation, and how is the optimization budget controlled?

Algorithm 1 computes advantages prior to augmentation. Do the permuted samples directly reuse these advantages, or are advantages recomputed using the critic’s predictions on the permuted states? In addition, compared with the no-augmentation variant, does augmentation alter the number of optimizer updates, the minibatch size, or the frequency of data reuse? These details are important for understanding the relationship between Theorem 2 and the actual algorithm.

2. Which component primarily accounts for the gains from augmentation?

For the Carry task, where the gap between the full method and the no-augmentation variant is largest, could the authors separately test actor-only augmentation and critic-only augmentation under matched environment interactions and optimization budgets? A comparison with a permutation-invariant critic would also be informative. Such experiments would help distinguish gains arising from critic symmetry, differences in actor updates, and effects of changes in the optimization procedure.

3. Do the comparative results remain valid when the core mechanisms of TeamHOI are retained?

Has it been verified that the adapted TeamHOI can learn the required walking and contact behaviors on H1? For Carry and Push, if teammate-conditioned inputs and interaction-state-dependent AMP masking were retained, what results would be obtained under a comparable training budget? This would help determine whether the reported gap originates from the proposed method or from the specific manner in which the baseline was adapted.

4. What evidence would isolate the distinctive role of per-joint skill blending in collaboration?

Could the authors conduct a controlled comparison on Carry or Push against a hierarchical control method that possesses comparable locomotion and manipulation capabilities but does not employ learnable per-joint blending? This directly affects the assessment of the distinctive contribution, since Move uses only a single primitive, whereas the primitive-removal ablation changes the motor capabilities available to the method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.