acceptodds
Withdrawn from ICLR 2027

GroupACT: Functional-Group Coordination for Robot Action Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large-scale robot models learn diverse motion patterns from data collected across embodiments. However, robot embodiments, particularly mobile manipulation and multi-arm platforms, comprise heterogeneous joints with distinct functional roles and motion characteristics. These differences shape the temporal and spatial structure of joint trajectories. Incorporating this structure as a physical prior can guide the learning of coordinated actions across heterogeneous control variables. In this work, we assign each joint a group label based on its functional role and motion characteristics and organize joints into action groups. Using the articulated structure of each embodiment, we further construct a joint-group graph that explicitly represents the relationships between joints and their functional groups. We then introduce GroupACT, a flow-matching policy that incorporates these structural priors into its action expert through group-mediated block and group-query tokens. The group-mediated block is designed for integration into Transformer-based action experts. Experiments on mobile manipulation in simulation and bimanual manipulation on a physical robot show that GroupACT outperforms the evaluated baselines on multiple tasks. These results support the effectiveness of functional-group priors for learning heterogeneous actions across the evaluated embodiments and tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, robot action model, embodied AI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NHE5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GroupACT, a VLA framework that incorporates functional-group structure into robot action generation. The method organizes robot joints into functional groups and introduces group-mediated Transformer blocks to model intra-group interactions and inter-group coordination. Experiments on MS-HAB SetTable and real-world bimanual manipulation demonstrate promising improvements over existing baselines.

Critical strengths
  1. Explicitly modeling functional relationships between heterogeneous robot joints is meaningful for coordinated manipulation.

  2. The proposed group-mediated blocks provide a structured mechanism for joint-level and group-level information exchange.

  3. GroupACT achieves strong performance on MS-HAB, and the ablation studies demonstrate substantial benefits from introducing group-mediated layers.

Critical weaknesses
  1. Limited evidence for the central functional-group contribution. Although the proposed group-mediated architecture improves performance, the experiments do not isolate the benefits of functional grouping from those of general hierarchical attention or additional architectural capacity. This weakens the evidence supporting the paper's main methodological contribution.

  2. Insufficient experimental generalization. The evaluation is limited to two robot platforms with separate experimental settings, and the real-world results are relatively limited. The lack of cross-embodiment evaluation makes it difficult to assess the general applicability of the proposed method.

Questions for authors
  1. Methodological novelty and related work. The proposed approach is closely related to existing structure-aware robot learning methods. In particular, Structural Action Transformer [1] already introduces joint-wise action trajectories with functional and kinematic embeddings, while HEX [2] explores body-part-aligned representations and coordinated whole-body modeling. Could the authors clarify the methodological contribution of GroupACT beyond existing structural action representations and body-part-aware modeling approaches?

  2. Effectiveness of functional grouping. The ablation studies demonstrate the effectiveness of group-mediated layers, but do not establish whether the proposed functional grouping is superior to other grouping strategies. Could the authors compare functional grouping with random grouping, kinematic grouping, and alternative group configurations? Parameter-matched comparisons would also strengthen the conclusions.

  3. Limited experimental generalization. The experiments cover one simulated mobile-manipulation embodiment and one real-world dual-arm platform, without joint cross-embodiment training or transfer evaluation. Could the authors further validate whether the proposed architecture generalizes across different robot configurations and action spaces?

  4. Real-world evaluation. The real-world evaluation includes only two tasks and does not demonstrate consistent improvements over π0.5 across all stages. Could the authors provide more evaluation trials, statistical analysis, and controlled comparisons with comparable training settings?

[1] Structural Action Transformer for 3D Dexterous Manipulation. CVPR 2026. [2] HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation. arXiv 2026.

Reviewer_dqzqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GroupACT, a vision-language-action policy that incorporates robot functional groups into action generation. Action coordinates are assigned to predefined groups, such as the base, arms, and grippers. The action expert combines ordinary Transformer layers with structured blocks that perform attention within groups, aggregate joint features into group tokens, exchange information between groups, and broadcast the resulting context back to joints. The policy uses a pretrained vision-language backbone and a standard flow-matching objective. Experiments on MS-HAB mobile manipulation show improved success rates over baselines and variants without the proposed modules. Real-world evaluations cover two dual-arm storage tasks, with average complete-task success matching the π₀.₅ baseline.

Critical strengths
  • The architecture provides a concrete way to incorporate robot functional groups into action generation while retaining direct interactions between joints.
  • Simulation results show substantial improvements. GroupACT achieves 72.0% average success, compared with 55.6% for AC-DiT and 52.54% for its variant using ordinary Transformer layers.
  • The method is demonstrated on a real dual-arm robot using 64 task demonstrations, achieving average complete-task success comparable to a fine-tuned π₀.₅ baseline.
Critical weaknesses
  • Task selection does not directly test the central motivation. The experiments do not isolate situations requiring strong coordination between functional groups. In particular, the real-world tasks largely involve sequential arm movements, providing limited evidence for the claimed coordination benefits.
  • The source of improvement remains unclear. Removing the proposed modules reduces simulation performance, but the absence of alternative groupings and parameter- or compute-matched controls prevents attributing these gains specifically to meaningful functional grouping.
  • Real-world evidence is limited. Evaluation covers two similar storage tasks, with average complete-task success matching π₀.₅. No real-world module ablation establishes the benefit of the proposed structure.
  • Task presentation is insufficiently informative. The small simulation images in the appendix make it difficult to assess whole-body motion and interactions between functional groups. Clearer visualizations and coordination analysis would strengthen the connection between the tasks and the method.
Questions for authors
  1. Which evaluated tasks require tightly coupled coordination between functional groups? Is the improvement larger on tasks requiring simultaneous base, torso, and arm movements than on tasks that can be completed through sequential movements?

  2. What evidence supports the chosen functional grouping? Would random groups of the same sizes, alternative body partitions, or learned groups achieve similar performance?

  3. Do the improvements remain when the ordinary Transformer baseline is matched in parameter count and computational cost? The current module-removal ablations do not fully separate the effect of grouping from additional model capacity.

  4. Can a real-world ablation using the same backbone without group-mediated blocks establish their contribution? For cup storage, what causes the advantage in object placement to disappear during the final closing stage?

  5. How many real-world trials were conducted for each task and method, and what uncertainty accompanies the reported success rates? Were both methods evaluated under matched initial conditions?

  6. Could clearer task visualizations show the robot’s overall motion and the coordination between functional groups? A comparison with the ordinary Transformer variant would help illustrate which coordination failures the proposed modules address.

Reviewer_LjbKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a specially designed architecture for flow-matching-based robot action generation models to better encode heterogeneous robotic body structures. The key idea is that functional groups of robot joints should be encoded separately to improve action generation.

Essentially, the authors divide robot joints into different functional groups, such as arms, grippers, and mobile base, and use a graph-based structure with group-level attention to coordinate the action generation.

And they hope this structure can better handle heterogeneous robot actions. Experiments on mobile manipulation and real-world bimanual manipulation have shown some promising results.

The authors also agree they should do more experiments on more robot embodiments in the last section.

Critical strengths
  • I think the research problem is interesting and seems relatively novel. However, robot morphology and graph-based policy learning have already been well studied, and some previous works have demonstrated impressive cross-embodiment generalization (I will talk about these papers in the next section). So while the specific architecture may be new, I am not convinced that it brings any fundamentally new capabilities beyond what existing methods have already achieved.

  • The paper is well written, and the proposed architecture makes sense. Using functional groups to coordinate different joints is intuitive, and the experiments show some promising improvements.

  • I appreciate that the authors conducted real-world bimanual manipulation experiments, beyond just simulation.

Critical weaknesses

For ICLR papers, we need a paper both strong in the novelty and experiment part, which we treat as "significant contribution". However, I feel this paper does not fall into this category.

Here are my suggestions.

(1) Very limited novelty compared with preivous work.

My main concern is the novelty of the proposed method. Using robot morphology, joint structures, and functional groups to guide policy learning has already been well studied. For example, NerveNet (Wang et al., ICLR 2018) and Body Transformer (Sferrazza et al., CoRL 2024) have explored very similar ideas using graph-based representations and structured attention.

More importantly, SAT (Lei et al., CVPR 2026, https://arxiv.org/abs/2603.03960) has already proposed joint-wise action representations with functional and kinematic information, and uses flow matching for action generation. It also studies cross-embodiment skill transfer with heterogeneous pretraining data. This seems particularly close to the proposed method. This paper appears on Arxiv on Mar 4th, 2026, while our ICLR 2027 submission DDL is Sep 18, 2026.

I understand GroupACT introduces a new group-based attention module, which is different from these earlier methods. But is this really a significant methodological contribution, or mainly a different implementation of structural attention in the action expert? I am not fully convinced.

(2) Even I agree being a reviewer should be more generous and supportive to our authors. Then, this novelty is still acceptable if and only if the authors have done comprehensive experiments with strong baselines. But I did not find those evidence either.

My biggest concern, then, is the expeirments do not seem to demonstrate a clear cross-embodiment generalization.

Cross-embodiment action generation has been a well-studied problem, and some previous works have already demonstrated quite impressive results. For example, this paper "Towards Embodiment Scaling Laws in Robot Locomotion" (Ai et al., CoRL 2025) investigated around 1,000 robot embodiments and demonstrated generalization to unseen robots.

Compared with these works, the experiments in this paper are quite limited. The authors only evaluate separately trained policies on two robot platforms, without any cross-embodiment joint training or transfer experiments.

Of course, I don't expect the authors to reproduce experiments on 1,000 robots. But given the motivation of learning heterogeneous robot actions, I would expect at least some evidence showing that the proposed structure benefits learning across different robot embodiments.

Otherwise, the experiments only demonstrate that functional grouping helps policy learning. This is a much narrower contribution.

(3) Seems baselines are cherry-picking, and missing some strong baselines.

The paper mainly compares against recent action generation models such as ACT, Diffusion Policy, RDT, and other regular VLA models.

However, there are many traditional methods for learning policies with robot morphology and graph-based structures. These are actually more directly related to the core idea of this paper.

For example, Body Transformer would be a very relevant comparison.

SAT is another important baseline, especially since it already uses joint-wise action tokens and flow matching.

It's quite strange for me because the authors actually cited "Body Transformer" but somehow, ignore this baseline in the experiments. Can the authors clarify this issue?

Even if these methods have different architectures, I think the authors should at least compare their structural modules under a similar policy learning setup. Otherwise, it is difficult to judge what is actually new and better about the proposed group-based attention.

(4) The ablation study can be better designed

The authors show that removing group-mediated layers reduces the success rate, which is good.

But does this improvement actually come from grouping joints according to their functional roles?

For example, what if we randomly group the joints while keeping the same group sizes? Or group them based on their kinematic connections? Would the performance be similar?

These experiments are important because functional grouping is supposed to be the main contribution of the paper.

Overall, the idea is quite good, but the work seems unfinished.

Questions for authors

check relevant questions raised above. No need to repeat here.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.