acceptodds
Withdrawn from ICLR 2027

Arm-wise Compositional Generalization in Dual-Arm Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generalization in multi-arm collaboration can be studied as composing familiar atomic skills in new ways across arms. However, existing evaluations offer limited insight into which training and architectural choices support this ability under different coordination requirements. We introduce ACG-Bench, a benchmark for Arm-wise Compositional Generalization that provides a common testbed for studying skill recomposition in dual-arm policies. It contains 23 task–condition pairs across 8 task families, with 6 in-domain conditions and 17 unseen compositions covering reordering, synchronization, their combination, and cross-task composition. All methods receive the same per-arm atomic prompts, and success requires achieving the task goal while satisfying physical milestones and specified order or timing constraints. Using as a common vision-language-action backbone, we compare representative data-augmentation and architectural strategies with shared source data and a common evaluation protocol. Our architectural study examines arm-token grouping, skill-specific LoRA adapters (SkillLoRA), and arm-wise attention (AWA), highlighting the complementarity of skill-conditioned parameters and attention structure. Combining these choices yields AE-VLA, which achieves 21.53% generalization success in simulation, compared with 2.94% for Single , 3.06% for MA-VLA, and 5.53% for two independently controlled policies. On physical SO101 robots, AE-VLA reaches 39.00% mean success across five unseen conditions, compared with 10.00% for the strongest baseline. These findings provide empirical guidance for designing dual-arm policies that generalize beyond fixed training routines.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-language-action, Compositional Generalization, Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9o53OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether dual-arm policies can execute familiar skills in combinations not seen during training. It introduces ACG-Bench, which tests changes in execution order, synchronization, and cross-task combinations. A shared scheduler provides the current instruction for each arm, so the evaluation focuses on execution rather than planning.

The proposed AE-VLA combines arm-specific action-token groups, skill-specific LoRA adapters, and an arm-wise attention mask on a shared π0.5 backbone. In simulation, it achieves higher average success on unseen compositions than Dual π0.5, the best-performing of the three main execution baselines. On real robots, it achieves 39% versus 10% for Dual π0.5 across five unseen conditions. AE-VLA has lower mean in-domain success than Dual π0.5 in both settings.

Critical strengths
  1. S1: The benchmark checks whether the robot follows the required order and timing; it also checks whether the robot reaches the goal. This catches failures that a final-state check could miss. All methods are told what each arm should do, so the comparison focuses on how well they carry out those instructions.

  2. S2: AE-VLA improves average success on unseen compositions in both simulation and real-robot tests. In the simulation ablations, adding SkillLoRA or AWA alone gives a small gain on unseen compositions. Adding both gives a much larger gain.

Critical weaknesses
  1. W1: The ablations show that the full model performs better on unseen compositions, but several changes are tested together. The authors acknowledge this limitation. Further controls would help explain the gains:

    • AWA changes several attention connections at once. It blocks direct attention between the arms' action tokens, restricts access to the other arm's local observations, and prevents attention to later action positions within each arm. The ablation does not separate these effects.
    • Without AWA, right-arm action tokens can attend to left-arm action tokens, but left-arm action tokens cannot attend to right-arm action tokens. This asymmetry makes the comparison harder to interpret.
    • SkillLoRA adds skill-specific adapters, extra parameters, and a skill-classification loss. The experiments do not separate the effects of these changes.
  2. W2: The contribution beyond prior work and some evaluation details need clarification.

    • MA-VLA already tests unseen skill combinations, including changes in order and synchronization. SkillVLA studies skill reuse and controls communication between arms. The paper explains how the methods differ, but it is still unclear what ACG-Bench adds to existing tests and what we learn from AE-VLA that prior work has not shown.
    • In simulation, synchronization windows range from 15 to 45 seconds. The paper does not explain how these compare with typical skill durations or whether tighter windows change the results.
    • Two Dual π0.5 conditions had their success scores changed after a manual collision check. The paper does not say whether all methods were checked in the same way.
    • For the real-robot tests, the paper does not clearly explain how skill completion is detected or how instructions are updated. It is also unclear whether either step involves human input.
Questions for authors
  1. Q1: Could you test the following action-token attention settings while keeping all attention connections involving observation-prefix tokens fixed?
    • Block attention between the arms' action tokens, while keeping attention within each arm unchanged.
    • Prevent each action token from attending to later positions within the same arm, while keeping cross-arm action attention unchanged.
    • Allow both arms' action tokens to attend to each other, while keeping attention within each arm unchanged.

How does each setting affect success on unseen compositions? The third setting would help check whether the current one-way access between the two action-token groups affects the results.

  1. Q2: How does SkillLoRA compare with the following alternatives?

    • Use one adapter shared by all skills, with the same number of added parameters.
    • Keep the skill-classification loss, but remove the skill-specific adapters.
    • Choose the adapter directly from the skill named in the prompt, instead of using the learned router.

These comparisons would help show whether separate skill adapters and learned routing are needed, and how much the extra parameters and classification loss contribute.

  1. Q3: Please clarify the collision checks and the real-robot evaluation:

    • Were successful runs from all methods checked for collisions in the same way?
    • How many apparent successes were removed in each of the two affected Dual π0.5 conditions?
    • On real robots, how is skill completion detected and how are instructions updated? Does either step involve human input?
  2. Q4: Please clarify the benchmark's added value and synchronization thresholds:

    • What does ACG-Bench test beyond the settings already studied in MA-VLA? What additional design insights does AE-VLA provide beyond SkillVLA?
    • How were the synchronization windows chosen? How long do the relevant skills usually take, and do tighter windows change the relative performance?
Reviewer_sAD6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Arm-wise Compositional Generalization in dual-arm vision-language-action models, focusing on whether familiar arm-level skills can be reused under unseen coordination requirements such as reordering, synchronization, and cross-task composition. To evaluate this problem, the authors introduce ACG-Bench, which tests familiar skills under unseen compositions with different order, timing, and cross-task requirements. They further propose AE-VLA, which combines arm-token grouping, skill-specific LoRA adapters (SkillLoRA), and arm-wise attention (AWA) within a shared π0.5 policy.

Experiments in simulation and on physical SO101 robots show that AE-VLA achieves substantially better generalization performance than the evaluated baselines, reaching 21.53% success in simulation and 39.0% in real-world evaluation.

Critical strengths
  • The paper identifies an important and relatively underexplored problem in dual-arm manipulation: whether familiar arm-level skills can be recombined across two arms under unseen coordination requirements. While compositional generalization has been widely studied in single-arm manipulation, this work specifically focuses on skill composition across two interacting arms with changing execution orders and coordination patterns.

  • ACG-Bench evaluates arm-wise compositional generalization under fixed per-arm atomic skill instructions provided by a common scheduler. This design largely removes high-level planning and skill decomposition as confounding factors, allowing the benchmark to focus more directly on whether a policy can execute unseen dual-arm skill compositions.

  • AE-VLA shows substantial improvements over the evaluated baselines in both simulation and real-world dual-arm experiments. The consistent gains across these two settings provide encouraging evidence that the proposed design improves compositional execution beyond a single evaluation environment.

Critical weaknesses
  • While Can in Basket involves clear complementary arm roles, tasks such as Stack Bowls and Stack Cubes appear achievable through sequential single-arm manipulation when considering their final goals alone. As a dual-arm benchmark, ACG-Bench would benefit from more tasks with explicit two-arm dependencies, such as jointly lifting an object, exemplified by RoboTwin’s lift_pot, or direct object handovers.

  • AE-VLA underperforms Dual (\pi_{0.5}) on Burger & Fries under both Sync and Sync+Reorder, and its in-domain performance also trails Dual (\pi_{0.5}) or MA-VLA on several tasks. For example, its in-domain CCSR on Object in Cabinet is 21%, compared with 59% for Dual (\pi_{0.5}). Although the paper acknowledges this trade-off, could the authors explain which design choices contribute to the in-domain degradation and whether it can be mitigated without compromising compositional generalization?

  • Although SkillLoRA relies on predicted skill IDs during both training and inference, the paper does not report routing accuracy on in-domain or unseen compositions. Without per-arm and per-skill routing analysis or an oracle-routing comparison using the prescribed skill labels, it remains unclear how much task failure stems from incorrect adapter selection rather than action execution. Such analysis would clarify the router’s reliability and its impact on overall performance.

Questions for authors
  • The paper describes state-based prompt updates, but what specific completion checks trigger transitions between atomic skills during AE-VLA execution? When one arm finishes before the other, are their prompts updated independently or only after both arms satisfy the phase requirements, and how are waiting, retries, timeouts, or termination handled if one arm fails? Please clarify which decisions are handled by the common scheduler and which are left to the learned policy, including how skill completion is detected in simulation and on the physical robots.
  • Appendix B.3 explains that the ([A_L,A_R]) layout and standard block mask allow (A_R) to attend to (A_L), but not vice versa, whereas AWA blocks both cross-arm action-attention directions. Why was this asymmetric connectivity retained for the non-AWA variants rather than allowing symmetric cross-arm access? Does reversing the block order or using symmetric attention affect the results, particularly under Reorder conditions?
Reviewer_Lns9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether dual-arm VLA can reuse familiar per-arm skills under unseen coordination requirements (reordering, synchronization, sync+reorder, and cross-task composition), which the benchmark specifies to the policy through per-arm prompt sequences. It introduces ACG-Bench, comprising 23 task–condition pairs across 8 task families (6 in-domain conditions and 17 unseen compositions). A common rule-based scheduler supplies per-arm atomic prompts, and success requires the goal plus correct order and timing. All methods are fine-tuned from the same π0.5 checkpoint. Combining arm-token grouping, skill-specific LoRA adapters (SkillLoRA) and arm-wise attention (AWA) yields AE-VLA: 21.53% success vs. 2.94–5.53% for baselines in simulation, and 39% vs. at most 10% on real SO101 robots, with cost to in-domain success.

Critical strengths
  1. The problem is well posed and relevant, and the evaluation targets it directly. A rule-based scheduler separates execution from planning, and CCSR rejects rollouts with wrong order or timing. The finding that the π0.5-based baselines nearly fail at recomposition is informative.
Critical weaknesses
  1. The evidence for the main claim is weaker than presented. The contributions of this paper are empirical, so the conclusions are only as strong as these numbers are robust.

(a) Gains are concentrated in the bowl tasks. The 21.53% vs. 5.53% in the abstract is an average over 17 held-out conditions. On the 11 conditions outside the three bowl-related families, AE-VLA averages 6.7% vs. 3.3% for Dual π0.5 and wins only 5 of 11. The improvement is therefore far less general than the average suggests.

(b) Weak in-domain performance. Success is low even under training conditions (Single π0.5: 9% on Burger & Fries, 14% on Object in Cabinet). Held-out failures may therefore reflect weak basic skills rather than poor compositionality, and conclusions drawn from such policies may not hold for well-trained ones. Moreover, AE-VLA, which adds SkillLoRA and AWA to Token Group, lowers average in-domain success further (from 45.00% to 27.17%).

(c) No variance estimates. Results appear to come from single training runs without confidence intervals. On Push Cubes/In-domain, Single π0.5 scores 50%, while all six other variants score 3–21%. In particular, adding token grouping to Single π0.5 improves five of the six in-domain tasks but drops Push Cubes from 50% to 3%. One change that helps five tasks but collapses on one is hard to explain by architecture alone and suggests large run-to-run variance.

  1. The design insights are not established.

(a) Unresolved confounds. SkillLoRA adds both capacity and a router-supervision loss. AWA changes both cross-arm and within-chunk attention. The paper acknowledges both (L257–258, L421–423) but runs no controls that separate them.

(b) Interface confound. The evaluation supplies two per-arm prompts, a format π0.5 was not designed for, and the paper does not say how Single π0.5 and MA-VLA receive them. SkillLoRA's per-arm routers and AWA's mask both tie each arm's prompt to that arm's action tokens. Part of the gain may therefore come from better binding of instructions to arms, rather than from better compositionality.

This paper's third contribution is design guidance. Without these controls, it is unclear which choice matters and why.

Questions for authors
  1. Are the results from single training runs?
  2. How are the two per-arm prompts fed to Single π0.5 and MA-VLA?
  3. On SO101, who switches the per-arm prompts, and how are timing and collisions judged?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.