acceptodds
Under review as a conference paper at ICLR 2027

VLA-Fabric: Learning Interaction for Composable Multi-Arm VLA Agents

Abstract

Vision-language-action (VLA) models are rapidly expanding what individual robots can accomplish, yet many real-world manipulation settings require multiple arms to operate as a coordinated system. Existing approaches organize multi-arm coordination in different ways, including centralized joint policies, high-level task or role assignment, and locally executed policies that react to teammates. However, how complete action-producing VLAs should explicitly interact with one another during action generation remains largely unexplored. We introduce VLA-Fabric, which organizes one complete VLA per arm while preserving local perception and action ownership, and enables coordination through learned interaction across policy boundaries. We first systematically study where such interaction should occur, how peer information should be aggregated, and how the interaction should be learned, revealing complementary coordination functions spanning shared context, peer-specific latent information, and action-stage interaction. We then re-instantiate these functions in a structurally different VLA architecture, showing that the coordination principle is not tied to a particular tensor layout or backbone implementation. Finally, we compose teams of three and four arms and evaluate them in simulation and physical experiments. On all four physical tasks, Full interaction improves success over independently trained local policies by 16–40 percentage points. Together, these results suggest that explicit interaction among complete VLA policies offers a scalable path toward networked embodied intelligence.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Multi-Arm Manipulation, Multi-Robot Coordination, Inter-Policy Interaction, Robot Learning, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_yQP6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper mainly studies how multiple pretrained vision-language-action models can coordinate during multi-arm manipulation. The paper proposes VLA-Fabric, which assigns one complete VLA to each arm and exchanges information between the models through shared context, intermediate features, and action-stage interaction. The authors compare different interaction designs, implement the method in SingleVLA and π₀.₅, and evaluate it on bimanual and larger-team tasks in simulation and on physical robots.

Critical strengths
  1. The paper compares different interaction locations, fusion operators, and fine-tuning strategies, which helps in understanding how these choices affect coordination.
  2. The experiments cover two VLA architectures and include three- and four-arm tasks in both simulation and the real world. The handover and shared-object manipulation results provide useful evidence that the method can support coordinated execution.
Critical weaknesses
  1. The Full results in Tables 1A and 1B are different. For example, SingleVLA-Fabric achieves 80.0% versus 75.0% on Box and 70.0% versus 78.0% on Shoes. The Box result for π₀.₅-Fabric also changes from 80.5% to 79.0%. Appendix A.5 mentions a separate evaluation batch for the SingleVLA ablations, but does not clearly explain what changed between the two evaluations. Since Table 1B only disables interaction during inference without retraining, it is unclear to me whether the same trained checkpoint was used in both parts and what caused the different results.
  2. Tables 2 and 3 only compare Full with Independent, both of which already use a separate VLA for each arm. These comparisons show the benefit of information exchange, but it remains unclear whether a single VLA receiving all arms’ observations and states could achieve similar results by jointly predicting their actions. Moreover, the four-arm system uses four complete VLA copies and additional interaction modules. The paper does not sufficiently report their parameter count, computational cost, memory requirements, or communication overhead, so it is difficult to assess whether the performance justifies the additional resources. The bimanual comparisons in Table 1 do not answer this question for the larger-team and real-world tasks.
  3. The practical benefit of modularity remains unclear to me. Sections 5 and 7 emphasize keeping local action dimensions unchanged and re-instantiating coordination across VLA architectures. However, Appendix C.1 states that each task and team configuration is fine-tuned separately, and the SingleVLA and π₀.₅ versions use architecture-specific interaction implementations. These results support reuse of the design principle, but do not establish how much adaptation effort is saved when changing the team configuration or VLA architecture. Could the authors clarify what can be reused and what needs to be redesigned or fine-tuned?
Questions for authors
  1. Could the authors explain why the Full results differ between Tables 1A and 1B? Please clarify whether the same checkpoints, test conditions, and inference settings were used. If different checkpoints were used, I would appreciate an explanation of why this was needed for the fixed-weight ablation study.
  2. I would suggest adding a joint VLA baseline that receives all arms’ observations and states and jointly predicts their actions. The comparison should use the same observation sources, demonstrations, and evaluation conditions, with comparable pretraining and training budgets. Reporting the total parameter count, training compute, peak memory usage, and inference latency would also help in assessing the performance relative to the computational cost.
4 comments
AuthorsOpenReview ↗

Response Part 1/3 | W1-W2 | Evaluation and deployment costs

Thank you for examining the experiments. Please allow us to briefly restate the paper’s main contribution.

VLA-Fabric explores a way to organize complete VLAs for multi-arm learning through explicit exchange of their internal representations. We establish the interaction architecture in SingleVLA, transfer it to the different computation of π0.5, and apply it to teams of two to four arms in simulation. The performance comparisons evaluate this initial design under familiar VLA fine-tuning objectives. The architecture gives us a basis to combine improvements in local policies, exchanged representations, and communication efficiency, with room to improve coordination and task performance.

During the discussion period, we consolidated the reviews and added experiments to address them. We completed four larger-team joint π0.5 evaluations, two CHORUS configurations, bimanual π0.5 Independent and joint controls, and wrist-only vision evaluations. We draw on these experiments below to address the specific points in your review. We will use the ten-page main text and appendix in the discussion revision to explain the method, protocols, and results. Thank you for guiding these additions.

Weaknesses

W1. Thank you for asking us to clarify Appendix A.5. During development we trained multiple Full checkpoints with the same interaction design. For SingleVLA Box and Shoes, Table 1A reports the capability runs. In Table 1B, we selected another Full checkpoint for each task to check whether reliance on the exchange paths also appeared in a different trained instance of the same design. Those diagnostics used a separate batch of test conditions. Q1 details the checkpoint and condition records, including the π0.5 Box repeats.

Within Table 1B, each removal uses the corresponding Full checkpoint and test conditions. On Handover Mic, Full π0.5-Fabric achieves 97.5%. With Private K/V removed, the same policy achieves 11.0%. Independent trains a separate policy per arm without interaction modules. The Frame4 results in Q2 illustrate how Full and Independent compare on a task that requires the arms to move a shared object together.

W2. Thank you for asking whether one VLA could achieve comparable performance with more arms. Table 1 compares bimanual performance. To address the larger-team question, we added a joint π0.5 on each larger-team simulation task. One model sees all arms' observations and states and outputs their actions together.

MethodFrame3Stack3Arch4Frame4
Fabric Full81.5%91.0%5.0%97.0%
CHORUS67.5%73.5%0.0%11.0%
CHORUS-Augmented80.0%94.5%6.0%21.5%
Role-mixed Fabric82.0%81.5%0.5%66.5%
Joint π0.574.0%36.0%0.0%73.5%

Following suggestions from other reviewers, we also evaluated both CHORUS configurations and Role-mixed Fabric. We include these results here for your reference.

The results show that a joint model can learn some of these tasks, with larger gaps on tasks such as Frame4.

The native action head also constrains which teams a single model can control. Four 7-dimensional arm actions use 28 of π0.5's 32 output dimensions, while a fifth would require 35 with the same interface. SingleVLA's 10-dimensional head was designed for one arm, so it would need a different output interface even for joint bimanual control. Fabric adds one complete VLA per arm and keeps the existing local action heads. The four-arm Fabric configuration has about 14.067B total parameters, or 3.517B per device with one VLA per arm, versus 3.403B for joint π0.5. Q2 gives the training measurements and a communication result from our separate optimization study.

AuthorsOpenReview ↗

Response Part 2/3 | W3 | Modularity and reuse

Weaknesses (continued)

W3. Thank you for asking what carries over. We chose π0.5 after studying SingleVLA because their internal computation differs substantially. As Figure 3 shows, SingleVLA first processes observations in its VLM, then fuses the readouts before separate local action decoders run. In π0.5, Common acts on the multimodal prefix, peer Private K/V condition the Action Expert, and action features are exchanged as the Action Expert computes each flow step. The transfer shows that the same three functions can be placed at different computational boundaries.

To apply the design to a new backbone, we identify the features that can serve as shared context, peer-specific information, and action-side intent. The placement, operators, and learned weights are then tailored to that backbone. Across team sizes, each arm keeps its observation and action interfaces, while the peer set and some fusion components change and the team is fine-tuned. At fixed team size, Shared π0.5-Fabric reuses one checkpoint per role across four bimanual tasks. This gives us a concrete way to extend the team by keeping the existing arms’ input and action interfaces and fine-tuning the team to coordinate with the additional arm. Our experiments establish the adapted teams’ task performance. Thank you for raising the question of how much of a trained team can be reused when adding an arm. It gives us a concrete direction for studying how to preserve the existing arms’ skills as the team expands.

AuthorsOpenReview ↗

Response Part 3/3 | Q1-Q2 | Checkpoints, joint π0.5, and resource measurements

Questions

Thank you for specifying which evaluation settings and resource measurements would make the comparisons more useful.

Q1. For SingleVLA Box and Shoes, the A and B entries use different Full checkpoints and condition sets. On Box, the Full run in A achieved 160/200. The Full run in B achieved 150/200. On Shoes, the Full run in A achieved 140/200. The Full run in B achieved 156/200. For each removal in B, we hold the checkpoint, test conditions, and other inference settings fixed relative to its own Full reference.

For SingleVLA Mic, both entries point to the same archived checkpoint path and use different condition sets. The Full run in A achieved 168/200. The Full run in B achieved 163/200. For π0.5 Box, we verified that the original runs used the same recorded checkpoint, seed, conditions, and Full inference settings. The run in A achieved 161/200 (80.5%). The run in B achieved 158/200 (79.0%).

To check where variability can arise, we held the weights, input, and actual flow noise fixed. Two fresh inference processes still differed slightly in their outputs, with maximum normalized action differences of 0.00277 for the left arm and 0.00168 for the right. In a check with compiler autotuning disabled, two fresh processes produced identical outputs for that input. Rendering the same simulator state without advancing it also produced occasional RGB differences of one intensity level (1/255). These diagnostics point to numerical differences between inference processes and small rendering changes as possible contributors to the Box score differences.

We reloaded the original π0.5 Box checkpoint and evaluated it twice on the same 200 conditions. The two repeat evaluations achieved 77.5% and 79.0%, respectively. Across repeats, the weights, initial positions and velocities, proprioceptive states, and first planning-step sampling keys matched. The repeats show that success counts can fluctuate slightly under the same recorded setup. We retain both original table entries.

Q2. The joint π0.5 uses the shared scene image, all wrist images, and all arm states from the same observation sources as Fabric. We adapted image preprocessing to retain those views and used the native 32-dimensional action head for joint prediction.

TaskFabric FullIndependentJoint π0.5
Frame381.5%76.5%74.0%
Frame497.0%39.0%73.5%

Full outperformed Independent on both tasks. The joint results in W2 are close to Full on Frame3, with a larger gap on Frame4.

For the joint comparisons we used the same demonstration source, base π0.5 checkpoint, and update budget as Fabric Full within each task.

The four-arm Frame4 Full run on four A800 GPUs averaged 2.09 seconds per optimizer update near the end of training and reached a peak JAX allocator footprint of 31.60 GiB per GPU. Those measurements characterize training. For inference, we can share the communication and planning measurements from the separate study below.

Communication optimization for VLA-Fabric is the focus of a separate study of ours. In four-agent Eagle2/SingleVLA Take Photo, the RAW and optimized runs used the same checkpoint and 50 initial conditions. Team-wide logical payload fell from 310.785 to 34.535 MiB per planning round (88.89%), and mean planning latency fell from 875.5 to 322.4 ms per round. These measurements are for that Eagle2 setup. We appreciate your attention to this practical question and look forward to sharing the broader communication results in that study.

Reviewer_yQP6OpenReview ↗

Comment by Reviewer yQP6

Thank you for the response and additional experiments. The explanation of Table 1 and the new joint π₀.₅ comparisons address several of my concerns. However, the training and inference costs relative to the joint policy remain insufficiently evaluated. It is also still unclear how much adaptation effort the proposed modularity saves when changing the team configuration, since no corresponding reuse experiment is provided. I therefore maintain my original rating.

Reviewer_UuHdOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how multiple arms, each controlled by its own complete VLA policy, can be coordinated through learned interaction, as an alternative to absorbing all cross-arm dependencies into a single joint policy.

The proposed framework, VLA-Fabric, keeps one VLA per arm and connects these local policies through three interaction functions, namely a shared semantic context formed by combining the agents' common representations, a peer latent context in which each agent attends to the keys and values of its peers, and an action-level fusion of the agents' action-side states. The specific design is selected through a controlled study over where the interaction enters the policy, how peer information is aggregated, and how the interaction is trained, conducted by decomposing TwinVLA into two single-arm policies on two ALOHA tasks. The same functions are then re-instantiated in pi0.5 and extended to teams of three and four arms.

The authors evaluate the approach on bimanual simulation tasks, on three- and four-arm simulation tasks, and on four physical three- and four-arm tasks, where interaction improves success over independently trained local policies by 16 to 40 percentage points.

Critical strengths

The first strength is that the paper tackles an important and timely problem with a well-motivated approach. Keeping each arm's VLA complete while letting the policies exchange internal representations is a clean alternative to an ever larger joint policy, and the controlled study that decomposes TwinVLA yields findings that are useful beyond this method, such as that peer keys and values alone already help and that adding more interaction pathways does not by itself improve coordination. This is consequential because, together with the successful transfer of the same design to π0.5, it makes the paper a useful step in a direction the community will need to explore.

The second strength is the breadth of the evaluation, in particular the physical experiments with teams of three and four arms, which are rarely demonstrated. Interaction improves over independent local policies on all four physical tasks, with the largest gains on tasks that involve handovers or a shared object, and the same pattern appears in simulation. This is consequential because the consistent link between physical coupling and the benefit of interaction directly supports the central premise of the paper.

Critical weaknesses

The first critical weakness is that the method, and in particular how it differs from a joint policy, is not described clearly. The interaction functions are introduced mostly in abstract terms, and some key operators remain ambiguous even in the appendix, such as what is exchanged when the Common computation is synchronized in the RAW variant. Since pi0.5-Fabric synchronizes at every layer and flow step, it is unclear in what sense the agents remain local, which matters because the modularity and composability claims depend on this distinction.

The second critical weakness is that the comparisons supporting the central claim are incomplete. pi0.5-Fabric is compared with joint policies only through results from other papers with older backbones and different protocols, and some compared models differ in training budget or recipe, as the appendix acknowledges. Without a same-backbone joint baseline and matched training conditions, it is hard to tell how much of the improvement comes from the interaction itself, although these comparisons seem feasible and would make the case considerably stronger.

Questions for authors

Clarification requests

  1. Could the authors define precisely what operator-level synchronization of the Common computation means in the RAW variant, which tensors are exchanged at each synchronization point, and whether the Common stream consequently becomes identical across agents? It would also help to explain how this differs from the AVG variant and from the joint attention used in TwinVLA.

  2. Which inputs are assigned to the Common and Private streams in each task, and is the shared global camera view, which all agents receive in the physical experiments, always part of the Common stream? How would VLA-Fabric behave in settings without any shared observation, where each agent only has its own wrist view?

Analyses that could change my assessment

  1. How does a joint bimanual pi0.5 policy trained on the same data compare with pi0.5-Fabric under the same protocol, and how does the independent pi0.5 baseline perform on all four bimanual tasks? Relatedly, what do the authors think is lost when TwinVLA is decomposed, given that SingleVLA-Fabric remains below TwinVLA on Shoes Table?

  2. Do the advantages of the Common-first recipe and of Full over Independent, in particular on Frame4, hold when the compared models share the same training budget and adaptation recipe?

  3. Since each team is currently fine-tuned separately, could the authors show whether trained agents can be reused when a team is extended from three to four arms, or when one agent is replaced, without retraining the whole team? This would directly support the composability claim in the title.

4 comments
AuthorsOpenReview ↗

Response Part 1/3 | W1-W2 | Information exchange and joint baseline

Thank you for a thoughtful review and a clear summary. You captured our aim of connecting complete arm-local VLAs, transferring the interaction design across backbones, and testing it with larger teams. We appreciate your recognition of this direction and your constructive suggestions.

We share your interest in how this direction can develop. Using familiar VLA fine-tuning objectives, our exploratory study shows that explicit exchange can support coordination with two to four arms, with results close to established methods on several bimanual tasks. This gives us a basis for improving the policies, communication, and team organization.

During the discussion period, we consolidated the reviews and added experiments to address them. We completed four larger-team joint π0.5 evaluations, two CHORUS configurations, bimanual π0.5 Independent and joint controls, and wrist-only vision evaluations. We draw on these experiments below to address the specific points in your review. We will use the ten-page main text and appendix in the discussion revision to explain the method, protocols, and results. Thank you for guiding these additions.

Weaknesses

W1. Thank you for asking what the policies exchange and what remains local. Each arm in Fabric runs a complete VLA with its own weights, assigned observations, action state, and output head. It can run on its own device and generates its own action chunk. A centralized joint policy processes all arms' observations and states in one model and predicts their actions together. Fabric connects the arm-local computations through explicit exchange of internal activations.

Common aligns features from the shared scene and instruction by averaging the outputs of selected operators. Private lets each arm use its own query to attend to peers' keys and values. At the action stage, SingleVLA fuses readouts before local decoding. In π0.5, each arm's local action query attends to peer action K/V, and a gated residual updates its action features at each Action Expert layer and flow step. These exchanges create dependencies during cooperative inference, while each VLA retains its own computation and action output. Our answer to Q1 identifies the exact RAW tensors and explains how they are averaged.

W2. The appendix flags two comparisons with different total training budgets. The SingleVLA design study uses 40k updates for Single-stage and 50k for Common-first, including initialization (Appendix A.3). On Handover Box, π0.5 Independent used 40k updates and Full used 50k (Appendix B.4).

Tasks differ in difficulty, how tightly the arms must coordinate, and how much variation the policies encounter. We choose the demonstration count and training budget for each task accordingly. For example, Arch4 requires learning an ordered assembly sequence across varied initial conditions. For the four larger-team joint controls, we use the same task demonstrations, pretrained π0.5 base, and total update budget as the corresponding Fabric Full runs.

Thank you for suggesting a same-backbone joint baseline. We trained and evaluated one joint π0.5 for each of the four larger-team tasks.

We also include the two CHORUS configurations and Role-mixed Fabric alongside Full and joint π0.5. Each result summarizes 200 trials.

MethodFrame3Stack3Arch4Frame4
Fabric Full81.5%91.0%5.0%97.0%
CHORUS67.5%73.5%0.0%11.0%
CHORUS-Augmented80.0%94.5%6.0%21.5%
Role-mixed Fabric82.0%81.5%0.5%66.5%
Joint π0.574.0%36.0%0.0%73.5%

The joint π0.5 results show that a single model can coordinate several arms on some tasks while learning more demanding team behaviors remains difficult. Our answers to Q3 and Q4 discuss the comparisons and training setups in more detail.

A practical constraint is that the joint model's output grows with the team. Four 7-dimensional arm actions use 28 of π0.5's 32 native output dimensions. Adding a fifth arm with the same interface would require 35. Backbones designed for one arm can encounter this constraint sooner. For example, SingleVLA's native 10-dimensional head would need a different output interface to predict both arms' actions. Fabric can add an arm and its peer connections while keeping each existing arm's input and output interfaces.

AuthorsOpenReview ↗

Response Part 2/3 | Q1-Q2 | RAW synchronization and observations

Questions

Thank you for asking us to clarify which tensors the policies exchange and what each arm observes.

Q1. RAW averages forward-pass activations at aligned Common positions, using separately parameterized VLAs. For two arms, z_common = (z_left + z_right) / 2. In SingleVLA, the exchanged outputs are pre-attention normalization, the Q/K/V projections, attention output projection, pre-FFN normalization, and FFN output. π0.5 synchronizes the same seven operator outputs at Common positions in its multimodal prefix. Both arms receive the same value at each synchronized Common slice. Between those points, each arm computes with its own weights and retains its Private features and action state.

SingleVLA's AVG variant averages only Common K/V before Private attention. RAW also averages other operator outputs, including Common Q. The receiver's Private and action queries stay local. In TwinVLA, joint attention mixes the two branches' features within the policy. In Fabric, selected representations cross the boundary between complete local policies.

Q2. Each arm's VLA directly receives the global scene image, the task instruction, and only that arm's wrist image and proprioceptive state. This assignment is the same for Full and Independent. The global image and instruction supply Common features. Each arm's own wrist image and state supply Private features. Information specific to another arm enters the policy through the exchanged representations.

With wrist-only vision, each arm mainly sees its gripper and nearby workspace. The team can still exchange learned features from those local views. In Table 1B, removing Common exchange lowered SingleVLA-Fabric on Handover Box from 75.0% to 38.5% even with the global image still present. The wrist-only runs below test a different question about the image input itself.

We trained π0.5-Fabric with each arm receiving only its wrist image, local state, and task instruction directly. All three learned interaction paths remained enabled.

TaskWrist-only FullGlobal-view Full
Handover Box48.0%80.5%
Frame496.0%97.0%

The Box global-view score is from Table 1A. The two Frame4 runs used the same training and evaluation setup. This shows that the team can coordinate from local visual inputs on Frame4.

Thank you for suggesting this experiment. It helped us see how the usefulness of a shared view can vary by task. In Frame4, the key challenge is for the arms to grasp the frame at about the same time and move it together. The wrist views may already give each arm much of the information it needs to position its gripper and maintain its grasp, while the exchange paths help the arms coordinate. This could explain why the added global view made little difference in Frame4. We retain the shared global camera in our general setup because it gives each arm scene context beyond its wrist view and supports coordination across tasks with different visual demands.

AuthorsOpenReview ↗

Response Part 3/3 | Q3-Q5 | Bimanual controls, training, and reuse

Questions (continued)

Thank you for specifying the comparisons that would help separate the effects of policy design, training recipe, and checkpoint reuse.

Q3. We evaluated joint π0.5 and Independent on all four bimanual tasks using our demonstrations and evaluation conditions. Joint π0.5 sees both arms' observations and predicts both actions. Independent fine-tunes a separate π0.5 for each arm without interaction modules.

TaskSubmitted FullIndependentJoint π0.5
Handover Box80.5%63.5%55.0%
Shoes Table94.5%78.5%87.0%
Handover Mic97.5%98.5%99.0%
Scan Object33.0%23.0%35.5%

We used the existing Box Independent result from Appendix B.4. Full achieved the highest score on Box and Shoes. On Mic, all three scores were close. On Scan, joint scored highest, followed by Full and then Independent.

Decomposing TwinVLA into complete local policies removes the route through which joint attention and the shared decoder directly use both arms' intermediate features. Each Fabric arm has its own path from observation to action and accesses cross-arm information through selected features. If those features omit details needed for coordination, some fine-grained dependencies can be harder to recover. That is a plausible explanation for the Shoes Table gap (70.0% for SingleVLA-Fabric and 84.8% for TwinVLA), and it points to room to improve the exchanged features.

Q4. Common-first had a substantial effect in our SingleVLA design study. As described in Appendix A.3, Single-stage uses 40k updates and Common-first uses 50k in total, including its Common-only initialization. The comparison therefore reflects the complete adaptation recipes, including the difference in total budget.

When we transferred the design to π0.5-Fabric, we found that Full could be trained directly, as in Handover Mic and Frame4. We suspect that π0.5's stronger pretrained manipulation capabilities make the preliminary stage less important. This suggests that the benefit of the schedule can depend on the starting VLA.

On Frame4, Full reaches 194/200 and Independent reaches 78/200. Both use the same demonstrations, pretrained checkpoint, and 200 evaluation conditions.

Q5. Thank you for distinguishing checkpoint reuse from architectural reuse. Table 1A's Shared setting shows that each role can reuse one checkpoint across four bimanual tasks. Based on our experience with supervised VLA fine-tuning, we can continue training from existing role checkpoints while retaining some of the policies’ earlier skills. To extend a three-arm team, we can initialize the existing arms from their checkpoints, add a fourth policy and its peer connections, and continue fine-tuning. Including demonstrations from the earlier tasks can help preserve those skills during adaptation. A role replacement similarly changes the peer information available to the team. We greatly appreciate this suggestion. It gives us a concrete direction for future work on how policy adaptation, skill retention, and coordination can be evaluated together as teams change.

Thank you for the response and the additional experiments. I also checked the revised manuscript as well.

Same-backbone joint and independent comparison on the bimanual tasks, together with budget matched joint policies on the larger teams, supporting the comparison. But still common-first and signle-stage still differ in total updates so the effect of the shcedul cannot be saparated from teh effecto of the budget. And I would agree that skill retention or replacing remains open as future work.

Given above, I maintain my rating. Thanks.

Reviewer_x4k5OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VLA-Fabric, an architecture and training paradigm for coordinating multiple complete, arm-local vision-language-action (VLA) policies via learned inter-policy interaction. Instead of a single joint policy or only high-level role assignment, each arm runs its own full VLA (with local perception and action ownership), and coordination is achieved by exchanging internal representations through three complementary interaction functions: shared semantic context, peer latent context (private K/V), and action-intent coupling. The authors (i) conduct a controlled architecture study to identify effective interaction entry points, aggregation operators, and adaptation procedures; (ii) re-instantiate the same interaction roles in a substantially different VLA backbone (π0.5) to demonstrate cross-architecture transfer; and (iii) scale from two to three and four arms in simulation and on hardware, showing substantial gains over independently trained local policies.

Critical strengths

The work proposes a scalable and composable path to multi-arm coordination that can benefit from future improvements in single-arm VLAs. This separation of concerns—local capability vs. team interaction—has practical importance for building larger, heterogeneous systems.

Critical weaknesses

It is not fully clear if RDT-1B/π0/TwinVLA numbers are reproduced under identical data and observation setups or borrowed from prior work. Adding decentralized-with-observation baselines (e.g., CHORUS) or LatentToM-style message exchange under the same protocol would strengthen claims.

Questions for authors
  1. In RAW Common “operator-level synchronization,” are model parameters shared or only activations exchanged? Could you clarify the precise boundary (and any tied weights) to rule out implicit parameter coupling?

  2. How sensitive are the three interaction roles to model scale, dataset size (e.g., fewer than 50 demos), and domain randomization? Any signs of overfitting to particular coupling patterns?

3 comments
AuthorsOpenReview ↗

Response Part 1/3 | W1a | Published references and bimanual controls

Thank you for a thoughtful review and a precise summary. You captured why we separate local capability from team coordination, and why that separation lets teams benefit as individual VLAs improve.

Our contribution is an initial architecture for explicit exchange between complete VLAs. With standard VLA fine-tuning objectives, we obtained competitive results on several bimanual tasks and demonstrated coordinated execution with teams of up to four arms. This gives us a basis for improving the individual policies and the features they exchange.

During the discussion period, we consolidated the reviews and added experiments to address them. We completed four larger-team joint π0.5 evaluations, two CHORUS configurations, bimanual π0.5 Independent and joint controls, and wrist-only vision evaluations. We draw on these experiments below to address the specific points in your review. We will use the ten-page main text and appendix in the discussion revision to explain the method, protocols, and results. Thank you for guiding these additions.

Weaknesses

W1a. Thank you for asking where the comparison numbers come from. The RDT-1B, π0, and TwinVLA entries in Table 1A use published results. For Box and Shoes we used the numbers collected in the TwinVLA paper, which also provides TwinVLA's Mic and Scan results. The π0 and RDT-1B entries for Mic and Scan come from the RoboTwin leaderboard. We used published results consistently because our preliminary reruns did not reproduce every reference score. For the bimanual π0.5-Fabric policies trained per task, we used 50 demonstrations per task, the same count as TwinVLA's single-task Tabletop-Sim setup. The published numbers serve as references alongside our controlled interaction experiments. Appendix B.3 identifies their sources and settings. To complement these references, we have trained and evaluated our own joint π0.5 on the larger-team tasks. We also completed Independent and joint π0.5 controls on our bimanual tasks using our demonstrations and evaluation conditions.

AuthorsOpenReview ↗

Response Part 2/3 | W1b | Multi-arm baseline comparisons

Weaknesses (continued)

W1b. Thank you for suggesting CHORUS as a decentralized comparison. CHORUS is a multi-robot manipulation policy that shares VLA weights across roles. Each robot acts from its own observations and an identity prompt. Information about teammates is inferred from the local camera views, and the policies exchange no internal representations at runtime. VLA-Fabric studies how explicit exchange of internal features can support coordination among complete VLAs. Comparing the two gives us a way to examine these different approaches to multi-arm coordination.

We evaluated CHORUS on the same four simulation tasks, following its published local-observation, shared-policy setup and principal LoRA training settings. Each arm receives its own workspace and wrist views, local state, and a role prompt.

Fabric also receives a shared scene image, so we evaluated CHORUS-Augmented with the same shared scene and arm-local inputs as Full and a Fabric-style fine-tuning recipe that unfreezes selected backbone layers. The policy still shares weights across roles and sends no messages.

Your suggestion to evaluate CHORUS also prompted us to try a preliminary variant of Fabric. Inspired by how CHORUS learns from demonstrations across robot roles, we trained each Fabric policy on examples from multiple roles and their corresponding arm-local viewpoints. The policies retained separate weights and all three interaction paths. We call this variant Role-mixed Fabric.

We are interested in how VLAs can learn from multiple roles and viewpoints during training. This first experiment gives us a starting point for studying how to sample experience across roles and how to adapt the information the policies exchange. Thank you for suggesting this comparison. It led us to a question we are excited to pursue in future work.

Success rate over 200 trials per task.

MethodFrame3Stack3Arch4Frame4
Fabric Full81.5%91.0%5.0%97.0%
CHORUS67.5%73.5%0.0%11.0%
CHORUS-Augmented80.0%94.5%6.0%21.5%
Role-mixed Fabric82.0%81.5%0.5%66.5%
Joint π0.574.0%36.0%0.0%73.5%

LatentToM offers a useful perspective on learned coordination through ego and consensus representations. It uses visuomotor diffusion policies, while our study examines exchange between complete, pretrained VLAs. The paper describes a general multi-agent formulation and demonstrates it with two arms, which is also the setup in the released training configuration. Running it on our three- and four-arm tasks would require extending that implementation and integrating our task data, control interfaces, and closed-loop evaluator. A same-backbone comparison would also require integrating the latent-coordination mechanism into a pretrained VLA and validating that adaptation. This is beyond the scope of the current revision, so we focused the additional decentralized experiments on the two CHORUS configurations above.

We also evaluated a complementary centralized baseline. A single joint π0.5 receives the shared scene image and all arms' wrist images and states, then predicts all arms' actions together.

TaskJoint π0.5Fabric Full
Frame374.0%81.5%
Frame473.5%97.0%

On the bimanual tasks, joint π0.5 achieved 55.0% on Box, 87.0% on Shoes, 99.0% on Mic, and 35.5% on Scan.

AuthorsOpenReview ↗

Response Part 3/3 | Q1-Q2 | RAW synchronization and sensitivity

Questions

Thank you for asking both where the synchronization happens and what the current experiments tell us about the robustness of the three interaction roles.

Q1. Each arm owns a complete VLA with its own parameter tensors and local action head. The reported configurations have no tied weights across arms. The arms can start from the same pretrained checkpoint, but each has its own copy that can adapt separately. RAW synchronizes forward-pass activations. The exchanged values are outputs of the participating VLA modules.

For two arms, let z_left and z_right be the activation slices at corresponding Common token positions for one operator. RAW replaces both slices at that boundary with their mean, z_common = (z_left + z_right) / 2. In SingleVLA, the Common positions come from the global image and instruction. The synchronized outputs include the pre-attention normalization, the Q, K, and V projections, the attention output projection, the pre-FFN normalization, and the FFN output. In π0.5, the same idea applies at Common-owned positions of the multimodal prefix. Equal activations at a synchronization point come from the averaging operation. The modules that produced those activations retain their own weights.

The other channels have different boundaries. For Private exchange, a receiver uses its own query to attend to peer Private K/V. In π0.5, the action-side query is also local and attends to peer action K/V through a separate gated branch. This distinction matters because Common Q can participate in RAW averaging even though the receiving Private and action-side queries stay local.

Q2. Thank you for asking how the interaction roles respond to changes in data, model scale, and the environment. We agree that these are useful tests of how well the exchanged representations generalize to new coupling patterns.

In this exploratory study, we used standard VLA backbones and existing benchmarks to vary where interaction enters the policy, what is exchanged, and how features are combined. We compared Common K/V averaging with synchronization of intermediate operator outputs, tested different ways to fuse action-side features, and carried the three interaction roles from SingleVLA to π0.5. In the fixed-weight π0.5-Fabric ablation, removing Private K/V reduces Handover Mic success from 97.5% to 11.0%, while Shoes Table changes from 94.5% to 93.5%. This illustrates how the trained policies rely on peer information differently across tasks.

We trained π0.5-Fabric Full on a fixed subset of 25 of the original 50 demonstrations for each of Frame3 and Frame4. Each comparison uses the same 200 evaluation conditions.

Task25 demonstrations50 demonstrations
Frame379.0%81.5%
Frame491.0%97.0%

Halving the demonstration count lowered success by 2.5 percentage points on Frame3 and 6.0 points on Frame4 in these runs. Both policies retained much of their performance, with greater sensitivity to this reduction on Frame4.

Thank you for pointing us in this direction. A next step is to test the individual paths under changes in model scale, teammates, and environments, and use those results to improve the exchanged features. This can inform further work on communication efficiency and how teams adapt.

Reviewer_SLozOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes VLA-Fabric, a framework for multi-arm coordination that aims to enable coordinated action generation through explicit information exchange among independent VLA policies. The authors investigate different interaction mechanisms, including shared context, peer-specific latent features, and action-stage interaction, and evaluate these mechanisms across different VLA architectures. Furthermore, the framework is extended to three- and four-arm systems, with its performance evaluated through both simulation and real-world robotic experiments. The experimental results show that the proposed method achieves higher success rates than independently trained local policies across multiple collaborative tasks.

Critical strengths
  1. The paper focuses on explicit interactions among multiple independent VLA policies, providing a modular approach to multi-arm cooperative manipulation.

  2. The authors systematically investigate different information exchange locations, aggregation mechanisms, and training strategies, and analyze the contributions of individual interaction modules through ablation studies.

  3. The proposed method is validated on two different VLA architectures, SingleVLA and π0.5, demonstrating a certain degree of architectural generalizability.

  4. The experimental evaluation is relatively comprehensive, covering dual-arm, three-arm, and four-arm cooperative tasks, including both simulation and real-world robotic experiments.

  5. In some multi-arm tasks requiring close coordination, the proposed method achieves higher task success rates than independently trained VLA policies.

Critical weaknesses
  1. The paper lacks a clear explanation of how the Common and Private representations are constructed, particularly how they correspond to different VLA architectures. Although the interaction formulas between these representations are provided, a complete description of their sources and computation processes is missing.

  2. Although each VLA maintains an independent action output, the dense interactions across layers and Flow Matching steps introduce substantial coupling among the VLAs during inference. As the number of robots increases, the computational cost of attention, feature communication, and synchronization overhead may also increase. The paper lacks an efficiency analysis and does not sufficiently demonstrate the practical advantages of the proposed approach over a centralized joint policy. Furthermore, separate fine-tuning is still required for different tasks and numbers of robots, limiting the practical composability of the framework.

  3. SingleVLA-Fabric underperforms TwinVLA on some tasks. For example, the success rate on Shoes Table drops from 84.8% to 70.0%. In addition, π0.5-Fabric performs slightly worse than π0 on Handover Mic (97.5% vs. 98.0%). Although the latter difference is small, the paper should provide a more thorough discussion of when the proposed interaction mechanisms are effective and why certain tasks do not benefit from them.

  4. The All-off setting in Table 1B directly disables the interaction modules in a fully trained model, rather than training a model without interactions. Therefore, the observed performance degradation may be influenced by the distribution mismatch between training and inference, and cannot directly establish the effectiveness of the interaction mechanisms. The paper only reports the results of independently fine-tuned π0.5 on Handover Box in the appendix, while corresponding comparisons on other dual-arm tasks are missing.

  5. The proposed method is compared only with TwinVLA, lacking comparisons with other existing multi-arm cooperative manipulation methods.

Questions for authors
  1. Could the authors clarify how the Common and Private representations are constructed, particularly across different VLA architectures?

  2. Given the extensive interactions among different VLAs during inference, what fundamentally distinguishes the proposed framework from a centralized joint policy?

  3. Why SingleVLA-Fabric underperforms TwinVLA on many tasks?

5 comments
AuthorsOpenReview ↗

Response Part 1/4 | W1-W2 | Representations, efficiency, and reuse

Thank you for reading the paper so carefully and for recognizing the systematic study of interaction mechanisms, their transfer across VLA architectures, and the larger-team experiments. Your questions about the exchanged features and the cost of exchanging them show where we could have been clearer.

This exploratory study introduces explicit exchange of internal representations among complete arm-local VLAs. Using familiar VLA fine-tuning objectives, Fabric coordinates teams of two to four arms and approaches established methods on several bimanual tasks. The results vary by task, with room to improve both the local policies and what they exchange.

During the discussion period, we consolidated the reviews and added experiments to address them. We completed four larger-team joint π0.5 evaluations, two CHORUS configurations, bimanual π0.5 Independent and joint controls, and wrist-only vision evaluations. We draw on these experiments below to address the specific points in your review. We will use the ten-page main text and appendix in the discussion revision to explain the method, protocols, and results. Thank you for guiding these additions.

Weaknesses

W1. Thank you for asking how the representations are formed. Each arm runs its own VLA with separate model parameters and no weight sharing across arms. The messages come from outputs of internal VLA modules. The policies average corresponding activations or use selected peer features in their own computations.

In SingleVLA, the shared scene image and instruction supply Common tokens, while each arm's wrist image and state supply Private inputs. RAW Common averages corresponding operator outputs, including Common queries. AVG averages Common keys and values. A local Private query can attend to peer Private K/V, and the action readouts are fused before each arm's local decoder. In π0.5, RAW averages corresponding activations at Common positions in the multimodal prefix. Peer Private Prefix K/V condition the local Action Expert, and a gated attention residual uses peer action-side K/V at each Action Expert layer and flow step.

W2a. We appreciate your attention to the practical costs of inference. The four-arm π0.5-Fabric configuration has about 14.067B parameters across the team. In a one-device-per-arm deployment, each device holds only its own VLA, about 3.517B parameters per role. The devices exchange selected features so the local policies can coordinate. This distributes where the weights reside, while peer attention, data transfer, and synchronization still add work at inference time.

We study communication optimization in a separate project and can share one result from that work here. In four-agent Eagle2/SingleVLA Take Photo, the RAW and optimized runs used the same checkpoint and 50 initial conditions. Team-wide logical payload fell from 310.785 to 34.535 MiB per planning round (88.89%), and mean planning time fell from 875.5 to 322.4 ms per round.

W2b. Thank you for raising the practical side of composability. In the Shared setting of Table 1A, each role uses one checkpoint across four bimanual tasks. The separately fine-tuned larger teams reuse the interaction design and local interfaces. Adapting existing role checkpoints to new tasks and team sizes opens up future work on team composition and on preserving skills as roles change.

AuthorsOpenReview ↗

Response Part 2/4 | W3-W4 | Task coordination and Independent controls

Weaknesses (continued)

W3. Thank you for drawing attention to these performance differences. We agree that the methods perform differently on Shoes Table and Handover Mic. The two tasks place different demands on coordination, which helps explain when explicit exchange is useful. We developed Fabric to let complete VLAs coordinate when an arm needs to respond to what its teammates see or do. That is particularly important as more arms participate in a shared manipulation.

We included Shoes Table as an example of a task with relatively low interaction demands. Each arm can largely complete its assigned manipulation from its own observations, with limited need to adjust its actions in response to the other arm. Indeed, π0.5-Fabric still succeeds in 85.0% of trials with all exchange disabled. Exchange can help on this task, but the local policies can already complete many episodes on their own. We discuss in Q3 how the differences between the architectures of SingleVLA-Fabric and TwinVLA can also contribute to the gap on Shoes Table.

Handover Mic illustrates a different requirement. The receiving arm must judge when the microphone is presented and time its grasp to the other arm’s motion. Peer representations can help the receiving policy decide when to act. Removing peer Private K/V lowers π0.5-Fabric’s success rate to 11.0%, even though its Full result is close to π0. The ablation shows how much the trained Fabric policy relies on exchange for this handover, while the comparison with π0 shows how the complete policies perform on the task.

Your question points to a gap we would like to address in future work. We still lack a systematic way to classify VLA tasks by whether the arms need to interact, what each arm needs to infer about its teammates, and how tightly their actions must be coordinated. Building such a taxonomy and developing ways to measure the strength of these dependencies would help determine when to communicate, what information to exchange, and how precisely to synchronize. We see this as a necessary step toward extending embodied systems to a wider range of tasks and larger teams.

W4. Thank you for pointing out the training-to-inference mismatch. Full uses the interaction modules in training and inference. All-off keeps the Full architecture and weights and disables exchange at inference. It probes the trained model's reliance on exchange, and the drop can also reflect the distribution shift you noted. Independent answers a separate question about performance when the policies are trained from the outset without inter-policy interaction modules. Each arm keeps its assigned observations and runs without exchange at either stage.

We have now completed the bimanual π0.5 Independent comparison.

TaskFabric FullIndependent
Handover Box80.5%63.5%
Shoes Table94.5%78.5%
Handover Mic97.5%98.5%
Scan Object33.0%23.0%

These independently fine-tuned controls complement the All-off results and the larger-team comparison in Table 2. Full achieved higher success rates on Handover Box, Shoes Table, and Scan Object. On Handover Mic, the Independent policies already achieved a high success rate.

AuthorsOpenReview ↗

Response Part 3/4 | W5 | Multi-arm baseline comparisons

Weaknesses (continued)

W5. Thank you for encouraging a broader comparison. Table 1A includes bimanual results for TwinVLA, π0, and RDT-1B. Larger-team comparisons primarily use Independent, with fixed-weight All-off on four-arm Take Photo in Appendix C.3. We agree that an external baseline would complement these controls.

We evaluated CHORUS on the same four simulation tasks, following its published local-observation, shared-policy setup and principal LoRA training settings. Each arm receives its own workspace and wrist views, local state, and a role prompt.

Fabric also receives a shared scene image, so we evaluated CHORUS-Augmented with the same shared scene and arm-local inputs as Full and a Fabric-style fine-tuning recipe that unfreezes selected backbone layers. The policy still shares weights across roles and sends no messages.

Your suggestion to broaden the comparison also prompted us to try a preliminary variant of Fabric. Inspired by how CHORUS learns from demonstrations across robot roles, we trained each Fabric policy on examples from multiple roles and their corresponding arm-local viewpoints. The policies retained separate weights and all three interaction paths. We call this variant Role-mixed Fabric.

We are interested in how VLAs can learn from multiple roles and viewpoints during training. This first experiment gives us a starting point for studying how to sample experience across roles and how to adapt the information the policies exchange. Thank you for encouraging these comparisons. They led us to a question we are excited to pursue in future work.

Success rate over 200 trials per task.

MethodFrame3Stack3Arch4Frame4
Fabric Full81.5%91.0%5.0%97.0%
CHORUS67.5%73.5%0.0%11.0%
CHORUS-Augmented80.0%94.5%6.0%21.5%
Role-mixed Fabric82.0%81.5%0.5%66.5%
Joint π0.574.0%36.0%0.0%73.5%

We appreciate your suggestion to compare with a broader range of cooperative manipulation methods. Beyond the two CHORUS configurations, we considered LatentToM and DuoMind. LatentToM provides a public implementation of a two-arm diffusion-policy setup. Evaluating it on our three- and four-arm tasks would require extending the peer coordination mechanism beyond two arms, adapting the training data and observation and action interfaces, and integrating and validating a closed-loop evaluator. A comparison with the same pretrained VLA would also require implementing and validating its latent coordination mechanism in that backbone.

DuoMind’s public release provides the RoboPoly tasks, demonstrations, and benchmark tools. We could not locate a runnable implementation of the full DuoMind policy in that release. Such an implementation would be needed to adapt and evaluate the policy on the same tasks and under the same conditions as Fabric.

The limited number of learning-based methods we can readily evaluate on these tasks reflects how early this direction is. We intend to continue developing multi-arm VLA coordination and to broaden the comparisons as more implementations become available. For this revision, we added the two CHORUS configurations and evaluated a joint π0.5 policy on the same four simulation tasks. The CHORUS configurations test shared policies that coordinate through observations, and joint π0.5 tests centralized prediction of all arms’ actions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.