acceptodds
Under review as a conference paper at ICLR 2027

CoLA: Multi-VLA Coordination via Learned Adapters

Abstract

Vision-Language-Action (VLA) models are typically deployed as isolated single-agent policies. When several such agents share a workspace, they can coordinate implicitly, by observing one another, or explicitly, by exchanging messages, but it remains unclear when explicit communication becomes necessary. We introduce CoLA (Coordination via Learned Adapters), which keeps each agent's VLA backbone frozen and trains a lightweight message channel and a per-arm diffusion action head on top, with fully decentralised execution. Across three tasks that differ in how much each agent can observe, the channel matters only when information is private. On an occluded tray handover, where the box's colour marker is visible to one agent only, CoLA reaches 76.0% success versus 1.3% with messages zeroed throughout training. On two-arm and three-arm handover tasks, removing messages has no measurable effect with a shared overhead camera (90.6% vs. 88.3% at two arms; 63.5% vs. 60.0% at three), but lowers success on the two-arm task's wrist-only variant (74.8% vs. 64.3%). This architecture generalises across different VLA backbones as well. With a frozen π0.5 backbone on the same occluded tray task, CoLA matches the overall success rate of a LoRA-fine-tuned baseline (37.3% vs. 34.7%) using a fraction of the trainable parameters. When CoLA places the box, it selects the correct tray 71.8% of the time, whereas the independently fine-tuned baseline, whose placing arm never sees the marker, selects it at chance level (34.7% against 33.3%).

Primary area:
applications to robotics, autonomy, planning
Keywords:
multi-agent coordination, vision-language-action models, learned communication, robot manipulation, imitation learning, partial observability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4nC2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CoLA, a framework for decentralized multi-robot coordination using frozen Vision-Language-Action (VLA) backbones. The method introduces lightweight learned communication adapters and per-arm diffusion-based action heads, trained jointly through behavior cloning on multi-robot demonstrations. Each robot independently processes its local observations, exchanges learned latent messages with its partners, and generates coordinated actions without a centralized controller. The authors evaluate CoLA on three simulated manipulation tasks under four observation configurations, including an occluded tray handover, two-arm handovers with wrist-only or shared overhead observations, and a three-arm handover chain. The main empirical finding is that learned communication provides substantial benefits when task-relevant information is private to individual agents, while its additional contribution is limited when agents have access to shared overhead observations. The paper also evaluates CoLA with a frozen π0.5 backbone, showing improved conditional target selection over an independently LoRA-fine-tuned baseline, although the overall task success rates are comparable. The work aims to investigate when explicit communication is useful for decentralized coordination while preserving pretrained VLA representations.

Critical strengths

S1. A well-defined and relevant research question. The paper addresses an important practical question in multi-robot learning: when does explicit inter-agent communication improve decentralized coordination over implicit observation alone? Rather than treating communication as universally beneficial, the authors examine its dependence on information availability. This distinction is meaningful for systems built from pretrained VLA models, particularly when modifying backbone parameters is undesirable or computationally expensive.

S2. Useful experimental decomposition and transparent reporting. The paper includes a communication ablation in which messages are zeroed during training and evaluation, allowing the contribution of communication to be examined separately from some of the other architectural components. The per-stage success rates in Table 10 are particularly helpful for distinguishing physical handover failures from target-selection errors. I also appreciate that the authors explicitly acknowledge several limitations of their results, including the unexpected failure of the zero-message model during the tray handover and the lack of a clear overall success advantage in the π0.5 experiment. The additional cross-backbone experiment and conditional success analysis strengthen the empirical presentation.

Critical weaknesses

W1. The main communication ablation does not fully establish the proposed causal explanation. The strongest reported result is on the occluded tray handover, where overall success decreases from 76.0% to 1.3% when comparing the full CoLA model against a separately trained variant with messages zeroed throughout both training and evaluation. However, Table 10 shows that the physical handover completion rate also decreases dramatically, from 92.7% to 8.0%. In comparison, the independent head-only baseline achieves an 88.7% handover rate without communication. This raises an important concern: the large overall performance difference cannot be attributed solely to the inability to transmit the hidden color information. It may also reflect more general coordination information, differences in learned action generation, or training instability in the zero-message model. The authors acknowledge this issue, which is appreciated, but the current evidence does not fully resolve it. A stronger no-message control capable of completing the physical handover, together with more targeted interventions on message content, would substantially improve the causal interpretation. The methodological novelty also needs clearer differentiation from simpler alternatives. In the tray task, the central private variable has only three possible values. An explicit color-label message, a predicted task-state message, or an oracle-information baseline would help determine whether the learned 64-dimensional communication representation provides benefits beyond sharing readily identifiable task information.

W2. Baseline differences and limited robustness evidence weaken the generality of the conclusions. The proposed approach differs from the Octo baselines in several components beyond communication, including the action-generation head, proprioceptive encoding, action horizon, and optimization procedure. In particular, Section 4.2 states that the head-only Octo baselines use a newly initialized proprioceptive tokenizer that remains frozen, whereas CoLA trains its proprioceptive encoder. This difference may disadvantage the head-only baselines. Although the message-zeroed CoLA variant controls for these architectural differences, its severe handover failure on the tray task complicates the interpretation of the largest reported improvement. Further investigation of training stability or a stronger, carefully tuned non-communicating baseline with matched action-generation and proprioceptive components would help clarify the contribution of communication. The experimental evidence for robustness is also limited. The reported results are based on a single trained replicate per method and experimental configuration, without repeated independent training seeds. Furthermore, the three-arm CoLA results are evaluated on only one 200-episode block. Table 11 shows that the 32-dimensional message variant achieves only 42.0% success, compared with 73.3%-76.0% for other tested dimensions. The same table reports substantial degradation under altered lighting conditions. Finally, the claim that communication is unnecessary under shared observations should be interpreted within the tested camera configurations, rather than as a general conclusion about fully or partially observable multi-robot systems. The paper also reports a nominal 5 Hz action-chunk inference schedule without measuring complete online inference latency, which limits the evidence for real-time deployment.

Questions for authors

Q1. What causes the severe physical handover failures in the message-zeroed model? On the occluded tray task, the message-zeroed CoLA model completes the handover in only 8.0% of episodes, compared with 92.7% for full CoLA and 88.7% for the independent head-only baseline. Could the authors clarify whether this behavior arises from missing coordination information, optimization instability, or degradation of the learned action policy? A more targeted experiment separating physical handover performance from hidden-color communication would be particularly informative. Clarification of this issue would substantially affect my assessment of the paper's main claim.

Q2. Does learned latent communication provide advantages over simpler task-state communication? For the occluded tray task, could the authors compare CoLA against a baseline that transmits the marker color directly, or uses a lightweight classifier to predict and transmit the color? Similarly, for the wrist-only handover, how does communicating a predicted object pose compare with the learned message? I would also be interested in whether the authors can provide a matched independent-policy baseline with the same diffusion action head and trainable proprioceptive encoder. This would help separate the benefit of communication from that of the revised action-generation pipeline.

Q3. How stable are the findings across training runs and observation conditions? Could the authors clarify the large performance drop for the 32-dimensional message ablation and, if available, provide results from independent training seeds? Additional clarification of the three-arm evaluation variability and the substantial performance degradation under altered lighting would help establish the robustness of the conclusions. For the π0.5 experiment, a no-message CoLA ablation would also help isolate the communication contribution from differences in manipulation reliability. Finally, reporting measured end-to-end online inference latency would help substantiate the deployment-related claims.

2 comments

Response to Reviewer 4nC2 (1/2)

We thank the reviewer for the detailed and fair review. All new results use 5 independent training seeds (mean ±\pm SD). We also designed a new variant of the occluded tray task with a randomised starting position. All tray results below are on the randomised variant of the task, where CoLA reaches 89.3±3.789.3 \pm 3.7% vs. 19.9±3.219.9 \pm 3.2% without messages.

W1 / Q1: Why does the message-zeroed model fail the handover?

Why the handover failed for message-zeroed. First, the submitted result came from the original fixed start task. After running the original task with 5 seeds, handover completion ranges from 47.3% to 98.0% with messages and from 1.3% to 38.0% without, so the submitted 8.0% came from a fragile, high-variance setting. However, the model without messages still completes fewer handovers in every seed. Second, a consistent gap still remains on the randomised start variant of the tray task (97.1±2.997.1 \pm 2.9% vs. 66.7±6.966.7 \pm 6.9%), and our experiments point to missing information about the partner's state. On two arms handover task with overhead, replacing each arm's learned message with its partner's joint angles and gripper opening is comparable to CoLA (83.9±4.283.9 \pm 4.2% vs. 80.0±5.680.0 \pm 5.6%, 95% CI [−6.9,+14.8][-6.9, +14.8]), against 55.8±3.355.8 \pm 3.3% without messages. Moreover, delaying CoLA's messages by one policy query lowers tray handover completion from 96.7% to 82.1%, so arm B relies on up to date message content to complete the handover.

Targeted intervention on message content. From the moment arm B first touches the box, we replace A's message with a recorded message from a demonstration of a different colour. Handovers still complete 96.4±3.396.4 \pm 3.3% of the episodes, and B places the box in the new colour's tray (demonstration colour) in 94.7±1.494.7 \pm 1.4% of them, and in the original tray in only 3.2±1.23.2 \pm 1.2%. Arm B received the correct colour for roughly the first half of the episode, yet the later message still determines which tray arm B chooses.

W1 / Q2: Learned message vs. simpler task-state communication.

We replace A's learned message to B with a hand-built vector, a one-hot colour, and optionally together with Arm A's normalised joint angles and gripper opening, zero-padded to 64d. Arm B's message to A, the action head, and all other modules are the same as CoLA. True colour rows are trained from scratch with the colour read directly from the simulator. The classifier is a small MLP on A's wrist features (rollout accuracy 82.6% for colour only, 97.5% for colour + state). The classifier experiments reuse the true colour checkpoints, so the gap from the true colour rows is caused by purely misclassification.

Table 6: CoLA vs. explicit colour messages (tray task)

What B receives from ASuccess Rate (%)Δ\Delta vs. CoLA (points) [95% CI]
CoLA (learned message)89.3±3.789.3 \pm 3.7–
True colour71.1±4.171.1 \pm 4.1−18.3-18.3 [−24.4,−12.2][-24.4, -12.2]
Classifier colour67.7±4.667.7 \pm 4.6−21.6-21.6 [−30.0,−13.2][-30.0, -13.2]
True colour + A's state98.4±0.998.4 \pm 0.9+9.1+9.1 [+4.5,+13.6][+4.5, +13.6]
Classifier colour + A's state95.3±1.795.3 \pm 1.7+6.0+6.0 [+1.2,+10.8][+1.2, +10.8]

Colour alone underperforms CoLA because due to lower handover rates. Adding A's state fixes this problems and exceeds CoLA by 6-9 percentage points. A hand designed message can therefore outperform CoLA if it is crafted to tailor to the task specific hidden information. CoLA's advantage is that the same module is used unchanged across tasks and backbones (Octo and π0.5\pi_{0.5}), without the colour labels or specifying what to send.

Table 7: Box-position messages on the two arm handover, wrist camera (task success %, 5 seeds)

What each arm receives from its partnerSuccess (%)Δ\Delta vs. CoLA (points) [95% CI]
CoLA85.2±6.985.2 \pm 6.9–
True box position81.1±3.481.1 \pm 3.4−4.0-4.0 [−11.2,+3.1][-11.2, +3.1]
Predicted box position80.9±3.380.9 \pm 3.3−4.3-4.3 [−14.8,+6.2][-14.8, +6.2]

For the wrist only handover, we replaced each arm's learned message with the box's position (x, y, z), either the true position from the simulator, or position predicted by a small regressor on the sender's own wrist-camera features and joint state. Everything else matches CoLA's architecture. Both variants reach about 81% success, 4 points below CoLA, a difference that is not significant, and both clearly outperform the model without messages (68.6±6.168.6 \pm 6.1%). The predicted position performs as well as the true one (−0.3-0.3 points, [−5.9,+5.4][-5.9, +5.4]). Thus, handcrafted, task specific messages are able to achieve similar performances to CoLA. However, CoLA is task agnostic, and able to enable coordination across diverse tasks. CoLA is re-evaluated here through the same evaluator as the new variants (85.4±6.285.4 \pm 6.2% reported).

Response to Reviewer 4nC2 (2/2)

Q2 / W2: Matched independent-policy baseline.

This is CoLA (message zeroed) in the paper. CoLA (message zeroed) uses the same diffusion head, trainable proprioception encoder and training recipe and hyperparameters. With 5 training seeds: CoLA outperforms the independent policy baseline, CoLA (message zeroed), across all 5 tasks.

W2: Frozen proprioceptive tokenizer in head-only baselines.

This is a fair point. Our claims, however, rest on CoLA vs. CoLA (message zeroed). The full fine-tuning baselines train all parameters in Octo and is a stronger comparison to CoLA. For example, the decentralised full finetuning baseline matches CoLA on two arm handover, overhead camera (84.7±6.484.7 \pm 6.4% vs. 80.0±5.680.0 \pm 5.6%), while CoLA is more efficient to train.

Q3: Stability across runs and conditions.

Seeds. All methods now use 5 training seeds (0, 1000, 2000, 3000, 4000).

32d message. The 42.0% was single-run noise on the original task. On the randomised start variant of the tray task: 86.5±4.786.5 \pm 4.7, 89.2±3.989.2 \pm 3.9, 89.3±3.789.3 \pm 3.7 and 88.9±2.788.9 \pm 2.7% for 16d, 32d, 64d and 128d.

Three-arm. Each seed is now evaluated on 1,000 episodes. Seed variance is large on the overhead variant with CoLA ahcieving 67.2±11.667.2 \pm 11.6% success rate and no message model reaching 19.1±7.219.1 \pm 7.2%.

Lighting. Performance of CoLA in the hidden tray task still degrades under unseen lighting: 72.0±8.572.0 \pm 8.5% (bright), 70.9±5.970.9 \pm 5.9% (dim), 53.1±8.353.1 \pm 8.3% (dark), vs. 89.3%. Most of the loss is in the handover (70.5% completion in the dark condition), while if the box reaches a tray, arm B chooses the correct tray 78-87% of the time, against 33.3% by chance. We attribute this mainly to the frozen backbone's visual features and we will report it as a limitation in the paper.

π0.5\pi_{0.5} without messages. 21.6±2.021.6 \pm 2.0% vs. 93.1±4.993.1 \pm 4.9% for CoLA on the same frozen backbone (+71.5+71.5 points).

Q3: Inference latency.

300 calls per arm at batch size 1, after 30 warm-up calls. Each call covers the backbone on the camera frame, state embedding, message encoding/decoding, and 10-step diffusion sampling of a 10-action chunk.

Table 8: Inference Latency of CoLA

Backbone \quadGPUOne armBoth arms
OctoRTX A600079–80 ms158 ms
OctoA100 40GB89–92 ms183 ms
π0.5\pi_{0.5}RTX A6000189–193 ms377 ms
π0.5\pi_{0.5}A100 40GB \quad223–231 ms \quad456 ms

With Octo, one arm's call (~80 ms) is under the 200 ms query period, leaving ~120 ms for message exchange, and both arms fit on one A6000. π0.5\pi_{0.5} fits only with one A6000 per arm, where CoLA head ran on CPU, contributing 60-90 ms latency. We have not measured a real network link, however, robustness to simulated message delay and dropout (including CoLA trained with random delay and loss) is in our response to Reviewer qkUS.

W2: Communication is unnecessary under shared observations should be interpreted within the tested camera configurations

We agree. With 5 seeds, the no message CoLA variant underperforms in a shared overhead camera from 80.0±5.680.0 \pm 5.6% to 55.8±3.355.8 \pm 3.3% (two arms) and from 67.2±11.667.2 \pm 11.6% to 19.1±7.219.1 \pm 7.2% (three arms). CoLA outperforms its matched no message variant in every setting tested in the paper, with the largest performance gained when the task-relevant variable is private to one agent (tray: +69.5+69.5 points). We will revise the paper's claims accordingly and scope them to the tested camera configurations.

Reviewer_qkUSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The proposed CoLA enables multiple robot arms to cooperate by exchanging compact learned messages while keeping their pretrained VLA backbones frozen. Only communication modules and new action heads are trained from demonstrations. Its main finding is that communication helps most when one robot possesses information its partner cannot see. On a handover task with a hidden colour marker, CoLA achieves 76% success, compared to 22.7% for the strongest independent baseline. With a shared overhead camera, removing messages has little effect. Experiments cover two- and three-arm simulated tasks and two backbone models. The approach offers efficient training, but remains untested on physical robots and assumes instantaneous, synchronized communication.

Critical strengths
  1. It investigates when communication is useful, showing that its value depends on whether task-relevant information is private or already shared.
  2. Retraining the same architecture with messages zeroed helps distinguish communication benefits from improvements due to the new action-generation pipeline.
  3. Hidden-colour prediction probes and above-chance tray selection support the claim that agents transmit information their partners cannot directly observe.
Critical weaknesses
  1. Zeroing messages reduces tray-task success from 76% to 1.3%, but also collapses handover completion. That drop cannot be attributed solely to losing hidden-colour information.
  2. CoLA changes the action head, gripper prediction, proprioception encoding, and action chunking. Its gains over independent VLA baselines therefore do not isolate the communication mechanism.
  3. Every method uses a single training run; some key results also use only one evaluation block. Evaluation variability does not establish robustness across training seeds.
  4. Experiments cover three scripted simulation tasks, with only one explicitly testing hidden semantic information. Broader tasks are needed to establish when communication is necessary or redundant.
  5. There are no physical-robot experiments, communication is synchronous and delay-free, and pair-specific decoders scale quadratically with team size.
Questions for authors
  1. Why does zeroing messages collapse handover completion, and how can you isolate the benefit of transmitting hidden-colour information?
  2. How does CoLA compare with independent policies using the same action head, proprioception encoder, and training recipe?
  3. Are the main findings consistent across multiple independent training seeds?
  4. Does communication remain useful across more varied tasks with partial observability and larger robot teams?
  5. How robust is CoLA to communication delays and dropouts, and does it transfer to physical robots?
  6. What causes the reduced manipulation reliability with π0.5\pi_{0.5} despite successful information transfer?
2 comments

Response to Reviewer qkUS (1/2)

We thank the reviewer for the constructive feedback and the suggestions. All results currently use 5 independent training seeds (mean ±\pm SD). We designed a new variant of the tray task with a randomised start pose, and all tray results below are on this randomised start variant of the task.

W1 / Q1: Why did zeroing messages also collapses the handover. How can the benefit of transmitting the colour be isolated?

With 5 seeds on the randomised start variant of the tray task, the effect is smaller than in the single-run submission but still present. In the new tray task, the handover completion is 97.1±2.997.1 \pm 2.9% with messages and 66.7±6.966.7 \pm 6.9% without.

Why the handover collapsed. First, the submitted result came from the original fixed start task. After running the original task with 5 seeds, handover completion ranges from 47.3% to 98.0% with messages and from 1.3% to 38.0% without, so the submitted 8.0% came from a fragile, high-variance setting. However, the model without messages still completes fewer handovers in every seed. Second, a consistent gap remains on the randomised start variant of the tray task (97.1±2.997.1 \pm 2.9% vs. 66.7±6.966.7 \pm 6.9%), and our experiments point to missing information about the partner's state. On two-arm overhead, replacing each arm's learned message with its partner's joint angles and gripper opening is comparable to CoLA (83.9±4.283.9 \pm 4.2% vs. 80.0±5.680.0 \pm 5.6%, 95% CI [−6.9,+14.8][-6.9, +14.8]), against 55.8±3.355.8 \pm 3.3% without messages. Moreover, delaying CoLA's messages by one policy query lowers tray handover completion from 96.7% to 82.1%, so arm B relies on up to date message content to complete the handover.

Isolating colour transmission. From the moment arm B first touches the box, we replace A's message with a recorded message from a demonstration of a different colour. Handovers still complete (96.4±3.396.4 \pm 3.3%), and B places the box in the new colour's tray (demonstration colour) in 94.7±1.494.7 \pm 1.4% of the rollouts, and in the tray matching the marker (original colour) in only 3.2±1.23.2 \pm 1.2%. This means that arm B relies heavily on the updated message for the task. We also compare against hand built colour messages in our response to Reviewer 4nC2.

W2 / Q2: Comparison with independent policies using the same components and recipe.

That comparison is CoLA (message zeroed) in the paper: same action head, proprioception encoder, chunking and training hyperparameters, with messages zeroed in both training and evaluation. With 5 seeds it reaches 55.8±3.355.8 \pm 3.3% on two-arm overhead (CoLA 80.0±5.680.0 \pm 5.6%), 68.6±6.168.6 \pm 6.1% on two-arm wrist (CoLA 85.4±6.285.4 \pm 6.2%) and 19.9±3.219.9 \pm 3.2% on the tray task (CoLA 89.3±3.789.3 \pm 3.7%). We will state this more explicitly in the paper.

W3 / Q3: Are the findings consistent across seeds?

Every method now has 5 training seeds, with 1,000 evaluation episodes per seed on the two arms and three arms handover tasks and 150 on the tray task. One finding in the paper is corrected, we had stated that communication does not improve success under a shared view. With 5 seeds, removing messages lowers two-arm overhead success from 80.0±5.680.0 \pm 5.6% to 55.8±3.355.8 \pm 3.3%, where we had reported 90.6% vs. 88.3%.

The corrected findings are consistent across all 5 seeds, every CoLA vs. no-message difference is significant. Evaluation is not deterministic, so success rates vary between re-evaluations of the same checkpoints. Re-evaluating the five CoLA tray checkpoints gives 87.1±5.787.1 \pm 5.7% against the reported 89.3±3.789.3 \pm 3.7%, which does not affect comparisons with large differences

W4 / Q4: More varied tasks and larger teams.

Within the current tasks, the benefit of messages stays true in all five settings (two arm overhead and wrist, three arm overhead and wrist, hidden tray), and with both Octo and π0.5\pi_{0.5} backbone on the tray task. We agree that three scripted tasks might not be able to establish when communication is necessary in general, and we will state this in the limitations. Additionally, we all also state that teams larger than three arms is untested.

Response to Reviewer qkUS (2/2)

W5 / Q5: Delays and dropouts, physical robots, decoder scaling.

Delays and Dropouts. CoLA trained with a perfect channel is fragile to delays and dropouts that it never saw in training. With messages delayed by one policy query (200 ms), success drops from 87.1±5.787.1 \pm 5.7% to 62.1±9.662.1 \pm 9.6% on the hidden tray task, and from 80.6±5.680.6 \pm 5.6% to 32.0±28.532.0 \pm 28.5% on two arms handover task, with overhead camera.

Training with impairments solves most of the performance drop. We retrained CoLA (5 seeds) with each message delayed by up to 2 queries (400 ms) and up to 25% of messages lost, a lost message being replaced by the last one received (hold).

Table 5: Delay and Dropout Result Table (Success Rate %)

Test conditionCoLA (Clean) \quadCoLA (Impairment Trained)Difference [95% CI]
Tray task
Clean channel87.1±5.7\mathbf{87.1 \pm 5.7}75.1±11.075.1 \pm 11.0−12.0-12.0 [−20.4,−3.6][-20.4, -3.6]
Delay 1 query (0.2 s)62.1±9.662.1 \pm 9.680.8±9.2\mathbf{80.8 \pm 9.2}+18.7+18.7 [+10.2,+27.2][+10.2, +27.2]
Delay 2 queries (0.4 s) \quad26.1±6.726.1 \pm 6.778.4±7.3\mathbf{78.4 \pm 7.3}+52.3+52.3 [+40.4,+64.1][+40.4, +64.1]
25% loss, hold63.3±2.963.3 \pm 2.975.5±13.175.5 \pm 13.1+12.1+12.1 [−1.2,+25.4][-1.2, +25.4]
50% loss, hold42.3±6.342.3 \pm 6.372.8±7.4\mathbf{72.8 \pm 7.4}+30.5+30.5 [+23.1,+38.0][+23.1, +38.0]
Two-arm overhead
Clean channel80.6±5.680.6 \pm 5.683.8±4.983.8 \pm 4.9+3.2+3.2 [−2.4,+8.8][-2.4, +8.8]
Delay 1 query (0.2 s)32.0±28.532.0 \pm 28.578.1±5.2\mathbf{78.1 \pm 5.2}+46.1+46.1 [+6.9,+85.3][+6.9, +85.3]
Delay 2 queries (0.4 s)23.5±27.723.5 \pm 27.763.5±5.863.5 \pm 5.8+40.0+40.0 [−0.3,+80.2][-0.3, +80.2]
25% loss, hold52.9±9.852.9 \pm 9.879.3±7.0\mathbf{79.3 \pm 7.0}+26.5+26.5 [+8.7,+44.3][+8.7, +44.3]
50% loss, hold34.2±13.134.2 \pm 13.172.4±5.9\mathbf{72.4 \pm 5.9}+38.1+38.1 [+15.8,+60.5][+15.8, +60.5]

Task success(%), mean ±\pm SD over 5 seeds.

Clean trained CoLA evaluation numbers here come from a separate evaluation of the same checkpoints, so they differ slightly from the 89.3% / 80.0% reported in the main results table. Even with 50% of messages lost (twice the maximum training rate), the impairment trained CoLA keeps 72-73% success on both tasks. On a clean channel, it matches clean CoLA on two arm handover (+3.2+3.2, [−2.4,+8.8][-2.4, +8.8]) but is lower on the tray task (−12.0-12.0, [−20.4,−3.6][-20.4, -3.6]). Under an out of distribution 1s delay, it reaches 33.1% on the tray task and 48.6% on two-arm.

Physical Robots. Sim to real transfer is outside the scope of this paper, which studies the effect of a communication channel in multiagent VLA collaboration. we will state this as a limitation.

Pair-specific decoders grow quadratically with size. We also state that pair-specific decoders grow quadratically with team size as a limitation. Possibility of sharing decoders across partners is a promising direction we have not yet tested.

Q6: Why is manipulation less reliable with π0.5\pi_{0.5} despite successful information transfer?

This does not appear on the randomised start variant of the tray task with 5 seeds, so we attribute it to the original task's demonstrations rather than the backbone and CoLA. On the randomised start variant of the task, CoLA on a frozen π0.5\pi_{0.5} completes the handover in 96.7±4.596.7 \pm 4.5% of episodes and successfully choose the correct tray in 93.1±4.993.1 \pm 4.9% of the episodes, slightly above the Octo version (89.3±3.789.3 \pm 3.7%). Decentralised LoRA fine-tuning of π0.5\pi_{0.5} completes the handover equally often (96.4±2.996.4 \pm 2.9%) but selects the tray at chance (34.8±2.334.8 \pm 2.3%), for 33.6±3.033.6 \pm 3.0% success.

Reviewer_qdrnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when explicit inter-agent communication improves coordination among vision-language-action (VLA) policies, particularly under partial observability. CoLA keeps each agent’s VLA backbone frozen and trains a compact message encoder, partner-specific message decoders, and a diffusion action head on top. Each agent broadcasts a latent message derived from its local observations and proprioception; received messages are decoded and, together with the agent’s own representation, condition a temporal U-Net that generates action chunks. Training combines a diffusion noise-prediction loss with a binary gripper classification loss. Experiments on three simulated tasks (an occluded tray handover, a two-arm handover with wrist-only or overhead views, and a three-arm handover chain), with Octo-Base and, on one task, π0.5 backbones, suggest that communication is most useful when task-relevant information is private to one agent, and largely redundant when agents share a view of the workspace.

Critical strengths

The research question of when explicit communication is necessary for coordinating frozen policies is clearly posed and relevant to multi-robot deployment.

Critical weaknesses

While not directly on VLA, multi-agent communication is a long-standing research problem. [1-3] The key result from this work basically reveals that an explicit channel matters in proportion to how much task-relevant information is private to one agent. This aligns with common intuition. It would be more interesting to see if there is a quantification metric to show when and how much communication is required.

[1] LEARNING WHEN TO COMMUNICATE AT SCALE IN MULTIAGENT COOPERATIVE AND COMPETITIVE TASKS, ICLR2019 [2] Learning Attentional Communication for Multi-Agent Cooperation, Nips19 [3] TarMAC: Targeted Multi-Agent Communication, ICML2019.

The comparisons cover only fine-tuning variants of the backbone (head-only and full fine-tuning, decentralised and centralised) and a zeroed-message ablation. No existing method for multi-agent coordination or learned communication is evaluated. Moreover, the zeroed-message CoLA already outperforms these baselines on the handover tasks, so most of the reported gains reflect differences in the action pipeline, not in coordination. As a result, it is difficult to judge how CoLA compares with established alternatives.

Questions for authors

This task carries the headline result (76.0% vs. 1.3%) and the π0.5 comparison. In the released code, the box position range is fixed at (0, 0) (data_collection/handover_marker/scripted_policy.py), the evaluator resets without start-pose randomisation, and the marker colour is assigned deterministically. I think it's worth demonstrating the generalization by testing more init states.

How would the authors compare the proposed method with other multiagent communication framework?

4 comments

Response to Reviewer qdrn (1/2)

We thank the reviewer for the constructive feedback and for reading the released code. All results below use 5 independent training seeds (mean ±\pm SD), with 150 evaluation episodes per seed on the tray task and 1,000 on the two arms and three arms tasks.

W1: The finding is intuitive, but a quantification of when and how much communication is needed would be more interesting.

We agree that "communication helps when information is private" is the expected direction. However, three of our findings are less intuitive:

  1. The benefit is not limited to private information. With a shared overhead camera, removing messages lowers success from 80.0±5.680.0 \pm 5.6% to 55.8±3.355.8 \pm 3.3% at two arms, and from 67.2±11.667.2 \pm 11.6% to 19.1±7.219.1 \pm 7.2% at three arms. This corrects the single-run result in our submission, with 5 seeds, communication improves coordination success rate even when observations are shared.

  2. Communication carries more than the private variable. Replacing A's message with the one-hot true marker colour gives 71.1±4.171.1 \pm 4.1% success, below CoLA's 89.3±3.789.3 \pm 3.7%, and handover completion falls from 97.1±2.997.1 \pm 2.9% to 75.6±5.375.6 \pm 5.3%. This means that the hidden tray task requires more than just sharing of private information.

  3. The message causally determines the target. From the moment arm B first touches the box, we replace A's message with a recorded message from a demonstration of a different colour. B places the box in the new colour's tray (demonstration colour) in 94.7±1.494.7 \pm 1.4% of handovers, and in the tray matching the marker (original colour) in only 3.2±1.23.2 \pm 1.2% of the handovers.

Quantifying when communication is needed. We added a second marker on the box's top face that arm B can see in a share qq of episodes. The benefit of communication falls as the private information becomes observable:

Table 3: Communication benefit vs. colour visibility to arm B (tray task, 5 seeds)

Colour visible to B \quadCoLANo messages \quadDifference(points) [95% CI]
0% of episodes89.3±3.789.3 \pm 3.719.9±3.219.9 \pm 3.2+69.5+69.5 [+65.0,+74.0][+65.0, +74.0]
50% of episodes88.0±4.488.0 \pm 4.436.4±3.836.4 \pm 3.8+51.6+51.6 [+44.5,+58.7][+44.5, +58.7]
100% of episodes96.5±3.096.5 \pm 3.0 \quad61.7±3.861.7 \pm 3.8 \quad+34.8+34.8 [+27.7,+41.9][+27.7, +41.9] \quad

At q=0.5q = 0.5, the gap is larger in episodes where B cannot see the colour (+61.3+61.3 points) than where it can (+41.9+41.9). The gap that remains at q=1q = 1 reflects the partner-state information discussed in our response to Reviewer qkUS.

Q1: Fixed initial states in the tray task.

Thank you for the suggestion. We originally fixed arm A's starting position to isolate the task to transferring private information. After analysing the results, we realised that small drifts from the demonstrated trajectories during rollouts compounded into failed handovers or misplaced boxes. This is because the arms only learned a single fixed trajectory per colour. Therefore, we design the new variant the task with a randomised starting position for arm A in the expert demonstrations. On this variant of the hidden tray task, CoLA reaches 89.3±3.789.3 \pm 3.7% vs. 19.9±3.219.9 \pm 3.2% without messages. With a frozen π0.5\pi_{0.5} backbone, CoLA reaches 93.1±4.993.1 \pm 4.9%, vs. 21.6±2.021.6 \pm 2.0% without messages and 33.6±3.033.6 \pm 3.0% for decentralised LoRA fine-tuning.

Response to Reviewer qdrn (2/2)

Q2 / W2: Comparison with existing multi-agent communication methods.

We added two external comparisons, each with 5 training seeds (0, 1000, 2000, 3000, 4000):

MIMIC-D: coordination without learned messages [1]. A decentralised diffusion policy per agent, conditioned on its own observation and its partners' states, with no message channel (no communication). This tests whether sharing partner state is enough. We also run a privileged variant that receives the true box pose instead of camera images.

TarMAC-style channel [2]. We replace CoLA's message modules with a TarMAC-style channel (without TarMAC's recurrent state or one-step delay). Each receiver's incoming message is an attention-weighted average of all arms' 64-d values, including its own, weighted by how well its 16d query matches each sender's 16-d signature. Backbone, action head and training hyperparameters are identical to CoLA.

Table 4: Multiagent Baseline Comparison Table (Success Rate %):

MethodTray (rand. start)2-arm overhead2-arm wrist3-arm overhead3-arm wrist
CoLA \quad89.3±3.789.3 \pm 3.780.0±5.680.0 \pm 5.685.4±6.285.4 \pm 6.267.2±11.667.2 \pm 11.674.1±6.574.1 \pm 6.5
TarMAC-style71.2±8.671.2 \pm 8.684.0±6.984.0 \pm 6.985.8±5.385.8 \pm 5.365.0±17.065.0 \pm 17.073.9±8.173.9 \pm 8.1
MIMIC-D33.6±2.633.6 \pm 2.671.1±3.671.1 \pm 3.671.4±1.771.4 \pm 1.766.4±2.966.4 \pm 2.967.7±2.967.7 \pm 2.9
MIMIC-D, privileged \quad30.0±3.630.0 \pm 3.674.3±3.174.3 \pm 3.174.3±3.174.3 \pm 3.170.9±1.970.9 \pm 1.970.9±1.970.9 \pm 1.9

Privileged MIMIC-D uses no camera, so its overhead and wrist results are the same. The full table, including the Octo fine tuning baselines, is in our general response (Table 2).

When information is private, CoLA outperforms. On the tray task, CoLA outperforms the TarMAC-style channel by 18.1 points (paired 95% CI [3.8,32.5][3.8, 32.5]). TarMAC completes the handover as often as CoLA (98.8% vs. 97.1%) but selects the correct tray less often given a handover (72.1% vs. 92.0%), so the gap lies in carrying the hidden colour. MIMIC-D completes almost every handover (99.9%) but picks the tray at chance (33.6%), as expected since arm B has no access to the colour.

For Handover tasks, CoLA is comparable to the baselines. At two arms with an overhead camera, the TarMAC-style channel has the highest mean (84.0%), but its difference from CoLA is not significant (+4.0+4.0 points, [−0.2,+8.2][-0.2, +8.2]), nor is CoLA's lead over MIMIC-D (+8.9+8.9, [−2.0,+19.9][-2.0, +19.9]). With wrist cameras, CoLA and TarMAC are comparable (85.4% vs. 85.8%), and CoLA outperforms MIMIC-D (+14.0+14.0, [5.2,22.8][5.2, 22.8]). On the three-arm chain, the differences are small and not significant, apart from a marginal lead over MIMIC-D with wrist cameras.

W2: Zeroed-message CoLA already outperforms the baselines, so gains reflect the action pipeline.

With 5 seeds, CoLA without messages outperforms only the head-only baselines. It is below decentralised full fine-tuning on two-arm overhead (55.8±3.355.8 \pm 3.3% vs. 84.7±6.484.7 \pm 6.4%) and on the tray task (19.9±3.219.9 \pm 3.2% vs. 34.7±2.634.7 \pm 2.6%). Decentralised full fine-tuning of Octo also matches full CoLA on two-arm overhead (84.7% vs. 80.0%). CoLA's advantage against full finetuning is that the backbone stays frozen. CoLA outperforms the decentralised full finetuning for the 3 arm handover tasks (67.2% vs. 35.8%), as well as the tray task (89.3% vs. 34.7%). Every claim we make about communication is compared against CoLA without messages, which share the same action head, proprioception encoder and training hyperparameters with CoLA.

References

[1] D. Dong, M. Bhatt, S. Choi, and N. Mehr, “MIMIC-D: Multi-modal Imitation for MultI-agent Coordination with Decentralized Diffusion Policies,” in 2026 IEEE International Conference on Robotics and Automation (ICRA), IEEE, June 2026, pp. 17193–17200. doi: 10.1109/ICRA57385.2026.11697073.

[2] A. Das, T. Gervet, J. Romoff, D. Batra, D. Parikh, M. Rabbat, and J. Pineau, "Tarmac: Targeted multi-agent communication," in Proceedings of the International Conference on Machine Learning (ICML), PMLR, 2019, pp. 1538–1546.

The authors have partially addressed my concerns. I would raise my score to 3.

Nevertheless, I suggest the authors clarify whether learning the message itself offers a distinct advantage over simpler ways of sharing task-relevant partner information. And provide more analysis of the message itself rather than only showing superior results.

We thank the reviewer for the follow up and for raising the score.

Does learning the message offer a distinct advantage over simpler ways of sharing task-relevant partner information?

A hand-built message matches the learned message (CoLA) only when it contains every variable the task needs, and those variables are either given or predicted by a model trained on their labels. CoLA is able to remove these requirements by extracting the variables from what the sender already observes (its frozen backbone feature and joint state) using the demonstrated actions.

Table R9 collects every hand-built message we tested (Tables 3 and 11 in the paper, and Section 4.3, Q4).

Hand-built messageTaskSuccess (%)vs. CoLA [95% CI]
Partner's joint angles and gripper2-arm HO (OH)83.9±4.283.9 \pm 4.2+3.9 [-6.9,14.8]
True box position2-arm HO (W)81.1±3.481.1 \pm 3.4-4.0 [-11.2,3.1]
Predicted box position2-arm HO (W)80.9±3.380.9 \pm 3.3-4.3 [-14.8,6.2]
True colourTray71.1±4.171.1 \pm 4.1-18.3 [-24.4,-12.2]
Classifier colourTray67.7±4.667.7 \pm 4.6-21.6 [-30.0,-13.2]
True colour + A's stateTray98.4±0.998.4 \pm 0.9+9.1 [4.5,13.6]
Classifier colour + A's stateTray95.3±1.795.3 \pm 1.7+6.0 [1.2,10.8]

A hand-built message that contains every variable the task needs matches the learned one (partner state, box position) or beats it (colour + arm A's state, by 6 to 9 points). We do not claim that a learned message beats a well chosen hand-built one.

However, the hand-built message fails if it misses a variable. With colour alone (the private variable for the tray task), success drops by 18 to 22 points. This is mostly due to the handover failures (75.6% vs. 97.1% handover completion). Sharing partner state without the colour (MIMIC-D, which conditions on the partners' proprioception) completes the handover in 99.9% of episodes but chooses the tray at chance, reaching only 33.6% success (Table 9 in the paper).

Additionally, the hand-built messages that work need extra supervision unless they are given the true values. For example, the colour classifier is trained on the simulator's colour labels, and the box-position regressor on the simulator's box positions. In contrast, CoLA learns its message from the demonstrated actions alone, with the same architecture in all five settings and on both backbones. The distinct advantage of learning the message is therefore that it does not require knowing in advance what to send, or labels for it.

More analysis of the message itself.

We thank the reviewer for the additional suggestions. In the paper, we analysed the message directly:

What the message contains (Section 4.4, Table 14). Before arm B chooses a tray, the colour is at chance (33%) in B's own input but decodable at 100% from the decoded message it receives. With messages zeroed it stays at chance, so the colour reaches B only through the channel.

Whether arm B actually acts on the message (Section 4.4). If, from the moment arm B first touches the box, we replace arm A's message with one recorded for a different colour, arm B places the box in the swapped colour's tray in 94.7±1.494.7 \pm 1.4% of completed handovers, and in the tray matching the scene in only 3.2±1.23.2 \pm 1.2%.

The message carries time-varying partner state (Table 12). If the message encoded only a constant per episode, delaying it would not matter. On the two-arm overhead handover, where there is no private colour, a one-query delay (0.2 s) lowers success from 80.6% to 32.0%. Thus, arm B relies on the message being in sync with its partner's current state.

Does the message matter if the variable is no longer private? (Table 3, top). As B sees the colour itself in more episodes, the gap between CoLA and its message-zeroed model shrinks from 69.5 to 34.8 points but does not close. The remaining gap is consistent with the partner-state component above.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank the reviewers for their insightful reviews. In response, we have strengthened the evaluation and added the requested comparisons.

1. Multi-seed evaluation All results now use 5 training seeds (0, 1000, 2000, 3000, 4000), reported as mean ±\pm SD.

2. Randomised start variant of the hidden tray task In the original task, arm A has the same starting position, so the demonstrations contained only one handover trajectory per colour. Rerunning it with 5 seeds shows very high variance across training seeds (range across seeds in parentheses):

Table 1: Fixed Start Hidden Tray Task (Success Rate %), 5 training seeds

Original task (fixed start)CoLACoLA, no messages
Task success (%) \qquad41.9±17.1(17.3-64.0)41.9 \pm 17.1 \quad (17.3\text{-}64.0) \qquad2.5±2.4(0.0-6.0)2.5 \pm 2.4 \quad \quad (0.0\text{-}6.0) \qquad
Handover (%) \qquad72.5±19.8(47.3-98.0)72.5 \pm 19.8 \quad (47.3\text{-}98.0) \qquad13.6±16.0(1.3-38.0)13.6 \pm 16.0 \quad (1.3\text{-}38.0) \qquad
Correct tray | handover (%) \qquad56.3±14.2(36.6-71.6)56.3 \pm 14.2 \quad (36.6\text{-}71.6) \qquad24.5±25.2(0.0-66.7)24.5 \pm 25.2 \quad (0.0\text{-}66.7) \qquad

The submitted 76.0% came from a single training run with seed 0. Because the policies learned fixed trajectories, small drifts during the handover compounded and moved them out of the training distribution. This causes arm B to freeze in many rollouts, or the box was released too early or too late, hence a low handover rate. We therefore randomised arm A's starting position in the demonstrations, so that the policies learn from a range of handover trajectories. On the randomised start variant, CoLA reaches 89.33.7% against 19.93.2% without messages, with low variance across seeds. All hidden tray results are below in the full results table.

3. We revised the claim that communication is redundant under a shared view. With 5 seeds, removing messages lowers success under a shared overhead camera from 80.05.6% to 55.83.3% at two arms, and from 67.211.6% to 19.17.2% at three.

New baselines (5 seeds each, trained on the same demonstrations)

  • MIMIC-D [1]. A decentralised diffusion policy for each agent that conditions on its own observation and its partners' states, with no message channel. It tests whether sharing partner state is enough to coordinate. A privileged variant receives the true box pose in place of camera images.
  • TarMAC-style channel [2]. CoLA with its message modules replaced by a TarMAC-style channel, without TarMAC's recurrent state or one-step delay. Each receiver's incoming message is an attention-weighted average of all arms' 64d values, including its own values, weighted by how well its 16d query matches each sender's 16d signature. The frozen backbone, action head and training recipe are unchanged, so only the communication channel is swapped.

Table 2: Full Results Table (Success Rate %), 5 training seeds

MethodTwo-arm (overhead)Two-arm (wrist)Three-arm (overhead)Three-arm (wrist)Hidden tray (wrist)
CoLA (ours)
CoLA80.0 ± 5.685.4 ± 6.267.2 ± 11.674.1 ± 6.589.3 ± 3.7
CoLA, no messages55.8 ± 3.368.6 ± 6.119.1 ± 7.246.7 ± 7.219.9 ± 3.2
Octo fine-tuning
Octo decentralised, head only48.4 ± 7.752.0 ± 5.54.5 ± 1.620.3 ± 4.818.3 ± 3.1
Octo decentralised, full fine-tune84.7 ± 6.475.9 ± 15.935.8 ± 12.716.4 ± 5.334.7 ± 2.6
Octo centralised, head only †39.4 ± 19.623.4 ± 8.7––18.4 ± 8.5
Octo centralised, full fine-tune †84.6 ± 7.777.7 ± 7.4––98.1 ± 1.1
Multi-agent baselines
MIMIC-D (partner states + camera)71.1 ± 3.671.4 ± 1.766.4 ± 2.967.7 ± 2.933.6 ± 2.6
MIMIC-D, privileged (partner states + true box pose) ‡74.3 ± 3.174.3 ± 3.170.9 ± 1.970.9 ± 1.930.0 ± 3.6
TarMAC-style channel84.0 ± 6.985.8 ± 5.365.0 ± 17.073.9 ± 8.171.2 ± 8.6

Task success (%), mean ± SD over 5 seeds. † Sees all arms' cameras and true arm states. ‡ Uses the true box pose instead of cameras. Rows marked † and ‡ use privileged information, so they are upper references and are not bolded. –: not run. Note the tray task here is the randomised start variant.

References

[1] D. Dong, M. Bhatt, S. Choi, and N. Mehr, “MIMIC-D: Multi-modal Imitation for MultI-agent Coordination with Decentralized Diffusion Policies,” in 2026 IEEE International Conference on Robotics and Automation (ICRA), IEEE, June 2026, pp. 17193–17200. doi: 10.1109/ICRA57385.2026.11697073.

[2] A. Das, T. Gervet, J. Romoff, D. Batra, D. Parikh, M. Rabbat, and J. Pineau, "TarMAC: Targeted multi-agent communication," in Proceedings of the International Conference on Machine Learning (ICML), PMLR, 2019, pp. 1538–1546.

Dear reviewers,

Thanks for serving as a reviewer. As the discussion period comes to a close and the authors have submitted their rebuttals, I kindly ask you to take a moment to review them and provide any final comments.

If you have already updated your comments, please disregard this message.

Thank you once again for your dedication to the OpenReview process.

Best,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.