- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DuoMind is a distributed hierarchical framework for two-robot coordination. Each robot uses a VLM orchestrator for high-level decisions and message generation, together with a VLA policy for continuous low-level control. Robots exchange structured natural-language messages about intentions, subgoals, and beliefs. The paper also introduces RoboPoly, a seven-task manipulation benchmark, and evaluates on eight RoboTwin tasks adapted to distributed arm control. Evaluation uses 400 rollouts per task, direct VLA baselines, communication ablations, and an alternative action model. Results show improvements on several tasks, particularly Cook Pot, where communication increases success from 1.00% to 39.25% within the hierarchical system.
Critical strengths
-
The framework connects language-based coordination to closed-loop continuous manipulation rather than evaluating high-level plans alone. Tasks such as handovers and jointly lifting a pot require coordination to survive execution, making the system and benchmark relevant to robot foundation-model research.
-
The communication ablation supplies concrete evidence that messages can matter within the proposed system. The large Cook Pot improvement is particularly consequential because it corresponds to a task with a clear synchronization requirement, rather than relying only on qualitative examples of cooperation.
Critical weaknesses
-
The main comparison confounds high-level orchestration with predefined task structure and supervision. The prompt on page 20 supplies arm-specific candidate stages and their order, requires sequential advancement, and forbids skipping or returning to earlier stages. On RoboPoly, DuoMind also uses stage-level training instructions while direct VLA baselines use whole-task instructions. Consequently, the results do not establish how much improvement comes from VLM reasoning rather than the supplied stage program, finer supervision, or progress rules.
-
The specific contribution of semantic communication is not established beyond access to peer information. A no-message ablation demonstrates that information sharing helps, but not that rich natural-language messages outperform simple stage and completion signals. Since the orchestrators also share a global camera, the benefit should be distinguished from compensating for otherwise invisible physical state. This distinction is central to the paper's positioning around distributed semantic coordination.
Questions for authors
-
Clarification of the controller and comparison. Please provide the complete candidate-stage sequences and the precise forced-advancement rule. Is there a comparison using the same low-level weights, stage supervision, and ordered candidates with a simple visual stage controller? If not, which evidence separates autonomous orchestration from execution of the supplied stage program?
-
Clarification of communication and observations. Which inputs are private to each orchestrator, and which global or wrist-camera observations are supplied to the VLM and VLA separately? Do existing results show that intention/subgoal/belief messages add value beyond compact stage identifiers and completion signals?
5 comments
Response to Reviewer 9vFJ (2/3)
Weakness 2
The specific contribution of semantic communication is not established beyond access to peer information. A no-message ablation demonstrates that information sharing helps, but not that rich natural-language messages outperform simple stage and completion signals. Since the orchestrators also share a global camera, the benefit should be distinguished from compensating for otherwise invisible physical state. This distinction is central to the paper’s positioning around distributed semantic coordination.
Thank you for this insightful comment on semantic communication. We acknowledge that a short signal can be sufficient for simple synchronization movements. However, DuoMind targets coordination across complex, long-horizon tasks, where each robot requires richer shared information to reason about its peers’ near-term intentions and expected outcomes. This motivates our inclusion of intention, subgoal, and belief fields in the communication messages. To better quantify the contribution of each component, we add ablation experiments that individually remove the intention, subgoal, and belief fields.
Table 2. Ablation study on communication components. Success rates (%) across RoboPoly tasks.
| Method | Cook Pot | Prepare Snack | Clean Table |
|---|---|---|---|
| DuoMind w/ | 39.25 | 18.00 | 24.75 |
| No intention | 0.00 | 18.00 | 19.00 |
| No subgoal | 0.00 | 16.00 | 13.00 |
| No belief | 0.00 | 19.00 | 20.00 |
| No communication | 0.00 | 13.50 | 17.25 |
The results show that removing individual message components can lead to performance degradation, as agents lose important information about their peers' intentions, goals, or beliefs needed for effective coordination. This is particularly evident in Cook Pot, where successful execution requires precise synchronization between the two robots. This further highlights the importance of our message component design in enabling effective multi-robot coordination.
We apologize for the confusion caused by the term "global camera," which may misleadingly suggest access to complete global information. In fact, this camera is positioned at the side of the workspace and provides only a limited side view of the environment, rather than full observability of the entire workspace or access to privileged physical states. To avoid this misunderstanding, we have renamed it the "shared camera" throughout the paper.
This camera configuration is fully compatible with our distributed framework, as it provides a common visual observation of the shared workspace without compromising the decentralized nature of decision-making. While the shared camera allows agents to partially observe their peers' behavior, it does not directly reveal their internal intentions, subgoals, or beliefs, which are explicitly conveyed through inter-agent semantic communication. Importantly, our no-message ablation retains identical camera inputs, isolating the benefit of explicit communication from visual access. Furthermore, the additional communication ablations described above quantify the contributions of individual message components, providing a more detailed assessment of the effectiveness of semantic communication in distributed coordination.
Response to Reviewer 9vFJ (3/3)
Question 1
Clarification of the controller and comparison. Please provide the complete candidate-stage sequences and the precise forced-advancement rule. Is there a comparison using the same low-level weights, stage supervision, and ordered candidates with a simple visual stage controller? If not, which evidence separates autonomous orchestration from execution of the supplied stage program?
As detailed in Appendix I, we provide the orchestrator with the following system prompt to enable candidate-based instruction selection:
This ManiSkill task supplies an arm-specific candidate set. The L1/R1-style prefixes are metadata tags, not part of the low-level instruction. Select exactly one candidate, but put only the sentence after its colon in low_level_instruction; do not invent, paraphrase, include the tag, or select another arm's instruction.
Permitted candidates for this <ARM_SIDE> arm:
<THIS_ARM_CANDIDATES>
Here, <ARM_SIDE> and <THIS_ARM_CANDIDATES> are predefined in the task configuration. For example, in Food Serve, the candidate instructions are:
L1: pick up mug and place on tray
L2: pick up bread and place beside mug on tray
R1: pick up bread and place in the middle
We also specify the following stage-advancement rule as provided in Appendix I:
Progress rule: do not keep issuing a subgoal that is already complete. Review the current image and your own history when available. If visual evidence or your own prior belief says the current subgoal's observable result is complete, you MUST select a different permitted candidate that advances the unfinished high-level task. Otherwise repeat the current candidate. Do not advance merely because time passed, and visually verify claims from the other agent.
In addition to these prompt-level instructions, we introduce a stagnation-prevention mechanism that triggers instruction advancement when the orchestrator repeatedly selects the same instruction due to the fault from VLM's limited visual capability. For example, we set the threshold to 6 high-level planning steps for Food Serve and 10 steps for Prepare Snack, preventing prolonged stagnation at a single stage.
Regarding orchestration, we adopt candidate-assisted instruction selection to address the challenges of visual grounding and reasoning in complex multi-robot tasks, particularly given the limitations of the compact VLM used in our original experiments. Candidate-based selection is also a common approach to structuring VLM-based robotic decision-making. Furthermore, our GPT-6 Astra experiments discussed in Weakness 1 demonstrate that a stronger reasoning backbone can operate without predefined candidates or stage-progression constraints, further validating the effectiveness of our framework when equipped with a more capable orchestrator. We emphasize that our primary contribution lies in the hierarchical framework for distributed multi-robot coordination, while the performance reported in our original experiments is partly constrained by the reasoning capabilities of the VLM available at the time.
Question 2
Clarification of communication and observations. Which inputs are private to each orchestrator, and which global or wrist-camera observations are supplied to the VLM and VLA separately? Do existing results show that intention/subgoal/belief messages add value beyond compact stage identifiers and completion signals?
Thank you for the opportunity to clarify the observation and communication setup. We apologize again for the misleading term "global camera," which we have now renamed "shared camera" throughout the paper. This camera is positioned at the side of the workspace and provides only a limited side view, rather than complete global information. Both the orchestrator (VLM) and the action model (VLA) receive observations from this shared camera and the robot's own wrist camera, which remains private to each robot. Importantly, DuoMind does not require a shared camera, as each robot can independently obtain visual observations of the workspace. In our experiments, we use a shared camera to reduce the number of cameras and computational overhead. Therefore, this observation configuration is fully compatible with our distributed multi-robot framework.
The ablation experiment for individual components in the shared messages are presented in our response to Weakness 2.
Response to Reviewer 9vFJ (1/3)
We sincerely thank Reviewer 9vFJ for their constructive feedback and valuable suggestions. Below, we address the reviewer's comments and provide additional clarifications on the key concerns raised.
Weakness 1
On RoboPoly, DuoMind also uses stage-level training instructions while direct VLA baselines use whole-task instructions. Consequently, the results do not establish how much improvement comes from VLM reasoning rather than the supplied stage program, finer supervision, or progress rules.
Thank you for your insight on this important point. On one hand, we adopt an orchestrator with candidates to address the challenges of visual grounding and reasoning in complex multi-robot tasks, particularly given the limitations of the compact VLM used in our original experiments. Candidate-based selection is also a common approach to structuring VLM-based robotic decision-making [1][2], as low-level action models may struggle to reliably execute arbitrary instructions beyond their training distribution. On the other hand, recent advances in foundation models, such as GPT-6 Astra, have introduced stronger visual and embodied reasoning capabilities, potentially reducing the need for predefined task structures. To investigate this, we add experiments using GPT-6 Astra as the orchestrator, without any candidate lists and sequential advancement, and compare against the constrained setting under matched execution budgets. The experiment results shown as below:
Table 1. Comparison with GPT-6 orchestrator. Success rates (%) across three RoboPoly tasks.
We observe that GPT-6 Astra performs below the original Qwen3-VL-4B orchestrator, while still surpassing the π0.5-only baseline. Without candidate constraints, GPT-6 Astra produces more diverse instructions, some of which deviate from the action model's training distribution and are difficult to execute reliably. This indicates that the performance difference stem partly from the action model's limited ability to follow unconstrained instructions, rather than from the orchestrator's reasoning capability alone.
Meanwhile, the improvement over the VLA-only baseline provides further evidence of the effectiveness of DuoMind's distributed hierarchical coordination. These findings also emphasize that overall performance depends on the compatibility between high-level reasoning and low-level execution. A stronger foundation model does not necessarily yield better results when its instructions are not well aligned with the action model's capabilities. Nevertheless, DuoMind's modular design allows both components to be improved independently, suggesting that integrating more capable orchestrators with better-aligned action models could further enhance multi-robot coordination.
[1] Nasiriany et al. PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs. Secs. 3.1–3.2. Paper
[2] Buoso et al. Select2Plan: Training-Free ICL-Based Planning through VQA and Memory Retrieval. IEEE RA-L, 10(11), 11267–11274. Sec. III. Paper