- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MotorMind, a framework that uses a frozen general-purpose VLM for zero-shot robotic manipulation. It assigns the VLM multiple roles for planning, action generation, monitoring, verification, and memory, and connects these roles to a deterministic controller through mid-level actions and asynchronous execution. The paper evaluates the framework on LIBERO-PRO, adaptive manipulation tasks, and a real xArm6 robot, reporting task success and execution efficiency.
Critical strengths
-
The paper presents a complete closed-loop manipulation system without task-specific policy training. Its ability to revise actions based on execution feedback, together with the real-robot experiments under human perturbations, provides useful evidence that general-purpose VLMs can support physical manipulation.
-
The evaluation considers execution time alongside success rate. This is consequential for a framework involving repeated VLM calls, as it makes the trade-off between manipulation performance and inference overhead more explicit.
Critical weaknesses
-
The physical execution process is insufficiently specified. The paper describes mid-level actions and controller checks, but does not clearly explain how visual observations are translated into robot-frame motions or how target distance and clearance are obtained. This makes it difficult to reproduce the system and assess how much of its performance comes from the VLM versus the controller and its associated rules.
-
The near-zero performance of most direct zero-shot VLA baselines limits the conclusions supported by these comparisons. Some configurations involve embodiment transfer, so their failures may reflect control-interface or input-distribution mismatch rather than task-level generalization alone. Although the paper also evaluates agentic baselines, a matched comparison using the same VLM and controller would provide clearer evidence for the proposed harness.
Questions for authors
-
Execution clarification: How are visual observations and mid-level actions converted into robot-frame motions? What information is used for target-distance and clearance checks, and which parts require calibration, geometric estimation, or manually specified rules?
-
Baseline clarification: How did you verify that the near-zero VLA results are not primarily caused by embodiment, action-normalization, or control-interface mismatch? Which evaluated configurations support a task-generalization comparison independently of these factors?
-
Controlled comparison: Have you evaluated a simpler synchronous closed-loop agent using the same VLM, observations, controller, and execution budget? Such a comparison would materially affect my assessment of whether the proposed multi-role asynchronous design is responsible for the gains.
6 comments
As the discussion period draws to the close soon, we would greatly appreciate any further feedback or follow-up you may have regarding our rebuttal. If our response has satisfactorily addressed your major concerns, we would also be grateful if you could reconsider our merit and update the rating.
We thanks again for your evaluation, suggestion and your time! Have a wonderful weekend.
Following up on our response
Hi Reviewer X1xu,
We sincerely thank the reviewer for their feedback and suggestions! We have carefully addressed all three concerns through additional clarifications and experiments:
- Execution transparency: We expanded the implementation details, including calibration, geometric estimation, controller checks, and the respective responsibilities of the VLM and controller.
- Baseline validity: We clarified the impact of embodiment mismatch of VLA evaluation, and provided additional comparisons to better assess task-level generalization.
- Controlled ablations: We conducted matched experiments using the same VLM, observations, controller, and budget, demonstrating an improvement from 45.0% to 66.7% success over the synchronous baseline, alongside further component ablations.
We hope these revisions and new results have addressed the reviewer's concerns and strengthened the paper. We would be very grateful if the reviewer could kindly revisit their assessment once they got chance. We truly appreciate the reviewer's time, consideration, and valuable feedback throughout the review process.
Response to Reviewer X1xU
We thank the reviewer for the precise questions and for recognizing the closed-loop system and the efficiency evaluation. We answer each question below.
We have updated the PDF with more detail (Section 3; Appendix C, Table 8, Fig. 8; Appendix D,; Appendix J)
In short, each control cycle has three steps:
We also provide more details on the three ingredients the reviewer asks about:
Everything else is decided by the VLM: task decomposition, target localization, action selection, monitoring and completion judgment are all calls to the same model. No learned detector, segmentation model, IK solver or motion planner is used.
We also add the VLM versus controller results in Table 11: In 18,241 proposed actions, 68.4% are executed, 29.0% are rejected before execution, and 2.6% are not reached. A rejection ends the batch, and its reason goes back to the VLM, which writes the next proposal. The checks bound and filter individual motions; they never choose the target object or the plan. The prompts for every role are in Appendix J.
We’d like to clarify that the near-zero results of zero-shot VLA exactly show the generalization limitation of VLA.
With the same evaluation code, observation transforms and action conversion, the LIBERO-adapted checkpoints reach high success: 98.3% (π0.5), 100.0% (MolmoAct2) and 98.3% (OpenVLA-OFT). Each zero-shot model runs with its own released preprocessing, action decoding and normalization, listed per model in Appendix E.2.
We further analyze the rollouts of each zero-shot VLA results. The base checkpoints do not produce coherent, target-directed motion on the LIBERO arm. This points to an embodiment and action-distribution mismatch, not a task-level error. It is the limitation that motivates our approach: a VLA policy needs embodiment-specific adaptation before it can act, whereas MotorMind's VLM-facing interface is unchanged between the simulated Franka and the real xArm6.
In addition, we provide the comparisons independent of embodiment mismatch. The LIBERO-adapted VLAs run on their own embodiment, so their perturbation results isolate task-level generalization:
Adapted VLAs lead when the task matches their training distribution. When object positions or the task itself change, they fall to 10–37%, and MotorMind leads at 51.7% and 58.3% with no task-specific training. The agentic baseline CaP-X does not depend on VLA transfer and uses stronger models, yet it reaches 13.3% base and 19.2% under perturbation, against 66.7% and 53.8% for MotorMind.
We additionally add a Serial execution ablation. It keeps the same VLM, observations, prompts, action interface, controller and budget, and removes only concurrency: monitoring and memory updates run between steps. Results on 60 paired episodes, with the full system re-run for this comparison:
The asynchronous design accounts for a 21.7-point gain and saves 80 s per episode. The mid-level action interface accounts for 26.7 points over 6D pose and 48.4 points over 2D points. Within the roles, removing replanning lowers success to 36.7%, and removing the Verifier lowers it to 60.0%.