- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper focuses on that basic vision-language models are hard to translate to robot control though it has world knowledge, and proposes Show-Harness, a model-free semantic action interface. It compresses the control space into 15 discrete semantic units: 6 translation units, 6 incremental rotation units, 1 GRASP/ 1 RELEASE, and 1 DONE. The model only chooses the direction of movement, while the movement range is determined by translators written for each robot, which also enforce workspace and safety constraints.
Then, the paper builds a closed-loop shell consisting of 9 plugins from perception, reasoning, and action, and presents two usage modes: ZS uses Gemini-3.1 Pro to directly output semantic units via prompts with no training, and FT fine-tunes only the linear layers of Qwen 3.5-2B using LoRA (about 3% of parameters, with the visual encoder frozen).
However, there are already works like Guava (2026), which proposes a general Harness framework and semantic action abstraction, and DSD (2026), which decomposes direction/scale at the action token level. These works are not discussed in the Related Work, nor included as compared experiments.
Critical strengths
Main results show average success rates across tasks of ZS 89.0% and FT 86.0% (strongest baseline RATS 57.0%), across environments 100.0% and 88.0%, and across robots 93.0% and 87.0%. The paper also provides horizontal ablations for six types of action interfaces, cross-design experiments between the interface and the shell, factor experiments for action naming and rule prompts, and step attribution experiments. References are timely, and comparison methods are taken from work in the past two years.
Critical weaknesses
- Key conclusions lack uncertainty and pairing significance. The main table contains only single-point estimates with 10 trials per task and only 20 trials across environments. At 10 trials, the 95% normal approximation half-width for individual success rates can reach approximately 0.3, while the Wilson interval half-width is about 0.26. The difference between 6/10 and 10/10 for the same model cannot determine whether it stems from task effects or sampling fluctuations, yet the summary employs strong terms like "generalize robustly." It is recommended to supplement uncertainty intervals for each task and include paired tests and effect sizes under shared layouts for key comparisons, thereby narrowing the summary conclusions accordingly..
- The action interface ablation lacks external method comparisons. Table 3's interface ablation only compares the L0–L4 abstraction layers, without incorporating complete strong baselines such as π0.5, GR00T, H-VLA, G-VLA, CaP-X, and RATS under the same control variables for interface-level comparison. All six layers were implemented by the authors, with their performance ceiling depending on prompt format, action granularity, and skill library coverage, making it difficult to rule out the possibility of weaker implementations serving as controls. After introducing specialized insertion skills for high-level skills in the paper, the fine-grained task performance improved from 42.5 to 62.5. It is recommended to include at least one published external method with similar interface hierarchy, retraining the controls under the same task and budget conditions.
- The metric "step" has inconsistent cross-method definitions. One step for semantic agents corresponds to a single model decision (a semantic unit), while π0.5 and GR00T adopt the "predict 8 action chunks, execute 4 and replan" mechanism (A.2.9). The main text never defines whether their "step" refers to the number of replans or the number of executed actions at 10 Hz. The 50-step limit and timeout failures thus represent different physical execution volumes for the two method types, raising doubts about the comparability of success rate figures. It is recommended to clarify the definition of "step" for each method and unify the limit criteria, or align comparisons under the same model call budget.
- The fine-tuning mode underperforms the zero-shot mode in fine-grained and reasoning tasks. After reducing the step size, the zero-shot performance improved from 60% to 80%, while fine-tuning only increased from 40% to 65%; in reasoning tasks, the two modes scored 85% and 10%, respectively, with fine-tuning reaching only 70% even after external subtask instructions. The paper does not explain this discrepancy. Since low-cost deployment is one of the two core claims, it is recommended to supplement a detailed comparison of the two modes under identical prompts and interfaces, as well as their performance on reasoning probes before and after fine-tuning, to distinguish between the backbone capacity limit and the erosion of instruction-following capability.
- The version reproducibility of closed-source models requires clarification. The paper itself demonstrates that the directional mapping accuracy of open-source models with 300 billion parameters under the same interface is significantly lower than that of state-of-the-art interface models, indicating that the benefits of the interface still largely depend on the backbone strength. Since such services cannot be frozen long-term, version information is essential for assessing transferability. It is recommended to list the complete version identifiers and access times for each model, along with at least one minimal reproducible configuration based on publicly downloadable backbones.
- Expressions such as "cross-robotic" and "capable of operating robots" exceed the scope of evaluation. The actual assessment is confined to desktop single-arm and dual-arm parallel gripper grasping and placing tasks, and the paper acknowledges that it has not yet been validated on humanoid robots or dexterous hands. However, the title and conclusions employ broader coverage, including terms like "expensive robot-specific pre-training." The more diverse scenarios in the appendix are qualitative examples. It is recommended to limit the conclusions in the abstract and introduction to desktop parallel gripper grasping and placing tasks and explicitly enumerate more complex robots as limitations.
- Two settings of cross-object migration are presented side by side but without emphasizing the differing strengths of migration evidence between them. The zero-shot mode only requires replacing the translator to switch to another robot, while the fine-tuning mode jointly trains on data from both robots, representing migration evidence of different intensities. Additionally, the paper has observed that the migration loss for orientation-related units is significantly lower than that for grasping-related units, which serves as strong supporting evidence for its design claims yet is only briefly mentioned. It is recommended to move the mechanism of defining orientation in the image coordinate system and its sub-item data to the method section, along with detailing the composition of training data for each row.
- The residual failure of this article itself has not been attributed for analysis. The paper breaks down the failure of competitive methods into specific causes, but it still has about 10% failure in cross task analysis and does not provide an equally granular composition; Due to the mechanistic claims of gradually correcting errors and terminating invalid searches through interfaces, the distribution of residual failures directly determines their functional boundaries. Suggest supplementing the proportion of failure modes by task category and distinguishing interface limitations from sensing limitations such as lack of force and tactile feedback.
- The default activation strategy and respective costs of plugins have not been fully disclosed. The paper has provided independent contributions by closing each plugin one by one, but has not explained the model call and word element overhead brought by each plugin. This not only affects the selection in resource constrained scenarios, but also makes it difficult for readers to judge the level that can be achieved with the minimum engineering investment. Suggest adding the average number of steps, model calls, and inference latency when each plugin is enabled and disabled.
- The human baseline lacks expert remote control. Among the three operators, one without robot experience completed the vast majority of tasks after 15 minutes of explanation and 5 practice sessions, but lacked the reference of skilled operators, and this score may only reflect the lower difficulty of the tasks. Suggest adding a set of experienced operator results using conventional remote operation or graphical end control.
- Several technical details need to be clarified. Firstly, the step size is inconsistent between the main text and the appendix, and a complete set of steps and switching criteria need to be provided; Secondly, the paper states that the demonstration can be reused across ontologies, but the real machine corpus is recorded with a uniform step size. It needs to be explained how to maintain consistent displacement when the step size does not match; Thirdly, the paper highlights the relative advantage of the visual language action model, which does not require action heads or special lexical elements. However, similar works that output discrete motion lexical elements are also listed. It is suggested to rephrase this as not requiring new action tokenizers and regression heads.
Questions for authors
None.
3 comments
Follow-up Response to R3 dPER
We sincerely thank the reviewer for raising the score and for the follow-up feedback. Our initial response focused on reporting the experimental results for each point; below we connect the core insights across these results and clarify how the issues in the Paper Summary are addressed.
Paper Summary and W2: Guava, DSD and Published Systems
The comparisons with Guava and DSD (Sec. 2, Tabs. 3 and 7) share one insight: our gain over both comes from leaving motion magnitudes to the interpreter.
- Guava. At the framework level, Guava couples its harness with external perception models (SAM3 segmentation and depth back-projection) and object-centric tools mixed with metric poses. In Tab. 3, these lift Guava above our bare-pose layer L1 (65.8% vs. 50.0%), confirming their value for coarse transport (87.5%, against 97.5% for ours). On fine tasks, however, object tools cannot bypass sub-centimeter alignment: Guava must still output metric coordinates and drops to 37.5%, whereas Show-Harness, with no external perception model, reaches 80.0% by keeping the VLM in a closed directional loop and leaving step sizes to the interpreter.
- DSD. DSD decouples direction and scale at the action-token level. Rebuilt in our training pipeline (Tab. 7), this factorization lifts binned tokens from 8.9% to 30.0%, confirming the benefit of separating direction, while our semantic units reach 66.1% because the model no longer predicts a scale at all.
W4, W5 and G1: How ZS and FT Complement Each Other
Rather than competing pipelines, ZS and FT play complementary roles on the shared action space, as the new probe, open-weight and dynamic-task results show:
- ZS (Gemini-3.1 Pro) brings frontier reasoning with no robot data: 85% on reasoning tasks and 91.5% on the reasoning probes.
- FT (Qwen3.5-2B) is a minimal reproducible open-weight configuration that runs locally at 39 ms per decision. It nearly matches Gemini on the Plate tasks (45/50 vs. 48/50, W5) and leads on moving targets (45/60 in simulation and 17/20 on the truck, against 31/60 and 9/20 for π0.5, G1).
- Fine and reasoning tasks (W4). Adding back the inputs omitted by FT's minimal prompt narrows the gap to ZS on both fine tasks (65% → 70% with proprioception text, vs. 80% for ZS) and reasoning tasks (10% → 70% with the planner's subtasks, vs. 85% for ZS). The probes before and after fine-tuning (2B: 47.0% → 43.0%; 9B: 63.5% → 61.0%) show that the remaining reasoning gap comes mainly from the small backbone's capacity, while LoRA fine-tuning costs only 2.5 to 4 points.
W1, W3, W6 to W11: Plugin Costs and the Revised Manuscript
- Plugins (W9). Enabling Multi-View Guidance, Proprioception, Action History or Failure Recovery reduces both calls and episode time (28 to 34 → 26 calls; 5.6 to 7.0 → 5.4 min), because early state feedback and grasp rollback prevent oscillation and unguided wandering after missed grasps.
- Manuscript. The revised manuscript defines steps and timeouts for every method (Sec. 4.1), explains cross-embodiment transfer (Sec. 3.2, App. A.2.10) and reports paired tests and intervals (App. A.3.3). The abstract, introduction and Limitations now state the tested scope (tabletop manipulation on single- and dual-arm parallel-jaw robots), and the abstract's cross-task, cross-environment and cross-embodiment claims rest on these tests (every gain over the strongest baseline is statistically significant, p < 3e-4).
If any specific question remains, we would warmly welcome the chance to address it. The detailed recommendations have already helped us strengthen the paper substantially; we hope the analyses above and the revised manuscript resolve the remaining concerns, and would be grateful if they could be considered in the final assessment.
Final Manuscript Update to R3 dPER
We sincerely thank the reviewer again for raising the score and for the detailed recommendations. Before the discussion closes, we would like to briefly note one final update in the uploaded PDF:
- Following the recommendation to align the conclusions with the evaluated setting, we have further updated Sec. 4.3 and the Conclusion, which now match the abstract, introduction and Limitations in consistently stating the tested scope (tabletop manipulation on single- and dual-arm parallel-jaw robots).
We will also be happy to incorporate any further suggestions into the final version of the paper. We are grateful for the thorough feedback, which has helped us strengthen the paper, and would be deeply grateful if the updated manuscript could be reflected in the final assessment.
WR
It added some experiments or only some tables, but "only add", also some new problems were brought from them. Also some issues in the summary is not answered.