- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper provides a suite of tools for VLM-based robot control systems, including spatial localization, depth measurement, object tracking, and grasp candidate generation, enabling the model to execute operations based on tool feedback. The trajectories resulting from successful tool invocations are then used to fine-tune smaller Qwen models. The authors evaluate the approach across LIBERO-PRO, RoboSuite, and RoboTwin, covering scenarios such as the deployment of frozen models, student model training, and cross-robot transfer.
Critical strengths
-
The authors distinguish between visible surfaces and the full pose of an object, between measurement hypotheses and executable grasps, and between historical identifiers and currently valid coordinates. These design choices help mitigate the risk of a VLM treating uncertain perceptual outputs as definitive facts that are a feature of practical value for real-world systems.
-
This paper is well-structured, clearly presents detailed information about the model, and provides plenty of clear figures and diagrams. The technical details in the supplementary material effectively complement the main text.
Critical weaknesses
-
Some aspects of the experimental design are not fully considered. In the RGB baseline, the episode terminates immediately upon a done signal, whereas K1 receives a native "task success" determination and allows for corrective actions in the event of failure. The RGB-only Astra baseline uses different control, memory, and completion-feedback mechanisms, while the RGB-only Qwen RUA baseline has fewer observation inputs and suffers from frequent invalid action outputs. These differences make it difficult to isolate the contribution of the proposed perception tools.
-
The transferability to real-world hardware remains unclear. The experiments rely on two somewhat idealized assumptions: first, that the simulation environment pauses during model inference and tool invocation, which does not apply to the real world; and second, that the done signal, indicating task completion, is directly available from the simulator, whereas such feedback is typically absent in real-world settings.
Questions for authors
Q1. Fairness of experimental comparison. The RGB baseline terminates upon done, while K1 receives task-success feedback and can continue execution. Could the authors provide an ablation to isolate the contribution of perception tools from differences in feedback, control, and memory?
Q2. Fairness of the RGB-only RUA baseline. The RGB-only RUA baseline has fewer observation inputs, and some of its evaluations fail due to invalid outputs. Would matching the input modalities and using constrained decoding lead to a fairer comparison?
Q3. Real-world applicability. The experiments pause simulation physics during inference and perception queries and rely on simulator-provided task-completion feedback. How would the system handle inference latency and task-completion detection on real robots?