- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes VLAs-as-Tools, a hierarchical system for long-horizon manipulation in which a high-level VLM agent (e.g., Gemini 3 Flash) decomposes tasks and handles replanning, while a VLA executes bounded subtasks as "tools." Two components are introduced: (1) a VLA tool-family interface, where each call is a (tool-family label, scene-grounded instruction) pair (e.g., grasp('bowl')), and the VLA returns a scalar per-call progress signal from an auxiliary head, which a threshold-based monitor uses to trigger replanning; and (2) Tool-Aligned Post-Training (TAPT), which segments demonstrations into invocation-level clips using an automatic VLM-assisted labeling pipeline (11 tool families), routes each family through its own LoRA adapter on a shared backbone, adds a progress-regression loss, and optionally runs subtask-level GRPO with simulator-state completion predicates. The paper evaluates OpenVLA, OpenVLA-OFT, and π0.5 on LIBERO-Long, an 8-task single-arm subset of RoboTwin, CALVIN, RoboMemArena, RoboLab-120, and a self-built counterfactual suite (LIBERO-IF), as well as three real-world Franka tabletop tasks (20 trials each). Reported gains include +3.6 to +5.2 points on LIBERO-Long, up to +35.5 points on RoboTwin-SA, +15.0 points in Non-biased Rate on LIBERO-IF, and an increase from 16.7% to 58.3% in average real-world success over a fine-tuned π0.5.
Critical strengths
-
The evaluation is broad and covers multiple backbones. The method is tested on three VLA backbones and six simulated benchmarks, plus real-robot experiments, and it improves consistently over the corresponding single-policy baselines. The appendix also contains useful practical results: adding a tool family with a new adapter preserves earlier skills (97.6% retention vs. 86.4% for sequential fine-tuning, Table 8), and progress-triggered replanning cuts VLM calls by more than two orders of magnitude compared with periodic VLM monitoring (Table 14). This breadth is the main argument for acceptance, because it shows the recipe is not tied to a single model or benchmark.
-
The invocation-fidelity study addresses a real failure mode. LIBERO-IF (Table 3) directly measures whether a VLA follows the invoked command rather than reproducing training-task behavior. The component ablation shows that each part of TAPT, especially the tool-family adapters (40.6 → 54.0 Faithful Rate for OpenVLA-OFT), improves instruction-faithful execution. This is the most targeted evidence in the paper, and it supports the claim that the executor has to be trained to be called reliably.
Critical weaknesses
- The paper combines known components without a focused scientific question. Hierarchical VLM planning over learned skills (SayCan, Inner Monologue, RT-H, Hi Robot, π0.5's high-level subtask prediction), per-skill LoRA experts (TAIL, AtomicVLA), VLM-orchestrated policy primitives (RoboClaw), and subtask-level progress or RL signals all have close precedents. The formalization in Sec. 3 is a generic description of hierarchical control and yields no testable insight. The paper's own ablations also weaken its more distinctive parts: under TAPT, the interface adds only about 1 point on LIBERO-Long (Table 4), a coarse taxonomy performs almost as well as the proposed one (Table 7), and the choice of VLM planner hardly matters (Table 13). The paper reports system-level gains, but the reader learns little that transfers beyond this particular pipeline.
- The evidence does not isolate the contribution against fair or strong baselines. The only hierarchical baseline sends subtask calls to a VLA trained on full-task instructions, so its drop in performance is expected. The paper never tests a planner combined with a single VLA trained with standard SFT on the same segmented data, nor existing hierarchical VLA systems such as π0.5's native high-level inference or Hi Robot. TAPT also receives extra DROID-split mid-training that the baselines do not appear to receive, the largest gain comes from RoboTwin-SA, an 8-task subset the authors selected themselves, and no variance is reported. The three real-world tasks are simple cube/bowl tasks, and the largest gain ("Conditioned") comes from observation aliasing that any stateful planner resolves by design. The real-robot results therefore show that hierarchy helps a memoryless π0.5, not that TAPT or the progress interface does.
Questions for authors
What is the success rate of a "planner + single VLA SFT on the same invocation-segmented data" baseline, with no tool-family adapters, no progress head, and no DROID-split post-training, using VLM-based or timeout-based subtask termination? This is the most direct test of whether the TAPT-specific components matter. A related question: what happens if the same baseline is run on the real robot?
Thank you again for your thoughtful feedback, and if you have a chance to review our rebuttal, we’d be happy to clarify any remaining questions.