- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper tackles the challenge of reducing end-to-end interaction time between a user and a VLA policy based on typing or speaking the language command. The authors propose Premover, a lightweight module on top of a frozen VLA, based on a contrastive focus map and action-readiness gate, that begins execution from a partial streaming instruction, once its informative enough for the policy to begin executing action chunks; it continues updating as more instruction tokens stream in. They validate in LIBERO, VLA-Arena, and the SO-101 arm and modestly decrease interaction time (6-18%).
Critical strengths
Action-readiness gate: the manuscript's most convincing result is the ablation done in Table 4 that shows the action-readiness gate carries most of the benefit, raising success rate from 58.4 (acting immediately with naive streaming) to 92.2 (with very minor improvement in clock-time). It suggests that we can start acting earlier for simple pick and place tasks.
Critical weaknesses
Missing baselines: there are a few baselines that are not present which I think are necessary to assess the benefits conferred via Premover. Listed is one below and one in the questions.
(i) VLM as Gate: firstly, I think using an LLM directly as the readiness gate should be included. While readiness labels are annotated offline with Qwen judging when a prefix is sufficient, the obvious baseline is to use the same VLM online as judge to gate when an instruction contains enough information to begin execution. If this works, the action-readiness gate adds little, beside being lightweight
Minor Improvements: the gains reported (6-8% in simulation and 18% on real-world) are minor. This would be fine for complex tasks, but in pick and place tasks where the task instructions clearly delineate progression throughout a task, e.g., I can perform 'pick up' before determining where to 'place', the improvements are weak.
Questions for authors
Can you include the VLM as Gate baseline? that I mention above. Also, can you consider this ablation:
(Ablation): Full Prompt + Focus-Map: on the real-robot, Premover is more successful than full-prompt (86% vs. 81%), which doesn't make sense because all we are doing is starting earlier. My hunch is that this is due to visual reweighting with the contrastive alignment objective. Without this ablation, I can't tell how much gain comes from early execution.
5 comments
Author Response [2/2]
[Question 1]
Can you include the VLM as Gate baseline?
[Response 1]
Thank you for raising this question. We have addressed it in our response to Weakness 1 above.
[Question 2]
Full Prompt + Focus-Map: on the real-robot, Premover is more successful than full-prompt (86% vs. 81%), which doesn't make sense because all we are doing is starting earlier. My hunch is that this is due to visual reweighting with the contrastive alignment objective. Without this ablation, I can't tell how much gain comes from early execution.
[Response 2]
We appreciate the suggestion to separate the effects of visual reweighting and early execution. To address this concern, we have evaluated a Full-prompt + Focus-map control that uses the complete instruction with the same trained focus-map reweighting, including the contrastive alignment objective.
| Task | Full-prompt | Full-prompt + Focus-map | Premover |
|---|---|---|---|
| Block→Cup | 65 | 70 | 75 |
| Cat→Cup | 95 | 95 | 90 |
| Cup→Cup | 90 | 95 | 95 |
| Push Cup | 85 | 90 | 90 |
| Drawer | 70 | 80 | 80 |
| Mean | 81.0 (81/100) | 86.0 (86/100) | 86.0 (86/100) |
Full-prompt + Focus-map improved mean success from 81.0% to 86.0% (+5.0 pp), matching Premover. This shows that focus-map reweighting improves task success even when execution starts after the full instruction, rather than the gain coming from early execution alone. Premover also achieved lower mean Wall than Full-prompt + Focus-map (49.7 vs. 56.9 s, a 12.7% reduction), while maintaining the same mean success. Thus, Premover retains a timing advantage beyond the success improvement provided by focus-map reweighting.
We have added these control results to Section 4.5.2 and Appendix F.1, with successful-episode Wall also reported in Appendix F.2.
Thanks for the additional experiments. The LLM/VLM (alternative streaming) baselines in Table 7 address my first concern. The Full-prompt + Focus-map control answers my question, but it shows the real-robot success gain comes from the focus map, not early execution (86.0% for both). So early execution saves only 12.7% rather than the reported 17.9%, and the new pre-motion analysis suggests the focus map accounts for at least part of this. It would be good to report this in the abstract and introduction, since they currently overstate the effect of early execution. My original concerns about the size of the gains and the reliance on pick-and-place instruction structure remain, so I am keeping my score.
Author Response [1/2]
Dear Reviewer N8Vh,
Thank you for your constructive review and suggestions. We have evaluated an online VLM-as-Gate baseline and a Full-prompt + Focus-map control, and address the magnitude of the latency gains below. We have revised the manuscript accordingly and highlighted the changes in blue for ease of reference. For clarity, we address the individual points within each comment separately below.
[Weakness 1]
[Response 1]
Thank you for suggesting this baseline. We have evaluated Qwen3.5-2B, the same model used for offline readiness annotation, as an online readiness gate. Unlike offline annotation, the online VLM judges whether the current instruction prefix contains enough information to begin execution using only the words received so far and the current camera observation. Premover achieved higher success and lower Wall than VLM-as-Gate, with substantially less GPU memory. All methods are evaluated with π0.5 on LIBERO at the speaking rate, following the setting of
Table 2.Although Qwen released execution earlier than Premover (6.5 vs. 8.2 words), its success rate was 8.28 pp lower (89.29% vs. 97.57%). Including Qwen inference overhead, its Wall was also higher (8.10 vs. 7.49 s), nearly the same as Full-prompt (8.17 s). Qwen additionally required 5.0 GiB of GPU memory, compared with 13.6 MB for Premover's heads.
These results show that using Qwen directly as an online readiness gate does not match Premover's success–latency trade-off in this setting. The learned readiness gate therefore offers benefits beyond its smaller memory footprint.
We have added this baseline and its failure analysis to
Section 4.5.4.[Weakness 2]
[Response 2]
We agree that the average latency improvements are modest in some settings. However, the gains vary considerably across tasks and instruction-delivery rates. In simulation, Premover reduced mean Wall by 5.9-8.3% at the speaking rate, with larger reductions of 11.0-23.5% at the typing rate. On the real robot, the average reduction was 17.9%, with larger improvements on individual tasks, including 38.5% on Cup→Cup (53.5 to 32.9 s) and 26.4% on Block→Cup (66.6 to 49.0 s).
Our primary goal is to demonstrate that the instruction-delivery interval, which conventional VLAs spend waiting, can be used for robot execution without substantially compromising task success. The achievable gain naturally depends on the task and how much execution can overlap with the remaining instruction. Our results demonstrate that this previously unused interval offers a practical opportunity to reduce end-to-end interaction time, even for relatively simple manipulation tasks.
We have discussed the factors affecting the available overlap in
Section 5.(Our response continues in the next comment.)