- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents Fast-VLA, an efficient vision-language-action framework that decouples episode-level task understanding from continuous robot control. The proposed method invokes a frozen 4B vision-language model (VLM) once at the beginning of each episode to construct a fixed, multi-layer key-value task memory from the language instruction and initial visual observations.
During execution, a lightweight online actor processes current visual observations, short-term visual history when available, and proprioceptive feedback. Through learned action queries and cross-attention mechanisms, the actor retrieves task-relevant information from the cached memory and predicts action chunks in a single forward pass without repeatedly invoking the large VLM.
The authors evaluate Fast-VLA on RoboTwin 2.0, LIBERO, and LIBERO-Plus. The proposed framework achieves competitive manipulation success rates while substantially reducing amortized inference latency and online GPU memory consumption. Additional ablations investigate the importance of task memory, online observations, memory refresh frequency, and different memory encoders.
Critical strengths
-
The paper addresses an important practical limitation of current VLA architectures: the substantial computational overhead associated with repeatedly executing large vision-language backbones during robot control. The proposed separation between episode-level task understanding and lightweight online control is intuitive and well motivated. By reusing task representations across action predictions while maintaining access to current visual and proprioceptive feedback, Fast-VLA offers a promising direction for reducing inference costs without completely sacrificing closed-loop control. The overall framework is also relatively straightforward, making the proposed approach potentially attractive for practical robotic applications.
-
The reported experimental results are impressive, particularly on RoboTwin 2.0, where Fast-VLA achieves 92.46% and 92.40% success in the Clean and Randomized settings, respectively. Despite maintaining competitive task success rates, the method achieves a substantial reduction in inference latency compared with several strong VLA and WAM baselines. The reported 10.9–17.1× reduction in amortized inference time, together with the reduced online GPU memory footprint, demonstrates a compelling performance-efficiency trade-off. Importantly, the authors include the one-time VLM prefill cost in their amortized latency measurements, making the reported efficiency gains more informative than comparisons based solely on the online actor's forward-pass latency.
-
The paper provides evaluations across three established manipulation benchmarks, including robustness assessments on LIBERO-Plus. The ablation studies are particularly useful for understanding the proposed architecture. Memory swapping experiments demonstrate that the online actor relies on episode-matched task representations, while comparisons involving different memory constructions and encoders help establish the contribution of the multi-layer KV memory. Furthermore, the memory refresh experiments provide evidence that repeatedly invoking the VLM is not necessary to achieve competitive performance in the evaluated settings. Overall, these experiments provide meaningful empirical support for the proposed efficiency-oriented architecture.
Critical weaknesses
-
My primary concern is the absence of real-world robotic experiments. Although the simulation results are promising, the paper does not provide experimental validation on physical robotic systems. This limitation is particularly important given the paper's central motivation of enabling efficient and responsive robot manipulation. While simulation benchmarks provide useful measurements of policy performance, they cannot fully capture the practical challenges encountered during real-world deployment, including perception uncertainty, control inaccuracies, hardware communication overhead, and unexpected environmental disturbances. Moreover, the reported inference efficiency is measured through offline profiling on an A100 GPU rather than through end-to-end execution on a physical robotic platform. Consequently, it remains unclear how the proposed computational improvements translate into practical control performance and responsiveness in real-world settings. I believe that real-world validation is particularly important for an efficiency-oriented VLA framework, as computational speedups alone do not necessarily guarantee improved robotic execution. Even a relatively small-scale evaluation on physical manipulation tasks, accompanied by measurements of actual policy latency and execution success, would substantially strengthen the practical significance of the paper.
-
The proposed framework constructs task memory once at the beginning of an episode and keeps it fixed throughout execution. Although the online actor continuously incorporates current observations, the paper provides limited analysis of scenarios where the environment changes substantially after the initial memory construction. For example, unexpected object movements, execution failures, or interruptions may introduce discrepancies between the initial task context and the current environment. The existing robustness evaluations are encouraging, but they primarily consider perturbations present when the episode begins, rather than controlled changes introduced during execution. Additional experiments involving mid-episode disturbances would help clarify the limitations of maintaining fixed task memory.
-
The reported RoboTwin configuration predicts and executes 50 actions before replanning. While this execution schedule contributes to the low amortized inference cost, it also limits the frequency at which the policy can respond to new observations. This raises questions about how well the proposed approach would perform in manipulation tasks requiring more frequent feedback, particularly those involving precise contact, unexpected execution errors, or dynamic object interactions. Although the paper provides detailed inference latency measurements, it does not sufficiently investigate how the performance-efficiency trade-off changes with shorter action execution horizons. Such an analysis would provide a more complete understanding of the practical benefits of Fast-VLA for responsive robotic control.
Questions for authors
-
Could the authors provide real-world robotic experiments to validate the proposed framework? Given that the primary motivation is improving inference efficiency for robot manipulation, it would be particularly valuable to evaluate Fast-VLA on a physical robotic platform and report both task success rates and actual inference latency. Ideally, such experiments would include comparisons with representative VLA baselines under the same hardware and control conditions. This would help establish whether the substantial computational advantages demonstrated in simulation translate into meaningful benefits during real-world execution.
-
How does Fast-VLA handle significant environmental changes after the initial task memory has been constructed? For example, if a target object is unexpectedly moved during execution, can the online actor adapt reliably using only current visual feedback and the fixed task memory? Could the authors provide controlled experiments involving mid-episode perturbations, execution interruptions, or unexpected object movements to investigate this behavior?
-
How sensitive is the proposed method to the number of actions executed between replanning steps? In particular, could the authors evaluate shorter execution horizons, such as 10 or 20 steps on RoboTwin, and report the corresponding task success rates and inference latency? This would help clarify whether the proposed efficiency advantages can be maintained in settings requiring more frequent feedback and responsive closed-loop control.
Response to Reviewer 2i2J
We thank the reviewer for the concrete experimental suggestions, which we followed. The General Response summarizes the new real-robot and dynamic-object results; here we report the key numbers and address the execution-horizon question.
Q1 / W1. "Could the authors provide real-world robotic experiments ... report both task success rates and actual inference latency ... comparisons with representative VLA baselines under the same hardware and control conditions."
Yes (General Response 1; §4.5, Table R3, Figures R1–R3, App. F.6). On an SO-101 arm with one RTX 3090, all methods are trained on the same 100 demonstrations, execute 50 actions per call, and are evaluated from the same 20 initial scenes, each reproduced with IoU > 0.8 for every object before the trial starts:
Fast-VLA matches π0.5 and exceeds DM0.5 by 10 points, with 10× and 17× lower per-call latency.
Q2 / W2. "If a target object is unexpectedly moved during execution, can the online actor adapt reliably using only current visual feedback and the fixed task memory?"
On the real robot we observed both the ball and the target box being moved during execution (App. F.6, Figure R3). When a failed grasp pushed the ball aside, Fast-VLA, with its memory still built from the first frame, re-grasped the ball from its current observations and completed the task; when the held ball pushed the target box, it released the ball into the box at the box's new position. Our 100 training demonstrations contain no such disturbances. In comparable trials, π0.5 did not re-grasp a pushed-away ball, and DM0.5 pushed the box away with an empty gripper; neither completed the task (Figure R3, rows C–D). In simulation (General Response 2), on DOMINO (35 RoboTwin tasks in which objects keep moving), Fast-VLA with memory fixed at the first frame reaches 34.2%, second only to the dynamics-specific DynamicWAM (38.2%) and well above PUMA (17.2%), π0.5 (9.6%) and OpenVLA-OFT (9.1%) (§4.4, Table R1). With another episode's memory in 10% of training samples, success with mismatched memory rises from 6.26%/8.36% to 82.90%/81.70% (§4.4, Table 4).
Q3 / W3. "How sensitive is the proposed method to the number of actions executed between replanning steps? ... evaluate shorter execution horizons, such as 10 or 20 steps on RoboTwin."
We re-evaluated the paper checkpoint on all 50 RoboTwin tasks (Clean and Randomized, 50 episodes per task and setting), predicting 50 actions and executing the first k before the next actor call, with memory still built once per episode (§4.4, Table R2, App. F.3):
Mean success stays within 92.82–94.06% across horizons, so Fast-VLA can re-observe the scene as often as every 10 steps without refreshing its memory or losing success. Because the VLM is not re-run, shorter horizons only add actor calls: at k = 10 the cost is 5.01 ms/step, still 3.2× lower than DM0.5 at its 50-step schedule (16.069 ms/step, Table 2).
The revised PDF marks all changes in blue. We hope these results address the reviewer's main concerns and would be grateful if the reviewer would reconsider the score.