acceptodds
Under review as a conference paper at ICLR 2027

QFast: Fast Quantized Inference for Robotics Vision-Language-Action Models

Abstract

Vision-Language-Action (VLA) models have demonstrated strong performance in robotic manipulation by unifying visual perception, language understanding, and action prediction within transformer-based architectures. However, their deployment on resource-constrained edge devices faces critical challenges: inference latency and memory requirements that prevent real-time operation. Existing quantization methods prioritize accuracy preservation and memory compression while largely neglecting inference speed—a bottleneck for real-time robotic control requiring sub-100ms latency. We present QFast, a quantization inference system designed to accelerate VLA models on edge devices. QFast employs speed-oriented quantization that balances accuracy and latency through module-specific bit-width selection, per-token weight quantization with dynamic per-channel activation quantization, and selective preservation of precision-sensitive layers. To exploit hardware capabilities, we implement optimized GEMM kernels tailored to diverse VLA matrix dimensions, enabling native low-bit computation on Tensor Cores. Evaluated on Pi0.5 and OpenVLA, QFast achieves 1.3 end-to-end speedup with up to 3.6 memory reduction while maintaining task performance on LIBERO benchmarks. QFast represents the first quantization system enabling hardware-accelerated low-bit inference for VLA models, making real-time edge deployment practical. Code is available at https://anonymous.4open.science/r/QFast-D157/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Vision-Language-Action (VLA) models, Inference Accleration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ppcAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

QFast is a quantization inference system for vision-language-action models. It combines per-output-channel weight quantization, dynamic per-token activation quantization, cuBLASLt INT8 matrix multiplication, and custom quantization and rescaling kernels. The evaluation covers π0.5 and OpenVLA-OFT on LIBERO suites. Table 1 reports 1.22× and 1.31× policy-inference speedups at 8 bits, respectively, with average success rates close to FP16. The 4-bit configurations provide greater memory reduction but smaller speedups and larger success-rate reductions.

Critical strengths
  1. The evaluation jointly considers inference speed, memory, and policy effectiveness. The reported 8-bit configurations improve policy-inference speed by 1.22× and 1.31× while keeping average success rates within approximately one percentage point of FP16. Evaluating task success makes the results more relevant to robotic deployment than kernel timing alone.

  2. The paper provides useful measurements of the limits of low-bit acceleration. Appendix E includes matrix-shape sweeps and repeated timing measurements, including configurations where W8A8 is slower than FP16. Appendix F connects these measurements to the vision encoder, language prefix, and action expert. These results help identify where quantization overhead can erase arithmetic savings.

Critical weaknesses
  1. Deployment claims and optimization benefits are insufficiently validated. Experiments use A800/H100 GPUs without absolute policy latencies or edge-device measurements. Comparisons with optimized VLA inference backends such as FlashRT are missing, leaving QFast’s practical advantage in latency, memory usage, and task success unclear. A basic cuBLASLt W8A8 baseline and ablations of the custom optimizations are also needed to isolate their contribution to the reported speedups.

  2. The 4-bit execution path and its performance trade-off are insufficiently explained. Table 1 does not distinguish W4A4 from W4A8, and the paper does not explain why its 4-bit configurations achieve lower speedups than W8A8. This makes it difficult to assess whether the limitation comes from the quantization format or its implementation.

  3. Task-performance validation is limited to near-ceiling LIBERO results. FP16 success rates already average 96.2% and 97.0% (Table 1), potentially obscuring quantization-induced failures on harder tasks. A more challenging benchmark is needed to assess performance preservation under precise manipulation or longer sequences of dependent actions.

Questions for authors
  1. Would QFast’s latency–accuracy advantage persist on a Blackwell GPU such as the RTX 5090 when compared with native FP4 inference? Since FP4 differs from the paper’s INT4 format, could the authors discuss how hardware support and numerical format affect this comparison?

  2. Does QFast preserve task success on a more challenging benchmark than the evaluated LIBERO suites, particularly for tasks requiring precise manipulation or longer sequences of dependent actions?

  3. What are the absolute policy-inference latencies corresponding to Table 1? FlashRT (an optimized real-time inference engine for VLA models) already reports 44 ms for π0.5 on Jetson AGX Thor, achieving a deployment target of <100 ms. What advantage does QFast offer over this optimized backend in latency, memory usage, or task success?

  4. What weight and activation precisions are used in the 4-bit configurations in Table 1? Do they use native INT4 Tensor Core computation, and what explains their lower speedups than W8A8?

1 comment

The authors did not submit a rebuttal during the author response period. Therefore, my original score remains unchanged.

Reviewer_mhKhOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors introduce a quantization framework for improving inference speed of VLAs. To be specific, they apply int4 or int8 quantization to the specific model components with dynamic activation, and provide custom GPU kernels, while preserving even higher precision for some layers. The key contribution claimed by the authors is an inference framework tuned by VLA workloads that can turn quantization into acceleration of inference speed instead of just reducing memory usage. They conduct experiments on LIBERO benchmark using pi0.5 and OpenVLA-OFT.

Critical strengths
  1. The motivation of the paper is clear where they target a practical bottleneck that using low-bit quantization to reduce both inference latency as well as memory usage for VLAs.

  2. This paper provides an interesting evaluation on GPU GEMM microbenchmarks for a quick glance of both behavioral impact and the proposed int8 acceleration.

Critical weaknesses
  1. My main concern is novelty and contributions of this work. The proposed QFast is an engineering pipeline of well-studied quantization and kernel techniques (INT8 Tensor Core GEMM, dynamic activation quantization, mixed precision and custom low-bit kernels). The only possible contribution I can figure out is the VLA-based integration and tuning. However, the authors failed to clearly isolate what is actually novel over the existing VLA-based quantization methods in either related work and experiment sections under matched conditions. The lack of the report of comparing with the existing work on speeding up VLAs and low-bit quantization makes me hard to point out its contributions.

  2. All experiments are evaluated only on LIBERO benchmark. This is clearly not sufficient for a quantization work, especially considering the key contribution is an engineering integration. The evaluations should be at leasts conducted on RoboCasa, RoboTwin, and RoboDojo benchmarks. Moreover, there is no experiments on real robot settings, which is important for a work that claims their contributions on deployment.

  3. The evaluation metrics are poorly defined. To be specific, the authors do not clearly detail the timing boundaries for its policy-level end-to-end speedup or fully describe how the measurements can be reproduced. Its kernel section also reports conflicting peak and anchor speedups and latencies.

  4. The authors mentioned that all reported results using 100 episodes per suite, however, there is no statistical significance included (e.g., uncertainty estimates and significance tests for the repeated experiments). Some individual suite scores fall several percentage points below FP16, so the reported aggregated success rates do not support the main claim that quantization can preserve the task performance.

  5. In the paper, the authors label pi0.5 as a 0.5B-parameter model. How come? Pi0.5 is based on a 3B PaliGemma backbone plus a LLM-based action expert. They also misattributes the pi0.5 reference, and describes its flow-matching action generation as autoregressive.

  6. To be honest, this paper gives me the impression that the whole work is an auto-research using Claude. Although I have no direct evidence, the writing styles of both paper and code are very Claude-style. This is not a problem for a work, but the authors should clearly provide a feedback of in what percentage of the paper is built upon GPTs, and which numbers are reported by GPT. This is important, since the recent GPTs could fabricate numbers even though you ask them to directly copy from shells.

Questions for authors

See all my points in Critical Weaknesses.

2 comments

The authors did not provide any rebuttal during the response period. Thus, I will keep my original evaluation.

Thank you for your time. We did not submit a formal rebuttal because we had no access to a physical robot and could not complete real-robot experiments within the review period. We appreciate the opportunity to discuss our work with you.

  1. Motivation. Our goal is to release an open-source VLA quantization method that delivers measurable inference speedups through low-bit arithmetic. This goal guided the design of each component. Currently available open-source VLA quantization implementations largely rely on fake quantization, which simulates the effects of quantization in floating point. For example, the released QuantVLA script uses W4A8 fake quantization. A separate issue in the QVLA repository raises questions about whether its released code matches the method described in the paper. QuantVLA script, QVLA issue #5

  2. SimplerEnv results. In our additional evaluation on SimplerEnv’s google_robot_move_near task, optimized W8A8 achieved a 1.119× speedup over FP16, with both configurations achieving 70% success. We used the same 20 seeds (0–19) for both configurations and capped each episode at 80 steps.

    ConfigurationSuccess rateInference latency after warm-up (mean ± SD)Speedup over FP16
    FP1670% (14/20)194.733 ± 1.939 ms1.00×
    W8A870% (14/20)174.043 ± 2.095 ms1.119×
  3. Remaining comments. We will address points 3, 4 and 5 of the review in the revised manuscript.

Reproducibility and AI use. We did not use an automated AI research workflow for this study, and all reported results are genuine and reliable. Our open-source repository provides a script that lets others rerun the experiments end to end with a single command and independently check the results. Our manuscript’s AI-use statement explains how we used AI tools.

We have found AI tools very helpful in our own research. As we learn how best to use them, we hope that care and rigor will continue to guide our work as a research community.

Reviewer_wtsfOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem that, although quantization can reduce the memory footprint of Vision-Language-Action (VLA) models, additional computational overhead, such as dequantization, may increase inference latency. To address this issue, the paper proposes QFast, a quantized inference system that prioritizes inference speed. The system employs weight and activation quantization techniques, combined with some implementation optimizations to improve inference efficiency. The paper evaluates QFast on π0.5 and OpenVLA-OFT, achieving inference speedups of 1.22× and 1.31×, respectively, under 8-bit quantization. It further analyzes the acceleration achieved across different matrix dimensions and model components.

Critical strengths

The paper addresses a practically important problem. Although quantization can reduce the memory footprint of VLA models, additional computational overhead, such as dequantization, may increase inference latency, limiting its practical benefits. This issue is particularly important in robotics, where inference latency directly affects control frequency and real-time operation. Therefore, prioritizing inference latency rather than memory compression alone is a well-motivated and practically meaningful research objective.

Critical weaknesses
  1. Unclear Contributions and Very Limited Novelty
  • Sections 3.1-3.4 mainly follow standard quantization procedures, while Section 3.5 primarily presents implementation-level engineering optimizations. Although the paper emphasizes a latency-first design philosophy, the methodology does not clearly explain how this philosophy translates into technically novel methods. Furthermore, the paper does not sufficiently clarify the fundamental differences between the proposed system and existing high-performance quantized inference solutions.
  • Several existing VLA quantization methods that have demonstrated inference acceleration are not included in the experimental comparisons. For example, QVLA has already reported inference speedups. Therefore, the paper's claim of being "the first demonstration of actual acceleration for quantized VLA models" appears insufficiently justified.
  1. Insufficient Experimental Validation
  • The paper emphasizes deployment under resource-constrained conditions in its motivation. However, the experiments are conducted on GPUs such as the A800. The lack of evaluations on representative edge devices, including inference latency, memory usage, and actual runtime performance, leaves the claimed deployment benefits insufficiently validated.
  • Section 3.5 introduces several implementation-level optimizations. However, the paper does not sufficiently demonstrate the performance improvements contributed by each optimization, nor does it provide a detailed analysis of the specific conditions under which each optimization is effective.
  • The paper mainly evaluates the performance preservation of quantized models using task success rates on LIBERO. Although success rate is an important task-level metric, for models with already high success rates, small performance differences may not adequately capture behavioral changes introduced by quantization. Therefore, evaluations on additional simulation and real-world robotic benchmarks could provide a more comprehensive assessment of QFast's ability to preserve model accuracy.
Questions for authors
  1. Compared with standard W8A8 quantized inference approaches, what are the specific technical innovations introduced by QFast? How is the proposed latency-first design philosophy reflected in these innovations? How much does each innovation contribute to the overall inference performance?
  2. Existing VLA quantization methods, such as QVLA, have already reported inference speedups. Why does this paper still claim to provide “the first demonstration of actual acceleration for quantized VLA models”? Could the paper further clarify how this claim is distinguished from prior work?
  3. Is QFast effective on actual edge devices? Have the authors conducted any related experiments or deployment attempts, and are there any results available?
  4. Have the authors evaluated QFast on real robotic systems? If so, have they measured the relevant performance metrics during real-world deployment?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.