- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
QFast is a quantization inference system for vision-language-action models. It combines per-output-channel weight quantization, dynamic per-token activation quantization, cuBLASLt INT8 matrix multiplication, and custom quantization and rescaling kernels. The evaluation covers π0.5 and OpenVLA-OFT on LIBERO suites. Table 1 reports 1.22× and 1.31× policy-inference speedups at 8 bits, respectively, with average success rates close to FP16. The 4-bit configurations provide greater memory reduction but smaller speedups and larger success-rate reductions.
Critical strengths
-
The evaluation jointly considers inference speed, memory, and policy effectiveness. The reported 8-bit configurations improve policy-inference speed by 1.22× and 1.31× while keeping average success rates within approximately one percentage point of FP16. Evaluating task success makes the results more relevant to robotic deployment than kernel timing alone.
-
The paper provides useful measurements of the limits of low-bit acceleration. Appendix E includes matrix-shape sweeps and repeated timing measurements, including configurations where W8A8 is slower than FP16. Appendix F connects these measurements to the vision encoder, language prefix, and action expert. These results help identify where quantization overhead can erase arithmetic savings.
Critical weaknesses
-
Deployment claims and optimization benefits are insufficiently validated. Experiments use A800/H100 GPUs without absolute policy latencies or edge-device measurements. Comparisons with optimized VLA inference backends such as FlashRT are missing, leaving QFast’s practical advantage in latency, memory usage, and task success unclear. A basic cuBLASLt W8A8 baseline and ablations of the custom optimizations are also needed to isolate their contribution to the reported speedups.
-
The 4-bit execution path and its performance trade-off are insufficiently explained. Table 1 does not distinguish W4A4 from W4A8, and the paper does not explain why its 4-bit configurations achieve lower speedups than W8A8. This makes it difficult to assess whether the limitation comes from the quantization format or its implementation.
-
Task-performance validation is limited to near-ceiling LIBERO results. FP16 success rates already average 96.2% and 97.0% (Table 1), potentially obscuring quantization-induced failures on harder tasks. A more challenging benchmark is needed to assess performance preservation under precise manipulation or longer sequences of dependent actions.
Questions for authors
-
Would QFast’s latency–accuracy advantage persist on a Blackwell GPU such as the RTX 5090 when compared with native FP4 inference? Since FP4 differs from the paper’s INT4 format, could the authors discuss how hardware support and numerical format affect this comparison?
-
Does QFast preserve task success on a more challenging benchmark than the evaluated LIBERO suites, particularly for tasks requiring precise manipulation or longer sequences of dependent actions?
-
What are the absolute policy-inference latencies corresponding to Table 1? FlashRT (an optimized real-time inference engine for VLA models) already reports 44 ms for π0.5 on Jetson AGX Thor, achieving a deployment target of <100 ms. What advantage does QFast offer over this optimized backend in latency, memory usage, or task success?
-
What weight and activation precisions are used in the 4-bit configurations in Table 1? Do they use native INT4 Tensor Core computation, and what explains their lower speedups than W8A8?
The authors did not submit a rebuttal during the author response period. Therefore, my original score remains unchanged.